My camera is sending XAVC 4K 30fps 100mbps via CamLink 4K HDMI>USB adapter. Quicktime records ProRes422 of the feed at 29.97fps. VDMX will encode 27.5 ish if I designate the source as the CamLink4K, ie upstream of any VDMX processing. I have disabled every plugin and tried every combination of throttling FPS and smoothing input FPS, trying different video codecs in the recorder settings and also in the global settings. The frame rate goes down more, the more layers of VDMX it is going through (ie 22fps ish if I record the layer, 20fps down to 15fps depending on other settings when I record the master output). FX are disabled everywhere in the path. I am trying to record the 4K stream and only one layer, to troubleshoot. Is VDMX doing something massive in the background, or what? Why does it seem I am experiencing a resource crunch? Activity Monitor says I still have 50% of CPU idle. Please share insights. TIA
UPDATE: I found a couple active plugins and disabled them and this brought the encoded FPS up to 26 when recording the layer and 28 when recording the master output. The FX in question were layer masks of left mix on the right mix, and vice versa. So, not actually doing anything when only one source is active.
Also FYI I previously ejected every media element so things weren’t playing in the background. This was not what made the difference, I did that prior to my earlier tests. So something is still knocking it down but not much. Also FYI during these tests I did not get warning messages about dropping frames, which I usually do with a serious overload.
The movie recorder is trying to record frames at the max rate that your computer can support. Which video codec are you using and what are the specs of your computer?
Also, what is the speed of the drive you are writing to? Is it also your internal SSD that everything else is running off of, or a separate drive?
Also, do you have it so that VDMX is importing the media after you finish recording it?
Lastly, as an alternative you can try the syphon recorder Syphon Recorder | Syphon
Thanks for these tips, I’ll dig in. I have a macbook pro M1 max with internal 1TB SSD. Everything is on it, no partitions (when I got it, the VDMX team reported testing on the M1 and it could run 100 layers of 4K simultaneously, so honestly I am surprised to be hitting a wall).
The problem seems to arise differently under different circumstances, but I think it boils down to converting my 4K XAVC source to HD then doing any processing at all, and writing a h264 file in real time. So I will see what happens when I configure all sources every time (I was having success leaving all sources at their highest frame rates and resolutions and relying on VDMX to convert down when mixing, but this new issue mostly means I have to give closer attention to the specs, I think).
I am not importing the media post recording. The drive has 300GB of free space currently out of 1TB, so a housecleaning is due soon, but usually I don’t see any issues until there is less than 10% remaining..
I have also noticed that when issues do arise, Activity Monitor tells me that I have more than 50% idle CPU so it seems weird that I am having a resource crunch. FWIW the M1 Max has more GPUs than CPUs and I don’t think I can designate them in VDMX to do any of the heavy lifting (say, GPU doing the plugin processing and leaving CPU to write the file) - please correct me if I’m wrong and it is possible!).
I have not yet tried Syphon recorder but I downloaded it. I’m really not doing that much, just capturing a video, maybe applying a LUT (but it didn’t help when I removed that). I feel like I should be able to do ten times as much processing without lifting a finger. I am using VDMX5 btw and my OS is Monterey - I can’t update or all my firewire audio gear has to be replaced.
109 layers of HAP HD actually, but that also required multiple external SSDs because drive bandwidth is the limiting factor. The internal SSD is somewhere between 3000-5000MB/s read write, and most external SSDs are anywhere between 1000-2000 MB/s. External hard drives are usually 200MB/s or less. So which drive you read and write to makes all the difference.
If you’re asking a lot from the internal SSD it can be a bottle neck since your system is using it to function. And most SSDs start slowing down after 80% is full. 
And if your system is low on memory, it uses the internal SSD for SWAP (as memory) which cuts down on your SSD speeds even more.
Lots of variables there.
I also believe that capturing and encoding to H264 in real time isn’t the best option. The encoding will take more time and processing than starting with a lossless codec, even if the file size is larger. Later, you can re-encode to whatever is most convenient for your needs.
Otherwise, it’s clear there’s a bottleneck, and the limitation isn’t the software in this case.
The thing is, when I am shooting, I shoot 3-4 hours in a session, so if I record in ProRes etc the data size would be astronomical. But I need to be able to capture a lot of footage once my scenes are set up. Usually I am working in HD and don’t experience bottleneck issues. If I look at the common element in all the videos that are dropping frames, it’s that I’m ingesting 4K XAVC and mixing it with HD and other resolutions (lately just XAVC sources but sometimes webcams or NDI too). Scaling those up doesn’t seem to bog the system down, but scaling 4K down really seems to. I am also sending XAVC from the camera but I could send mp4, however I’m hoping to spec up not down when possible. Sometimes I record in the camera at the same time.
Can you tell me this: in the general settings, the “record” codec was set to PJPEG and I never changed it until yesterday. I tried the other options with no better outcome. But should this setting match the setting in the movie recorder plugins? Or should I do my best to match to what the camera(s) is sending (and if they are not the same, then what?) I assume ingest conversion adds a level of processing.. I believe the HDMI feed from Sony cameras is 4:2:2 so maybe the ingest should be ProRes but I write the file as h264? Thoughts on this chain?
EDIT: I’m reading that the Hap format engages the GPU; would this enough take load off the CPU to resolve any resource crunch, and in doing so, would it reduce the difference in image quality between ProRes and Hap vs all the processing being done in the CPU? Basically I want best quality but Hap is described as speed over quality - presumably only when it has to choose? So if the work is spread out then no crunch.. I suppose both are better than h264 so maybe I’ll experiment with Hap.
How are you getting the data from your camera into the computer?
I’m using the HDMI out into a CamLink 4K capture device then USB 3 into a USB-C hub. I sometimes have a webcam plugged into the same USB-C hub, and some MIDI controllers but my other HDMI feed (HD not 4K) is coming in through a different USB-C port on the mac. Should be well within bandwidth, I think all the specs are about 5gbps and I’m only sending 100mbps from the camera. I think. That might be the internal record data rate bc I’ve seen up to 550mbps when I record ProRes in quicktime. In any case, still well under 5gbps going into any port.
Mini suggestion: Check if all your input sources that are not in use are properly deactivated (in the input sources inspector). If you have multiple cameras connected, including built-in cameras and NDI sources, they might be invisibly active (so VDMX can use them instantly) which means they are consuming their full bandwidth which could explain the strange framerate.
I’m working with many cameras and whenever I don’t need some of them, completely disabling them in the inspector instantly improves framerates. This might not solve it for you, but just jumped out at me reading your last post, maybe its other sources using the bandwidth.
If this is the case, then it doesn’t matter what settings you adjust on the cameras, that won’t change how your computer is handling it while capturing with the CamLink 4K over HDMI. CamLink acts like a UVC web camera and simply imports your HDMI stream. It will take up some resources, but changing XAVC or H.264 on you camera shouldn’t have anything to do with your cameras HDMI output.
If you have multiple camlink devices, I would attempt to plug them into individual thunderbolt 4 ports to avoid USB bandwidth issues. This is more common in USB 2.0, but the quality of your USB C hub could also make a difference.
Does this sound right: camera sends 4:2:2 feed over HDMI. CamLink turns it into a webcam ie USB data procotols, bandwidth specs. VDMX “recording” preference is an ingest setting basically, so if I choose ProRes there, and I choose ProRes as the Movie Recorder format, then there is no codec translation math to bog down the CPU. Is that right?
And then if I want to work with multiple sources in HD, not 4K (I only have one 4K cam) then I should just set the A cam settings to HD (I can shoot at 120p in HD but only 30p in 4K).
I can also set my Sony cam to mp4 as a recording format. I don’t know for sure if that makes the HDMI feed mp4 formatted as well, but if I am using iPhones etc as extra sources, I might as well ingest as h264 and record as h264. Like, for sketches etc. I think using a UHD source is adding math right off the bat if I’m going to mix down to HD, so then I should try to config the cam feed to HD if possible.
Your camera might send 4:2:2, but CamLink is most like 4:2:0 so you’re not getting the most out of your cameras unless you record directly onto them. What you do have some say over is Camlink resolution and frame rate. If your camera supports HDMI 4K60, then you should be able to bring in the higher framerate and resolution into VDMX.
For the movie recorder, if you use H.264 you are going to use CPU power, if you use HAP, it will be more on the GPU. HAP, Prores, and other larger codecs will use up more bandwidth on your internal or external storage (SSD or HDD). Other than output resolution and frame rate, your cameras settings have little to do with what happens on your computer. It doesn’t matter if you change your camera record settings to h264 or XAVC, that only relates to recording on the camera itself, not passing a signal through Camlink.
If you use an external drive, make sure it is fast enough with the Black Magic Disk Speed test app.
If you are working with a 4K HDMI signal, then make sure VDMX’s composition resolution and the movie recorder resolution matches the incoming signal to avoid scaling up or scaling down the content. Make sure if you choose HAP or ProRes that your drive (internal or external) is fast enough.
I don’t know what you mean by 120p. Usually, it’s the resolution first and frame rate after the p e.g. 1080p30 or 4Kp60. Also, the p should mean progressive, so 30 fps, or 60fps. Something like 59.94i or 29.97 may not be progressive, but interlaced (less common / likely with newer cameras).
You can check your camera manufacturer specs, but setting your Camera to 120 fps, won’t matter if the camlink is limited to 60fps. Does that make sense?
We are mainly troubleshooting your camlink and the VDMX settings (movie recorder, preferences), based on your system setup. The camera could be a $12 usb camera and still be having the same issues if the computer is limited in some sort of way.
P is just an abbreviation or fps (in this instance). I looked into the CamLink specs and it converts down to 4:2:0 at 4K but maintains 4:2:2 at 1080p. The camera records 4:2:0 internally (plus it stops every 30 mins, annoyingly). So in theory the HDMI stream is a better option than the internal recorder, they recommend sending HDMI for capture to an external recorder for highest spec.
The CamLink does top out at 60fps when receiving HD. I also learned (with an ugly shot) that shooting 120fps on the cam forces a closer crop.
I’m trying to do 2 things: shoot a single 4K shot (I find I’m preferring 24fps to 30 anyway) OR shoot a mix, including sources that are 1080p and sometimes even lower (eg iPhone screen capture). I think I just have to do more tech config each time than I hoped. Also I’m doing fine capturing 4K with Quicktime, if it’s just one cam. So I have that option too, and maybe I’ll leave WDMX set up for the 1080p collages.
With the M1 it sounds like it might be about finding a balance. Are you also running the Elgato camlink software hub while running VDMX? Are any other applications running that you don’t need opened (such as a web browser)? I’m assuming you have at least 16GB of memory, but you haven’t specified.
I would try the following since you’ve only got three thunderbolt 4 ports / USB C.
-
Port 1. Plug in your CamLink source (set your camera to output at 1080p to avoid VDMX doing the downscaling).
-
Port 2. Plug in your iPhone or iPhone HDMI capture device, or other HDMI feed.
-
Port 3. Plugin your USB C hub with the webcam and an external SSD drive.
In VDMX, set the movie recorder to output to the external SSD drive.
In theory, this should help balance the load across your ports (just in case), and exporting to the external SSD should help if there are any issues with your machine needing additional resources.
Stop Recording Directly to H.264. Think of H.264 as a “delivery” format, like for uploading to YouTube. It’s not meant for live recording because it’s very hard on your computer’s main processor (the CPU).
Right now, your M1 Max is trying to do three huge jobs at once:
-
Mix your 3 camera feeds.
-
Shrink the 4K camera down to HD.
-
Squish that final HD mix into a complex H.264 file.
Your computer just can’t squish the video fast enough before the next frame of video arrives. That’s why it’s dropping frames.
You need a codec that is “easy” to record with. Your M1 Max can handle these much better.
-
Try Hap: Hap is designed for VDMX. It uses your powerful M1 Max graphics card (GPU) to do the work, which takes all the pressure off your CPU.
-
Try ProRes LT: This is another great choice. Your M1 chip is specially built to handle ProRes formats very, very fast. It will be much easier on your system than H.264.
The file sizes will be bigger, but your recording will be smooth. You can always re-compress the final, smooth recording into an H.264 file after your show is over.
If you are recording your main output + three separate movie recorder feeds, then split the load between your internal and external SSDs. See if the dropped frames only happen on the internal drive, or both. (So if you are recording four sources at once, try sending two to the external drive, and two to the internal.)
Lastly, also try the syphon recorder and see if that helps. Since it is a completely separate app, if it is experiencing the dropped frames when recording the main output, then it might be related to the hardware.
Let us know if that makes sense to you?
Thanks