Sounds like you’ve figured this out on your own, but for anyone who stumbles into this in the future, here is a useful thread,
tl;dr – use Syphon whenever possible, it passes image textures directly between apps on the GPU which is going to be the most efficient and highest quality transfer.
When using NDI, there is an option to use the RGB mode that is higher quality, but uses more CPU time / more bandwidth.
In general it is best if only one of your apps is going fullscreen for output – some apps (such as VDMX) can be configured to render only to offscreen buffers for when you are only going to be outputting to Syphon or NDI. Not all apps will let you do this though, depending on how they work, they may require some level of drawing to screen in order for render upkeep…
(eg in the case of VDMX you can go into fullscreen mode and disable all of the displays)