Here is my approach:
Conceptually I think your stumbling block is not dedicating a source layer to Only be the Mask, and “Hide” it.
And then you call/apply/refer to that Masking layer in your ForeGround(FG) Layer.
You can duplicate said origin source layer to have it be the mask and the FG layer if need be, but those are two distinct layers.
See attached project.
Notes:
Basic Chroma Mask.
Most of the work happens in the Mask layer. I’m choosing red to be the Masking color form the Default Color Bars source. In the FX options you can see Chroma determines what is the mask. If Color Replacement is used as a Mask then the issue is that Whites also conatain some aspect of RGB, consequently Whites and Reds become the masking color (even greys actually). This could be a good thing if you want two colors to be a mask since White and Red are both letting through a layer, aka being the Alpha. The benifits of Chroma Mask is that White is treated as a color and blocked.