Something is causing the system to offload some of the workload to memory.
If your card is running at x8, then I would move your Intensity to another slot (#7, instead of #5).
Maybe something like
1: Mini recorder
3: Mini recorder
5: empty (putting a card here will cause GPU to run at x8, not x16)
7: Intensity 4k
What resolutions are you trying to capture? You have to think about bandwidth issues being caused too.
If your card is running at x8, PCIE 2.0 at x8 has 4GB bandwidth max
1920x1080@60Hz 8bit color just over 4Gbps bandwidth ~4.4
So then let's say you're only capturing 1280x720@60 8bit color, ~2.6Gbps
So 2 mini recorders trying to capture 720p would put you at 5.2Gbps, which is still above the 4GB bandwidth cap for PCIE 2.0 at x8
Because resolume has native support for BMD, means they capture to the GPU, but since the GPU can't process all of that information it has to go somewhere. Next best thing, memory