NDI support

[leadedge]

Interesting idea to use FFGL alfaleader.

I have been looking at "NDI to Spout" and "Spout to NDI" with a view to inter-machine sharing. I have a local copy working but have yet to explore performance and whether there is any benefit in doing any more.

I ran across a few problems related to sender management and YUV to RGBA conversion which I resolved and think are working. The SDK examples are not very helpful in that regard. I wonder how you have handled that.

[alfaleader]

Both maybe also ndi to spout and syphon to spout

[Joris]

Alfaleader, I think you accidentally that whole sentence.

[alfaleader]

Sorry, was working and did a quick post with my phone (and didn't see the reaction from leadedge).

The both was an answer to the mac or windows question

I started with a spout version but quickly switched to FFGL. I think the main problem is getting the texture from your GPU RAM to your RAM (for the performance). I also got a YUV/RGBA conversion that works now but still have to test for performance. Will keep you posted (I'll PM) when I did some more tests.

[leadedge]

Alfaleader - there is potential here but I have found no means of communication between developers on the NDI website. I have changed email recently and just now updated my Resolume account.

I have also written a YUV to RGBA converter function. It seems a waste to use it with the optimizations to transmit the image in the first place. Even so it seems fast enough. I am not sure what happens to alpha. And I also find a strange image format that is twice the width it should be. I think this is a basic function and an optimized converter should be part of the SDK.

Once I have the texture to send as usual, I get it into a pixel buffer using glGetTexImage, but it is bgra so I have to use GL_BGRA_EXT. I mention that here because I notice a colour shift. The colour is different in the NDI Video Monitor. I am not sure where this is happening.

[alfaleader]

Might be a problem with the YUV color spaces. NDI uses BT.601 for SD, Rec.709 for HD resolutions and Rec.2020 for UHD resolutions. Maybe that's why the colors are a bit off

[leadedge]

Something like that I guess. I noted the difference with SD and HD after studying YUV a bit more. But this isn't using my YUV422_to_RGBA function, it is just a simple sender using the standard NDI SDK functions to send a frame so I expected the same colours at the other end. I am sure that OpenGL is not changing anything. There's not much I can do about it without some time consuming conversions. A bit of a problem in practice. Interested to know your results.

I have done some profiling at 1920x1080 on the same machine. Total 16msec (minimum) per frame and a whopping 14msec (maximum) for the YUV to RGBA conversion. So that is the bottleneck. I have narrowed it down to a clamp function (<255 and > 0) otherwise I can achieve around 8-9msec per frame locally.

[alfaleader]

I seem to be getting the biggest performance hit when loading the texture to CPU RAM (around 13ms) and then the NDI sending at around 9ms (I just send RGBA values, not using YUV for the moment. But maybe the NDI libs convert it to YUV internally).

I don't seem to have color issues while using RGBA with NDIlib, don't really know what went wrong with you but with the use of RGBA I don't hava any issues.

Now I'm trying to optimise the program, it's now just a quick dirty prototype (but it works ). Also seems that NDIlib is a lot more efficient while using YUV, so maybe I can get better performance while using this. The GPU texture to CPU data might also be faster in other ways.

[leadedge]

At the moment I am going the other way because sending is no problem. I have found that receiving RGBA data does not work, it just comes in as YUV anyway. That's the reason for the conversion function. but I will check again. I think this could be a bug.

Potentially YUV422 is half the amount of data but the frame seems to be packed out to full width. Perhaps the compression ignores the padding. It would be the most efficient for transmission, but the conversion to rgba at the other end has to be equally efficient.

Pixel data from CPU to GPU texture is 1.5msec (1920x1080). I have not measured the other way yet.

I have had trouble with sender management. It appears that while new senders are discovered, ones that close are not removed from the list.

[alfaleader]

Hmm don't really know where your problem might be. I'm having neither of those problems (with visual studio 2015), can't really tell what the problem might be. Is your reciever set up good (there is a bool flag somwhere to ask the lib to give you RGB instead of YUV)? Is the sending format set up good (shouldn't make a difference, I get auto YUV to RGB conversion when testing the reciever with the NDI Test Pattern App and I guess that sends YUV)?

My pixel data from CPU to GPU takes 6ms, how did you manage to get it done so fast?