Input Device Benchmarks

Bro, does your rig even lift?
User avatar
Arvol
Knows Resolume better than the devs
Posts: 1058
Joined: Thu Jun 18, 2015 17:36
Location: Oklahoma, USA

Input Device Benchmarks

Post by Arvol » Thu Sep 21, 2017 23:48

Hey Guys,
I had some downtime today so I thought I would create a small benchmark thread for input devices.
This is being made to compare Resolume 5 with Resolume 6. I benchmarked everything on both versions so we can compare performance.

Here are the devices I tested with:
Black Magic Ultrastudio SDI
AJA U-Tap SDI
Magewell SDI 4K Plus
NDI (Direct PC to PC)

The 2 computers I used were:
2017 Razer Blade (GTX 1060), Running a single layer at 140+ fps
2015 HP crap i5 Laptop with integrated Intel HD GPU, Running a single layer at 33 fps

Both PC's had their screens AND composition set to 720p.
All the input devices were configured for 720p@60
I was coming out of my PC's HDMI port into a Black Magic HDMI to SDI Micro Converter for all of these tests.

Here are the results:

Resolume 5:
AJA U-Tap SDI - 7 Frame Delay

Black Magic Ultrastudio SDI - 4 Frame Delay

Magewell SDI 4K Plus - 4 Frame Delay

NDI - 22 Frame Delay


Resolume 6:
AJA U-Tap SDI - 4 Frame Delay

Black Magic Ultrastudio SDI - 4-6 Frame Delay

Magewell SDI 4K Plus - 4 Frame Delay

NDI - 7-10 Frame Delay


I didn't test with Spout as there really isn't that many applications developed for it as there is Syphon, And I don't use OSX anymore.
All of these tests were done in Arena, with no background applications running at the moment of testing.

What I would like to see, is some more benchmarks added to this thread.

Here are a few devices I had in mind:
Syphon/Spout
Datapath Input Devices
Black Magic Decklink Input Devices
Black Magic Mini Recorder

If anyone has some of these devices (or any other input devices you would like to add to the list), and has the time to do some benchmarks, I would love to see how everything is stacking up in version 6 from version 5.

I'm really impressed with the improved performance with the AJA U-Tap. Well done guys!

Warspite
Hasn't felt like this about software in a long time
Posts: 136
Joined: Thu Nov 01, 2012 14:36

Re: Input Device Benchmarks

Post by Warspite » Fri Sep 22, 2017 23:10

Is input from AJA smooth?
My AJA behaves like there is non constant fps with frames dropped. Blackmagic is smooth.
Also, your resultsare higher than mine, so this delay depends on system and cannot be compared directly.
On MacBook i have this numbers
Aja U-TAP gives me 4 to 5 frames delay (0.16-0.2 of a second). Taking photos i admit that i get 4 frames delay more often, so delay is around 0.17 of a second.

Blackmagic gives 1 to 2 frames of delay (0.04-0.08 s). Most of a time 2 frames, so delay is around 0.07 sec.

So, Blackmagic gives 2,25x times less delay.

Warspite
Hasn't felt like this about software in a long time
Posts: 136
Joined: Thu Nov 01, 2012 14:36

Re: Input Device Benchmarks

Post by Warspite » Fri Sep 22, 2017 23:12

Ok, your delay is measured at 60 fps. So when we convert to milliseconds, results are comparable, that is great.

Warspite
Hasn't felt like this about software in a long time
Posts: 136
Joined: Thu Nov 01, 2012 14:36

Re: Input Device Benchmarks

Post by Warspite » Fri Sep 22, 2017 23:16

How you measured delay from SDI input? We can consider that video-output from videocard has 0 delay, but i do not think this is correct for native SDI output on capture card or with help of some sort of converter/scaler.
Last edited by Warspite on Fri Sep 22, 2017 23:20, edited 1 time in total.

Warspite
Hasn't felt like this about software in a long time
Posts: 136
Joined: Thu Nov 01, 2012 14:36

Re: Input Device Benchmarks

Post by Warspite » Fri Sep 22, 2017 23:20

Also, i have a question - slices in advanced output are rendered top to bottom or bottom to top. On some complex setups your get delay from this, and it is good to keep live input slices close to render start point.

User avatar
Arvol
Knows Resolume better than the devs
Posts: 1058
Joined: Thu Jun 18, 2015 17:36
Location: Oklahoma, USA

Re: Input Device Benchmarks

Post by Arvol » Fri Sep 22, 2017 23:30

This was done at 60fps, so 1 frame is 00:00:00:01 (Hr:Mn:Sc:Ms).
I had a computer with a Timecode Counter running and outputing via HDMI into a BMD Micro HDMI-SDI converter, into my Input Cards.
Both computers are side by side and I took pictures with my phone and counted the delay from computer to computer.
Pretty straight forward.

Resolume was setup with a single slice in ASS. Nothing to crazy, Again... This is a benchmark, I want to know what the basic setup results will produce, all the extra fanciness can be calculated later. But if it's going to add that much more of a strain on your system, knowing what cards work best when there isn't any strain should make your purchasing decision a no-brainier at that point :P :lol:

User avatar
Oaktown
Resolume honorary member
Posts: 2766
Joined: Tue May 08, 2012 15:19
Location: Oakland, CA

Re: Input Device Benchmarks

Post by Oaktown » Mon Sep 25, 2017 15:02

Dinga, what kind of network were you using when you did your NDI testing? Was it a dedicated 1G wired network?

User avatar
Arvol
Knows Resolume better than the devs
Posts: 1058
Joined: Thu Jun 18, 2015 17:36
Location: Oklahoma, USA

Re: Input Device Benchmarks

Post by Arvol » Tue Sep 26, 2017 18:43

Oaktown wrote:Dinga, what kind of network were you using when you did your NDI testing? Was it a dedicated 1G wired network?
It was a direct link from pc to pc via a cat6 cable. Gigabit connection on my Blade, I'm not sure about the NIC on my HP, I Just got a Blade back from tour, so I can try to re-do this from Blade to Blade in a week or so when things slow down.

If anyone else has the time and wants to try it out, that's kind of the point of this thread, so we can get everyone's input and updated benchmarks :)

Warspite
Hasn't felt like this about software in a long time
Posts: 136
Joined: Thu Nov 01, 2012 14:36

Re: Input Device Benchmarks

Post by Warspite » Mon Dec 11, 2017 00:44

Tested Blackmagic and Aja U-Tap with Resolume 6.
Source - Macbook HDMI Out at 720p50, receiver - X99 based Rack PC.
Built into resolume 5 test card with 25 fps timecode

Aja U-Tap - 1-2 frames delay at 25 fps (around 60ms).
Blackmagic Mini Recorder HDMI input - 2-3 frames delay at 25 fps (around 100ms).
Blackmagic Mini Recorder SDI input with Decimator HDMI to SDI converter - 2-3 frames delay at 25 fps (around 100ms). So Decimator SDI-HDMI converter does not add any latency.
Blackmagic Decklink Duo 2 SDI input with Decimator HDMI to SDI converter - 2 frames delay (around 80ms).
So Aja is now even better than Blackmagic. What a twist! Bravo Resolume Team!
But also Blackmagic is 1 frame behind to my previous tests. Just wander if this change is up to new version of Blackmagic Desktop Video Software or because of Resolume 6.

User avatar
drazkers
Wants to marry Resolume, and Resolume said "yes!"
Posts: 943
Joined: Wed May 18, 2011 10:54
Location: Brady V up in Canada

Re: Input Device Benchmarks

Post by drazkers » Mon Dec 11, 2017 03:52

Warspite wrote:Tested Blackmagic and Aja U-Tap with Resolume 6.
Source - Macbook HDMI Out at 720p50, receiver - X99 based Rack PC.
Built into resolume 5 test card with 25 fps timecode

Aja U-Tap - 1-2 frames delay at 25 fps (around 60ms).
Blackmagic Mini Recorder HDMI input - 2-3 frames delay at 25 fps (around 100ms).
Blackmagic Mini Recorder SDI input with Decimator HDMI to SDI converter - 2-3 frames delay at 25 fps (around 100ms). So Decimator SDI-HDMI converter does not add any latency.
Blackmagic Decklink Duo 2 SDI input with Decimator HDMI to SDI converter - 2 frames delay (around 80ms).
So Aja is now even better than Blackmagic. What a twist! Bravo Resolume Team!
But also Blackmagic is 1 frame behind to my previous tests. Just wander if this change is up to new version of Blackmagic Desktop Video Software or because of Resolume 6.
May be slightly more accurate if those tests are done at 50 or 60 FPS. Gives a little more resolution in your ms, as i suspect both devices have less latency in ms.

Post Reply