Premultiplied alpha

FFGL, OSC, GLSL. If you like abbreviations, this is the forum for you
User avatar
subpixel
Hasn't felt like this about software in a long time
Posts: 142
Joined: Thu Jun 05, 2014 09:32
Location: Melbourne, AU

Premultiplied alpha

Post by subpixel » Fri Oct 06, 2017 04:22

So, I released a couple of FFGL effects recently and burned myself by not accounting for, what seems to be the rule, effect inputs and outputs assuming premultiplied alpha.

It seems to me that there are a couple of issues with this:
  • the alpha channel is "clean", in so far as you can set/receive any value and it just is that value, but RGB values are all scaled by the alpha, meaning, that RGB values with, say, 10% alpha, are basically bit-crushed down to a much lower colour resolution (error is 10x)
  • every effect that does something with colour has to do a division at the start and a multiply at the end; perhaps not the biggest deal in the world but seems like an unnecessary waste
What's the story? Is straight alpha bad for something? What are the factors that lead to choosing premultiplied or straight alpha?

Side question: I noticed the alpha type selector is missing for clips. Is there a way to detect what kind is used from the content media?

-subpixel

Joris
Doesn't Know Jack about VJ'ing or Software Development and Mostly Just Gets Coffee for Everyone
Posts: 5180
Joined: Fri May 22, 2009 11:38

Re: Premultiplied alpha

Post by Joris » Fri Oct 06, 2017 17:11

You can write books on the straight vs premult debate. Technical minded people will generally prefer straight ("Ze valuez are ze most precious. Ve must have them in ze highest prezision!"), practical minded people will generally like premult ("hurr durr why is this outline all shiny when I gon and made it all invisible?")

Neither option is inherently better than the other. In Resolume we haven't always been consistent, mostly because we're also regular people.

We're trying to be consistent now and work with premultiplied alpha internally. The reason for this is that for most compositing operations on a gpu, premult is slightly faster. Most blends can be done in shaders by simple mathematical operations on the RGBA values. The upside is you don't need to premultiply the rgb with its alpha first. The downside is that color effects do require their operators be premultiplied in order to work correct.

We figured that blending happens more than color operations, so there you are.

Some info on why premult is better for compositing textures together: http://www.realtimerendering.com/blog/g ... plication/

Some info on why straight is better for color correction: https://compositing.wordpress.com/2010/ ... emultiply/

Joris
Doesn't Know Jack about VJ'ing or Software Development and Mostly Just Gets Coffee for Everyone
Posts: 5180
Joined: Fri May 22, 2009 11:38

Re: Premultiplied alpha

Post by Joris » Fri Oct 06, 2017 17:22

Alpha channels are not labeled, so the only way to detect straight is to cycle through all the pixels and hope you'll find one where alpha > rgb. If alpha <= rgb, you can assume premult. This gets even hairier when you remember that you can premult with other colors than black, and at that point you just want stop thinking about the topic altogether and get back to fixing more accute problems.

http://stackoverflow.com/questions/9147 ... lied-alpha

Joris
Doesn't Know Jack about VJ'ing or Software Development and Mostly Just Gets Coffee for Everyone
Posts: 5180
Joined: Fri May 22, 2009 11:38

Re: Premultiplied alpha

Post by Joris » Fri Oct 06, 2017 17:28

If a clip with alpha does not offer you the dropdown to change it (check the clip properties, which are folded in by default in 6), that sounds like a bug and I'd report it as such.

User avatar
subpixel
Hasn't felt like this about software in a long time
Posts: 142
Joined: Thu Jun 05, 2014 09:32
Location: Melbourne, AU

Re: Premultiplied alpha

Post by subpixel » Mon Oct 09, 2017 05:17

Good article. I see mention of going to 16bit for colour accuracy (makes sense).

Re selector box, a good chance I didn't expand the properties. I'll check that.

Regarding the DXV(3) codec: does it store Premultiplied alpha or straight? And can DXV store 16bit colour/alpha?

User avatar
drazkers
Wants to marry Resolume, and Resolume said "yes!"
Posts: 955
Joined: Wed May 18, 2011 10:54
Location: Brady V up in Canada

Re: Premultiplied alpha

Post by drazkers » Mon Oct 09, 2017 06:02

I'm just enjoying the reading and learning of this thread.

Joris
Doesn't Know Jack about VJ'ing or Software Development and Mostly Just Gets Coffee for Everyone
Posts: 5180
Joined: Fri May 22, 2009 11:38

Re: Premultiplied alpha

Post by Joris » Mon Oct 09, 2017 06:38

DXV doesn't care about premult or straight. DXV is a container for pixel data. What you want that data to mean is irrelevant at the storage level. If you want to render with straight alpha, that's fine as long you tell Resolume it's meant to interpret those values as straight when reading the data..

Think of it like doing an operation in after effects where you change colors, so that you're storing the color as cmy values instead of rgb. As long as you then have an effect in Resolume on every clip that sets those values back to the expected rgb, it will all look okay. DXV doesn't need to know what you stored.

DXV is 8 bit only. This does make it harder to use it for operations that require precise color information, like UV mapping. Being VJ software, we're fine with opting for speed over accuracy for the time being.

User avatar
drazkers
Wants to marry Resolume, and Resolume said "yes!"
Posts: 955
Joined: Wed May 18, 2011 10:54
Location: Brady V up in Canada

Re: Premultiplied alpha

Post by drazkers » Tue Oct 10, 2017 01:27

Joris wrote:DXV doesn't care about premult or straight. DXV is a container for pixel data. What you want that data to mean is irrelevant at the storage level. If you want to render with straight alpha, that's fine as long you tell Resolume it's meant to interpret those values as straight when reading the data..

Think of it like doing an operation in after effects where you change colors, so that you're storing the color as cmy values instead of rgb. As long as you then have an effect in Resolume on every clip that sets those values back to the expected rgb, it will all look okay. DXV doesn't need to know what you stored.

DXV is 8 bit only. This does make it harder to use it for operations that require precise color information, like UV mapping. Being VJ software, we're fine with opting for speed over accuracy for the time being.
I was about to email about this but i might as well kinda side track here.

So DXV3 is 8bit, was DXV 2.2 6 bit? I see that we have a 16bit high quality mode in 6 now. But most consumer cards only output 8bit and Quadros/Firepros send out 10 or 12 bit colour. I remember back in the windows xp days we had 16 and 32 bit colour modes and that made a difference so i feel like similar terminology is getting cross between two things. Even when your screen can only show 8bit it is useful to have that extra data when streching to discourage banding.

So why 16 bit high quality instead of 12? Future proofing? Or just having that extra resolution when effecting? Do we need something like Pro Res 4444 and a Quadro to truely use it?

Joris
Doesn't Know Jack about VJ'ing or Software Development and Mostly Just Gets Coffee for Everyone
Posts: 5180
Joined: Fri May 22, 2009 11:38

Re: Premultiplied alpha

Post by Joris » Tue Oct 10, 2017 10:42

To be 100% correct, DXV uses several variants of DXT, which doesn't really deal in bits. For instance, this is the wikipedia entry on DXT1:
DXT1 (also known as Block Compression 1 or BC1) is the smallest variation of S3TC, storing 16 input pixels in 64 bits of output, consisting of two 16-bit RGB 5:6:5 color values {\displaystyle c_{0}} and {\displaystyle c_{1}}, and a 4x4 two-bit lookup table.
https://en.wikipedia.org/wiki/S3_Texture_Compression

So it's not necessarily 8 bit, it can store values with more or less precision depending on the other pixels in that frame. That has always been the case for all version of DXV. DXV3 just has a more efficient compression algorithm to determine what to keep and what to toss.

The main reason for having a 16 bit internal mode is to pave the way for things like UV mapping, but you can already see a difference when rendering gradient sources. Also when mixing and compositing multiple gradients, having higher bit depth means you don't crush things as much. This does come at a hefty performance hit, so handle with care.

To confuse things even more, we're talking about 8/16 bits *per channel*, which is different from the 24 or 32 bits *per pixel*, which are used to refer to RGB (at 8 bpc) or RGBA (at 8 bpc).

chenthemagician
Hasn't felt like this about software in a long time
Posts: 133
Joined: Fri Aug 22, 2014 15:25

Re: Premultiplied alpha

Post by chenthemagician » Tue Oct 10, 2017 17:09

I'm loving the learning process....head hurts a little bit, but i learned a lot here....

thanks Joris
HP Omen
8 GB RAM
i7 7th gen 2.8 GHz
Win 10 Home
Nvidia 1050 GTX

Asus G751J - 860M
i7 4710HQ 2.5 GHz
8 GB RAM

Livid OHM RGB Controller

Creating Solutions, Tailored to your needs!

Repping from the 876 - Jamaica "Out of Many, One People"

Post Reply