360° video FFGL plugin for resolume?

FFGL, OSC, GLSL. If you like abbreviations, this is the forum for you
lightbx
Hasn't felt like this about software in a long time
Posts: 144
Joined: Thu Oct 29, 2015 21:05

Re: 360° video FFGL plugin for resolume?

Post by lightbx »

I know its not the same, but I've had to "map" VR to rectangular walls for tech conferences, and my quick and dirty solution was to stream VR apps over Airplay to an Apple TV, which was piped into Resolume via Blackmagic Intensity 4K. It worked really well, pleasd client and guests alike. Just saying, when pressed for time and budget, you can focus on the overall effect and sometimes come up with simple solutions.

Another idea I played with but never implemented was to just project rectuangular 360-degree footage (with panoramic warping) on the walls and working with the Slide effect plugin. Was thinking an OSC slider could control the position parameter, hence "spinning" the world around you.

With the FFGL plugin, as well as mapping onto a dome (which I hear is tough in Resolume), it sounds like you are pushing for something much more cutting edge. Best of luck, excited to see what comes out!

Sadler
Hasn't felt like this about software in a long time
Posts: 240
Joined: Tue Sep 20, 2011 23:55

Re: 360° video FFGL plugin for resolume?

Post by Sadler »

Not FFGL but works in ShaderLoader...

Code: Select all

float PI=3.14159265;
float TWEAK_X_ORIGIN=0.25;
float ZOOM=0.4;

//tools
vec3 rotateXY(vec3 p, vec2 angle) {
    vec2 c = cos(angle), s = sin(angle);
    p = vec3(p.x, c.x*p.y + s.x*p.z, -s.x*p.y + c.x*p.z);
    return vec3(c.y*p.x + s.y*p.z, p.y, -s.y*p.x + c.y*p.z);
}

void mainImage( out vec4 fC, in vec2 fX ) {

	iResolution.x = iResolution.x * (iResolution.y/iResolution.x);
	
    //place 0,0 in center from -1 to 1 ndc
    vec2 uv = fX.xy/iResolution.xy*2.-1.;
    
    // Correct Horizontal Mirroring
    uv.x = uv.x*-1.+(PI*TWEAK_X_ORIGIN-1.);
    
    vec2 CAMERA = vec2 (iMouse.x/iResolution.x*2.-1.,iMouse.y/iResolution.y*2.-1.);
    CAMERA *= iResolution.xy;
    
    //to spherical
    vec3 camDir = normalize(vec3(uv.xy, 1. + (ZOOM*2.0-1.) - (uv.x/PI*uv.x/PI) - (uv.y/PI*uv.y/PI)));
    
    //camRot is angle vec in rad
    vec3 camRot = vec3(CAMERA.yx * PI / iResolution.xy, 0.);
    
    //rotate
    vec3 rd = (rotateXY(camDir, camRot.xy));
    
    //radial azmuth polar
    vec2 RadAzPol = vec2(atan(rd.z, rd.x) + PI, acos(-rd.y)) / vec2(2.0 * PI, PI);

    //this is not a radial texture but whatever
    fC = texture2D(iChannel0, fract(RadAzPol * vec2(1.0, 1.0)));
}
Say thanks to artnik over in the MMV forums.
https://magicmusicvisuals.com/forums/vi ... ?f=4&t=913

BlueMD
Met Resolume in a bar the other day
Posts: 3
Joined: Sat Sep 24, 2016 11:40

Re: 360° video FFGL plugin for resolume?

Post by BlueMD »

Sadler wrote:Not FFGL but works in ShaderLoader...

Code: Select all

float PI=3.14159265;
float TWEAK_X_ORIGIN=0.25;
float ZOOM=0.4;

//tools
vec3 rotateXY(vec3 p, vec2 angle) {
    vec2 c = cos(angle), s = sin(angle);
    p = vec3(p.x, c.x*p.y + s.x*p.z, -s.x*p.y + c.x*p.z);
    return vec3(c.y*p.x + s.y*p.z, p.y, -s.y*p.x + c.y*p.z);
}

void mainImage( out vec4 fC, in vec2 fX ) {

	iResolution.x = iResolution.x * (iResolution.y/iResolution.x);
	
    //place 0,0 in center from -1 to 1 ndc
    vec2 uv = fX.xy/iResolution.xy*2.-1.;
    
    // Correct Horizontal Mirroring
    uv.x = uv.x*-1.+(PI*TWEAK_X_ORIGIN-1.);
    
    vec2 CAMERA = vec2 (iMouse.x/iResolution.x*2.-1.,iMouse.y/iResolution.y*2.-1.);
    CAMERA *= iResolution.xy;
    
    //to spherical
    vec3 camDir = normalize(vec3(uv.xy, 1. + (ZOOM*2.0-1.) - (uv.x/PI*uv.x/PI) - (uv.y/PI*uv.y/PI)));
    
    //camRot is angle vec in rad
    vec3 camRot = vec3(CAMERA.yx * PI / iResolution.xy, 0.);
    
    //rotate
    vec3 rd = (rotateXY(camDir, camRot.xy));
    
    //radial azmuth polar
    vec2 RadAzPol = vec2(atan(rd.z, rd.x) + PI, acos(-rd.y)) / vec2(2.0 * PI, PI);

    //this is not a radial texture but whatever
    fC = texture2D(iChannel0, fract(RadAzPol * vec2(1.0, 1.0)));
}
Say thanks to artnik over in the MMV forums.
https://magicmusicvisuals.com/forums/vi ... ?f=4&t=913
Hi Sadler, thanks for pointing out this possible solution. would u mind writing a couple of steps on how to get this shader running with an equirectangular image/video in resolume? i tried to load it but would only show a white screen and i cant find where to load the image/video

my project involves projecting 360° videos onto a surround screen mapped with 4 projectors. Thanks!

BlueMD
Met Resolume in a bar the other day
Posts: 3
Joined: Sat Sep 24, 2016 11:40

Re: 360° video FFGL plugin for resolume?

Post by BlueMD »

markbres wrote:
hive8 wrote:I need a little more info to get started on this.

The clips you be loading are equirectangular panorama video clips?
And you want a view placed on that so you can look at a section of that panorama.
Also the panorama will need to be mapped on a sphere.

View movement you want left, right, up, down, zoom in / out?

I hope i got that right, please let me know if something is not right.
Can you give me a demo clip for testing.
sounds great! camera inside the sphere mapped equirectangular panorama video clips.
possible parameters:
-camera rotate x, y, z (look left-right, look up-down, roll angle)
-camera field of view (degree)
-camera or sphere move x, y, z
-sphere rotate x, y, z
-sphere size/radius
-sphere resolution/detail?

yes you look at a section. camera could also be on the outside of the sphere and look at it.
Hi guys, just wondering if this actually has become a reality? =)

Sadler
Hasn't felt like this about software in a long time
Posts: 240
Joined: Tue Sep 20, 2011 23:55

Re: 360° video FFGL plugin for resolume?

Post by Sadler »

BlueMD wrote: Hi Sadler, thanks for pointing out this possible solution. would u mind writing a couple of steps on how to get this shader running with an equirectangular image/video in resolume? i tried to load it but would only show a white screen and i cant find where to load the image/video

my project involves projecting 360° videos onto a surround screen mapped with 4 projectors. Thanks!

Here's what I did to get it working in Resolume:
Grab yourself an equirectangular image or video.
Save the shader code to a text file.
Download ShaderLoader from github
Copy the plugin to vfx (or your own dir)
Copy the SpoutPanel to Resolume app dir
Start Resolume
Drag ER media to clip slot
Drag ShaderLoader onto same clip
Start the clip by clicking on it (important)
Select shader and locate saved text file
Use X mouse to rotate horizontally
Use Y mouse to move up and down

I have just done all of these steps right now using the code above using a demo version of Arena 5.0.4. Note, you can also apply the effect as an effect clip and it will apply to media in the layer below it..

lightbx
Hasn't felt like this about software in a long time
Posts: 144
Joined: Thu Oct 29, 2015 21:05

Re: 360° video FFGL plugin for resolume?

Post by lightbx »

Is the ShaderLoader FFGL plugin from Github Windows only? It seems the plugin is only available a .dll file.

Is there a way to test this out on a Mac? Would love to get even a prototype working.

User avatar
francoe
Is seriously in love with Resolume. Met the parents and everything
Posts: 378
Joined: Thu Nov 27, 2008 23:30

Re: 360° video FFGL plugin for resolume?

Post by francoe »

Lightbx, have no time to deal with it right now, but maybe some of you...
On windows is pretty easy to build a ffgl plugin based on the shader code, since the shadermaker from leadedge is there. I've made a couple of plugins, and I tell you, when all is setted up correctly then is just a copy/paste/compile. Magic! :D
There's an xcode project file and makefile to build the plugin on osx, buy never tryed by mylself.
If someone have time and a little skill, check it out.
https://github.com/leadedge/ShaderMaker

lightbx
Hasn't felt like this about software in a long time
Posts: 144
Joined: Thu Oct 29, 2015 21:05

Re: 360° video FFGL plugin for resolume?

Post by lightbx »

Ah, thank you for getting back so fast!

I've been at it for about a day now. I have gotten as far as copying the above shader text into the source code of the ShaderMaker XCode project, and then building a .bundle titled "360-Viewer." I then copied the .bundle into /plugins/vfx within the Resolume application folder (the destination with all of the other vfx).

However, this new effect does not appear within the Resolume Effects menu. I have tried dragging and dropping the .bundle on top of my clip, but to no avail.

Will keep plugging away and let you know if I can get it working.

BlueMD
Met Resolume in a bar the other day
Posts: 3
Joined: Sat Sep 24, 2016 11:40

Re: 360° video FFGL plugin for resolume?

Post by BlueMD »

EASIEST SOLUTION FOR 360 VIDEO or IMAGE

Just apply a Stingy Sphere effect on top of the source equirectangular image/video. Then:

Sphere Size: 1
Extrusion: 0
Zoom: 1
Resolution: 1

And feel free to play with the rotations =)

Next up, synchronize 4+ spheres with the same content input on each, but outputting a different "face/section" of the sphere to achieve a full surround image output for 4+ projectors.

Babcom
Hasn't felt like this about software in a long time
Posts: 97
Joined: Tue Sep 13, 2011 08:30

Re: 360° video FFGL plugin for resolume?

Post by Babcom »

Thanks for that BlueMD : )

Now I just need a camera that can record equirectangulars... Do you guys have any clue of one?

Post Reply