Controlling 3D model in realtime based on input?

Just let it all out, buddy. You're among friends here.
Post Reply
cade
Met Resolume in a bar the other day
Posts: 1
Joined: Tue Feb 12, 2019 13:56

Controlling 3D model in realtime based on input?

Post by cade » Tue Feb 12, 2019 14:01

This is more of a general setup question than a Resolume question, but I'm wondering how I would approach controlling a 3D model (OBJ, particle cloud, etc) based on a keypress or midi input. Essentially playing the model as an instrument, using the given input to trigger a fractal or displacement (or modify any other property for that matter). I really am only looking to start with one variable and one input.

If anyone can push me in the right direction I'd be super grateful!

User avatar
NxKmn
Is taking Resolume on a second date
Posts: 23
Joined: Mon Jan 16, 2017 23:03

Re: Controlling 3D model in realtime based on input?

Post by NxKmn » Tue Feb 12, 2019 17:45

I think this example is what you're trying to do.

https://www.vfxartshop.com/product/kern ... aedd0e4327

Overall, you need to run your 3D model in a Graphics Engine.
Like Unreal or Unity.
The video generated in the Graphics Engine is shared via Spout to Resolume.

Then, we must find a solution to control all this.
I don't know if the graphics engines include MIDI instructions.

But in the example, it seems to be OSC Controls, integrated in the corresponding FFGL Plugin.
Then, it is always possible to set MIDI controls on the FFGL Plugin parameters.

*************

I have my own question on the subject:

Is it possible to mix textures in resolume.
To send them to the graphics engine. Resolume > Spout > Graphics engine ]
That it be applied to the 3D Model.
And to retrieve the textured 3D model in Resolume. Graphics engine > Spout Resolume ]

If anyone knows the answer :B

t13swift
Hasn't felt like this about software in a long time
Posts: 144
Joined: Thu Mar 26, 2015 14:00

Re: Controlling 3D model in realtime based on input?

Post by t13swift » Wed Feb 13, 2019 10:08

As NxKmn said for best results the model needs to be actually in 3D.

You could fake a rotation if the content of the 3D model doesn't need to change but only the camera angle, you could render out a pan or animation of the camera at 60 or more fps depending on how slow you need to rotate or move. Then map a midi input to the play head of the clip.
This would give you 1 axis / parameter of control.
The same could probably be done for any property in 3d provided it doesnt need to move when you arent directly controlling it.

Grimm90
Hasn't felt like this about software in a long time
Posts: 109
Joined: Wed Nov 29, 2017 19:08

Re: Controlling 3D model in realtime based on input?

Post by Grimm90 » Wed Feb 13, 2019 10:39

NxKmn wrote:
Tue Feb 12, 2019 17:45
I have my own question on the subject:

Is it possible to mix textures in resolume.
To send them to the graphics engine. Resolume > Spout > Graphics engine ]
That it be applied to the 3D Model.
And to retrieve the textured 3D model in Resolume. Graphics engine > Spout Resolume ]

If anyone knows the answer :B
I think so but i havent tried :

Make group 1 ,add layers , mix textures .
Go to advanced output and make a new screen , which is sending to Spout, make a slice & assign your texture group to that slice, now you are outputting your mixed textures to spout.
Open graphics engine, also for example Touchdesigner could be a good one here as graphics engine.
Load in .obj , set up camera & lights & render , bring the "spout in" in touchdesigner (there is a node for that in touch).
Then u gonna need some touchdesigner tinkering to get the spout stream in , into a material u can apply on the model .
Add a spout out on the render node =back into resolume

Sadler
Hasn't felt like this about software in a long time
Posts: 232
Joined: Tue Sep 20, 2011 23:55

Re: Controlling 3D model in realtime based on input?

Post by Sadler » Wed Feb 13, 2019 18:12

Skarabee uses Magic Music Visuals (MMV) to do his midi controlled visuals.
phpBB [video]


Texturing can be tricky but MMV supports 3D models directly and quite easily and they can be transformed and coloured etc. via midi/osc/audio. You can can also use 3D models in (currently very simple) particle system.

cjbor
Met Resolume in a bar the other day
Posts: 2
Joined: Thu Feb 14, 2019 11:32

Re: Controlling 3D model in realtime based on input?

Post by cjbor » Thu Feb 14, 2019 11:55

If your looking for a complex solution without any programming, I would recommend SMODE Studio.
https://smode.fr
Supported inputs are:
  • Midi
  • Osc
  • Audio
Another Solution would be to use Unity.
https://github.com/keijiro/KlakSpout
To send them to the graphics engine. Resolume > Spout > Graphics engine ]
That it be applied to the 3D Model.
And to retrieve the textured 3D model in Resolume. Graphics engine > Spout Resolume ]
It is possible! In SMODE you can implement that quite easily by setting the texture to a Spout input.

Post Reply