Page 1 of 1

Controlling 3D model in realtime based on input?

Posted: Tue Feb 12, 2019 14:01
by cade
This is more of a general setup question than a Resolume question, but I'm wondering how I would approach controlling a 3D model (OBJ, particle cloud, etc) based on a keypress or midi input. Essentially playing the model as an instrument, using the given input to trigger a fractal or displacement (or modify any other property for that matter). I really am only looking to start with one variable and one input.

If anyone can push me in the right direction I'd be super grateful!

Re: Controlling 3D model in realtime based on input?

Posted: Tue Feb 12, 2019 17:45
by NxKmn
I think this example is what you're trying to do.

https://www.vfxartshop.com/product/kern ... aedd0e4327

Overall, you need to run your 3D model in a Graphics Engine.
Like Unreal or Unity.
The video generated in the Graphics Engine is shared via Spout to Resolume.

Then, we must find a solution to control all this.
I don't know if the graphics engines include MIDI instructions.

But in the example, it seems to be OSC Controls, integrated in the corresponding FFGL Plugin.
Then, it is always possible to set MIDI controls on the FFGL Plugin parameters.

*************

I have my own question on the subject:

Is it possible to mix textures in resolume.
To send them to the graphics engine. Resolume > Spout > Graphics engine ]
That it be applied to the 3D Model.
And to retrieve the textured 3D model in Resolume. Graphics engine > Spout Resolume ]

If anyone knows the answer :B

Re: Controlling 3D model in realtime based on input?

Posted: Wed Feb 13, 2019 10:08
by t13swift
As NxKmn said for best results the model needs to be actually in 3D.

You could fake a rotation if the content of the 3D model doesn't need to change but only the camera angle, you could render out a pan or animation of the camera at 60 or more fps depending on how slow you need to rotate or move. Then map a midi input to the play head of the clip.
This would give you 1 axis / parameter of control.
The same could probably be done for any property in 3d provided it doesnt need to move when you arent directly controlling it.

Re: Controlling 3D model in realtime based on input?

Posted: Wed Feb 13, 2019 10:39
by Grimm90
NxKmn wrote:
Tue Feb 12, 2019 17:45
I have my own question on the subject:

Is it possible to mix textures in resolume.
To send them to the graphics engine. Resolume > Spout > Graphics engine ]
That it be applied to the 3D Model.
And to retrieve the textured 3D model in Resolume. Graphics engine > Spout Resolume ]

If anyone knows the answer :B
I think so but i havent tried :

Make group 1 ,add layers , mix textures .
Go to advanced output and make a new screen , which is sending to Spout, make a slice & assign your texture group to that slice, now you are outputting your mixed textures to spout.
Open graphics engine, also for example Touchdesigner could be a good one here as graphics engine.
Load in .obj , set up camera & lights & render , bring the "spout in" in touchdesigner (there is a node for that in touch).
Then u gonna need some touchdesigner tinkering to get the spout stream in , into a material u can apply on the model .
Add a spout out on the render node =back into resolume

Re: Controlling 3D model in realtime based on input?

Posted: Wed Feb 13, 2019 18:12
by Sadler
Skarabee uses Magic Music Visuals (MMV) to do his midi controlled visuals.
phpBB [video]


Texturing can be tricky but MMV supports 3D models directly and quite easily and they can be transformed and coloured etc. via midi/osc/audio. You can can also use 3D models in (currently very simple) particle system.

Re: Controlling 3D model in realtime based on input?

Posted: Thu Feb 14, 2019 11:55
by cjbor
If your looking for a complex solution without any programming, I would recommend SMODE Studio.
https://smode.fr
Supported inputs are:
  • Midi
  • Osc
  • Audio
Another Solution would be to use Unity.
https://github.com/keijiro/KlakSpout
To send them to the graphics engine. Resolume > Spout > Graphics engine ]
That it be applied to the 3D Model.
And to retrieve the textured 3D model in Resolume. Graphics engine > Spout Resolume ]
It is possible! In SMODE you can implement that quite easily by setting the texture to a Spout input.