Hi,
I'm doing a project using SimpleKinect processing code which sends out skeleton joint data via OSC.
My main aim is to project onto moving people. I don't know if processing hacking is the only way to go about it or if there is a more simple way to achieve this. I basically want to project the tracked skeleton onto an actual person, and then possibly use Resolume to achieve some cool effects.
Any suggestions and help would be greatly appreciated.
I'm not sure how much can be said about the tracking part of your project on these forums. But I've messed with live mocap into Resolume in the past using a solid backdrop which was keyed out for masks inside resolume.
I kinda can't remember how I went about capturing the screen of the mocap render into Resolume. I assume spout but I forget what I was able to get working on a windows computer to be a sender of the capture.
But another solution is just use a capture card.
The kinect has a kinda shallow depth to its senor and it will be pretty interesting getting perspective right your projector.
Edit: It was xsplit broadcaster that I think I used since there is no working OBS plugin for spout. So it went something like this.
Mocap suit (in your case kinect) into Mocap computer running its own software to interpret the what the hardware is sending. That computer was then networked to another computer where real time render software where a green screen layer was added behind the character that was being driven by the computer using the suit. Xsplit was ran on the same computer full screen capture of a 2nd monitor and then into Resolume. The green was keyed out and effects added or whatever. Doing all that takes a ton of system resources so it takes a boss computer or multiple computers.
I did something similar with simpleOpenNi, and in my experience, it took a while for ny kinect1 to recognise people and skeleton data takes some time to be created.
So I ended up using the depth data, by having an empty scene- without people - captured, and used all points which were different from that as the base to create a mask image, which was then sent to resolume via syphon.
The advantage using the depth data to create your mask is that you don't need a solid background.
The simplest way to get perspective right is to place your sensor exactly at your projector lens.
Then you'll have only to do some zooming to the mask image, and minor position adjustment.
of course if you want to have an image follow a person, you'll probably need the skeleton data.
Software developer, Sound Engineer,
Control Your show with ”Enter” - multiple Resolume servers at once - SMPTE/MTC column launch
try for free: http://programs.palffyzoltan.hu
ravensc wrote:
The simplest way to get perspective right is to place your sensor exactly at your projector lens.
Yeah I was thinking the same but since you get about 9ft in depth with a kinect v2 before inaccuracy it might have to be a short throw and weird things might happen towards the edges. But what else can you do really..
It sounds like a cool project though and using the depth to create a mask seems like what he wants.
My example was using fictional characters however so biped data was required.
You could transform the depth coordinates from the kinect to the perspective of the projector, knowing the sensor and projector position, and the depth information, that wouldn't be so difficult with vectors I think, and then you could theoretically place your sensor anywhere.
Software developer, Sound Engineer,
Control Your show with ”Enter” - multiple Resolume servers at once - SMPTE/MTC column launch
try for free: http://programs.palffyzoltan.hu
Hi I want to ask~ can i connect kinect to resolume, but to set it as a trigger ? exp, I have a show is different column, but i want to trigger it using kinect instead of midi or keyboard.
You'll need a third party solution to convert X,Y,Z values to usable MIDI, DMX, or OSC outputs.
I prefer Touch Designer.
I sent a Kinect out on a tour so the singer can push his hand forward to activate lasers and lights, then had his hand move left and right to move the lasers and lights to follow his hand.
All of this was ran through DMX and i just converted this X and Y of his hand to change the pan values of the fixtures, and the Z value was controlling the on and off state of the fixtures.
you have any tutorial or screen shot how to set this? actually i wanted to do a basic one that is ~ anyone walk by the sensor it will trigger the resolume button or play button, so if not detecting , it will stop the play or disappear
