Fake lip synch with Resolume: ideas?

[josheo]

Hi,

I've searched for similar projects but found nothing so far.

I want to be able to animate a mouth layered over other backgrounds that talks when i speak into the microphone.

I thought I could just record a clip of my mouth moving, with the first frame being the lips together, and then synch the play control to FFT but it doesn't work like that.

Does any body have any ideas or solutions?

I could synch the mouths size to FFT instead, so that it goes from being practically invisible to be larger.

Maybe this could be developed further by having an image of a still mouth that has the opacity FFT'd so that is disappears when the talking one appears...

Was thinking of mixing some Ableton based midi note stuff into this mess but not sure how it would work yet.

Anybody want to mind map with me?

[Basic]

are you wanting to do this live or in post production?

Lip synching is time consuming in post but I dont know if I've seen it live. Im not saying it can't be done though.

[fruitygreen]

Kinect possible solution.
Animata
NI-mate Blender Game Engine plugin

Pros: Realtime lip syncing. OSC controlled.
Cons:Require some way to capture video into Resolume. NI-mate is working on a syphon enabled update.

[Basic]

cool FruityG, I hadnt thought of the Kinect.

Have people been using Res and Kinect together?

[josheo]

That's an interesting idea, hmmmm, don't know if I can afford a Kinect yet...

I want to do this live. I thought of setting up a webcam and a green screen with a hole in it. Can you chroma key in resolume? or create image masks?

[josheo]

Kinect possible solution.
Animata
NI-mate Blender Game Engine plugin

Pros: Realtime lip syncing. OSC controlled.
Cons:Require some way to capture video into Resolume. NI-mate is working on a syphon enabled update.

just had a look at this, would be so so nice but maybe is stretching my technical/time limitations.

I can afford to look a bit worse maybe I can push myself though

[josheo]

I see that it does have chroma key, just tried it out, might work this way after all

[josheo]

I see that it does have chroma key, just tried it out, might work this way after all

[Basic]

you could always just create still images of the mouth shapes/sounds and play it live?

http://minyos.its.rmit.edu.au/aim/a_not ... es_01.html

that's possible and flexible and free!

[deepvisual]

if you scratch a clip in RT you'll get a much more realistic effect than trying to sync a RT animation to audio.

I did something similar here.

http://www.youtube.com/watch?v=LPhf-7KxCYI