Syncing it Up with Koan Sound
No show, these days, is about just the artist or the music or the visuals or the lights. What defines a great show is how all these different facets tie up together. It’s about skillfully intertwining the audio with the visuals, the lights with the sounds, the music with the mood. I like to imagine it as blurring the lines. In the most subtle and tasteful way. Presenting a full and complete piece to the audience. And what better way to do this, than with sync?
Now, you must know that not a day goes by when the elves at the Resolume HQ don’t think about sync. You see, Resolume is not just an enabler for VJs to perform cool tricks. The bigger picture is constantly at the back of everyone’s mind. And then somewhere along the road, good people like Leggy come along. And make it all worth it. When you use MIDI & SMPTE & ArtNet Merge at the same time, in the same show, its gotta be real juicy. We're thinking, “Dinner is served”
Kyle Roberts (aka Leggy aka mushpongo ) has been working hard at creating content, making edits & running all the video related stuff for Koan Sound and their latest project “Polychrome”. Apart from having the coolest nick name ever, we really like him coz he introduced Jim & Will (Koan Sound) to Resolume. And, before we knew it, their whole new show depended on it.
We caught up with Leggy & Jim + Will to talk to us about their new show.
First things first, tell us how you got into this world of visuals, Leggy.
My dad is an avid photographer/videographer so I was lucky enough to have access to a camcorder when I was young. I ended up making short films with whatever I could find round the house and then editing them on a single VHS deck.
After that I focused on music studying Music Technology at University; that’s where I was first introduced to After Effects which reignited my interest in visuals. During that time period both ‘WarpVision’ and ‘The Work of Director Chris Cunningham’ DVD’s were watched in heavy rotation so videos like Gantz Graf and Monkey Drummer influenced me. Regularly visiting Bristol also had a major impact as it’s where I first witnessed projection mapping within the context of club via the ANTIVJ collective.
I knew mapping/lighting was something I wanted to focus on and over the years I’ve thankfully landed work at various festivals including Glastonbury and Boomtown where I could explore this.
Working with KOAN Sound was a pleasure as I’ve been a fan of theirs for some time. As soon as I heard Polychrome it was clear they had put a massive amount of effort in to all aspects of the album including artwork so I was ecstatic to be asked to help out with content dedicated for the Live show.
Awesome. Tell us about Polychrome and how is it different from what you have done in the past.
In the past we’d displayed visuals whilst we were performing, but they were never linked to the music in any meaningful way. This show was all about showcasing visuals that were very closely tied to the music.
We’d been exploring macro photography to create all of the artwork for the album, and we wanted to take that one step further and experiment with creating visuals using the same technique.
From there we assembled a small team of visual artists and animators, and began to create a video for each of the audio tracks. It was a very hands-on process, and really fun to be involved so heavily with both the creative and technical side of the show.
It’s interesting how you’re from a music production background & so find many similarities between Ableton & Resolume. What are the similarities do you see & what makes Resolume so easy to use by producers?
Anyone who’s looked at the Session view within Ableton will recognize the layout of Resolume. Coupled with the drag and drop nature of both bits of software I think the learning curve compared to other visual software is rather gentle. In more ‘traditional’ DAW’s you would go through sub menus on each channel to allocate effects etc, within Ableton and Resolume you just drag them wherever you need them which encourages experimentation straight away. Also, the envelopes you can allocate to parameters introduced in Resolume 6 are visually similar to LFOs/Envelopes within popular workstation synths such as Massive and Serum, anyone familiar with them will quickly work out how they can be used.
We very quickly found parallels between Ableton and Resolume, in their ease of use, flexibility and workflow. Using Ableton’s MIDI envelopes to control Resolume was a revelation to us, and being able to program everything in real-time felt a lot like making music. Having the ability to tweak any number of visual parameters using automation inside Ableton is very powerful, and is similar to our process when working with just audio.
Let’s talk about Koan Sound’s new setup- from design to reality.
The setup used for the live performance was conceived and designed by Tim Smith. When the team first met up to discuss plans it became apparent that KOAN wanted to be ‘surrounded’ by the visuals so Tim drew up some ideas for curved/angled LED screens that sat behind the performers and we all agreed it was the right direction to go in.
Using Grandma2 OnPC, Ma3d and a simple video to CITP converter, we were able to pre vis the designs and collaborate on the final piece.
Lighting wise we wanted to be able to have an either-or situation, video only or lights only so the entire show wasn’t solely content focused.
Tim picked GLP X4 Bar 20s that sat above and below the video wall, Robe Pointes placed in between the gaps of the LED screen and stage right and left in order to highlight KOAN and finally SGM RGB strobes placed on and behind the structure to add some depth.
Regarding touring the plan is to have three tiers of stage design all based on Tims original setup that we can alter depending on the venue but FOH and routing will stay the same.
Tell us about the routing & the hardware for the show.
You have a FOH (Resolume) Laptop, a stage laptop and a playback laptop?
The stage laptop is sending MIDI note and CC data to the FOH laptop over network, in addition to MIDI from the instruments on stage.
The stage laptop is responsible for MIDI both in terms of VST playback in Ableton and sending data to alter parameters/trigger layers in the dedicated Resolume laptop at FOH. As KOAN could ‘play’ the visuals alongside their instruments it made sense to have a separate playback laptop to ensure a smooth show in case anything went awry, something Jon Savage, our experienced production and tour manager made sure of.
In terms of the hardware:
Resolume: GTX1080 MaxQ equipped W10 laptop
All of these laptops (alongside the GrandMA2) where connected via a network switch sending MIDI over LAN with the playback sending a separate LTC track over a stage box to the GrandMA2.
I hear that you are big on masks. I, personally, love the endless potential they provide. Tell us how you use them in your show.
When we were putting together the Arena composition it seemed like the easiest way to lay everything out was to use one deck per song, so that each track in the set had its own specific overlay and effect clips that could be independently adjusted.
In addition to selecting the deck with a note message at the start of each song, MIDI data is used to trigger masks and effects, as well as control a variety of parameters such as opacity, clip speed and start point.
The masks were also a way for us to showcase animations we’d made ourselves. Jim used Max/MSP/Jitter and Processing to create the overlays, rendering out lots of longer generative clips in high contrast. To make the show different every time, our generative masks run in the background, and by using a video router clip on a layer above, we found a way to trigger an opacity envelope on the mask without re-triggering the clip itself.
You also work with Artnet merge? It is such a great tool. It’s great to see more people use it to achieve sync between lighting & video.
We actually tested the Artnet merge last but it proved one of the easiest things to get going!
I created the X4’s within the fixture editor:
And, placed them along the top and bottom of the main output (around 20 pix in height) matching their position on stage.
The merge was done at the Artnet node using a HTP setup and a trigger channel on the MA to activate/deactivate the merge. This meant Tim could control brightness, pan, tilt and zoom of the X4’s whilst Resolume dealt with the mapping. We essentially extended the screen upwards and it added so much depth to the content.
We did hit some rather large delay when activating the merge but we put that down to the Artnet Node used.
Good to know that ArtNet Merge is not that complex.
Let’s talk about all that juicy content now. Obviously, you had multiple visual artists working on the show?
Yes we did. The team at KōLAB Studios (Sarah Koury, Alastair Brookes and Khali Ackford) used watercolor paints with various other fluids and materials to create abstract forms and movement. We used a Sony a7R III with a 90mm 2.8 macro lens to film everything at 120fps, and experimented with different lighting setups. Our friend Loz Blackwell then used this footage to map displacement of the album artwork. After Effects was used to synchronously time remap and transform a host of macro shots, using this as a displacement map to then apply to the artwork stills. Working with such high resolution photographs enabled the cropping and selection of different sections to match the motion of the underlying visuals.
In addition, the team at BEAMA Visuals (Ben Leonard, Leigh Powless and Logan Anderson) played a big role in the content generation. They come from a range of arts backgrounds (animation, painting, graphic design respectively) and each brought their unique take to the project.
Ben wanted to recreate the complexities of natural systems in a digital space, using Cinema 4D and X-Particles in order to achieve organic growth and fluid dynamics.
For Leigh the project offered the perfect opportunity to merge digital with analog, and she crafted much of her content by hand, mixing different pigments and additives to create organic patterns of color which were then brought into After Effects to be digitally manipulated and finished. "I was trying to create visuals that were sort of living, breathing paintings, playing with symmetry and rorschach style imagery to invoke a dreamy, surreal response in the audience."
My main duty content wise were visuals for the non-Polychrome songs that showcased KOANs back catalog so I wanted to create visuals that sat apart from the albums dedicated content but didn’t appear jarring.
Some of KOANs earlier work is more dancefloor focused so it gave me a chance to use some bold colours and quick edits.
The whole team shared a spreadsheet where we divided up the workload and decided what colour schemes each track should adhere to, this workflow meant we could maintain fluidity throughout the whole show.Visuals were mainly created within After Effects using the sound to keyframe tool to trigger effect parameters and layer opacities. KOAN rendered out separate stems (Bass Drum, Snare and Bass) to allocate different parameter changes to each resulting in tight visuals.
I also used C4D and Octane to create loops which were then manipulated in After Effects to create this bright warped shapes which KOAN liked the look of and I used this technique on a few songs.
So cool. We love it. The content is beautiful.
Tell us your favorite bits of the show. What do you love, what didn’t really work out?
We’d been relying on master tempo automation as a way of keeping everything on the grid in our onstage Ableton project, and using the Link protocol to sync the clocks of the two laptops. It had proven stable in the lead-up to the shows, but during one of the final rehearsals we discovered that these changes in tempo were causing the playback and LTC audio tracks to glitch.
We weren’t fully sure about the technical reasons for this, but thought it might have been because of the playback cursor on the synced laptop not moving smoothly through the changes in tempo, therefore causing the LTC being read by the media server and lighting desk to jump around. We had to completely scrap the tempo automation and manually move everything off of the grid in Ableton, which took ages!
The crowd at the show were so energetic, definitely a pleasure to watch people react to all the hard work the team put in.
The main issue for me was the dreaded ‘Optimus’ that cropped up on our Resolume laptop due to the inability to turn it off via BIOS. I had to use the workaround of creating the external display (in this case our LED wall) as the main display and everything ran smoothly after that.
Finally, we love how you’ve used all the different features Resolume has to offer to make your show. Talk to us about your favorite features. Is there anything you would like to see developed/worked upon?
The envelopes inside Resolume are amazing. It’s great being able to finely control any number of parameters, plus all of the different easing functions are really cool. Also how easy it is to repurpose content, by adding effects and quickly mirroring and inverting clips, as well as changing things like speed and scale.
The updates in 6 have been a pleasure to explore, mainly the envelopes and modular re-design to be able to alter your workspace. I’ve also used the DMX output intensely over the years running full lighting and visual shows from a single laptop; using simple animations you can come up with bpm synced chases that would take an age to program! In terms of future updates I would love to see the inclusion of basic 3D modeling and lighting.
Thank you for doing this interview, you guys.
Polychrome looks beautiful and we can’t wait to catch it live.
Meanwhile, give the album a listen here.
All photo credits: "Sarah Koury / KōLAB Studios"
Mushpongo Posted by shipra on Wednesday April 24, 2019 at 18:43
Login to the forum to comment.
This blog is about Resolume, VJ-ing and the inspiring things the Resolume users make. Do you have something interesting to show the community?
Send in your work!
- * Resolume 7.2.1 Maintenance Release
- * Resolume 7.2.0 Released! Clip Renderer, Recording & Conversion Improvements, 2 New Effects
- * New footage! toomuchlag, VJ Galaxy, Tomika
- * Visual storytelling with STVinMotion
- * Want Plugins? Get Juicebar!
- * A footage pack in memory of Jonathan Bernbaum
- * In Pursuit of Secret Mapping Experiment
5 * 6 * 7 * Ableton * ableton live * Adobe * Aerosmith * aliens * apparel * April * arena * Armin van Buuren * artist * artnet * av * AV * Beeple * Behind the scenes * behind the scenes * Benchmark * boompje * bugs * chaser * Club3d * coachella * comix * conspiracy * Content * controller * D-Fuser * Dan Wise * datapath * Denon DJ * dmx * doc optic * DXV * e2 * Eyesupply * FFGL * ffgl * Flash * Flying Lotus * free * FreeFrame * freeframe * gabe damast * ghostdad * gpu * Haloween * illenium * installation * interactive * Interactive * interview * Jobs * kbk visuals * laser * led * Leonardo * light festival * lights * madeon * manoeuvre * manuel rodrigues * marino cecada * Max Cooper * max for live * Max for Live * midi * mnvr * multiscreen * nature * Neal Coghlan * nicky romero * noisia * openFrameworks * OSC * patterns * pixelmap * plugins * Plugins * porter robinson * Processing * projection mapping * Projection Mapping * Quartz Composer * realtime * rebel overlay * review * Ricardo Arjona * richard de souza * rick jacobs * rigging * rjb visuals * rock * roy gerritsen * samson * sandy meidinger * sculpture * secret * showjockey * SMPTE * Spout * stage design * StageLinQ * storm festival * strip * STVinMotion * sync * Syphon * Tagtool * Total Unicorn * TouchOSC * touring * training * true colors * tutorial * TV-One * ultra * unity * Update * update * upgrade * v-squared * vello virkhaus * Vezer * video * vvvv * workshop * zedd