Silent is a two minute video created by combining frames from five classic silent films: Metropolis, Faust, Nosferatu, The Holy Mountain, and The Dragon Painter and put to the music of Charles Ives‘ Hallowe’en. The frames are chosen by custom software that compares data from each of the film’s soundtracks with the data from Ives’ music.

The software analyzes each film and records the audio (FFT) data and timecode for each frame. The final video is generated by processing an input soundtrack, in this case Hallowe’en, and finding the frames of film whose audio best fits that of the soundtrack.

Silent films were chosen as the source material because of their tight connection between narrative, visuals, and musical score. By using the soundtrack as the central driver of visual imagery, Silent inverts these relationships. This reversal allows forms typically associated with music-repetition, rhythm, movement-to express themselves visually.

Lucky Dragons
Mecca Vazie Andrews
Melina Ausikatis
Diaz
We Are The World
Rainbow Arabia (directed by Keith Musil)
Leyna Marika Papach
Nina McNeely
Renata Raksha
Peter Moran
Chandler McWilliams
High Places (Rob Barber)
Molly Shiot
Hawnay Troof (directed by Joseph Whitt)

Silent was created algorithmically using silent films and audio processing. More about the piece coming soon.

I’ll be performing my piece John Henry von Neumann at Machine Project from 10-6 on Sunday June 28th. Come watch me compete against a computer to complete a drawing in an eight-hour workday. I will use pen and paper while the computer uses a plotter. More info on the Machine Project site.

Week End (mise-en-scène) is a triptych of images generated using scenes from Godard’s film Week End. Each image was created through an accumulation of colors from frames of the film, where each point in the final image shows the average color value for that point over the course of the set of frames.

The first image was made using frames from the “Analysis” scene, in which one of the main characters, Corinne, wearing only bra and panties, retells a story of a sexual encounter while siting on a desk talking to an unnamed fully clothed man.

The second image combines every frame of title text and the interstitial text used to cut the narrative.

The third image was created from the frames of the famous traffic scene, a single long tracking shot depicting a traffic jam punctuated by a horrible accident.

It was important to keep the butterflies light and delicate–ideally as delicate as an actual butterfly–while allowing the wings to move in response to stimulus from the environment. We ultimately decided on laser-cut, heavyweight drawing paper to make the wings themselves, and we devised a mechanism using neodymium magnets and a servo for the wing movement.

Each wing was laser cut from the drawing paper, then fitted with 2 pairs of small neodymium magnets, making sure that the polarity on all magnets was kept consistent. Under the mounting board sat a much larger magnet oriented with the opposite polarity of the magnets on the wings. For example if the magnets on the wings all had the south pole facing down, the large magnet had south facing up. A hobby servo was used to rotate the large magnet so as it moved closer to the wings, the magnets on the wings were repelled causing them to close. All the servos were connected to a single computer using an Arduino board which was in turn connected to mac mini running a processing application which used a webcam mounted at the back of the room to look for movement in front of each butterfly. As visitors walked closer to get a better view, the butterfly in the pedestal they approached begins to flap its wings.

While working on the mechanism for the physical butterflies, we also started working on possible methods for patterning the wings. Andrew and robert had been working on a generative system based on simulated charged particles for some prints in another room of the gallery; and they modified some of these systems to generate point distributions that were in turn used to create voronoi diagrams fitted to the wing profile. The voronoi patterning eventually produced two of the families of butterflies. In addition to the “straight” voronoi, another version was created that fit b-spline curves to the voronoi cells.

Another set of patterns were produced using a fluid simulation and vector field visualization algorithm. The wing profiles were used as boundaries for fluid flow systems. This field was then fixed and rendered using fitted lines and circles.

The fifth pattern family was created by drawing random lines in the wing profile and placing circles at the point of intersection. The lines and intersections created cells that were then offset to create a structure for the wing.

Inspired by Benjamin Aranda and Chris Lasch’s book Tooling. I developed a variation of the cracking algorithm found in that book which introduced b-spline fitting to smooth the finaly output and give a more organic feel. The algorithm “cracks” a polygon by dividing it from and edge to its centroid, each resultant polygon is the put in a pool to be chosen at random and recursively cracked again.

The final patterning system was a simple random circle packing algorithm. Randomly sized circles were placed in the wing shape, if there was any overlap with the already existing circles, the new one was tossed and the process run again until the wing was filled. This algorithm produced a surprising degree of variation by altering the sensitivity of the overlap testing function.

A few members of each family were printed and mounted on the wall using insect pins, giving the whole thing and extra classic feel. The final movement turned out to be very creepy. Having them react to activity, not proximity, really helped add to the overall feel. As one approached a case it almost felt like as if they were flapping in hopes of escape, only to give up a few moments later.

Here is a quick video showing the movement and the space. You can see the mechanism in there too.

You can see some more photos on flickr.