project listing - Physical
Doppleganger Interactive Theater Residency
In collaboration with “Feed the Herd“, Doppleganger is a residency project at 3 Legged Dog through the month of July 2007. As part of the residency, I built hardware in the form of sensors and actuators for actors and the set. This was integrated with live video and visual effects through the Isadora environment. We were also able to sync the lighting cues with the physical computer/live video, further integrating the visual components of the show together through Isadora.
“Doppelganger is about two people who strive for material happiness, believing that superficial pursuits can bring personal satisfaction. After a life-changing event they must adapt and realize that their pleasure pursuits are only a single, small element of living, compared to the grand scope of the universe. For true fulfillment they must take a risk that contradicts popular logic and personal inhibitions to achieve a lasting understanding of happiness and the world. At its heart, Doppelganger is the simple story of a network engineer, George, and a human resources executive, Marcia, coming to terms with the death of their mutual acquaintance, Frank. Heath complicates this by creating a double world. Frank dies by falling to his death while joking with George, but simultaneously, the same Frank is having an affair in the office across the street with Marcia. Time and Space play tricks on the two protagonists, haunting, perplexing and tempting them to open up its mysteries and come to an understanding of Frank’s death and their own existence.“
To further augment the dream-scapes and alternate realities the characters in this play exist in, video has been an integral visual element which helps cement the fragmented thoughts and experiences as the narrative progresses. Additionally, as part of the residency, we’ve been exploring the uses of motion tracking and physical computing (in the form of accelerometer and force sensing electronics) to build relationships between the actors and their surround landscape on stage.
tags: 3ld live pcomp performance sensors theater video
Sight Gauntlet
A thought study and rapid prototype, Sight Gauntlet is a wearable sensory augmentation device.
Users wearing the gauntlet are given tactile feedback in the form of vibrating motors on the arm which relate to the distance from where they point their hand.
An array of vibrating motors on the arm mirror the distance, giving extra-sensory perception of the environment to the wearer. This project was a thought experiment, exploring how physical computing and wearable technology can augment and extend the day to day experience of a human being. It utilizes a very simple microcontroller, proximity sensor and motors to provide this simple interaction with the surrounding environment.
MS Gloves
A consultation for Wilson Built in Brooklyn, New York.
This was a short adaptation project from existing hardware. Wireless communication was needed between gloves, which simulate sensations to that of multiple sclerosis.
These modules were adapted from Velleman IR transmitter/receiver pairs, allowing vibrating motors to be triggered remotely by software. This software was integrated with visual and treadmill feedback, which was all synced to emulate an experience from the perspective of a person with MS.
Antikythera Mechanism Exhibit
In Collaboration with Rich Miller and John Schimmel, for installation in the Ancient Greece exhibit at the Children's Museum of Manhattan.
This project consists of two touch sensitive kiosks, which allow users to view and experience the antikythera device. This device is currently thought to be one of the earliest computers, designed by the greeks as a timepiece, it is one of the earliest uses of gears on record. In conjunction with HP research, I was involved in integrating the hardware with PTM technology HP had developed for viewing pre-photographed objects in a multitude of different lighting conditions.
Above is a picture of the installation in progress. It is now alive and active in CMOM, and will be installed for at least another year and a half.
Open Shutters: A Portable StoryBooth for New Orleans
Part of a collaboration and grant project with NYU's Interactive Telecommunications program and Xavier University in New Orleans. Inspired by co-collaborator Ruth Sergel and her previous work with "Voices of 911", we built a portable and easy to setup private space in which to record stories. In conjunction with Xavier University's art program, a small group of us designed this booth primarily of found objects and salvaged building supplies of post-Katrina homes. Two non-profit organizations, "Habitat Re-Store" and "Green Project", salvage old and historical buildings throughout New Orleans in hopes of reducing waste and saving historically hard-to-find supplies.
This project took three days, and the help of a variety of individuals in no particular order: Ruth Sergel, Roy Vanegas, Sonia Nelson, Pars Marash, Patricia Oakim, and Yonatan Kelib. It consists of multiple modular pieces, which can be assembled and disassembled quickly, making for a portable and versatile space in which to record and document the memories and stories of NOLA citizens.
Additionally, our trip to New Orleans was an effort to educate and help integrate participatory-media concepts into various institutions and non-profit organizations. This project, in its entirety, was made possible through the Nathan Cummings Foundation in New York.
tags: booth community participatory portable private video
Motion Sketches - Ongoing
This is an ongoing project, in which weekly physical interfaces and built and tested. The overall goal of this project is to acquire a large collection of working prototype interfaces, which can be tested and re-constructed together to find interesting and novel approaches to user interaction.
This is a quick sketch using a copper etched board,a small microprocessor, accelerometer and USB I/O
The recent development of the Nintendo Wii, along with the very recent Apple iPhone, has brought with potential for a rich (and potentially over-saturated) wave of gestural and kinetic user interfaces and experience devices. This project aims to join people who are building devices in this rush of new interfaces, but also build a language and learn from the process. These physical sketches are just that, quick and dirty tests and attempts. They are not meant to be final projects, but the knowledge gained from them as a whole is.
As an ongoing project, I wanted to give myself some constraints. First and foremost, the goal is that the capture device is as simple as possible, having a very simple set of components. These components may appear to be giving very simple information, but will generate very novel input when used in conjunction with each other. For this purpose, I've decided to begin by using nothing but a simple three-axis accelerometer. This allows me to read in these three values, and begin building software and firmware that will generate new results as an interface, while still retaining a sense of familiarity that allows users to easy understand how they use it.
tags: body gesture input interface motion physical
Phrases Collide - Fabric Instruments
Music in its digital form has taken the analogy of a wave or string. Once thought of as mystical and full of wonder, sound has lost most of the awe as its been converted into ones and zeros. In my approach to this project, I sought to connect the idea of a string of sound with a physical object that is controllable and shape-able.
Utilizing two pieces of rope, wrapped with a sleeve that has conductive thread sewn into it, these seven foot long ropes are given properties of a variable sensor. The performer can them move their hands, covered with conductive gloves, across these rope surfaces and generate values that are used to change sound.
Close up of the conductive sleeves surrounding the ropes, used to generate values to change sound.
These ropes are connected to software which stretches and changes pre-recorded and live sound, just like the fabric of the ropes are being distorted and moved. Using a technique called granular synthesis to grab tiny pieces of sound, these tiny "particles of sound" can be reattached to each other to slow down or speed up sound without affecting its pitch and timbre. Using the values from the conductive sleeves, the sound can be reconstructed as if it's inside of the rope itself.
tags: conductive fabric granular instrument interface music sound synthesis
Death of Sound
Overview
Created in the fall of 2005, Death of Sound is my initial exploration using artificial intelligence for computationally-driven musical instruments. Seven individual steel-stringed wooden instruments make up the project, in which the each are mounted to the wall. These all connect to each other through an eighth wooden box below. Programmed to be aware of people in the space, as well as each other, these seven simple tone makers make their own decisions on when to perform.
When one of the instruments chooses to make sound, it begins powering a motor spinning four rare-earth magnets. Based on the speed and the duration of that spin, the magnets induce vibration of various different tones, and assorted attacks and decays can be performed. Each of the seven are given certain characteristics when the composition begins. Over time, they use these characteristics to decide how to live, and begin a simple life-cycle until their compositional death.
Concept
Originally, upon their death, I had hoped to have circuitry which would destroy itself once its composition was complete, creating truly one-time instruments. After a prototype of this nature, however, I became attached to them and noticed others viewing/listening to them made a connection as well. In that regard, I lost my desire to see them physically die, and decided to save that element of the project for future exploration in other work. Rather, I became more interested in the story-telling elements of these instruments, as they live and act near each other. This lead to my further investigation of these instruments as objects with life characteristics, which grow and change over time and ultimately come to their end.
Compositionally, this project explores the sound created when performers argue, interact, and neglect each other. As these different instruments fight for attention over each other, very simple acts of motors spinning become much more complex anthropomorphic experiments. Most, if not all of this is done within the audience's mind.
As much as I try, as a programmer and circuit-maker, it is simply too complex to create human emotion in simple inanimate objects (at least intentionally). Ironically, the objects do a wonderful job of it on their own when viewers create their own interpretations. By creating objects which appear to speak the same musical language and have similar dimensions and placements, people begin to assign them similar emotional entitlements. This has fostered some incredibly simple, yet fascinating ideas about simulating intelligence, which I hope to use in future work.
Technical Details
This project is a combination of instrument design with a little electronic cocktail underneath. In its structural design, its a pretty simple piece. A single microcontroller guides and controls each instrument, through the spinning of the motor. Also, each instrument has a pre-amplification circuit and a microphone, which is then connected to an external sound source. This allows the instrument the ability to be amplified in the space beyond acoustic sound.
This shows a bit of the spaghetti of connections inside of the main control circuit (without the microcontroller)
Future Considerations
Reaction in the installation space fostered a completely different outlook on the project than I had begun with. Originally, my approach to this project was to explore the idea of death in inanimate objects. I wanted to see if an audience could sympathize with a simple mechanical tonal object. From a technical standpoint, I investigated ways in which to have circuits actually destroy themselves electrically. Through the use of nichrome wire, which is commonly used to electronically light fuses for fireworks, I found a viable solution. By supplying voltage to this wire, it becomes incredibly hot, and can short itself like a fuse as well as melt components and circuitry. As mentioned before, I decided not to use this concept in the piece, but it remains an area of exploration in future work of this kind.
After initial reactions from friends and peers, I have become much more interested in the mental reactions the audience performs during the course of an installation. I hope to employ ideas of endowing inanimate objects with other life characteristics, in an effort to give the audience further ways in which to associate with simple musical devices.
tags: artificial automata frequency intelligence interface magnetism music networked objects resonance sound strings
Wave Machine
A collaboration with Todd Holoubek, Dan Shiffman, Meghan Trainor, and Dimitri Negroponte. The Wave Machine is an experiment in taking an artificial mechanization of a real phenomenon and reproducing it in digital form.
.jpg)
Nine of the twenty one hand-etched boards
In today's world we are confronted with the unreal, primary as digital media. Through the media we are forced to accept the "essence" of what we would like to have. This project takes this idea to the test by reproducing the much loved "wave machine" as a tilting light grid. We are offering the essence of early nineties kitch as a series of ones and zeros.
.jpg)
The wave machine, in a test run
Inherent in this project, which uses the emphemeral and realitivly abstract medium of computer programming, we have sought to focus as much on the physical elements of the digital object. The wave machine is a mechanical abstraction of water. To create a digital form of this natural event implies a more ephemeral experience. However we have focused as much attention on the physical aspects ofcreating a digital experience as we have to the programming elements.
.jpg)
A view of the wave machine from behind, showing connections between the boards
Handcrafting our circuit boards, etching designs, words, and parts of the programming language we are using itself onto the boards, is in opposition to both the ephemeral and mass-produced elements of contemporary digital experiences.
tags: board circuit copper etch machine wave
Light Organ - Empire State Building Public Art
Currently an ongoing collaboration with Peter Coffin.
This project is an exploration of synesthesia as it is conceived in relation to sound and color. There has been extensive research in this area for hundreds years, with many different theories from widely popular to widely unknown scientists. Our project aims at controlling the colors and tiers of the empire state building from a single piano, and then hosting performances which would be available in different mediums for people to watch and explore the tonal space, both through sound and sight.
For the first part of the project, I worked on the electronics of three different five foot tall plexi-glass models of the building. These prototypes were connected to a midi-keyboard, where you could interact with the model in realtime, watch the colors change and interact to your performance. Below is a link to the preliminary code for the project, a demo of the top tier in action, and a look at the microcontroller and electronics under the hood.
Code - Top Tier Demo - Electronics
)){kind=link}
Can - N - String
This project is a collaboration with Liubo Borissav, ITP 2005.
As an exploration of networked "objects", this project had a simple mission: recreate the can and string phone, but replace the string with the internet. Our purpose was simple, to keep the metaphor and aesthetics as close to the same as possible on the outer edges, and employ the technology in the middle.
Our device did just that. Using two tin cans, with embedded electronics and micro-controller managed connections to each other, these two devices could be plugged into working cat-5 internet wall plugs, and locate and communicate with each other. Standard audio signal rates posed a problem on devices with so little room for compression and audio filtering, so our output on either end became nuanced and "colored" with the static that the internet often provides. We felt, in many ways, this was an added aesthetic, as it brings to light the distortions whatever technology you employ has.
Unfortunately, this project was never appropriately documented.
Social Pedagogy Instrument
This project was developed with Jon Kircherr and Michael Horan, as a collaboration at ITP, 2005.
The purpose of this project is to create a multiple-user interface for learning and exploring western music concepts, without the overhead of learning technical specifications of a classic instrument. Secondly, to create an instrument which needs more than one person to play, thereby shifting roles and responsibilities from standard scenarios of performance. Rather than one person striving to master a difficult instrument, this tool allows six users to distribute the complexities and concepts of music into manageable parts. This allows each person the opportunity to understand their individual role in relation to the entire group.