Imagine having a signature synthesizer sound that followed you around in the real world.
What if your sound could interact with others in a physical space?
What does it sound like flying around the room?
PollySynth combines simple UI with fun character design to virtually eliminate the learning curve associated with synthesizers and the technology involved.
PollySynth is a multiplayer polyphonic synthesizer.
Synthesizer voices are visualized as members of the PollySynth gang. Players fly these guys around and create audio havoc just by loading a website on their mobile and playing with the controls. Spatial audio effects add to the fun as characters zoom by.
PollySynth is currently available as a location specific experience. For inquiries please contact firstname.lastname@example.org.
Lately I’ve been making synthesizers using the Web Audio API and decided to build a simple monophonic synthesizer to get others started. I gave a talk about it last week at Converge SF, the video should be online soon. For now, here are my slides, code on GitHub, and the synthesizer in four parts. It’s fun to play on an iPad or iPhone (in landscape mode). I’ll be adding QWERTY keyboard and MIDI support for the piano keyboard soon as well as other synth modules.
Graphic designer Superdeux came to Secret Feature with an interactive project to be featured as part of his gallery show “I Wish I Could Talk” at The Summit SF. Shapes would be projected on a wall and users can interact with the shapes by moving in front of an XBox Kinect. Fun and intuitive. During the show’s opening reception the space in front of the projection would be clear and people would be free to play. During the exhibition interaction would be limited to people passing in front of the projection, colliding with the shapes.
ActionScript to C++
I have been a Flash developer since the early days and my programming skills have grown with the platform. I was hungry to try out a new language and this seemed like a great opportunity. Fascinated by the pieces Robert Hodgin had been creating in Cinder, I installed Xcode and began working through the Cinder tutorials and examples.
I spent a couple weeks haunting the Cinder Forums, watching videos, playing with example code, and reading a C++ book. I looked into 2D physics libraries in C++ and Box2D seemed like a good start since I had used the ActionScript version and Cinder Forum user Sansumbrella had worked out the integration with Cinder. Superdeux’s shapes needed to be circular and textured so I created a fork of Sansumbrella’s Box2D Sandbox and added circular and textured elements based off the existing BoxElement class- https://github.com/secretfeature/sc-Box2D
Kinect, OpenNI and OpenCV
Next I had to figure out the Kinect interaction. I was really impressed with the work people were doing using OpenNI, which matches body motion accurately using skeleton detection, but it requires users to pose with their arms in the air while it calibrates. This won’t work for the casual interaction we are looking for. After more digging around in the Cinder forum I found an example of hand tracking using the Kinect by Andy Berg- http://forum.libcinder.org/topic/simple-hand-tracking-with-kinect-opencv
This method uses the OpenCV library. Andy’s example is for a single user but it seemed possible to reuse as a multiuser/multipoint control system. I bridged the Kinect/Box2D worlds by dynamically adding and removing invisible Box2D elements that would be pulled toward the Kinect touch points, colliding with the onscreen shapes. This would allow multiple people to use the app and it would also provide the effect of collision as Summit customers walked by the projection. A debug shot here shows the control points in turquoise-
At The Summit
With a couple days to spare, I added a feature to take snapshots of people in front of the kinect along with a screenshot.
This being my first C++ app, was very rewarding. I am more than happy with the performance, the interaction is good and Superdeux’s graphics make it fun to watch. I’m excited to start my next Cinder project. Please go see Superdeux’s exhibit this month at Summit if you are in San Francisco. The interactive piece is on display in the evening.