The Lab

The Development of Cymata

With our new website up and running, we felt like it was a good opportunity to look back on the development of our most recent project, Cymata. What follows is a round-up of some of our thinking, early experiments, and discourse over the project’s evolution during the last 4-5 months. It’s been a pretty enlightening experience so far in thinking about how to express sound through new technology.

The Ideas

cymata ideas

With any project, whether its big or small, it’s always good to outline the overall goals and objectives at the start. It took quite a few meetings between Tim, Celia, and I around researching, writing, and sketching. But we eventually got to a good place with everything. The idea of generating and making paper models were a big inspiration for us, and we started this project thinking about how to give people the creative tools to make their own sculptures. This started us in the direction of making downloadable patterns that everyone could print out.

It wasn’t quite strong enough though. We started to think – what would make these objects more meaningful? Music came to mind.

Now we knew we wanted to work on a new project involving sound and making it visual. We also wanted to give this power to anyone, so they could make their own ‘sculptures.’ Having that control over a physical object, made from something you feel strongly about  – really excited us. So we repeated our stages of thinking again with some more researching, writing, and sketching. This time it was pretty focused on this idea though, and that allowed us to start thinking about ways to make it happen.

Starting with Processing

We needed something to program an application in, which could ideally give us a 3D model to print. Processing is a great tool for developing projects like this. I had some experience developing projects with it in the past, and the libraries available for audio analysis and visualization are well done. To get started and run a few prototypes in a couple weeks didn’t take much – just focus and keeping some good Processing reference books handy. Tim had a great book that assisted a great deal in developing tests for Cymata.

I had done a bit of audio reactive work with Processing, but this was a bit different because of the end result being a printable 3D model. Even with that in mind, we wanted to start simple. So to wrap my head around audio analysis, and how those related to changing visual things, I started with 2D experiments. One of the key things I wanted to get a handle on was how to get results over time from a complete sample of audio. So I started with a modified example that was built-in with Processing, and added some extra things to play with the visuals.

sound mountains

From 2D to 3D

I continued to do more experiments with 2D visualizations of music, this time adding some motion to the lines I was drawing. We wanted to figure out how we could affect shapes in motion, to later control through a simple interface with some parameters. I then started to experiment drawing simple 3D shapes alongside the 2D ones, which expanded as the music progressed. It was the first time a 3D form was thrown into the mix, and seeing something with volume react to music was pretty cool.

2D & 3D Visualization of Disclosure's "Latch"

Moving to 3D

By this time I had a good grasp of drawing shapes in 2D that reacted to music. It was time to move to try and move to full 3D, so we could have some more direct control over producing the final object. Processing has some great built-in 3D support, but we needed something that could give us more interesting results. I was lucky enough to find the hemesh library through some guidance from Matt Pearson, which gave us a whole new world to play in. With it I was able to start creating some basic examples with some of hemesh’s built-in geometry.

From here on, it was time to start working with randomizing the geometry, and getting into creating custom meshes. This enabled us to get a good preview of what the final object could look like. I started with a simple mesh plane, and added the snippets of code I had been building to change the geometry.

Screen Shot 2013-03-05 at 10.11.05 PM

Then it was time to add some modifiers, which is something you can make as a custom feature through the hemesh library. They’re essentially filters for the 3D model, warping it in different ways depending on what values you feed it. We started getting some really cool results.

Screen Shot 2013-03-12 at 9.46.58 PM

Screen Shot 2013-04-02 at 1.30.23 PM

 

Where We’re Going

We’ve gotten to the point where we have a great work in progress on how to make a 3D model in Processing react to music. But we still have development to do, and that’s mostly around making the application easy enough for people to use. We’re also working on creating a model that represents a complete song or audio sample, instead of just visualizing it in real-time.

We’re excited to keep developing this project all the way through, and we thought it’d be nice to share the knowledge that got us this far. You can check out out the latest version of Cymata on our Github page, and watch our blog for more updates on the process.

Drop a comment

Your email address will not be published. Required fields are marked *