Tuesday, May 26, 2009

Visual Effects as Audio Effects

Background
I wanted to use visual effects such as blurs, colour transformations and pinches as audio effects instead of visual effects.

Overview
I wrote a simple Max/MSP patch to help me out with this task. The idea is quite simple. The patch works in two phases. The first phase is to open up an audio file (of up to 8. 163 seconds in length). The patch then converts each individual audio sample into a pixel in an image of 600 by 600 pixels (this equals 360,000 samples), and saves off this image as an image file.

Each pixel represents the audio with a quality of up to 24 bits, spreading the information evenly over the three colour bands of red, green and blue like this:

COL: RED
BIT: MSB 23 22 21 20 19 18 17 16 LSB

COL: GREEEN
BIT: MSB 15 14 13 12 11 10 09 08 LSB

COL: BLUE
BIT: MSB 07 06 05 04 03 02 01 00 LSB

This image file can then be manipulated in an image editing program such as Photoshop. I have done all of my editing using Graphic Converter. The edited image can then be opened in the Max/MSP, which reads each pixel and converts it back to an audio file.

Max Patch
Download the patch here. Below are all of the screenshots of the Max/MSP patch.






Outcomes
Here are some resulting waveforms with their corresponding images. Naturally, the images have been compressed to be uploaded onto the web so there might be some artifacts not visible in the original, uncompressed versions.


Unedited audio (listen):


Gaussian Blur (listen):


Motion Blur (listen):


Glass Blocks (listen):


180 Degree Rotation and Zoom Blur (listen):



Vertical Crop (50%) and Vertical Rescale (200%) (listen):

4 comments:

nitro2k01 said...

Oh cool... A friend of mine did the same thing, but in reverse... He took an image and converted it to audio, fed it through livecut and other tools, then back to an image. The result can be seen here: http://www.oxo-unlimited.com/images/mynoise.jpgThis also reminds me that I need to upload my Bitmaps&Waves files that I did long ago.

Maxim said...

It might be interesting to use a Hilbert curve instead of mapping the samples onto the raster lines. The improved locality might allow 2D effects to work better.

poppi said...

Hey, these are really interesting (I particularly like "gaussian blur" :)

So when are you going to play out again, Sebastian?

Sebastian Tomczak said...

Hi Poppi! Thanks for your comment.

What do you mean with "play out"?