LEARN THE IMAGE
INTO SOUND PROCESS :
The concept of turning Image Into Sound can
be basically and simply divided in 2 steps. Step
1 – capturing the color information
of an image and Step 2 – feeding
the color information of this image into
the computer's sound card.
Before I continue to explain the Image Into Sound process in a more detailed
and technical manner, please make sure to read the Key Terms located on
the right side column of this this page. Also, please note that there are
infinite ways to convert images into sound, it all depends on the algorithm
of choice. The following "recipe" of this process is only one
of the many ways to convert Image Into Sound.
Please take note that I'm using Processing as the platform
to code my program and Bagel (Processing's compiler) to
compile the code into a Java Applet. In order to explain
the steps of this process in more detail, I will be showing
parts of the code used to create the Image Into Sound
applets, remember that this code is specific to the platform
of Processing, but the concept of it and logic used can
be applied to a number of other languages or platforms.
The code in the examples will contain formulas and algorithms,
but I will not explain the code in detail as the purpose
of this web site is just to explain the process of creating
image Into Sound and not teaching how to program.
1. Load an image file into memory by storing it into an variable
or array.
Because an image file is a stream of color data (pixel-color-information)
that represents an image, we can take that data and store
it into the computer's memory by assigning it to a variable
and or array. In this case, we will be using "BImage"
a datatype variable used by Processing to store image
data, the BImage object contains properties for the width
and height of the image, as well as an array called pixels[]
which contains the values for every pixel in the image.
BImage img;
img = loadImage("myPicture.jpg");
Line one declares a variable called "img" and
because it is declared as of the BImage type the computer
knows to expect color-data from an image file. In line
two, image file "myPicture.jpg" is transferred
and stored into the declared variable called "img"
with the use of the loadImage() function.
2. Separate pixel-data from the array in to Red, Green
and Blue values.
Separate the color-data, from the array that contains
the image-pixel information, into Red, Green and Blue.
Create 3 corresponding arrays that store each color value.
Thus each one of the 3 arrays will contain color values
corresponding to Red, Green or Blue in the range from
0 to 255. 0 being dark and 255 being bright.
color rgbCol = aimg.pixels[j];
float redCol = red(rgbCol);
float greenCol = green(rgbCol);
float blueCol = blue(rgbCol);
3. Convert the Red, Green and Blue array values from
0 to 255 into -1 to 1.
Convert the color values of each array from the the current
values of 0 to 255 into a range of 1 to -1, and store
the new values into 3 new arrays. We need this conversion
in order to plot each value, as a note, on to a sine wave
that the sound engine/sound card can play.
soundR[j] = (redCol-128)/128;
soundG[j] = (greenCol-128)/128;
soundB[j] = (blueCol-128)/128;
4. Play the -1 to 1 converted data by feeding it to
the sound engine/sound card.
Now that we have 3 arrays with the converted values of
-1 to 1, use the 3 arrays to feed the sound engine/audio
card these numbers and by doing so they will be plotted
into a sound wave and can be played and heard. In this
case I'm using Sonia the sound engine for processing and
the LiveOutput.data array is the array that need to be
feed the -1 to 1 values inside our 3 arrays. When this
is done then you are done!
void liveOutputEvent(){
for (int i = 0; i < num; i++){
LiveOutput.data[i]= soundR[i];
LiveOutput.data[i+1]= soundG[i+1];
LiveOutput.data[i+2]= soundB[i+2];
} |
|
KEY
TERMS :
• A computer file is a stream (sequence) of bits (information) stored
as a single unit in a particular way (format).
• An image file is a file that contains the data (information) needed
to present a graphical image. Image files usually store bitmap information. in
this case data presenting the RGB values of each pixel of the image.
• Algorithm is a finite set of well-defined instructions for accomplishing
some task which, given an initial state, will terminate in a corresponding recognizable
end-state.
• An Array is a place holder of information, it holds equally-sized
data elements, generally of the same data type. Individual elements are accessed
by index using a consecutive range of integers.
• RGB (color) stands for Red, Green, Blue. The three colors of light
which can be combined to produce any other color. In this case the RGB is mixed
by the computer monitor. Each color ranging from values 0 to 255 (0=dark to 255=bright).
• A Pixel is one of the many tiny dots that make up the representation
of a picture in a computer's memory, television screen or computer monitor.
declaring (a variable)
variable
Processing
Bagel
Compile
Java
Applet
Quick
Visual |
