Chapter 5: The Transformation of Sound by Computer

Section 5.1: Introduction to the Transformation of Sound by Computer

Although direct synthesis of sound is an important area of computer music, it can be equally interesting (or even more so!) to take existing sounds (recorded or synthesized) and transform, mutate, deconstruct—in general, mess around with them. There are as many basic sound transformation techniques as there are synthesis techniques, and in this chapter we’ll describe a few important ones. For the sake of convenience, we will separate them into time-domain and frequency-domain techniques.

Tape Music/Cut and Paste

The most obvious way to transform a sound is to chop it up, edit it, turn it around, and collage it. These kinds of procedures, which have been used in electronic music since the early days of tape recorders, are done in the time domain. There are a number of sophisticated software tools for manipulating sounds in this way, called digital sound editors. These editors allow for the most minute, sample-by-sample changes of a sound. These techniques are used in almost all fields of audio, from avant-garde music to Hollywood soundtracks, from radio station advertising spots to rap.

Soundfile 5.1
Jon Appleton:
"Chef d'Oeuvre"

This classic of analog electronic music was composed using only tape recorders, filters, and other precomputer devices and was based on a recording of an old television commercial.

Soundfile 5.2
"BS Variation 061801"

"BS Variation 061801" by Huk Don Phun was created using Phun’s experimental "bleeding eyes" filtering technique. He used only free software and sounds he downloaded from the web.

Time-Domain Restructuring

Composers have experimented a lot with unusual time-domain restructuring of sound. By chopping up waveforms into very small segments and radically reordering them, some noisy and unusual effects can be created. As in collage visual art, the ironic and interesting juxtaposition of very familiar materials can be used to create new works that are perhaps greater than the sum of their constituent parts.

Soundfile 5.3
Jane Dowe’s "Beck Deconstruction" piece

Playing with the perception of the listener, "Puzzels & Pagans" takes the first 2 minutes and 26 seconds of "Jackass" (from Odelay) and cuts it up into 2,500 pieces. These pieces are then reshuffled, taking into account probability functions (that change over the length of the track) that determine if pieces remain in their original position or if they don’t sound at all.

This was realized using James McCartney’s SuperCollider computer music programming language.

A unique, experimental, and rather strange program for deconstructing and reconstructing sounds in the time domain is Argeïphontes Lyre, written by the enigmatic Akira Rabelais. It provides a number of techniques for radical decomposition/recomposition of sounds—techniques that often preclude the user from making specific decisions in favor of larger, more probabilistic decisions.

Figure 5.1  Sample GUI from Argeïphontes Lyre, for sound deconstruction. This is time-domain mutation.

Soundfile 5.4
Argeïphontes Lyre

Sound example using Argeïphontes Lyre (random cut-up).

The original piano piece "Aposiopesi" was written by Vincent Carté in 1999. This recording was made by Akira Rabelais at Wave Equation Studios in Hollywood, California. The piece was filtered with Argeïphontes Lyre 2.0b1.

Sampling

Sampling refers to taking small bits of sound, often recognizable ones, and recontextualizing them via digital techniques. By digitally sampling, we can easily manipulate the pitch and time characteristics of ordinary sounds and use them in any number of ways.

We’ve talked a lot about samples and sampling in the preceding chapters. In popular music (especially electronic dance and beat-oriented music), the term "sampling" has acquired a specialized meaning. In this context, a sample refers to a (usually) short excerpt from some previously recorded source, such as a drum loop from a song or some dialog from a film soundtrack, that is used as an element in a new work. A sampler is the hardware used to record, store, manipulate, and play back samples. Originally, most samplers were stand-alone pieces of gear. Today sampling tends to be integrated into a studio’s computer-based digital audio system.

Sampling was pioneered by rap artists in the mid-1980s, and by the early 1990s it had become a standard studio technique used in virtually all types of music. Issues of copyright violation have plagued many artists working with sample-based music, notably John Oswald of "Plunderphonics" fame and the band Negativland, although the motives of the "offended" parties (generally large record companies) have tended to be more financial than artistic. One result of this is that the phrase "Contains a sample from xxx, used by permission" has become ubiquitous on CD cases and in liner notes.

Although the idea of using excerpts from various sources in a new work is not new (many composers, from Béla Bartók, who used Balkan folk songs, to Charles Ives, who used American popular music folk songs, have done so), digital technology has radically changed the possibilities.

Soundfile 5.5
"Dustiny"

Figure 5.2  Herbert Brün said of his program SAWDUST: "The computer program which I called SAWDUST allows me to work with the smallest parts of waveforms, to link them and to mingle or merge them with one another. Once composed, the links and mixtures are treated, by repetition, as periods, or by various degrees of continuous change, as passing moments of orientation in a process of transformations." Also listen to Soundfile 5.5, Brün’s 1978 SAWDUST composition "Dustiny."

Soundfile 5.6
Drum machine sounds

Drum Machines

Drum machines and samplers are close cousins. Many drum machines are just specialized samplers—their samples just happen to be all percussion/drum-oriented. Other drum machines feature electronic or digitally synthesized drum sounds. As with sampling, drum machines started out as stand-alone pieces of hardware but now have largely been integrated into computer-based systems.

DAW Systems

Digital-audio workstations (DAWs) in the 1990s and 2000s have had the same effect on digital sound creation as desktop publishing software had on the publishing industry in the 1980s: they’ve brought digital sound creation out of the highly specialized and expensive environments in which it grew up and into people’s homes. A DAW usually consists of a computer with some sort of sound card or other hardware for analog and digital input/output; sound recording/editing/playback/multi-track software; and a mixer, amplifier, and other sound equipment traditionally found in a home studio. Even the most modest of DAW systems can provide from eight to sixteen tracks of CD-quality sound, making it possible for many artists to self-produce and release their work for much less than it would traditionally cost. This ability, in conjunction with similar marketing and publicizing possibilities opened up by the spread of the Internet, has contributed to the explosion of tiny record labels and independently released CDs we’ve seen recently.

In 1989, Digidesign came out with Sound Tools, the first professional tape–less recording system. With the popularization of personal computers, numerous software and hardware manufacturers have cornered the market for computer-based digital audio. Starting in the mid-1980s, personal computer–based production systems have allowed individuals and institutions to make the highest-quality recordings and DAW systems, and have also revolutionized the professional music, broadcast, multimedia, and film industries with audio systems that are more flexible, more accessible, and more creatively oriented than ever before.

Today DAWs come in all shapes and sizes and interface with most computer operating systems, from Mac to Windows to LINUX. In addition, many DAW systems involve a "breakout box"—a piece of hardware that usually provides four to eight channels of digital audio I/O (inputs and outputs); and as we write this a new technology called FireWire looks like a revolutionary way to hook up the breakout boxes to personal computers. Nowadays many DAW systems also have "control surfaces" —pieces of hardware that look like a standard mixer but are really controllers for parameters in the digital audio system.

Figure 5.3  The Mark of the Unicorn (MOTU) 828 breakout box, part of a DAW system with FireWire interface.

Thanks to MOTU.com for their permission to use this graphic.

Figure 5.4  Motor Mix™ stands alone as the only DAW mixer control surface to offer eight-bank, switchable motorized faders. It also features eight rotary pots, a rotary encoder, 68 switches, and a 40-by-2 LCD.

Thanks to Sentech Electronics, Inc. for this graphic.