Digital Sound Separator

Peter Neubäcker, a former German guitar maker turned programmer, has done what many in the computer-music business believed impossible.

A new piece of software called Direct Note Access, first publicly demonstrated by Neubäcker and his company Celemony Software last month, will for the first time allow computers to analyze the digitized sounds of guitar or piano chords, or even multi-instrument recordings, and then extract and modify individual notes.

Computers have revolutionized the recording process, giving sound engineers wide latitude to manipulate notes recorded singly–to change their pitch, their tempo, or where they fall. But teasing apart notes recorded simultaneously, as in a six-string guitar chord, has never before been practical.

“In terms of sound processing, this is kind of the holy grail,” says Michael Bierylo, a guitarist and professor of computer music at Boston’s Berklee College of Music. “It’s something everyone more or less thought we couldn’t do.”

The new tool promises to give musicians and producers powerful new ways to manipulate recordings both old and new.

It might let studio engineers peer inside a chord-heavy rhythm-guitar part and nudge individual notes into tune. Or it could let them salvage unheard takes by classic musicians like Duke Ellington or Jimi Hendrix, left unreleased due to out-of-tune instruments or misplayed notes. It will certainly give musicians new ability to sculpt sound, such as prerecorded samples or loops, as if they were modeling clay.

Understanding exactly why this is such a breakthrough requires a quick tour of computer music’s limitations, however.

Digital audio recording faithfully reproduces the sound of a voice or instrument, just as a tape recorder would. Since the sound is stored digitally, computer tools have been able to stretch, bend, or change it–but only up to a point. Single notes, as in a vocal line, can be easily manipulated. Most big studios today use vocal pitch correctors to repair notes sung out of tune.

But record a guitar or piano playing chords, and things get more complicated. Existing computer recording devices treat these groups of notes as single sonic entities: changing the pitch or timing of one note requires changing the pitch or timing of all the others.

It’s that limitation that Neubäcker’s Direct Note Access appears to have overcome.

Originally a guitar maker, and with no formal training in physics, math, or programming, Neubäcker got involved in computer music after leading a local lecture group that studied the philosophy and physics of music. He cofounded Celemony in 2000, and a year later, the company released its Melodyne software package, a now widely used tool for analyzing and manipulating single-note audio recordings.

Neubäcker is modest about his own achievements, including the new Direct Note Access tool.

“It’s not a very sophisticated approach, with special mathematics, or a special method,” he says. “It’s more like diligently looking to see what is in there.”

“Looking to see” turns out to be what’s known as spectrum analysis–graphically or numerically representing a given sound’s complex blend of frequencies.

If you pluck the lowest string on a guitar, the sound you hear is dominated by a single fundamental frequency–in this case, an E. But you also hear a set of higher pitches, called harmonics or overtones. (It’s the idiosyncratic mix of overtones that makes an E played on the guitar sound different from one played on a trumpet or oboe.)

Neubäcker says that he spent time looking at spectral analyses, seeking “structures in the signal” that could identify a chord’s fundamental notes and link each of them with its own overtones. Because sound waves blend and interfere with each other, this is a hideously complicated task. But as Celemony’s demonstrations show, it’s one that Neubäcker has apparently accomplished.

Direct Note Access represents the notes of a chord graphically: they look like blobs of ink on a timeline. The blobs can be moved up or down to change their pitch, or back and forth to change their timing. An existing chord can be changed from major to minor; notes played at the same time can be strung out to occur in sequence, in what’s called an arpeggio.

Neubäcker is quick to admit that the software isn’t perfect. It works well with clean, unprocessed instruments, such as acoustic guitar or piano. It will work on a heavily distorted electric guitar but may, for example, read some high-pitched harmonics as separate notes.

Nor is the software quite the equivalent of a human ear, able to distinguish actual instruments from one another. Like an audio idiot savant, it can tease individual notes out of a complicated six-note chord. But feed it a recording in which two instruments are playing the same note–say, the trumpet and piano on an old Ellington record–and it will treat them as a single entity, rather than distinguishing their separate voices.

Like Celemony’s Melodyne, Direct Note Access is likely to find its way quickly into studios’ tool kits.

“I’ve seen the video, and I was blown away,” says J. Chris Griffin, a producer at the Cutting Room studio in New York, who has worked with such artists as Madonna and Kelly Clarkson. “I can’t wait to try it. I’m definitely planning on getting it.”

The tool may have cultural costs as well as benefits, however.

Already, vocal tracks on virtually all radio-ready albums are cleaned up; they’re perfectly in tune and in time but may lack the unique character of a vocal performance by, say, Billie Holiday or Bob Dylan. Direct Note Access’s power could accelerate that homogenization of pop music, says Berklee’s Bierylo.

Direct Note Access will be released commercially this fall as a plug-in compatible with most major audio-recording software packages. A stand-alone version will be released later.

But Neubäcker is already thinking about ways to expand the tool’s capabilities, perhaps even allowing it to recognize and distinguish the sounds of individual instruments.

“My approach is not to focus on making a product but to understand what sound is all about,” he says.