Audio Denoising

We have developed a fully automatic studio-quality audio denoiser. It removes stationary background noise while introducing minimal audible artefacts. Its quality is comparable with the best studio denoisers on the market and even outperforms them in tricky situations, all while being fully automatic.

Here’s an audio example.

Original

Denoised

You will find more audio examples at the end of this page. But first, let’s explain why we developed this.

Our Motivation

We want recordings made with Tape It to sound as good as possible. From a musician’s point of view, dealing with something so seemingly mundane as noise really shouldn’t be necessary. We’d like to have an algorithm that eliminates this problem - one that you can leave on anytime you record, until you forget that the problem exists. You should focus on your music.

Note that we’re not trying to remove abrupt sounds or a barking dog in the background - that would ruin all your field recordings, and make it impossible for you to record dog sounds. We’re just trying to remove stationary noise - noise that sounds like bad recording equipment or where the ambient noise level is too high. This includes amp hums, fans and other similar noise, which are difficult to avoid and degrade the quality of your recordings.

Current Solutions

Currently, you can either choose between high-quality studio denoisers that are complex to use, or automatic denoisers that suppress the noise, but leave clearly audible artefacts that are unacceptable for any audio recording use case where quality matters. Broadly speaking, the landscape is divided in two camps:

1. DSP-based methods that achieve high quality but require manual control. iZotope RX’s Spectral Denoise is the most prominent example in the industry. These methods are well understood and in use in studios today. Apart from requiring manual control, these methods need a noise-only section (noise profile) in the audio signal so that the algorithm can learn what to remove. While some products also have automatic modes (sometimes called “adaptive mode”), their audio quality is substantially worse compared to manual mode and usually not worth using. But sometimes you don’t have a noise-only section, or the noise changes over time (as is often the case in acoustic/unplugged recordings), and then you need something automatic.

   

   Here is the first example again, processed with iZotope’s adaptive mode:

2. Deep-learning based methods are new on scene and are more powerful. They can often remove more than just stationary noise, but may leave artefacts. Descript Studio Sound and Adobe Enhance methods are two prominent examples. All examples in this category are designed for speech, and the task given to the AI is “remove everything except the voice”. However, this approach doesn’t generalise to music. As a result, these methods are currently a “no-go” for any music applications. Here’s what happens if you put a guitar recording into Adobe Shasta (now called Adobe Enhance):

   Project Shasta by Adobe is normally meant for speech enhancement, but what if you give it a guitar recording instead? It'll try to convert the guitar to speech. It sounds like something music producers could experiment with. Listen: pic.twitter.com/OvG9siZqda

   — Iver Jordal (@iver56) September 1, 2022

   Here's our example recording, processed by Adobe Enhance:

Our Approach

We have developed two methods - one hybrid method that combines a DSP algorithm with a neural network, and one pure deep learning algorithm that we designed to work specifically for music. To train both systems, we recorded a large amount of real-world noise.

A scientific listening test is still running, and as such we cannot yet provide final results. But we can already make the following remarks:

• Our deep-learning method clearly outperforms existing deep-learning based products and publications.
• Our DSP-based method performs on par or better than existing DSP based methods, but is fully automatic.

It was important for us to investigate both directions to keep an open mind. When comparing our deep-learning and DSP denoiser, we find that

• The deep-learning denoiser can handle more challenging environments, but has moments where it performs rather badly. Like all deep-learning algorithms, it requires a lot of resources to run.
• The hybrid DSP denoiser performs exceptionally and reliably well. It’s also significantly lighter on resources, easily running at 40x realtime speed on regular CPUs without any optimisations.

Other advantages

Apart from being automatic, having excellent sound quality and being very resource friendly, our denoiser has several other highly useful properties.

• Because it is built on well-understood DSP methods, all automations can be visualised and easily edited in a DAW-like setting, just like automations of any other plugins.
• It doesn’t require a segment of only noise to generate a noise profile.
• Unlike with modern deep-learning methods, we can mathematically guarantee with our DSP approach that there are no surprising side effects.
• It adapts effortlessly to changing noise environments, which is often the case in long recordings.
• While we designed our method for offline use cases and gave it some look-ahead time to preempt changes in noise, it can easily be adapted to low-latency environments without a speed penalty. The only downside is that it will take a very short time to adapt to drastic changes in background noise (like a fan being switched on).

It’s also worth noting that the way we trained the DSP denoiser is novel in itself, and non-trivial. We will write about this in more detail in our publication, but as we’re still in the process of filing a patent, we have to remain tight-lipped for now.

Roadmap

We’ve finished the work on the algorithm and are currently conducting a scientific listening test. We aim to publish the results at this year’s AES conference in New York in October. The paper will also be available on arXiv. We will not publish our dataset or our code, however.

We will integrate the algorithm directly into Tape It and create a web-based tool for people to try the denoiser. If you want early access to this tool, let us know.

Licensing

We’ve already received inbound interest to license our denoiser, and are generally open to discuss licensing. Please get in touch with Thomas via thomas@tape.it and we can schedule a call.

Audio Examples

And finally, as promised, here are a few more audio examples:

Electric piano

Original

Denoised

Guitar and vocals

Original

Denoised

Moog

Original

Denoised