Building a Real-Time Audio DSP on a Raspberry Pi

You’ve probably heard the buzz about “real‑time audio processing” and thought it was something only big studios could afford. The truth is, with a Raspberry Pi and a bit of elbow grease, you can build a capable digital signal processor (DSP) right on your kitchen table. In this post I’ll walk you through a practical, step‑by‑step prototype that anyone with a modest hardware budget can try.

Why Real‑Time Matters

When you’re dealing with live audio—think guitar effects, voice assistants, or a simple noise‑cancelling loop—the delay between input and output must be barely noticeable. Anything over about 10 ms feels sluggish to the ear. The Pi’s quad‑core CPU and built‑in audio interfaces give us just enough horsepower to stay under that threshold, as long as we keep the software lean and the data path tight.

What You’ll Need

Before we dive in, let’s list the parts. I like to keep the bill under $100 so the project stays approachable.

Raspberry Pi 4 (4 GB model works fine)
Official 40 mm fan (optional but helps with thermal throttling)
USB audio interface with ASIO/ALSA support (a simple Focusrite Scarlett 2i2 is a solid choice)
Micro‑SD card (32 GB, Class 10)
Breadboard and a few jumper wires (for any extra sensors you might add later)
Power supply (5 V 3 A)

Software side:

Raspberry Pi OS (Lite version keeps the system light)
ALSA utilities (alsa-utils)
CMake and GCC for building native code
The Eigen library for fast matrix math (optional but handy)
A small audio framework like PortAudio or RtAudio (I’ll use RtAudio for its simplicity)

Setting Up the Pi

1. Install the OS and Update

Flash the Raspberry Pi OS Lite image onto the SD card, boot it up, and run:

sudo apt update && sudo apt upgrade -y

This pulls the latest kernel and library patches, which can make a big difference for low‑latency audio.

2. Enable Real‑Time Scheduling

Linux can give a process higher priority with the rtprio flag. Edit /etc/security/limits.conf and add:

@audio   -  rtprio   99
@audio   -  memlock  unlimited

Then add your user (default pi) to the audio group:

sudo usermod -aG audio pi

Log out and back in to apply the changes.

3. Install Audio Packages

sudo apt install libasound2-dev libjack-jackd2-dev libportaudio2 libportaudio-dev

If you prefer RtAudio, grab it from the GitHub releases and compile it with CMake.

Writing the DSP Code

H2: Choosing a Simple Algorithm

For a first prototype I like to start with a basic low‑pass filter. It’s easy to understand, and you can hear the effect instantly. The filter equation in the time domain is:

y[n] = a * x[n] + (1 - a) * y[n-1]

x[n] is the current input sample, y[n] is the output, and a controls the cutoff frequency. The math is just a couple of multiplies and adds per sample—perfect for real‑time work.

H3: The Processing Loop

Using RtAudio, the core of the program looks like this:

#include <RtAudio.h>
#include <cmath>

static double prevOut = 0.0;
static const double a = 0.1;   // adjust for cutoff

int audioCallback(void *outputBuffer, void *inputBuffer,
                  unsigned int nFrames,
                  double /*streamTime*/, RtAudioStreamStatus status,
                  void * /*userData*/)
{
    if (status) std::cerr << "Stream underflow!" << std::endl;

    float *in  = static_cast<float*>(inputBuffer);
    float *out = static_cast<float*>(outputBuffer);

    for (unsigned int i = 0; i < nFrames; ++i) {
        double curIn = static_cast<double>(in[i]);
        double curOut = a * curIn + (1.0 - a) * prevOut;
        out[i] = static_cast<float>(curOut);
        prevOut = curOut;
    }
    return 0;
}

The callback runs in a high‑priority thread that RtAudio creates for us. Because we only do a couple of floating‑point ops per sample, the CPU load stays well under 5 % on a Pi 4.

H3: Compiling with Low Latency Flags

When you compile, add the -O3 optimization flag and tell the compiler to use the ARM NEON SIMD extensions:

g++ -O3 -march=armv8-a+simd -std=c++11 -o dsp dsp.cpp -lrtaudio -lasound

The -O3 flag squeezes out every extra cycle, and NEON can process multiple samples in parallel if you later expand to more complex filters.

Testing the Prototype

H2: Measuring Latency

A quick way to check latency is to feed a short click through the system and record the output with a second audio interface. On my setup I measured about 7 ms round‑trip, which is comfortably below the audible threshold.

If you see higher numbers, try these tweaks:

Lower the buffer size in RtAudio’s RtAudio::openStream call (e.g., 128 frames instead of 256).
Make sure the Pi’s CPU governor is set to “performance”:

sudo cpufreq-set -g performance

Keep the fan on; thermal throttling can add milliseconds of delay.

H2: Adding a Second Stage

Once the low‑pass filter works, you can stack another stage to make a band‑pass or high‑pass filter. The code change is minimal—just keep an extra prevOut variable for each stage and adjust the coefficient a. Because each stage adds only a few more operations, you can comfortably run 4‑5 stages in real time.

Going Beyond: Real‑World Use Cases

Now that the basic pipeline is solid, think about what you actually want to do with it.

Guitar pedal emulator – swap the low‑pass for a distortion curve and add a simple envelope follower.
Voice activation – combine a high‑pass filter with an RMS level detector to trigger a smart‑home action.
Noise reduction – implement a spectral subtraction algorithm; the Pi’s FFT libraries (like KissFFT) run fast enough for 44.1 kHz audio.

All of these can be built on top of the same skeleton code, just by swapping out the processing function.

Tips for a Stable Prototype

Keep the audio thread isolated – avoid printing to the console inside the callback; it can cause jitter.
Use lock‑free buffers – if you need to pass data to another thread (e.g., UI), ring buffers work well.
Watch the temperature – the Pi can hit 80 °C under load. A small fan and a heatsink keep it in the sweet spot.
Document your coefficient calculations – it’s easy to lose track of what a means when you start mixing filters.

Wrap‑Up

Building a real‑time audio DSP on a Raspberry Pi is less about buying expensive gear and more about respecting the timing constraints of the audio path. By giving the processing thread high priority, using a tiny low‑pass filter as a starter, and compiling with aggressive optimizations, you get a responsive system that feels professional. From here, the sky’s the limit—add more stages, experiment with different algorithms, and maybe even package it as a DIY pedal board.

Happy hacking, and may your samples always stay in sync.