Creating the “Beatboard”

The Beatboard™ (Prototype v0.1)

Initial Inspiration

For my final project, I was inspired to construct an E-Z music making machine. For drummers like me, music with notes can be hard. I can wrap my head around simple roman numeral chord progressions, but when I want to make music live, I find my process of mapping harmonies to actual notes or chords on the piano to be prohibitively slow. By the time I can actually figure out where to put my fingers, I’ve already lost my creative train of thought. So, instead of going through any semblance of real musical training, why not engineer a solution to make being a musician a little easier? With the seedlings of an idea, I started to sketch out some concept designs. Here are a couple (out of many):

Initial design ideas

The main interface I converged on centered around an array of buttons that would allow me to select different diatonic chords of a certain key based on their root. I’d organize the buttons by ascending scale degree—instead of note name—so I’d be able to think in roman numeral harmonies as I played. Together with some knobs and sliders to tune different parameters, as well as two screens to display different instrument choices, the current key, and the current tempo, most of the interfaces I’d design looked were organized something like the following:

Main interface

I figured that this sort of instrument would best lend itself towards being more of a groove-maker or a drums&bass backtrack, so I started designing with that goal in mind. I also had several ideas for adding controls for some sort of lead voice (flex resistors, gyro sensors, a 12-channel capacitive sensor wired to an assortment of musical fruits), but I figured I’d save them for a later revision. I also realized I’d need a project name. “Beatboard” sounded nice to me, and since then I’ve stuck with it.

Choosing the Technology

Now that I had a somewhat fleshed-out idea of what I wanted to make, I started thinking about how I’d want to implement it. One important detail is that I wanted the box to be completely independent: plug in a speaker or a pair of headphones and play, no computers attached. This pretty much ruled out Arduino as the main brains of the machine, as implementing sound synthesis in bare-bones C on an 8-bit micro-controller with only a few KB of program memory sounded like a nightmare. Instead, I turned to Raspberry Pi. Pis are miniature single-board linux computers that can fit in the palm of your hand. They’re cheap ($10 for the model I’m using), accessible, and fit my project specifications very well: with an assortment of GPIO (General Purpose In-Out) pins, I’d be able control the hardware I needed, and as a computer running a full operating system, I’d be able to run our beloved SuperCollider as a synthesis engine. I’d also been wanting to become more comfortable with Raspberry Pis for a while, and this seemed like the perfect opportunity to do so.

With the project platform decided, I started organizing a general block diagram of the different components I’d need. Here’s an early draft I drew:

Block diagram

These groups components deserve some individualized attention, so I’ll go into a bit more detail about each one.

Hardware Components

The Raspberry Pi

Raspberry Pi Zero, with Quarter for size

I’d decided that I wanted to use Raspberry Pi for this project, but that still left me with the decision of which Raspberry Pi to choose. Just like Arduinos, Pis come in a variety of sizes and capabilities. Since I wanted this project to eventually fit in a small handheld box, I went with the Raspberry Pi zero. It’s quite tiny, and although it lacks a bit in processing power compared to its older brothers (more on that later), its still equipped with the same 40-pin GPIO as the larger models. You can get a Pi Zero for as little as $5, but I shelled out an extra $5 to get the version with WiFi built in—which came in handy as I was rapidly beaming it code updates from my laptop.

While Raspberry Pis are surrounded by a wonderful community of makers supplying countless documentation pages, tutorials, and help forums, I still had some trouble with some of the initial setup. After flashing a desktop-free, lite OS to the SD card (as I’d just be running the thing headless), I spent an entire day figuring out how to configure the Pi to automatically connect to my WiFi without hooking it up to a screen and keyboard. Several hours of pain and a few hundred Chrome tabs later, I found myself on the other end with a sufficiently self-initializing computer and a far more intimate understanding of linux network device setup than I had ever really wanted. After some extra hassle enabling kernel modules for the various hardware serial buses I was going to use, I had the brains of the project prepared for the time being.

The Pi connected to all its friends. How cute!

Audio Out – The Digital to Analog Converter

Choosing the Pi Zero came with one particular caveat: it has no audio jack! This left me with several options: I could 1) Extract the audio from its miniHDMI port, 2) Use the Pi’s PWM pins and a basic circuit to hack together an audio signal, or 3) hook up a DAC over some serial bus (DACs, or Digital-to-Analog Converters, convert a digital signal where audio is encoded as a stream of bits to an analog signal of varying voltage that can drive a speaker). Option 3 seemed like the the most reliable and easiest to implement, so that’s what I went with. While I could probably have hooked up an off-the-shelf USB audio adapter to the Pi’s single microUSB port, that option would have involved several adapters and felt a bit clunky. Instead, I found this awesome board built specifically for Pi Zeros. It includes a quality DAC and amplifier packaged into a compact PCB that plugs straight into the Pis GPIO header—no wiring necessary. It communicates to the Pi over I²S, a 3-wire serial protocol based on pulse code modulation. This was perfect for my purposes, as I’d soon learn that GPIO pins were a limited resource. I’d need to keep access to the Pi’s pins for the rest of the hardware, so I wouldn’t be taking advantage of its convenient plug-and-play functionality, but by looking at the board’s circuit diagram I was able to figure out which pins the board needed to function, and wired them up manually. Here it is, integrated into the current prototype of the machine:

I2S Audio Board

Analog In – Reading Potentiometers for Knobs and Sliders

Another important feature the Raspberry Pi lacks is an integrated Analog-to-Digital Converter (or ADC). ADCs encode voltages as digital values, and are essential for reading the types of knobs and sliders I would use for the project. These knobs and sliders are all potentiometers, variable resistors which can be hooked up as a simple voltage divider, which yields varying voltage as the potentiometer’s resistance changes. Rotational potentiometers can be used as knobs, and linear potentiometers can be used as sliders. In order to read these voltages, I used a small IC called the MCP3008, which can read 8 separate channels at 10-bit accuracy (encoding a voltage from 0-3.3V as number from 0-1023). This little chip communicates to the Pi over a serial protocol called SPI, a somewhat annoying and antiquated spec that eats up another 4 of my precious GPIO pins. The Pi includes integrated hardware for SPI, however, so communicating with the IC wouldn’t be too much of a pain (no bit banging needed!). In the end, I hooked up my 8 analog channels to five knobs, one slider, and the two axes of a joystick that I’d use to navigate beatboard’s menu. Here’s the chip, wired up to its children:

The MCP3008 ADC

The Screens

Here’s a surprisingly easy one! The two screens I chose to use communicate over the lovely I²C protocol (not to be confused with I²S, the sound protocol), which uses just two wires for data. The best part of this protocol is that practically limitless devices can be connected in parallel to the same two data lines—each device is given a unique 7 bit address that the host device can target messages to. This makes wiring a breeze, and saves on the GPIO pin usage! As an added bonus, both screens are OLEDs, giving them low power consumption and high contrast, making them easily readable even at low resolutions. Here are the two screens with some text on them. The left screen has the main instrument selection menu, while the right screen has some general system info. I didn’t have time to implement the right screen in the program, but I intend it to display the current key and tempo.

Monochrome OLED screens, with 128x64px (left) and 128x32px (right) resolutions

The Button Matrix

I’d designed a board that required an array of 21 (3×7) buttons, but I certainly didn’t have 21 GPIO pins left to read them. A common way to circumvent this issue is to wire the buttons in a matrix, connecting all of one side of the switches together in columns, and all of the other sides together in rows. That way, the Raspberry Pi can just read all the buttons with just 3 rows + 7 columns = 10 pins! This setup requires some intricate shenanigans to get a proper readout, however. On a given read cycle, the column pins are all set to inputs, while each row is sequentially set — bah! Too complicated to implement, here’s a python library that does it for us.

An early test prototype of the button matrix. Some button matrix designs require diodes for a proper readout, but the Pi can trigger its IO pins fast enough that they aren’t required.
A closer look at the button matrix in the current prototype

Putting it All Together

For my first prototype, I wired everything on a large breadboard just to make sure all the components would work together nicely. I’m currently powering it all from the 5V rail of my benchtop DC power supply, which is a convenient—albeit temporary—solution. While I definitely plan to solder everything together and put it in a fancy box with its own battery later down the road, this revision of the hardware was all I had time for in the scope of a couple week’s work. With a few of GPIO pins left over, I hooked up the switch that closes when you click the joystick down as a “menu select” button, and called it a day. I even still have enough pins for an extra row in the button matrix, or an entire second MCP3008 ADC! (Multiple SPI devices can share 3 of the data pins, but each requires its own “chip select” pin to specify who the host is talking to).

The result is the first prototype of Beatboard’s hardware, pictured at the top of this blog post.  Here’s my current working wiring diagram:

My working wiring digram, drawn over a printout of the Pi’s GPIO header

The Software

With the hardware complete, I was left with an entire second world of trouble: actually programming the system to do something. I’d never done any hardware interfacing with Raspberry Pi, so I started with the basics:

Pi Zero LED Blink Test (Click for video!)

Workflow

I was running the pi entirely headless, so after configuring it to automatically connecting to my home network, I’d ssh into it from my laptop for control. I found this to be much smoother than dealing with usb cables and the serial debug window required to work with Arduino. As it comes to the code it self, this project was written in Python (for hardware control) and SuperCollider 3 (for audio synthesis), along with several bash scripts for startup/automation. I kept everything in a GitHub repo, and would pull code to the Pi to execute after pushing changes from my laptop. This was a pretty convenient and easy to implement system, the only downside of which were abundant single-line commits every time I needed to fix a syntax error. (Most of the libraries I was using would only run on the Pi, so I couldn’t do any local tests beforehand).

A screenshot of my typical code upload workflow: SSH terminal on the left, GitKraken on the right

Environment Setup on the Pi

Setting up the dev environment on the Pi was mostly straightforward. Aside from some smaller details, the main challenge was setting up the system audio to work with my I²S DAC. Fortunately, Adafruit, the company I bought the board from included a helpful install script that only required some minor tweaking to get everything working smoothly. Oh yeah, aside from installing SuperCollider. That deserves its own section.

Installing SuperCollider

I’d gotten this far into the project without bothering to check if Supercollider would actually run on my Pi Zero. Genius. I was overjoyed to realize that SuperCollider installation on linux is notoriously difficult (according to this blog post by someone very special). In most cases, you have to build from source, which according to documentation on the SC3 GitHub takes multiple hours on the Pi Zero. Lovely. Luckily, the Raspberry Pi community comes to the rescue! User redFrick has published compiled standalone downloads for SC3 along with step-by-step installation instructions for every single Pi model. redFrick: I don’t know who you are, but I love you. The installation process came with only a few typical snags—my jack2 install conflicting with a apt package that came preinstalled on the system, and some audio bus conflicts with some of superfluous programs installed by Adafruit’s I²S DAC helper script to name a few. In the end, I had sclang and scsynth running on my Pi zero, with audio properly routing through I²S! Never in my life have I been more happy to hear a 440Hz sin wave.

Hardware Control with Python

All the button IO, SPI/I²C serial communication, screen drawing, and menu logic were implemented in Python. Most of the code was centered around Adafruit’s CircuitPython environment, which provides a set of hardware interfacing libraries that are a lot more convenient than the default RPi.GPIO library. All the code can be viewed in the project GitHub repository. (/python/beatboard.py would be a good place to start, if you’d like to take a look).

Python development environment. I used VSCode for everything except SuperCollider.

Audio Synthesis with SuperCollider

This project led me towards a lot of the parts of SC3 intended for live coding, which were super cool to dive into. I have an incredible appreciation for this language—I thought it was rather tedious at first, but I’m starting to really understand how powerful it is as a music creation tool. After a lot of internet searching, I became very close friends with Pdefs, Pdefn, and PatternProxy—all classes I hadn’t touched for any of the psets. On the macro scale, my SuperCollider script defines a lot of SynthDefs and a handful of different patterns, then plays them according to several control parameters that the script exposes to the network (more on that last part in just a moment). The SuperCollider portion of this project is actually still a bit rough around the edges, and I’m looking forward to optimizing and expanding a lot of the code. Several ways I’m looking forward to improve are: 1) saving large libraries SynthDefs to libraries of files on the Pi, 2) Unifying pattern implementation (PatternProxy vs Pdefn for live control? Pdef vs global variables? I’m still figuring out both what’s available and what I like the best). 3) Organizing a more thoughtful control API for other programs to interact.

A screenshot of some SuperCollider code

Bridging the gap between Python and SC3: Open Sound Control

One important task remained: I had to figure out how to get my Python and SuperCollider scripts to actually talk to each other. Luckily (and honestly, to my absolute surprise) there’s a standard communication protocol that SuperCollider uses called Open Sound Control (OSC). Open sound control messages can be sent over a simple UDP connection, and consist of a “target path”—specifying what should be a manipulated—followed by a list of arguments—specifying how the target should be manipulated. SuperCollider actually uses OSC to talk between sclang and scsynth. For my purposes however, I just needed to communicate to sclang, which can be achieved with an OSCdef, which allows you to bind a handler function to a specific OSC target. In the screenshot above, you can see my OSCdef for responding to messages that want to alter the pattern of Beatboard’s bass instrument. One annoying limitation I encountered is that SuperCollider doesn’t support sending arrays via OSC. As a hotfix, I decided just to send the root of each chord instead of a list of notes, but I’d like to come up with a better solution in the future. Meanwhile, on the Python side of things, sending OSC messages was super easy with the pyliblo library.

As an extra note, I looked through several libraries that try to essentially try to replace sclang entirely, though I found that for my purposes they were either too tedious, outdated, or simply unnecessary for this project. This project deserves special mention, though. It seems really cool.

Trouble In Paradise: scsynth Processing Issues!

As I was nearing my goals for this revision of the project, I hit a serious design flaw that I hadn’t anticipated. Long story short, the Pi Zero isn’t powerful enough to run a heavy load of synths and patterns on the SuperCollider server. It can run a simple drum pattern fine, but as soon as I’d throw in a bassline with several PatternProxies, the server would fall behind and the entire system would crash—even causing lag in my ssh terminal. Here’s a video of me demoing my SC3 script by manually sending it OSC messages from the Python console. The drums all work fine, but as soon as I layer on the bass, the whole thing lags and crashes. In the end, I came up with a simple, yet somewhat disappointing solution to this problem. In the current prototype, I run sclang and scsynth on my laptop, and let the Pi zero communicate to them over OSC. In the code, this was a super easy fix, as I just had to target my OSC messages to my laptop’s IP instead of localhost. However, this definitely broke on my goal of making a standalone device. In the future, I’m interested to see if the more powerful Pi’s could do a better job running things, and if optimizing my SC3 code could get things to run on the Pi Zero (PatternProxy seemed like the main culprit, while Pdefn was fine).

(alternative Google Drive Link, if embed doesn’t work)

Results!

Without further ado, here’s what Beatboard v0.1 is capable of!

(Alt Google Drive Link)

Reflection

With easily over 60 hours sunk into this project, I can definitely say that it taught me a ton. I’m a lot more comfortable with Raspberry Pi’s, I brushed up on my python, and I got to dive even deeper in SuperCollider. Overall, making the first prototype of Beatboard has been loads of fun, and I can’t wait to keep going with it. In particular, I’m looking forward to adding more instruments and patterns, adding more complex parameters for the knobs (did I hear user mappable?), and implementing seventh chords and secondary dominants to the extra button rows. For my next prototype, I’d also like to put everything in a nice box, and add some sort of lead instrument control. (Those musical fruits do sound like a lot of fun…)

First MIDI project!

Initial Ideas

My initial idea for this song—like all great ideas—started in the shower. I was jamming out in my head to a minor descending bass line in 7/8, and immediately I knew where my project would start. Soon after, I sat down with a synth app on my iPad (Synth One and Flynth are great, for those interested), and started to solidify my ideas. Music theory is pretty new to me, so it took me a little while to harmonize everything. Knowing I wanted the bass line to follow the classic 1 – 7(♭) – 6♮- 6(♭) – 5, I ended up with the minor progression: i – v6 – V6 / VII – VI – V. While I had initially imagined everything with jazz instrumentation, I eventually settled on a more electronic/”synthy” vibe. This was perhaps simply a product of the tools I was using to practice, but no matter. I created a new waveform project and set to work.

Flynth Additive Synthesizer App for iPad
AudioKit Synth One app for iPad

Meet MIDI!

The addition of MIDI to my composition toolkit immediately proved indispensable. I know it’s quite rudimentary, but it was really nice to be able bring my own musical ideas to fruition, rather than just mixing loops. I don’t have any sort of MIDI controller, so I tried to hook up my iPad as keyboard I could play. While I found a solution that works with Logic Pro, I couldn’t get anything to work with Waveform. Ultimately, I resorted to penciling in notes with Waveform’s built in MIDI editor. I found it useful to open clips in a separate window instead of using in-line track editor, which can be achieved by pressing the “pop out window” button on the top left of any MIDI clip. Here’s an example of what I was working with:

Waveform’s MIDI editor, with notes created via the pencil tool

Challenges with Synthesis

Ironically, however, the freedom of MIDI resulted in my biggest challenge with the project. I found it really difficult to accurately produce a sound I imagined with additive synthesis. Every time I wanted to introduce a new voice or instrument, I’d have a good idea of the sound I wanted, but had a lot of trouble translating that into the correct oscillator, filter, and LFO settings. I tried to combat this challenge with practice. I spent a lot of time with 4OSC and Subtractive, as well as the iPad apps mentioned above, simply jumping around presets, looking through how the sound was made, and tweaking things to see how each setting affected the instrument as a whole. (A few times, I found a sound I really liked on the iPad, but couldn’t easily translate it to 40SC or Subtractive because the interfaces are so different. Eventually I gave up on the non-Waveform synths from this frustration, even though they were quite convenient.) While I’d still say I’m far from comfortable, I’ve definitely started to get better—especially in 4OSC’s simple interface. At a few points, I was strongly tempted to look into virtual instruments, but I thought I’d leave that for my next project.

An example of a 4OSC sound I liked during this process

Digging into Composition

Pre-chorus, Chorus, and Rhythm/Meter Decisions

After what felt like a steep upfront time cost of getting my bearings in the world of MIDI, I was finally ready to start realizing the song I wanted to create. I started with a pre-chorus/chorus combo which followed the chord progression I had decided on earlier. I decided I wanted the odd meter of the chorus to stick out a bit more, so I wrote the rest of the song in 4/4, while in the chorus inserting an extra bar of 8/8 after every 3 of 7/8 to blend the worlds a little bit. In doing this, I signed myself up for the difficult task of convincingly transitioning between these two meters. After a few different approaches, I ended up using an isolated and syncopated pattern in the bass to erase some of the 4/4 context before entering the chorus.

On the left, the pre-chorus (4/4), and on the right the chorus (3x 7/8 + 1x 8/8), joined by a bar of syncopated 7/8 in the middle.
A snapshot of my tempo track, which required a lot of meter markers.
Meter changes can be added by clicking on your track BPM, then selecting “Insert Tempo Change at Cursor” in the properties panel.

Verses

Eventually, I realized that I had put a lot of attention into the middle of my song, without really thinking about how I wanted it to start. Since I was decidedly working in the world of 4/4 for the intro and verses, I thought it would be the right time to incorporate some loops. After quite a while sifting through Waveform’s search tool (something I would highly recommend, as it automatically adjusts to your tempo and key during preview) I came upon some pad, bass, and drum loops from Garage Band’s libraries that had a cool minor vibe. These went together quite easily, and with the addition of a buzzy synth melody I wrote in MIDI, I had the verse section.

The verse section in my track, made mostly of loops

Intro

Even as I tried to gradually layer tracks at the start of the verse, the start to the song was a bit abrupt. After trying a few different things, I decided to ease into the track with a smooth piano progression. One of the loops I had been using had a bass line that followed 1 – 6(♭) – 3(♭) – 1, which inspired the progression minor: i6 – iv – vii°6 – i. With a few suspensions and funky notes for some added spice, I ended up with something I liked. In order to transition in the verse, I used a sinusoidal AM LFO in 4OSC to add vibrato to the final i chord at the 1/16 note level (using the “sync” option in 4OSC). I used automation to gradually increase the depth of this effect at the end of the piano intro. This transition allowed me to play the chords at a bit slower of a tempo (80BPM), and use the 1/16 notes created by the LFO to establish the main 100BPM pulse. I though it all ended up sounding pretty cool, so I decided to use the vibrato sound as a pad throughout the entire piece to provide some unity.

Intro chords with AM LFO automation for vibrato

With another verse, an amped up prechorus using the lead instrument instead of lowpass keys, and a solo/chorus section, the content of the song was complete!

Edits and mixing

However, I had quite a ways to go before finishing. There were still several glaring issues with the track. First, I was having some trouble with the synthesis of my lead instrument: whenever multiple notes were playing, it seemed like they would cancel out and switch between each other. I spent a good while fussing with distortion, unison, detuning, and phase, as I suspected these were all culprits of some sort of phase cancellation between notes. I felt pretty stupid when I realized I just had the “mono” toggle on. To some relief, I did end up having to change the aforementioned settings as well, as the distortion and detuning didn’t sound great in a chord. The song then lacked some grunge, so ended up splitting my lead midi clips into two tracks: one with extra distortion and detuning for one note at a time, and another with less for multiple notes at a time.

Chorus clips split into two tracks

Second, even during later edits, the chorus didn’t sound nearly as full as the verses, and their overall vibes didn’t really match. This was one of my most difficult obstacles, stemming from how the piece was composed. Looking back, I think it would have served me much better to have planned out the entire song before jumping into the DAW. Writing the choruses before really knowing where I wanted to take the rest of the song resulted in somewhat disparate sections instead of a unified progression. Despite the deep roots of this issue, there were a few things I tried to help the chorus fit the vibe of the rest of the song. I added an extra ascending pad, panned different sounds left and right to spread everything out, messed with the different levels of all the instruments, and boosted the lows of the bass. These all helped the chorus sound a bit more full, and more in place.  I ended up putting a lot of automation on the oscilating pad, boosting it when isolated, and lowering it when layered with other sounds to keep the overall amplitude in check. One particularly fun edit in this phase was adding automation to the panning of an arpeggiated loop in the verse to make it feel like it was cycling around. It ended up looking something like this:

Takeaways

Overall, I really enjoyed working on this project. Music production is something that still feels incredibly out of my comfort zone, but I think for that reason I’m starting to have a lot of fun with it. I certainly still feel like there are several things wrong with my piece that I just don’t know how to fix, but just as often am I frustrated, I become excited with what have been noticeable improvements in my skills. Suffice to say, I’ve been having a great time learning more about music production. I’m particularly looking forward to branching out to less “electronic” sounds in my next project.

Here’s the final track!

Trying out Tracktion: Waveform Project 1

Getting Started

With essentially zero songwriting and digital music production experience, I was a bit daunted by this project at first. I didn’t really know where to start. So, to get a feel for things, I started by looking through some of the demo projects that came with Waveform 11. Clicking around an established mix proved a very useful way to orient myself to the program. I acquainted myself with the interface, learned about some of the more common plugins, and got to see automation in action. With a much more tangible goal in sight, I set out to start on my own production.

Looking for Loops

Without much of predetermined artistic vision (nor the practiced means to realize one I might have had), I started by searching through SSLIB for tracks to base my song off of. This proved to be incredibly difficult. I spent hours systematically searching through directories and jotting down notes as I listened through every different kind of sample I could imagine. Whenever I came across a sound I found cool, I’d copy it to my machine. This ended me up with a laughably eclectic collection ranging from spooky halloween EDM to to non-rhythmic robotic beeps and boops to chill lo-fi piano. Regardless, I’d managed to pare down to a much more manageable selection.

Putting it together

I found basing a song entirely off samples to be a significant artistic challenge. While I could imagine a lot of directions I would have liked to take each specific sample in a vacuum, I often felt as though the difficulty of finding a group of tracks that coordinated with one another prohibited me from taking the song in the direction I wanted to. I could hear the drum tracks, baselines, and melody accents I wanted to appear, but neither could I ever find any sample remotely close to what I wanted to search for, nor did I feel like I had the ability to synthesize my ideas using the software. (I tried using MIDI. It didn’t go incredibly well.) This was really frustrating. After a few hours of dragging random samples into an empty project to little avail, I put my computer down and decided to come back the next day.

With a fresh look at the project, I quickly came across my first breakthrough. With an energetic low-synth/bass lead, vibey drums, and some cool reverb-y accents, I had my first groove going that seemed to click. I even managed to find the track setting to pitch up every other two octaves, resulting in a simple (yet much more interesting) I-IV-I chord progression. Success! Well, not really yet. But it was a good start at least.

Recording

With some material to start with, I decided it was time to hit the recording studio. Equipped with some of my generous friend’s gear (Most importantly, a Blue Yeti Blackout microphone), we put together quite the satisfying setup:

The setup ™

I found the process of recording to be quite fun: with the Yeti on the lowest possible gain and set to cardioid (the Yeti has three condenser mics for multi-direcitonal selection), I’d record into ocenaudio, splice the sections I’d want to keep, and then normalize the amplitudes. Often I’d drape my duvet over my head and the mic to capture quieter sounds without any noise, but in almost all situations the Yeti produced pristine recordings with little interference.

With several friends eager to be featured in the song, I got together a few people to record some sounds for the track. Now, let’s play a game. What happens when you put four teenage boys in front of a microphone and tell them to make whatever sounds they want? Here’s the answer:

Plenty of material to work with (kill me now)

While this wasn’t exactly what I had been looking for, I held true to my word and made sure that the boys would be included in the final cut. Oh look, there goes my dignity.

Filters, mixing, and edits

After a bit more track aggregation, I was ready to start tuning the sounds I had mashed together into something a bit more artful. I lowpassed a bassline so that it would sit under the rest of the tracks. I put a narrow bandpass over a bothersome snare to make it a bit tighter. I added reverb to the leads in the chorus to make them more full. At one point, I came across a chorus plugin, and doubled one of the voice tracks I had recorded at a higher pitch. Certainly my favorite additions, however, were those that involved automation. In an early buildup, I decided to sweep open a lowpass filter to slowly introduce more and more of a drum track. At the same time, I decreased the bandwidth of a bandpass filter on the leads to let the drums pop, until reopening it, adding reverb, and removing compression after a sort of drop. These sorts of experiments proved to be a lot of fun, and certainly added a lot to the overall quality of the song.

Throughout this process, I ran into a lot that didn’t work, of course. At one point, I tried to split the bassline and melody of a single track with filters, then splice it in different ways to get some chord variation. While I managed to split the instruments, I couldn’t figure out how to get the hard breakpoints I had created to flow with the legato nature of the sample. Moments like these, though, were those that taught me most about the software. It’s certainly cliché to say, but I learned a lot from my failures.

Conclusions

In the end, I found this project to be a quite challenging, but very fun introduction to Waveform. To be frank, the results I ended up with are far from something I’d enjoy listening to. To offer some self criticism, there are certainly two ideas going on that (despite my best efforts) don’t mesh incredibly well. That being said, I feel like the spark is now there: I’m inspired to create new music, and I’m having a good time learning to do it. With a little more instruction (especially on MIDI), and a lot more experimentation, I’m excited to see where I’ll be able to take my artistic visions. For now, I can be proud of where I’ve started, and look forward to doing better on the next project.

Experiments with the iPhone 8’s Microphones

Initial Research

For this project, I was equipped with my trusty iPhone 8. Although Apple doesn’t publish any specs on its microphones, plenty can be determined with a little bit of digging. Several articles point to the iPhones XS and XR as the first Apple phones with stereo microphones, so my iPhone 8 will be restricted to mono (PhoneArena). Another article measures the frequency response and dynamic range compression of the iPhones 6s, 7, and X. While my phone wasn’t included in the study, the measurements give a pretty good picture of what we should be dealing with, as the iPhone 8 was released between iPhones 7 and X.

Frequency response

Frequency response of iPhones 6s (red), 7 (purple), and X (blue). (Credit: SignalEssence)

When it comes to frequency response, all three phones sport mostly flat profiles, save a roll off around 50-60Hz at the low end, and some funky behavior around 10kHz. All should record perfectly fine in the 100Hz to several-kHz range, but we may expect to see some distortion in the upper half of the (linear) spectrum (SignalEssence).

Dynamic Range Compression

Dynamic range compression of the iPhone at 3 levels: 90dBSPLA (reference), 96dBSPLA, and 99dBSPLA (Credit: SignalEssence)

To make sure the iPhones wouldn’t compress the dynamic range of loud samples, this study recorded samples at 90dBSPLA, 96dBSPLA, and 99dBSPLA (dB SPL A-weighted). All three phones elicited similar results, so the three lines on the graph show the loudness of the recorded audio for each sample. While the article notes that the iPhones began clipping above 99dBSPLA, the 96dBSPLA and 99dBSPLA samples were recorded with accurate dynamics relative to the 90dBSPLA reference (0dB input gain). This bodes well for our iPhone 8 (SignalEssence).

Recording Software

It wasn’t until I looked into the recording software itself that I could find out the specific bit-depth and sample rate my phone would be capable of. After some searching on google, I landed on the app Auphonic (Available on the iOS App Store). This free app presents a simple interface with all the settings we need to control, so choosing it was a no-brainer. The main interface of the app looks like this:

The recording interface, complete with a sound meter at the top
A view of the settings menu, including options to record from each of the iPhone 8’s three microphones
Options to output to .aac and .caf . I chose .pcm, as it’s equivalent to a .wav .
The options for sample rate on the iPhone 8
As we found before, the iPhone 8 can only record mono.
The bitrate options presented for the iPhone 8
Options for input gain. I chose -12dB, just to be sure that AGC would be off.

After looking through these settings, I found that my iPhone was capable of 24bit/48kHz on 3 (!) different microphones (front, bottom, back). To ensure Auto Gain Control (AGC) would remain off, I selected -12dB gain (the closest option to 0dB). With my research and setup complete, it was finally time to start recording!

Generating sounds

Actually, not so fast. Before I could record anything, I had to produce the sounds I was going to play: 15 seconds each of a sine sweep (100Hz-18kHz) and white noise. For this task, I turned to the application Ocenaudio, available on macOS, Windows, and Linux (website). The app has an intuitive interface that made everything super easy!

To access tone generation, go to Generation>Tones
Audio options for tone and noise generation
The interface for tone generation
The interface for noise generation

After opening Ocenaudio, I selected Generation>Tones to generate the sine sweep. After selecting options for audio quality (I selected 24bit/48kHz mono), I found the tone generation menu. With the settings pictured above, I pressed “OK”, then exported the file to full quality WAV (for playback) and 128kbit/s CBR MP3 (for web upload), and completed an FFT analysis on the clip (to be used later). The process was much the same to generate the white noise.

Recording

Now equipped with my audio files, I was ready to record them to see how my iPhone would perform. To spice it up a little, I decided to compare all three microphones on the device, each at a 2ft distance from my laptop speakers (MacBook Pro 16″, 50% volume).

Recording with the front mic
Recording with the back mic
Recording with the bottom mic

Results

After recording all 6 clips ( [front, back, bottom]x[sine sweep, white noise] ), I uploaded them to ocenaudio on my computer, cut the leading and trailing silence, exported to MP3 (for web upload), and performed an FFT analysis. All audio clips and graphs are shown below.

Sine Sweep

Sine sweep (original):

Sine sweep (front mic):

FFT on sine sweep (front mic)

Sin sweep (back mic):

FFT on sine sweep (back mic)

Sine sweep (bottom mic):

FFT on sine sweep (bottom mic)

Analysis: The intensity of the original sine sweep is flat over the 100Hz—18kHz range, until sloping down to lower intensities at 18kHz and above. I’m not exactly sure where these higher frequencies came from, but they seemed to be be picked up only by the back mic, while the bottom and front mics display a full cutoff at 18kHz. In regards to the shape of the recordings, the front mic seemed to capture a flat spectrum, while the intensity recorded by the back and bottom mics decreased asymptotically as frequency increased. Perhaps these decreasing profiles have something to do with lower frequencies being perceived as louder or echoing more in the room.

White Noise

White noise (original):

FFT of the original white noise

White noise (front mic):

FFT on white noise (front mic)

White noise (back mic):

FFT on white noise (back mic)

White noise (bottom mic):

FFT on white noise (bottom mic)

Analysis: While the initial recording is flat across the full frequency spectrum (low Hz to >25kHz), both the back and bottom microphones cut of around 20kHz, indicating either that they aren’t responsive to higher frequencies, or something about the room / distance from the speaker muffled out the high end. Meanwhile, the front mic recorded more lows (<1kHz)—though this likely could have been simply due to movement of the phone during recording.

Notes

For the life of me, I couldn’t figure out how to get the images to display next to each other. After the “align-left” jumbled with more than three images, I even tried injecting some CSS into all the <img> elements (to no avail). Does anyone know how to do this?

Works Cited

PhoneArena https://www.phonearena.com/news/Apple-iPhone-XS-MAx-records-stereo-sound-with-four-mics_id108769 

SignalEssence https://signalessence.com/can-you-use-an-iphones-internal-microphone-for-acoustic-testing-and-accurate-recordings/

Pierre Schaeffer and Musique Concrète: The Origins of Sampling in Modern Music

Nothing is more ubiquitous in the music we listen to today than the practice of sampling—the reuse of a portion of a song or other audio clip in a recording. From the image-invoking cash register and gun sounds in M.I.A.’s “Paper Planes,” to Nikki Minaj’s allusion to Sir Mix-a-Lot’s “Baby Got Back” in her “Anaconda”, to the instrument libraries which back nearly every song produced today, sampling defines the sound the music around us. 

The ability to splice together different audio recordings is a direct consequence of the development of computer music in the twentieth century. No other person deserves more credit for this advancement than Pierre Schaeffer, often considered the grandfather of sampling technology (Ankeny).

Pierre Schaeffer in his audio lab (c. 1948) (source)

Born in 1910 in France, Shaeffer wasn’t a trained musician or composer, but a radio engineer (Ankeny). Around 1948, while working at the Studio d’Essai (“Experimental Studio”) at the French Radio-Television system Radio-diffusion et Télévision Française, he invented a composition technique called musique concrète, or “concrete music,” in which tape recordings of natural, everyday sounds would be altered, cut, and spliced together to form novel music (Britannica Musique, Britannica Pierre). A talented interdisciplinary scholar, Schaeffer created musique concrète as a marriage of musical creativity and cutting-edge engineering in order to challenge traditional conceptions of composition and to advance electronic music technology (Patrick). 

On the creative side of things, Schaeffer sought to break down traditional conceptions of music, rather than producing notation for an assumed set of instruments and tonal system. Instead of relying on musicians to actualize his creative ideas, he pioneered a “backwards” approach to music, allowing the sounds around him to inspire and define his compositions (Henshall, Patrick). He was particularly fascinated by a concept he dubbed “reduced listening”, in which he would extract the rhythmic, musical aspects of familiar sounds from their common associations (Patrick). Instead of hearing a train roll into a station, he wanted to highlight the syncopation of the engine and the steam whistle, and the tonal dissonance of the crowd. This idea of “hearing without seeing” and “listening without reference” was first realized in Schaeffer’s piece “Etude aux chemins de fer,” where altered locomotive recordings create a jarring, unfamiliar soundscape (Patrick). In this first draft of musique concrète, Schaeffer struggles with his vision, at times evoking interesting progressions, while at others, seemingly playing back odd assortments of still-familiar sounds.

(alt link: https://youtu.be/N9pOq8u6-bA)

In order to dissociate sounds from their typical associations, Schaeffer pioneered many techniques to profoundly alter, rearrange, and assemble tape recordings. His approach consisted of three main components: acquisition of samples, sound manipulation, and tape aggregation (Musique britannica). First, a large assemblage of sounds would be recorded from the natural world. With samples ranging from kitchen utensils to fruits, these initial recordings would inspire the eventual tone of the piece (Howell). Then, the sounds would be altered by any number of complex means. Samples would be reversed, sped up, cut or extended, pitch altered, filtered, or even played and re-recorded in an echo chamber to add reverb (Britannica musique). In addition to the standard tape manipulation tools of the time, including mixers, a direct disc cutting lathe, and physical cutting and rejoining, Schaeffer invented several novel machines of his own to achieve these distortions (Patrick):

  • the morphophone, a delay and loop machine with ten tape heads
  • The phonogene: a device that could play a sample at multiple pitches with a keyboard
  • Amplification systems for experimentation with spatial sound
  • a triple-track tape recorder
The morphophone (source)
The phonogene (source)

Once the sounds had been altered, Schaeffer would physically splice segments of tape to form a single track. Tapes would be joined at different angles to create different “crossfades” between sounds, and the resultant long reels would be routed around improvised objects in order to feed them into the reader (Howell). These techniques involved a great amount of technical skill and precision.

Diagram of an oversized tape loop (Howell)
Crossfading, achieved by splicing segments of tapes at different angles (Howell)

In 1949, Schaeffer joined forces with Pierre Henry, a classically trained composer, to create the first great work of musique concrète, “Symphonie pour un homme seul” (Britannica musique). Recorded entirely from sounds produced by the human body, this ten-movement symphony exemplifies Schaffer’s “reduced listening” at the peak of his technological advances.

(alt link: https://youtu.be/MOYNFu45khQ)

While Pierre Schaffer’s music is certainly a far cry from modern pop and rap, the technological implications of musique concrète are lasting. As Jean-Michel Jarre, a former student of Schaeffer’s, said in 2007: “Back in the ‘40s, Schaeffer invented the sample, the locked groove — in other words, the loop […] It was Schaeffer who experimented with distorting sounds, playing them backwards, speeding them up and slowing them down. He was the one who invented the entire way music is made these days” (Patrick).

Works Cited

Ankeny, Jason. “Pierre Schaeffer: Biography & History.” AllMusic, www.allmusic.com/artist/pierre-schaeffer-mn0000679092/biography.

The Editors of Encyclopaedia Britannica. “Musique Concrète.” Encyclopædia Britannica, Encyclopædia Britannica, Inc., 27 Apr. 2018, www.britannica.com/art/musique-concrete.

The Editors of Encyclopaedia Britannica. “Pierre Schaeffer.” Encyclopædia Britannica, Encyclopædia Britannica, Inc., 15 Aug. 2020, www.britannica.com/biography/Pierre-Schaeffer.

Howell, Steve. “The Lost Art Of Sampling: Part 1.” Sound on Sound, SOS Publications Group, 1 Aug. 2020, www.soundonsound.com/techniques/lost-art-sampling-part-1.

Patrick, Jonathan. “A Guide to the Godfather of Sampling, Pierre Schaeffer.” The Vinyl Factory, The FACT Team, 10 June 2019, thevinylfactory.com/features/introduction-to-pierre-schaeffer/.