Digital Audio
How music works, using Ruby
Summarized using AI
How music works, using Ruby
by Thijs CadierIn his presentation at RubyConf 2022, Thijs Cadier explores the intricate relationship between music and programming by demonstrating how sound can be understood and manipulated through Ruby. The talk emphasizes how technology has fundamentally changed music, particularly with the evolution from live performances to recorded audio. Cadier outlines several key components that define music and highlights how Ruby can replicate industry-standard audio processing techniques.
Key Points Discussed:
- Understanding Music:
- Music as audio involves sound waves traveling through air, interpreted by our brain to create musical experiences. Cadier introduces fundamental musical concepts:
- Pitch: Frequency of sound waves; lower pitches (like bass) and higher pitches (like violins).
- Timbre: The unique quality of sound, illustrated by instrument varieties that contribute to musical depth.
- Tempo: The rhythm created from silence and sound interspersing.
Historical Context:
- Cadier provides a brief timeline leading to modern digital audio, starting with Edison’s wax cylinders, Berliner’s discs, and the transition to magnetic tapes; ultimately culminating in digital music with CDs and computer technology.
- Cadier provides a brief timeline leading to modern digital audio, starting with Edison’s wax cylinders, Berliner’s discs, and the transition to magnetic tapes; ultimately culminating in digital music with CDs and computer technology.
Using Ruby for Audio Processing:
- He introduces the wave file gem, a Ruby library for reading and writing WAVE files, serving as a tool for sound manipulation.
- Audio Amplification: Demonstrates amplifying sound samples and warns about clipping, which creates harsh distortion when signal peaks exceed allowed levels.
- Mixing: Merging audio tracks to enrich soundscapes, illustrated with an example combining two tracks.
- Compression: Reducing sound peaks for a balanced audio mix, common in radio and popular music.
- He introduces the wave file gem, a Ruby library for reading and writing WAVE files, serving as a tool for sound manipulation.
Sound Synthesis:
- Cadier discusses synthesizer sounds and the generation of noise, square waves, sine waves, and their combinations to create complex auditory textures and chords.
- Fourier Transforms: Encouraged for recognizing sound patterns, proving valuable for understanding complex sounds.
- Final Remarks: Cadier concludes with an example showcasing how basic waveforms can result in intricate musical pieces, and shares that he has created a cover of a song using samples generated during the talk.
- Cadier discusses synthesizer sounds and the generation of noise, square waves, sine waves, and their combinations to create complex auditory textures and chords.
Overall, the presentation effectively merges music theory with programming, empowering participants to grasp sound production techniques using Ruby.
00:00:00.000
Ready for takeoff.
00:00:16.940
Well, welcome to RubyConf! I'm really happy to be here.
00:00:19.740
It's nice to be in person. Today, I'm going to talk about music, specifically recorded music.
00:00:22.560
I'll discuss the kinds of tricks that producers and musicians use to make the sound good, and I'll explain that with some Ruby code.
00:00:35.640
What I like to do is use programming to understand the world. The world is a very complicated place, and as programmers, we have this amazing tool to actually comprehend things.
00:00:39.719
When you write code, you have to understand it; otherwise, you can't write it. This compels you to learn it, and that's what I did with music.
00:00:47.340
So today, I want to take you through a process where you don't just understand it but also try to write some code.
00:00:51.180
After a while, you start to grasp it, and you end up with hopefully some simple code and examples that I'll share with you.
00:00:59.160
Let’s begin with a very brief history of the technology behind music that has developed over the last several years, which will lead us to digital audio.
00:01:05.440
This is where we can start using Ruby to actually see what's going on.
00:01:10.680
There are several code examples accompanying this talk. You can find those here, and I will also post the slides after the talk.
00:01:15.900
If you want to follow along, you can quickly go to this link.
00:01:20.700
So, what is music? It is described by a few key aspects.
00:01:23.520
Audio, like what you are hearing right now, is a waveform moving through the air. Air molecules bounce against each other, and our ears perceive these vibrations.
00:01:32.520
The easiest way to visualize this is to compare it to waves in a pool of water, but instead of water molecules, we have air molecules.
00:01:38.520
These waves travel through the air from a sound source to your ears, where tiny hair cells called follicles vibrate in response to these sound waves.
00:01:45.720
Your brain then processes this signal to create the illusion of speech or music. So, there isn't anything inherently musical about these physical phenomena.
00:01:55.440
Your brain generates models to make sense of the incoming vibrations. Neuroscientists don't fully understand how this works yet, so there is a bit of magic involved.
00:02:05.280
Let’s explore the basic patterns that the brain uses to map these sounds.
00:02:10.680
The first important concept is pitch.
00:02:15.200
Pitch refers to the frequency of oscillations per second. A lower pitch corresponds to fewer oscillations, while a higher pitch corresponds to more.
00:02:21.879
For example, a bass guitar produces lower pitches, while a violin produces higher pitches.
00:02:28.200
Let’s listen to a basic pitch that sounds like this.
00:02:35.400
[Pitch Example]
00:02:37.860
We're all familiar with that sound.
00:02:41.760
The next element is timbre, which refers to the unique quality or color of a sound.
00:02:46.540
Timbre encompasses all the variations within pitches.
00:02:48.960
An example of timbre can be heard in a classic Beatles song from the 1960s, where different instruments contribute to the overall depth of the sound.
00:02:56.280
Finally, we have tempo, which is established through the presence of silence between sound waves.
00:03:02.400
This silence contributes to the perception of rhythm and can create a loop.
00:03:05.520
If we combine pitch, timbre, and tempo, we create a musical piece that sounds something like this.
00:03:13.280
Historically, music was always live; there was no recorded music until technology evolved.
00:03:20.760
This technological advancement changed the world in many ways, leading to the advent of recorded music.
00:03:27.480
Let’s quickly review the steps that led us to digital audio.
00:03:29.640
It all started with Thomas Edison, who developed a device that could etch sound onto a wax cylinder using vibrations from sound.
00:03:37.560
However, due to Edison's patent, this technology did not gain traction.
00:03:41.760
A different inventor, Berliner, proposed using discs instead of cylinders, which became the dominant format for recorded music.
00:03:49.860
After discovering magnetic tape technology, audio recording quality significantly improved, which facilitated multi-channel recording and transformed the music industry.
00:03:57.060
In the 1980s, music became digital, marking a shift from analog grooves to CDs and modern computer technology.
00:04:05.880
Now, we can use software to replicate tools that were once available only in large studios, making powerful audio production accessible.
00:04:14.640
Next, we’ll dive into what digital audio actually is and replicate some popular tools using Ruby to better grasp their functions.
00:04:20.760
Digital audio consists of samples; when we take a waveform, we can measure the intensity of the signal at specific points.
00:04:27.120
By using these samples, we can recreate and play back the sound.
00:04:35.880
Often, we process this data as a sequence of numbers, which allows for manipulation and playback.
00:04:42.060
In this presentation, I’ve chosen the wave file gem, a Ruby library for reading and writing WAVE files.
00:04:48.120
This gem enables us to read a WAVE file into a large array of samples, simplifying audio processing.
00:04:56.640
Let’s visualize this with an audio sample.
00:05:04.200
[Sample Visualization] Now that we have a representation of our audio data, we can start using some Ruby functions to analyze and modify these samples.
00:05:12.960
In fact, our first step will be to amplify the audio using Ruby.
00:05:20.640
By multiplying each sample by a factor, we can increase their amplitude and create a higher volume.
00:05:28.440
Let’s try this with a sound sample and listen to how the amplification affects the sound.
00:05:39.600
[Amplification Example] You can hear the increased volume, represented by peaks that rise higher than before.
00:05:47.160
However, if we amplify too much, we risk clipping, which distorts the sound and is generally undesirable.
00:05:57.960
Clipping occurs when the peaks exceed the limits of the waveform, leading to a harsh distortion.
00:06:03.420
Let’s listen to how clipping sounds.
00:06:10.380
[Clipping Example] This distortion is often used in genres like heavy metal to achieve a specific sound quality.
00:06:18.900
Next up, we’ll discuss mixing. Mixing involves merging audio tracks together to create more complex soundscapes.
00:06:27.840
When mixing, we can take several audio samples and sum them up to produce a new waveform that combines elements from each sample.
00:06:37.740
Let’s see how this works when we mix two audio samples.
00:06:44.040
[Mixing Example] You can hear how the combination of sounds builds a richer auditory experience.
00:06:52.200
Following mixing, we will cover compression, a technique used by audio engineers to enhance the overall loudness of a sound.
00:07:00.360
Compression targets the peaks of sounds to bring them down, which allows for an overall louder mix without distortion.
00:07:06.960
In radio or popular music, compression is employed extensively. Let’s see how we can achieve compression using Ruby.
00:07:14.520
The process involves defining a threshold that determines when a signal is too loud, and reducing the volume of those peaks.
00:07:26.340
After lowering the peaks, we can apply a makeup gain to raise the overall volumes while avoiding clipping.
00:07:35.520
Let’s listen to an example of this compression process.
00:07:46.800
[Compression Example] You can notice the difference in how much more balanced this sound is compared to the uncompressed version...
00:07:56.760
Now, let’s switch gears to sound synthesis. Many contemporary music pieces use synthesizers to generate sounds.
00:08:06.720
For example, we’ll start with generating noise through Ruby, which we can do using random number functions.
00:08:12.600
The output will appear chaotic, representing the randomness associated with noise.
00:08:19.560
Let’s visualize this noise sample.
00:08:26.640
[Noise Sample Visualization] You'll see how noise can form the basis of percussive sounds in electronic music.
00:08:33.840
Next, we’ll explore square waves, which represent the simplest oscillating sound.
00:08:39.360
Let’s create a square wave using Ruby and listen to its output.
00:08:47.760
[Square Wave Example] Although it produces a 'blunt' sound, it’s a fundamental element within electronic compositions.
00:08:54.480
Now, we’ll delve into sine waves, the core building blocks of many sounds.
00:09:03.180
Generating a sine wave involves using a mathematical function to plot samples in a circular motion.
00:09:10.440
Let’s visualize and listen to what a sine wave sounds like.
00:09:16.440
[Sine Wave Example] This smooth waveform is commonly used across various music genres.
00:09:24.000
Building on that, we can combine multiple oscillators at different frequencies to create chords.
00:09:32.520
Mixing these oscillators produces complex harmonic sounds that lend richness to our music.
00:09:39.420
Let’s hear how this combination sounds.
00:09:48.240
[Chord Example] This is how we can produce rich harmonic textures in our compositions.
00:09:55.920
If you’re interested in further exploration, I encourage you to look into Fourier transforms, where you can deconstruct complex sounds into their sine wave components.
00:10:02.640
This is valuable for recognizing patterns in sound and has applications in graph theory and data analysis.
00:10:09.840
Finally, let’s talk about creating tambre through the combination of different waveforms.
00:10:14.760
By mixing sine waves and square waves, we can generate unique wave shapes that sound richer and fuller.
00:10:22.920
Let’s listen to this final sound example that illustrates this mixture.
00:10:32.880
[Final Hybrid Wave Example] It’s fascinating how from simple building blocks, we can create intricate musical landscapes.
00:10:44.520
That brings us to the end of my presentation.
00:10:51.660
I appreciate your engagement and enthusiasm for this exploration of how music works.
00:10:55.740
As a fun note, I’ve even created a cover of my favorite song using only the samples generated here today.
00:11:06.000
[Musical Outro]
Explore all talks recorded at RubyConf 2022
+58