00:00:07.220
Let me see if I can get sand from here. I don't know. All right, there it is. So welcome! Welcome to Helsinki, a lovely place. I'm really happy to be here today. Today, we're going to be talking about music.
00:00:19.080
What I love to do is understand the world through Ruby. You know, the world is a really big, complicated place, and since we are programmers, we can use the languages we like to understand things a little bit better. Once you can write some code for something, you sort of actually get what it's all about. That's what I did with music.
00:00:37.739
I've been making music as a hobby for a long time, and this talk is sort of the intersection between my two main interests: programming and music. This is what I like to do—understand it by figuring it out through writing code. Now I'm here showing the stuff to you all.
00:01:06.780
We're going to cover a few things today. First, we'll have a brief history of music, highlighting how technology has played a big role in shaping that history. Then, we'll look at digital audio, how we can visualize it, do a few mixing tricks, and create some actual sounds. There are a whole bunch of Ruby codes for this talk, which you can find afterwards to play with if you're interested.
00:01:50.520
Let's get started by discussing what audio actually is. Audio is a waveform, and we perceive it as sound. To create music, our brain uses pitch, timbre, and tempo to create the illusion of music. Here is a sine wave, which is a waveform that doesn’t often exist in nature, but it's the simplest thing that can have a pitch. A waveform is similar to what you would see in a pool—it's waves of air molecules bouncing against each other and making their way through a space.
00:02:30.299
These bouncing air molecules are coming out of the speakers right now, creating a chain reaction of air molecules bouncing against each other in a wave, eventually reaching your ears. Inside your ears, there are tiny hairs called hair follicles that vibrate in response to these air molecules. This creates an electrical signal that travels to your brain, where it magically creates meaning out of these vibrations.
00:03:03.180
Humans can process language and interpret audio waves as music, but scientists haven’t fully figured out how this works yet. However, somehow, some magic happens in our heads, and this music that most people love just appears. The brain does this in three different ways.
00:03:49.500
The first way is pitch. Pitch is the number of times a wave oscillates. This is a pitch, and it’s the foundation of the musical scales we learn in primary school. The second aspect our brains use is timbre, which is what makes a sound specific, like what makes a violin sound like a violin. Lastly, we have tempo; if you take sound sources and introduce breaks in between them, we perceive that as rhythm. Combining these elements creates a simple piece of music.
00:05:00.479
Despite its simplicity, this piece already has all the components in place. Back in the day, music was always live, often around a campfire where everyone joined in to sing together. However, this concept has been somewhat lost as we moved everything into recordings.
00:05:44.940
The process really began in the early 20th century with Thomas Edison, a name you've probably heard of. He invented many things, one of which was the phonograph. You would shout loudly into that hole, and a tiny stylus would vibrate, making an etching on a cylinder.
00:06:15.720
Unfortunately, this phonograph didn’t catch on because Edison patented it, which made it exclusive to him, and others didn't want to pay him a license fee. A man named Berliner came up with the concept of the disc phonograph, which is how we ended up with records. If Edison hadn't patented his machine, we might have been using cylinders for 100 years instead of flat discs.
00:06:41.700
As we progressed, we began to develop technologies, especially during and after the Second World War. These advancements led to the creation of high-quality microphones and multi-track recordings. For example, consider the Beatles, who used multiple microphones and mixing desks.
00:07:13.620
Then we move into the 80s when everything became digital with the advent of CDs. For the younger people in the audience, CDs are something we used to play music from. Eventually, everything has moved into computers, and this brings me to the point of my talk today. Now, people use software to make music.
00:08:07.920
I will take a few components out of a program called Ableton Live and recreate them in Ruby. To do that, we first need to understand what digital audio actually is, which consists of many samples in a realm. If you look at the waveform, every little point represents where a sample is being taken. Essentially, digital audio measures the intensity of a waveform at given times, thousands of times per second.
00:08:57.060
When we recreate a waveform from these samples, the result is so close to the original that humans typically cannot hear the difference. These numeric samples can be positive or negative as the waveform oscillates around zero. The simplest format for audio is the WAV format. I'm using a Ruby gem to read WAV files, process them, and then write them back to disk, allowing us to manipulate them in Ruby.
00:10:04.620
This requires some helper code to read the WAV file and create a large array containing all the audio numbers. Once we have this big array, we can manipulate it. This could look like a second of audio; there are many numbers involved, and it's too much to grasp just by reading. Hence, we’ll create some visualizations using a gem called ChunkyPNG to generate images.
00:11:07.860
As we loop through all the samples, we draw a dot on a PNG image depending on whether it’s a positive or negative number. The images we create will represent real-world audio. For instance, here is an image of a noise made by a car passing by. The sine wave, on the other hand, appears more uniform, functioning as a mathematical function. To analyze audio more thoroughly, we utilize Ruby's enumerable method to compress data into manageable slices, which gives us clearer visual images.
00:12:26.760
With that foundation, we can begin manipulating audio. The first thing we’ll work on is making sounds louder, starting with a little drum loop. By comparing visual representations of the audio, we can see that raising peaks correlates to louder sound. To achieve this, we can process the samples by multiplying them by a ratio, writing the altered data back to disk for a louder output. However, if the multiplication exceeds the range, we encounter clipping, leading to distortion, which is evident in the sound.
00:13:40.860
This distortion is what audio engineers refer to when talking about clipping. Next, let's discuss mixing. Mixing involves combining multiple sound sources from separate channels into a more complex waveform. We have bass, drums, and piano tracks of the same length starting at the same time, which we can mix in Ruby by summing their data.
00:15:02.279
This is how we create mixed sounds in Ruby, which results in a combination of sample tracks. Thus far, it seems to work nearly effortlessly, highlighting the magic involved in audio manipulation, as it doesn't have to be overly complicated.
00:16:09.360
Now let's delve into compression, a technique that makes sounds louder by controlling their peaks. Compression has been used since the analog era, picking a threshold to identify and reduce the volume of these peaks to prevent distortion when amplifying the entire signal.
00:16:57.700
When applying compression in Ruby, we identify samples above a threshold, reducing their volume before amplifying the overall sound again. This two-step process enables us to achieve a compressed sound, causing the peaks to blend better with the rest of the audio.
00:17:54.660
At this stage, I believe I’ve made a simple compressor in Ruby. It may not be perfect, but here’s what it sounds like after applying the compressing method. You can hear the initial hit is clear, but there’s an extended ringing on the tail, which could become a trendy sound in the future.
00:19:20.480
Next, we’re exploring sound generation techniques. Historically, analog synthesizers used electrical circuits to generate waveforms. Today, we can recreate similar effects using Ruby, focusing on four primary approaches.
00:20:00.760
The first method is to create noise using Ruby’s random function within a certain range. This randomness can be visualized and utilized in various ways. For example, by applying filters, we can shape the noise into something more controlled, like a snare drum.
00:21:27.540
The second method involves generating a square wave, which can be achieved by creating an oscillator in Ruby. This oscillator produces alternating high and low values, which you can hear when combined with other samples.
00:22:26.820
The sine wave is another essential element in sound generation. By creating a sine wave using the principles of trigonometry, we yield waves that sound pure and distinct.
00:23:15.780
Combining multiple oscillators allows us to create chords, which consist of sounds at different pitches that harmonize together. In Ruby, mixing these oscillators gives us a richer sound, just as playing multiple strings on a guitar would.
00:24:13.920
At this point, we've covered the essentials. If we apply the Fourier Transform concept, we realize that all real-world sounds can be synthesized by combining multiple sine waves. This is valuable in data analysis and signal processing.
00:25:46.680
To add timbre, we can combine both sine and square oscillators, leading to an interesting mix. Even though the waveforms might not be aesthetically pleasing right now, creating these sounds using simple Ruby code is quite achievable.
00:26:17.400
To demonstrate further, I created a cover of what I believe is the best song ever made, showcasing the potential to create music with this approach. Thank you for joining my talk!
00:28:07.620
Please stay with us, and I’ll be happy to take your questions while we move on to the next segment.
00:28:19.020
The first question is about side chaining kicks to bass in Ruby. Sidechaining involves using a kick drum signal to control compression. If you have further questions, feel free to find me later.
00:29:08.100
There are common misconceptions about music, like the notion that musicians use esoteric math, while in reality, it’s quite simple. It annoys me when people complicate a straightforward process.
00:30:07.140
Another question is if it's possible to play with vocals in Ruby. Yes, the code can be applied to various audio forms. Regarding overtones, they are produced by vibrations in real-world mediums, and you can synthesize them in Ruby, although it can get complex.
00:30:49.380
Another question about applying attack and release times to compression—these concepts involve monitoring if signals stay high above the threshold. I initially ignored making that example, but it can certainly be integrated.
00:31:33.360
Lastly, someone mentioned Sonic Pi, an excellent tool for creating music in Ruby. I recommend trying it out.