00:00:15.800
Yes, my name is Joseph Wilk. I am a programmer, and today I am going to give you a live performance controlling visuals and music all live coded.
00:00:22.470
Everything is done live—nothing is pre-recorded, and nothing is pre-rendered. This is all done with live synthesis using the Unity 3D game engine.
00:00:35.910
I've never performed this before, so it's very improvised. It will probably go horribly wrong, and if it does, I will slam the laptop lid, and we will forget this ever happened.
00:00:43.739
So without further ado, I'll stop talking and start trying to make some noise. This is done using Sonic Pi, Unity 3D, and a whole ton of other tools, which I will show you in Emacs.
00:13:55.980
There is an exquisite blend between beauty and practicality; there is no reason to sacrifice either one. People who are willing to compromise on either do not really understand what computing is all about.
00:14:37.569
Now, let me reveal the magic of all that mess and how bad my code is. As you can see, Emacs is chugging because it can't handle rendering an OpenGL window while doing 3D graphics.
00:14:41.480
Let me just point out what I have here: Unity 3D is running in the background, and on top of that is a transparent terminal running Emacs. There are many layers to what I'm doing here; probably, Emacs is the place to start.
00:15:28.870
Emacs has direct controls to the graphics and visuals in Unity. There’s live synthesis through SuperCollider and Sonic Pi, as well as Ableton Live, which is controlling MIDI directed to things like my modular synthesizer.
00:15:41.209
When I say I am live coding, I don’t think of myself as a musician; rather, I see it as a musical instrument. Emacs runs a UDP server, allowing me to control speeds while triggering values. It broadcasts messages to both the game engine and the audio engine.
00:16:08.480
That’s the beauty of code—it allows me to manipulate many things through patterns and the execution of functions, which has many implications in the performance world.
00:16:14.930
Every time I run code in Emacs, it flashes, indicating it’s being sent to a Ruby server called Sonic Pi, which executes the Ruby code and sends messages to the graphics engine. What you see here are just functions executing various actions—some of them quite hideous, I admit.
00:16:36.620
As I mentioned, beneath it all, it’s quite messy. If I were being fancy, I would say I implemented macro precompilation in Ruby, but at its core, I end up using regular expressions to handle the data structure.
00:17:04.250
This is my personal Ruby—it’s how I express myself. It matters to me that I focus on details that may not matter to others. I also utilize a prefix notation and some other techniques that I'll skip over for now.
00:17:37.050
Here is Ableton Live, another crucial aspect of my setup. Unity 3D isn't open-source, but you can get a free license. Ableton is the only paid software I’m using; it’s essential for rendering lots of MIDI.
00:17:56.680
I wanted to show you what’s happening under the hood, as there’s a lot going on in Unity. You can't do everything in Ruby, though you could embed Ruby if you wanted.
00:18:04.340
What I am doing is building a scene with multiple controls, which are then sent using OSC (Open Sound Control)—the duct tape of all art performance work using computers.
00:18:21.060
This scene wasn't pre-designed; you'll see that there are many paths and choices. For example, I had a tree that I decided not to open, just to demonstrate what I composed without all the fancy features.
00:18:39.640
I will provide a link later so you can explore this for yourself—my real goal is to help you understand how to connect Ruby with Unity. What you see is effectively a patch graph with many OSC endpoints that look like URLs.
00:19:02.500
I’m sending Ruby messages from my code to specific OSC addresses, controlling various simulations with smooth transitions because jarring transitions are not friendly.
00:19:15.679
There are many messy bits in the code, which I probably won’t show you, but my goal is to give you something to take away. I haven't talked about this much before, so I wanted to share some links to help you explore this.
00:19:40.570
It's exciting to think about Ruby from this different perspective, considering programming as an expressive performance medium and how that shapes our ideas.
00:19:44.190
The first thing I perform is REPL-ELECTRIC, which is my live coding entity. I’ll share the source code of an album I created through Ruby code, which is a deviation from one of the tracks.
00:20:09.290
You can browse the source and see everything I worked on today. There was a playful incident involving Spotify and iTunes due to naming my album ASCII Art—I'll let you discover the details.
00:20:39.490
This is how I perform—with Sonic Pi, a free open-source tool using Ruby to live code music and audio. Sonic Pi is built to work seamlessly with various editors.
00:21:16.540
My goal is to visit schools and show children how programming languages can be expressive tools for creativity. I also work with professional composers for film and TV to demonstrate how programming can augment their creativity.
00:21:46.670
So, there’s a spectrum where I can engage with students and encourage them to open-source their music, empower them to make better music than I ever could, and potentially compose music at a professional level.
00:22:05.360
As I said, you can get a free license for Unity 3D, and while you can use it, there are many other alternatives like Ruby shoes or browser-based applications.
00:22:46.350
There's a vibrant community around live coding, and I’m happy to give recommendations for other tools to explore.
00:22:48.370
The most important library for enabling communication between Unity and Sonic Pi is called Clack. It’s not very well known, so I mention it because you'll need it to connect these two platforms.
00:23:15.860
By installing it, you’ll be able to send messages from Ruby to Unity, allowing for an exciting interaction between these two systems.
00:23:41.640
If you enjoy my music, thank you. I believe I might be the first person to publish an album on Spotify by executing Ruby code. You can find it on Spotify, Bandcamp, and other platforms.
00:24:10.410
I remember that each time I run my live coding session in Emacs, I make a Git commit. This means I can track the complete history of the performance and the creation process.
00:24:39.860
I will share this with people who are interested; please come see me afterward if you want to talk about any of this.
00:24:56.060
I'm very happy to share this with you and help you get started. Thank you very much, everyone.
