Programming as Performance

by Joseph Wilk

The video titled "Programming as Performance" features Joseph Wilk presenting at RubyConf AU 2015. The main theme revolves around the intersection of programming, performance, and creativity, emphasizing how coding can be expressive and dynamic, akin to performing arts.

Key Points Discussed:

Historical Context: The evolution of programming over the last 70 years, starting from early programmable computers. Wilk highlights the complexity of computational behavior and the challenges faced by programmers when instructing machines.
Three Perspectives on Programming: Wilk categorizes programming philosophies into computer science, engineering, and performance arts. While computer science leans towards mathematical models and precision, engineering emphasizes processes and testing. In contrast, performance arts focus on expression and the inherent risk involved in live performance.
Live Coding as a Form of Art: Wilk discusses live coding and how it differs from traditional programming. By performing code in real-time, programmers receive immediate feedback and engage audiences in unique ways. Examples include live coding music, where the creative process is shared and transformed into an interactive experience.
Case Studies and Examples:
- Leicester Project: A project demonstrating live coding with fast feedback to produce tangible outputs, enriching audience engagement.
- Code Poetry Jam: An event where writing code acts as a poetic expression, merging art and functionality.
- Sonic Pi: A Ruby-based platform that allows teaching programming through music, illustrating how accessible tools can make learning engaging.
Practical Application: Wilk showcases Sonic Pi as an educational tool, especially for children. He highlights how live coding can help in teaching programming concepts, making it accessible and relatable.
Blending Disciplines: The interaction between coding and music fosters collaboration between musicians and programmers, creating innovative performances and diverse outputs.
Future of Live Coding: The talk concludes with an optimistic view that live coding is evolving into a recognized form of art, and there are numerous opportunities for educators and performers to explore this creativity further.

Conclusions:
Wilk emphasizes that coding can transcend mere technicality and transform into a passionate expression through live performance. The integration of programming with art leads to engaging educational experiences and creative exploration of technology. The potential of programming as an expressive medium encourages both learners and presenters to embrace the unpredictability of live coding.

00:00:00.400 Programming is an odd thing. It's been about 70 years since the first programmable computer, the Z3, and still we grapple with the idea of computational behavior. This refers to when a human tells a machine what to do and, in many cases, what actually happens—the unseen ramifications of our decisions. These poor robots operate under very simplistic rules to avoid each other, yet they can get stuck in horrific complexity, which the programmer may not even realize.

00:00:14.000 However, we haven't been idle over the last 70 years. The fact that we are here programming indicates that we have developed many solutions to these challenges. We turn to computer science, where the general approach to the messiness of human interaction is to shift towards a more mathematical formal model. This allows us to create proofs and rely on specifications, which is fantastic and forms the basis of most programming languages. Yet, this approach tends to obscure the human element and doesn't address the fact that we can still make poor decisions; it's just that we can't completely wipe out our memory or our hard drive—unless we are using C.

00:00:42.160 The school of engineering focuses more on processes and practices. These are things we often discuss, such as unit tests, integration tests, regression tests, sandboxing, and encapsulation. We have developed various techniques to predict what a machine will do versus what a human instructs it to do.

00:01:03.920 Commonly, we consider the performer, who brings a very different perspective from the other two schools of thought. Instead of focusing on long-term building, the performer's emphasis is on expression and viewing code as the interface, rather than merely constructing an interface. The challenge lies in taming the 'tiger' of raw computational complexity on stage. It is terrifying, but it also brings the performance to life. There has to be an element of risk involved. This is why we attend live gigs; there is more passion and risk of failure. Strings can break, and many things can go wrong, and for me, that risk is a vital component of programming.

00:01:49.360 This brings us to the concept of live coding, which is simply writing algorithms and adding to them in real time. It's something that many of you have likely experienced; if you think of the Interactive Ruby Shell (IRB), you're live coding. However, we typically do not do this in front of an audience. Understandably so, but it is an incredible way to explore and experiment, providing immediate feedback. Something unique occurs when your code manifests itself in physical space.

00:02:31.680 For instance, let’s take an example from a project in Leicester. We have these instructions in a language, and while we have no context on what it generates, we know that in programming, fast feedback is crucial. If something goes wrong, you immediately see a test failure, which is very valuable. Performing code that produces tangible outputs—like playing music—enables the audience to derive their own meaning from the code. If I were to suddenly show you the output, and you can witness the results right next to the programming, these symbols transition from being mere random characters to possessing real significance. This is something I find interesting and am genuinely passionate about.

00:03:41.120 As programmers, we have immense power at our fingertips, with countless elements under our control. I feel that we are only beginning to scratch the surface of what we can do—not simply building engineering systems, but also how we express ourselves. Live coding can be likened to poetry. For example, an artist named Alex McLean created a project combining code and poetry, demonstrating that code itself can be literature.

00:04:02.960 There was a Stanford event in 2015 called the Code Poetry Jam, where people wrote code that functioned as poetry. It both executed the code and read aloud poetry, integrating the output with the code. This factor—having the code and being able to see the output in the real world—is powerful. Casey Sitcho, a researcher in New York, has been experimenting with the idea that you don't need a traditional computer to live code. She is projecting her code onto the ground, and the dancer responds to the instructions, making the performance not just about seeing the choreography but also about interpreting the artist's intentions.

00:04:47.440 There is a band called Slob that has been generating music and graphics, experimenting with diverse means of visualizing code during live performances. Performing live fundamentally alters everything. Many integrated development environments (IDEs) are not very capable of handling this dynamic nature. In one example, a performer employed a list language, using closures as bricks, where the flashing lines represent the connectivity of the music played—showing which elements are active. You can see bricks being moved around, representing scopes and closures. This leads to a significant insight: with live coding, the immediate feedback and immersion contribute to a more vibrant experience compared to traditional programming, which can often feel stale.

00:05:34.880 At Strange Loop 2014, Karen Myers showcased a live programmed robot. She had a drone flying and used Sphero balls—programmable robot balls—illustrating another avenue for live code. Many individuals are engaging in unique projects involving weaving, where they teach children programming through these live demonstrations. I firmly believe that live coding is evolving into an art form, although it currently exists somewhat in the niche of creativity.

00:06:10.480 My goal is to tackle the challenge of making code more meaningful to a wider audience. This means transforming random symbols that lack meaning into something relatable, thereby building audiences for code and helping it become a more accepted mainstream art form. We achieve this by introducing programming concepts to children through engaging activities. My daughter, for example, experiments with music, playing different parts of the keyboard and mapping sounds using Sonic Pi, a Ruby-based language that enables music creation.

00:07:11.760 Sonic Pi runs effectively on inexpensive Raspberry Pi computers, making it an accessible tool for education. Numerous teaching resources are available promoting the use of live coding to help children learn music theory without the burdens of mastering an instrument. Teaching programming this way is immensely creative, providing children with a fantastic opportunity to express themselves.

00:07:59.920 In the UK, many schools have started adopting this approach, and there are orchestras comprised of children engaging in live coding for music, which I find extremely exciting. If you are looking for the next Daft Punk, I believe it might be through live coding.

00:08:37.760 As I transition into my demos, I'd like to note something interesting about Sonic Pi. The platform brings together diverse groups of people who don't necessarily communicate effectively, fostering common ground through music and coding. I have attended music concerts and festivals, performing code, while musicians have also come to programming conferences, showcasing their art. This blend of diverse mindsets enriches both fields tremendously.

00:09:03.920 Now, let's dive into the live coding segment, which is fitting given the talk's focus. I anticipate that things will not go perfectly, but that's the beauty of live coding; embracing the unpredictability is what makes it special. Let's begin with Sonic Pi. My approach will be simple, as if we were young learners. I will aim for the fastest feedback and play notes consecutively, emphasizing the concurrent nature of the programming.

00:09:54.880 Interestingly, we can have multiple sounds playing concurrently without realizing it because music inherently functions on this principle. I will try a simple sampling approach and experiment to create a more musical experience.

00:10:09.600 For better sound control, I must adjust the music to run continuously, which requires managing its inherent concurrency. This iterative learning process reveals the importance of timing. Soon, I will introduce complexity, experimenting with live loops to enhance our outputs.

00:10:48.080 As I refine the production, synchronization becomes crucial. Learning how to coordinate concurrent threads presents new challenges but also provides engaging learning opportunities. The need for synchronized timing reveals foundational concepts in programming.

00:11:27.440 I factored in the feedback and am working on optimizing our code executions. By allowing live edits, I can demonstrate how to modify variables dynamically, showcasing the interactive possibilities within live coding.

00:12:35.200 As the performance continues, the exciting part lies in experimenting with the complexity of algorithms and iterating over various configurations. By working on simple coding concepts, we can gradually develop a richer and more intricate program.

00:13:47.360 Accessibility remains a priority. We aim to provide a variety of examples and solid documentation in platforms like Sonic Pi, making them user-friendly. For example, performers can quickly grab sample code and start playing around, exploring musical elements and creating engaging experiences.

00:14:23.680 This dynamic interaction between code and music echoes the feedback loop essential to learning music—it's all about hearing and experimenting. My own experiences working on this software have shown me how close the material relates to musical expression.

00:15:02.560 The programming language continues to break barriers, enabling more ruby programmers and musicians to engage with coding applications. I have witnessed children create fascinating drum sampling algorithms, showcasing their creativity.

00:15:37.720 To illustrate this creativity, I'm currently working on a poetic representation of sound. Rather than solely relying on code, I seek to express programming through poetry and music, merging these creative outlets to generate engaging pieces.

00:16:59.920 Through these artistic endeavors involving Sonic Pi, learners can grasp programming concepts creatively and innovatively. This methodology not only teaches foundational coding skills but also fosters an enjoyable atmosphere amid the learning process.

00:17:50.640 As I wrap up, it's evident that coding can be a passionate and exciting endeavor, much like performing music.

00:18:00.000 Thank you very much for your attention, everyone! It has been a pleasure.

00:18:14.000 DJ Joey will be available for weddings, bar mitzvahs, and school parties!

RubyConf AU 2015