00:00:00
Hello, everyone! My name is Russ Olsen. I've been part of the Ruby community for a long time and I work for a company called Hashrocket. I've written a couple of books that I’d like you to buy many copies of.
00:00:05
Before I really get started, I want to acknowledge that I helped organize another Ruby conference, Ruby Nation. It’s a real challenge getting everyone here; it takes more effort and aggravation than you could possibly imagine. So, I’d like to extend my thanks to the folks at Hashrocket and everyone else involved in this conference.
00:00:22
Now, onto the topic of the day. This morning, I’d like to take a step back and discuss the things we do as programmers and how we do them. In particular, I want to focus on insight and intuition in programming, starting with a fundamental question: Why? Why is the experience of programming what it is?
00:00:53
I don’t know about you, but here’s how I spend my work week: on Tuesdays, Thursdays, and Saturdays, everything is going brilliantly while I’m coding. But on Mondays, Wednesdays, and Fridays, I alternate between staring at the screen and banging my head against it. Programming is really hard – what we do is incredibly challenging.
00:01:14
So, the question is: why is what we do so hard? What makes it so difficult? If you recall when you were first learning to program, it was often a surprise. You start to learn programming, and you encounter trivial exercises that turn out to be really hard. The early days of programming were particularly surprising because software was invented to spare us from the complexities of programming computers by plugging in wires.
00:01:40
This is a picture of Gray Walter, a pioneer from the 1940s who was active in the early days of programming. He discovered something essential about why programming is so challenging. Gray Walter was trying to simulate animal behaviors, like seeking food and avoiding obstacles. He built the first robot of its kind, which he called a 'turtle' because it had a plastic or metal shell and resembled a turtle.
00:02:04
These turtles could navigate around by bumping into obstacles and backing up, much like a Roomba does today. One interesting feature Walter built into these turtles was a behavior where they would seek out a charging station when their battery ran low. A light above the charging station would guide them, and after recharging, they would continue to move around.
00:02:31
However, a fascinating observation occurred when he built two of these turtles. If you let them run simultaneously, one might end up dead while the other would continue to roam. Sometimes, both would stop working. This happened because, when both turtles got hungry at the same time, they would head for the charger. As they bumped into each other, they would essentially fight over the charger until one of them ran out of power.
00:03:16
The behavior that resulted from Walter’s design was not intentional; it emerged from the simplicity of his circuits. This, I believe, encapsulates the essence of programming – it is fundamentally about emergent behavior. We write code, hoping that from simple beginnings, we can build a complex accounting system or an online storefront.
00:03:56
The problem is, while we expect our code to lead to desired emergent behavior, there is also undesired emergent behavior, like having a non-functioning turtle. Controlling this emergent behavior is difficult; we are in the business of trying to make the right behaviors emerge while preventing the wrong ones. We like to think of ourselves as executing logical, semi-mathematical work in programming.
00:04:38
When we fix bugs, we often describe it as a deductive process, explaining how we arrived at the solution after reasoning through the problems. However, what truly happens is we often stare at the screen for days until suddenly, it all makes sense. We convince ourselves that we are strictly using logic when, in reality, our process can often feel more intuitive.
00:05:07
This brings me to the topic of those 'aha moments' that occur after we’ve spent time wrestling with problems. The question for us is: where do those moments come from? And how do we cultivate more of them? When we find ourselves stuck, how do we get unstuck?
00:05:36
Insight and intuition are not magical. Many of us treat them like magic when we experience a breakthrough after being stuck for days. However, they are connected to how our brains function – the left brain typically processes logical, language-related tasks, while the right brain is more creative, handling visualization and pattern recognition.
00:06:06
A key aspect often overlooked is that both halves of the brain share common resources. It’s akin to having a dual CPU setup but without sufficient shared bandwidth. Consequently, they can't operate at full capacity simultaneously, which is crucial to understand.
00:06:50
When working through problems, you have various facts floating around in your mind. For example, the notion that 'turtles seek chargers' or perhaps something simpler, like 'bacon is good.' There is a background process occurring in your mind where you pair these facts together until they eventually lead to some insight.
00:07:25
So, my first piece of practical advice is to select your facts carefully. It’s important not to narrow your focus too rigidly; instead, filter out the noise. In programming, there's an overwhelming amount of noise that can cloud your thinking.
00:08:01
Take, for instance, the Rails log file. In it, you typically encounter HTTP request data and ActiveRecord queries. These entries reflect Rails' original purpose of acting as a thin HTTP veneer over a relational database. However, most applications developed today are far more complex than this, and while we've become adept at handling them, we still often include excessive extraneous data that complicates debugging.
00:08:35
Instead, I suggest creating a log file dedicated only to higher-level concepts about what your application is doing. Isolate the relevant data that speaks directly to your application’s purpose. This way, you can avoid being distracted by irrelevant information that obstructs your insight.
00:09:16
If you pair program, I recommend adding periods of silence and reflection. When faced with a challenging problem, instead of continuing a fast-paced dialogue, take a moment to just think in silence. Often, creative ideas emerge during these quieter moments.
00:09:44
Another idea is to manipulate the facts you’re using. For instance, when dealing with test data in a system, instead of continually reusing the same set of sample data, change it up significantly. If you always use four fictional people, try using seven or insert real names to see if that triggers different insights.
00:10:09
You can also intentionally insert junk data into your system to see where the pieces fall. Though this might seem disorderly, it can sometimes reveal unexpected insights without serious consequences. Also, consider observing your log files in real time. This will allow you to witness the events as they unfold, providing a more convincing understanding of the data.
00:10:46
On a cautionary note, it’s essential to be wary of insights that strike suddenly, as these 'lightning bolt' moments may not always be reliable. Take, for example, the case of Percival Lowell, who was convinced that he saw a civilization on Mars based on the straight lines he observed through his telescope. However, his conclusions were founded on misinterpretations of his observations.
00:11:22
This goes to show that just because an idea comes to you with a flash of brilliance doesn’t mean it’s correct. We must be careful, especially when dealing with statistics. Intuitive beliefs often mislead us. Many people hold a fear of flying, for instance, due to a misreading of statistical safety data.
00:12:02
Our intuitions can often misguide us. For example, in programming discussions around dynamic versus static languages, static typing is sometimes perceived as inherently safer and more reliable. However, this doesn't capture the entirety of the coding experience as there are plenty of bugs typical in both kinds of languages.
00:12:35
The perfect storm for programming intuition occurs when someone asks you to optimize or speed up a piece of code. We are trained to dive into the code to find 'interesting' parts to improve. Yet, inspecting the code may not reveal where the program actually spends its time; instead, what’s more effective is simply measuring and profiling.
00:13:17
Profiling often uncovers unexpected performance issues that cannot be revealed just by code inspection. This illustrates that we need to focus primarily on what the system is doing rather than just what it looks like from the outside.
00:14:05
An important takeaway is that cultivating intuition and insight goes beyond facts and technical details. In our culture, we often separate people into categories of 'fact masters' and 'creative thinkers.' However, I believe successful creatives have an understanding of the facts, enabling better insights.
00:14:42
As Louis Pasteur stated, 'Chance favors the prepared mind.' You can boost your intuition by acquiring diverse knowledge, as suggested in the book 'Seven Languages for Seven Weeks'. While learning a range of programming languages is beneficial, exploring lower-level programming or even areas like electronics can also contribute to a deeper understanding.
00:15:21
Emergent systems are everywhere, even in music or art. The behaviors that arise from these systems can be understood similarly to programming. The more familiar you are with different emergent behaviors, the better you will become or recognize patterns in your programming challenges.
00:15:58
In summary, the job of a programmer is to create systems that yield interesting emergent behavior. All around us are systems producing complex emergent behavior – the environment, energy systems, education systems – and as programmers, we have unique insights into these frameworks.
00:16:30
We have much to contribute beyond just our job requirements. As a community, I encourage you all to reflect on how your work in programming intersects with broader systems of society. Thank you all for listening!
