00:00:15.599
Thank you very much! Oh, this thing works nicely. Can I get some slides?
00:00:22.800
Yesterday, I got a little push from the Montreal guys to fill in the missing slot, and right after that I found this post online on LinkedIn.
00:00:28.199
I won't disclose the name of the person who posted it, but the nice part about this is that if you look really closely, you can see I'm sitting here making this presentation. This was literally made here.
00:00:40.500
Alright, my name is unpronounceable and unspellable for English speakers, so I usually go by Chris. Today, I'm going to talk to you about feedback loops.
00:00:54.960
If you Google 'feedback loop,' you get many different definitions. One of my favorite definitions is that it's basically any system where the output affects the input for the next iteration. Every time we code, the innermost, most intimate part of coding is this particular feedback loop.
00:01:12.659
The code you run creates your output, and that's the start and stop.
00:01:17.820
This is the part when you think, 'When would you write the code in your mind and then put it into the computer?' Nowadays, you can write the code and execute it almost instantly, but it wasn't always like that. Let's take a step back to the 1960s.
00:01:35.400
Most of you probably know what this is. Yes, that's right — it's a punch card. If you had access to one of these back then, you were really lucky. Most of the time, though, you weren't so lucky.
00:01:53.040
You would write code like this. This lovely lady wrote the code for the Apollo Mission program, and this is the actual code. She printed it out and typed it in using a typewriter. Because she was lucky, she got to use a computer indirectly; she handed the pages to someone who typed them into punch cards.
00:02:05.159
That person would then give those punch cards to a computer operator. The computer operator was responsible for maintaining the computer, resetting its state. The reason wasn't a button; it was a series of switches. After that, they would run the code, compiling it beforehand and printing out the results, because, at the time, there were no screens, and you had to receive results via traditional mail. If you had this feedback loop set up, you would submit your code, which someone else would convert into punch cards, and then the computer would compile and run your code. Two days later, you would receive a letter stating that your code failed on the second line, and you would have to do it all over again.
00:02:39.060
Pretty neat, right? The feedback loop took days back then. If you were lucky and writing something important, it might be a matter of hours. But technology has improved.
00:03:28.680
During the 1970s, we had new, innovative devices. You could have something like this on your desk. You may know what it is — it's a terminal. Specifically, this is a vt100 terminal. The interesting part is that the vt100 is still present today, even in this MacBook.
00:03:45.379
This is also why the program is called 'terminal' — it’s still compatible. You can still use a vt100 to connect to a MacBook. The typical usage looked something like this. I want to point out that the important part isn't here; it's over here.
00:04:05.940
This is a traditional modem, connecting a microphone to a speaker via a traditional phone line. You had to be really quiet because any sound could disturb the connection. Code editing occurred line by line using this terminal since you couldn't afford to load the entire program at once due to memory constraints. Eventually, someone realized that as computers became more affordable, developers were becoming more expensive, so they needed to find ways to make developers more productive.
00:04:40.860
An editor called EX emerged; has anyone used it? Has anyone used VI? A couple of hands go up. VI wasn’t actually an editor back then; it was a mode for EX, which stands for 'visual.' This meant you could see the code on the screen and edit it, with instant changes, which was amazing!
00:05:07.320
At this stage, the feedback loop lengthened to hours because you had to wait for everyone else to compile their code and run it. But if you were very lucky, it could be minutes. Fast forward to the 1980s. Computers became affordable, at least somewhat. You could now have a computer on your desk, which allowed you to compile locally.
00:05:51.100
You could see the code compiling in real time, and if it failed, you would immediately see the result on your screen. You no longer had to wait for a queue; instead, you had a feedback loop now measured in minutes.
00:06:10.920
The significant point is that although we had minutes before, now those are our minutes. Now you could even have two computers if you were well off or had a particularly good employer.
00:06:18.419
However, the slowest part of this new feedback cycle was no longer the computer, but the developer. This was because you had to validate the output and determine if it worked, creating another delay.
00:06:35.220
At this time, a group of individuals created a language called AWK. One of the developers of AWK, in an interview, mentioned that they created a throwaway language — a language that you wouldn’t need to maintain anymore. However, people began using it.
00:06:58.800
There was one individual who attempted to write a program in AWK, which is rather tricky because it’s a text-parsing language. This developer spent three weeks trying to fix a bug and ended up blaming the developers for this experience.
00:07:33.660
As a result, the original developers of AWK decided that this was wasting the valuable time of developers, and they needed to address this issue.
00:07:57.639
In the 1980s, they created short programs to test the components they wrote, testing every feature.
00:08:09.480
This was before JUnit, TDD, and other methodologies, but they did it anyway. This actually made AWK the standard package on most operating systems. I know that macOS recently dropped it, but it still remains in older versions and existing distributions, such as Ubuntu.
00:08:56.460
So, with feedback loops, if you had tests that ran quickly, you could get results in minutes. If you had well-optimized tests, perhaps even under a second. I know a developer, Angie, who created an impressive performance-testing program, achieving testing times of approximately ten seconds.
00:09:30.780
A lot has transpired between the 1980s and present day. We have significantly better tools. Some concepts are quite old, like console debuggers, but we can use these concepts to showcase the code and experiment with various methods.
00:09:43.680
We can examine the state of running programs, modify them on the fly, engage in exploratory programming, and debugging. We can observe code execution dynamically and perform automated testing — all without requiring substantial input.
00:10:02.400
However, not everyone has access to these advancements. My question for you is: could you be stuck in the 60s? If you don’t run code yourself, this is a profound concern.
00:10:39.300
I mean, haven't we all experienced a pull request with syntax errors?
00:10:46.740
This represents a developer stuck in the 1960s, penning code and relying on someone else to execute it. Like: "Hey, can you check this for me?" The quintessential computer operator.
00:11:02.520
If you don’t run code on your local machine, or if your local machine is shared with others, you might indeed be stuck in the 1970s. I’ve seen many people pushing code to continuous integration and just waiting for results, rather than running the code locally, thinking, 'What’s the point? There’s already CI in place.' This results in waiting for others while sitting in queues.
00:11:46.920
If you are not conducting tests and manually clicking through functionality, usually on staging because of laziness or lacking a setup, you may be stuck in the 80s. Even if you're mixing and matching, you might find yourself working in an environment relevant to the 60s, 70s, or 80s. But it would be much better to do so in the present.
00:12:10.560
That being said, the feedback loop is crucial, and we should consider how to minimize it.
00:12:33.840
One simple answer is to throw money at the problem. It's the easiest solution. Computers are cheap, but developers are expensive. Buy a better computer!
00:12:58.920
The second option is to run things locally. This is simple in theory but can be difficult in execution for many.
00:13:10.920
Third, get your results more quickly, though this is easier said than done. Each of these requires more effort than the previous one.
00:13:47.880
But when examining your current feedback loop, remember that people often think the emphasis is on input, but in reality, it’s on the output. So let’s evaluate an average developer's salary.
00:14:06.840
Consider that a senior developer earns approximately $5,000 a month, right? Those of you who disagree should ask for raises!
00:14:40.800
Let’s say you work 20 days a month and have one day off. You run, let’s say, five complete test runs daily, and you can shorten those runs by ten minutes if you invest in a new computer.
00:14:54.060
A new MacBook could cost around $1,500 or maybe $2,000. In that case, it would essentially pay for itself in three to four months.
00:15:12.960
Also, don’t forget that continuous integration is still running on a computer. It’s not magic; it’s a cloud server. It costs money, so please keep track of how many builds are backing up in the queue. Evaluate the various pull requests that would get redeployed into the CI, as well as how much more expensive a faster CI server might be. Generally, it’s not that costly.
00:15:25.020
You can operate your own CI these days, with possibilities like self-hosted runners from GitHub. Simply plug in your machine, install Docker, and you’re good to go! You should only pay for the machine, and the service can be free.
00:15:46.320
If you reduce build times by ten minutes, that’s a savings of around $500, potentially allowing for a very nice server!
00:16:03.240
Having covered the easiest solution — spending money — the next option is to run stuff locally.
00:16:18.959
By running things locally, you allow for prototyping. This is a key aspect of having a local setup.
00:16:44.760
Often, with new projects, I see teams focusing heavily on the need for third-party services and deciding they can't run things locally due to that dependency.
00:16:56.280
For this reason, it's valuable to mock external services — you should not rely on third-party services for testing.
00:17:07.620
Someone mentioned Auth0 in the previous session, and while you might only have a test instance, having something local to test against is far better.
00:17:34.680
Secondly, make use of a REPL (Read-Eval-Print Loop). This isn't just for special cases; it can be used to assess various usage scenarios. If you need information about performance for an index or some table, interact directly with the production environment.
00:17:51.960
While you should exercise caution when doing this, it's commonly feasible to perform performance testing in production, particularly because replicating local environments can be difficult.
00:18:09.960
The third point is to push code often and deploy to production. Avoid keeping it all in pull requests; no one pays attention to those while developing new features. However, they will pay attention to the master branch.
00:18:34.680
It's better to deploy potentially unfinished code behind feature flags; you'll be able to experiment and see the results without waiting for someone else to give you input.
00:18:53.040
Simplifying your setup is important. Keep it simple because all complex projects begin in simplicity. If your setup is straightforward, new developers won’t have to navigate unnecessary complications to test things locally.
00:19:11.040
Even if you don’t use Docker for your application, try to dockerize your dependencies for exotic databases. Include defaults for their setup as well.
00:19:28.320
Reducing configuration steps is crucial. If I come across a README that takes an hour to read and still deal with complexities for setup, I'm at a disadvantage.
00:19:47.880
When joining a new project, please make sure it's straightforward to set up. Providing seed data for all applications is essential; if I need to create a new account and consistently reset the database, I'll become frustrated with the process.
00:20:14.760
And if you're using Docker, manage your images effectively. Containers can become heavy, just like a whale lifting things. Keep an eye on the code within those containers.
00:20:38.820
Moving to the more challenging aspects — I don’t want to see test suites that take longer than 15 minutes to run. If they do, I might switch to TikTok or another distraction. I can't focus when waiting on output.
00:21:01.680
If I jump into a project with a slow test suite, the first step for me involves checking what's causing the slowness.
00:21:14.520
This is about avoiding wasted developer time; remember that computers are patient, but developers are not.
00:21:32.040
Therefore, if you have loops in your tests where each test is generated from within the loop, someone may inadvertently add another element. This leads to redundancy, creating yet another slow-running test.
00:21:47.400
Also, file loading issues can slow tests. I've encountered codebases with sizable image files that slow down processes when running through CI. You don’t want to upload large files because they impact performance.
00:22:02.520
People often overlook the impact of file loading; this becomes evident when running tests that handle hefty image files. Local machines, which have faster disks, won’t exhibit the same problems you encounter under CI.
00:22:26.760
This becomes even clearer once you try to load those files in CI after running tests on local, causing slowdowns.
00:22:46.440
Another common practice is to run your tests in parallel, with a gem that creates isolated environments for each process. These tools run several tests simultaneously and gather results, giving a unified test output.
00:23:09.360
You can use timings from previous test runs to split them up more evenly, which is great. However, if one test takes an excess of time — like 17 minutes — you'll have delays that don’t help your feedback loop.
00:23:32.760
It's crucial to identify long-running tests as they can easily wind up consuming a disproportionate amount of time during testing.
00:23:56.520
For example, if you have a request test calling an endpoint that takes time, waiting for a primary assertion to pass could lead to delays.
00:24:14.760
Instead, that's when you want to aggregate failures; if one fails, that's not the end, and you'd still want to see the reports for failures.
00:24:31.920
Tools like TestProf exist to help during testing trials. It helps identify slow tests better so you can focus on optimizing those. Factor Cascades, for example, will let you know precisely which area is causing slowdowns.
00:24:54.960
Reducing the complexity of interrelations in your database can lead to quicker loading times, allowing you better performance in test runs.
00:25:17.760
Moreover, if possible, avoid database interactions altogether. Create objects that build rather than test directly in the database. This eliminates unnecessary interactions that add time.
00:25:40.200
As a result, consider your strategy while building; it can almost entirely remove the bottleneck you've created from database connections.
00:26:02.640
I know it's not easy, but you can create a lot of optimized code by closely analyzing what tests run and what functions need genuine queries.
00:26:25.320
I want to ask you: what will be next on your feedback loop shortening list?
00:26:40.200
Are there any horror stories you want to share? If you can't run your code locally, or if your test suite takes over 30 minutes, please raise your hand.
00:26:57.560
Let's share experiences; perhaps it might help someone down the road.
00:27:11.760
An individual discussed being tied to the CI, but without running code locally, they found CI runs faster for them.
00:27:29.800
However, that doesn’t prevent the possibility of bugs related to CI processes. It's difficult to catch bugs in settings not experienced locally.
00:27:45.280
Have you encountered false negatives due to unsynchronized updates?
00:28:00.600
Yes, I often left typos, leading to CI failures; you realize it instantly.
00:28:16.840
Did anyone else have stories to share? I'm aware it can be embarrassing to admit experiences, particularly in client projects.
00:28:36.000
However, recognizing the problem is crucial for you and your organization. Document the time wasted dealing with inefficiencies.
00:28:48.120
An individual shared an experience about running code locally and managing an extensive test suite while maintaining it `
00:29:02.520
An individual relayed they switched CI, utilizing pay-per-click and saving significant money. You might have remarkable results!
00:29:17.880
As you encounter discrepancies, you will find solutions to those problems and stabilize your test run. Keeping CI tests light and vast will help keep it manageable.
00:30:26.920
Now, I’d like to reference a Ruby gem called 'Crystal Ball.' It takes the code coverage for each test, and if you modify a file, it will provide a list of tests crucial to your changes.
00:30:51.720
This is not a complete solution because it's essential to run full coverage from time to time, but it helps you shorten your feedback loop.
00:31:35.360
Another option is to integrate a tool called Guard; it can track which files change and only seem important to rerun. It can monitor chances to ensure you’re quickly getting information.
00:32:06.600
In addition to your existing feedback management queues, identifying and monitoring changes only seconds after they're made will help alleviate a lot of breakdown pressure.
00:32:35.760
This involves using fixtures creatively, allowing for dependencies minimization while tracking changes and maintaining dependability.
00:32:56.760
A spotlight on unnecessary complexity and pooling will always signal you to audit how many objects you’re manipulating in tests.
00:33:12.680
Optimizing test scenarios must involve figuring out performance and reliability in context by considering slowed callback designs and business logic dependencies.
00:33:38.520
Lots of workflows demonstrate time wasted due to complex cascade callbacks, which send companies into a sink.
00:33:58.680
Utilizing tools creatively while adjusting coding workflows cultivates more agility through leaner builds — distilling reinvented feedback cycles.
00:34:58.520
Overall, this sums up what we’ve been witnessing in software development over the past several decades, reflecting on how we play ping pong between shifting resources, encapsulations, and logical maintenance needs throughout.
00:35:30.000
Thank you very much for your time, and let’s keep these conversations going!