00:00:16.720
Hi everyone, thank you so much for coming. The title of my talk is very evocative.
00:00:22.680
My name is Philip Mosa, but you can find me on various social media as Phil MV, as well as on Twitter.
00:00:28.640
I am one of the co-founders of Ruby.com, where we host the Ruby Security Advisory Database. If there's anyone in the audience who maintains gems, I would love to talk to you afterwards about how you handle security disclosures.
00:00:42.280
For my day job, I run a startup called App Canary. App Canary notifies you whenever you are running a vulnerable package in your servers or applications. We spend a lot of time thinking about whether you are running vulnerable software.
00:00:53.559
This talk is ultimately about a series of technical decisions we made while building the service at App Canary. To best understand our thought process, we should start by recounting the beginning of our incredible journey.
00:01:12.360
Back in 2012, I began a consultancy with a friend of mine, Max. We specialized in penetration tests, security code audits, MVPs, business automation, and advising teams on how to improve their software development processes.
00:01:29.000
One benefit of being a consultant is having a lot of control over your schedule. By November 2014, I had just spent two months working 12-hour days on the Toronto mayoral elections, and my co-founder Max had just enjoyed a magical summer at the Recurse Center.
00:01:40.920
As a result, I was really burnt out, while Max was looking for a challenge. We stumbled upon a market opportunity that could truly use our skills, and we decided to build a product.
00:01:53.680
We had some experience in this problem domain as we had previously built a free service called Gem Canary a few years earlier. To put it simply, an advisory may have many vulnerabilities, a vulnerability can exist in many packages, and each package can have multiple versions. We needed to keep track of all of this to inform users whether they were running a version that should not be in use.
00:02:22.840
As we looked around, Max suggested we use a system called Datomic. If you've never heard of Datomic, it's a fascinating key-value graph database where, instead of SQL, you write data logs, which is a kind of programming language.
00:02:35.239
On top of this, Datomic provides a free point-in-time database feature, meaning you can revert to any previous state of your data. Essentially, nothing ever gets deleted, making it a really cool tool to work with.
00:02:54.879
Given our struggles with the free service we built, we thought this technology might be really useful. Unfortunately, all the client libraries for Datomic had issues, but it worked remarkably well with Clojure, so we decided to use Clojure.
00:03:14.319
For those of you unfamiliar, Clojure is a functional programming language that features immutable data structures, providing you with asynchronous primitives for free. It runs on the JVM, which is touted to be web-scale.
00:03:39.519
Isn't it about time we all learned a Lisp? Shouldn't Lisp be this profound, enlightening experience that alters how you program for the rest of your life? Personally, I had been writing Ruby for five years, having built a dozen apps. All these reasons were compelling, but I was also driven by fear: was I stagnating?
00:04:06.480
When I started with Ruby, it represented cutting-edge web tech, but I worried it wasn't the same anymore. What happens to programmers at the age of 35? Do they vanish? I didn't want to spend the rest of my career writing JavaScript. The idea of investing in a new technology to expand my skills was appealing. So we thought, why not just go for it? We were going to run out of money in six months anyway.
00:04:32.280
We built it in Clojure, and it was a fantastic experience. For those already familiar with Clojure, I may be using the incorrect terminology, so please bear with me. Both Clojure and Ruby handle truthy values where things are either nil or false, and they both spend a lot of time dealing with hashes of symbols.
00:05:03.039
They both incorporate fun metaprogramming constructs. Now, why is Clojure appealing? The primary reason is functional programming. In Clojure, functions are the basic semantic building blocks. You're encouraged to encapsulate behavior into the smallest practical units.
00:05:21.360
In this example, we are filtering out all the even integers from an array by applying an even function. If you look closely, you might find it somewhat familiar. There's an old joke that Ruby is an acceptable Lisp, and a 2005 blog post supports this argument.
00:05:41.199
However, comparing the two, the Clojure code exhibits the more flexible behavior: the filter function operates without needing to know the specifics of the data structures. This shows the power of Clojure's functional capabilities—behavior can be combined without being tied to the properties of the numbers or items in the array.
00:06:19.520
When everything is a function, it often gives you the freedom to isolate and change behaviors without affecting the original structure. This greatly simplifies the development process. Another notable feature is immutability, which historically was quite costly until breakthroughs in 2002 made it feasible to include in common programming.
00:06:38.240
Immutability means that the state cannot change; any insert, update, or deletion results in a new structure. This is crucial since mutation can lead to complex behaviors in your code that can produce unforeseen bugs. In a language like Ruby, there's always uncertainty about whether an operation will mutate or copy an object.
00:07:24.919
For instance, if you have a list of strings and you assign it to another variable, the results of appending new values can be tricky to predict. Ruby developers deal with pass-by-reference or pass-by-value concepts, which complicates object mutations considerably.
00:07:51.040
The only way to ensure that objects are not modified is to create extensive workarounds in your code, which leads me to the conclusion that Ruby's mutability complicates things unnecessarily. In contrast, with immutability, the flow of state through your application becomes clearer and more predictable.
00:08:10.480
Take an example from an old app of ours: a route that processes a user and a database. The way my code is indented clearly shows what cannot affect the variables outside their lexical scope. The assurance that outer variables cannot be tampered with allows me to feel confident about code integrity.
00:08:39.840
On a side note, let me illustrate why Datomic is appealing. Let's say your boss comes up to you asking for a report based on last year's data. In traditional databases, depending on your design, it can be a nightmare to execute. With Datomic, I can grab the database’s previous state and rerun my report, and it works seamlessly.
00:08:59.920
Despite our enthusiasm for this new technology, I found myself reluctant to dive into user authentication and cookie management anew when we already had this experience building websites. So we built an API that communicates with Datomic, along with a Rails frontend for mundane tasks like user management and payment processing.
00:09:38.640
We began working on this in February 2015, and by May, to our astonishment, we got accepted into Y Combinator. This eased our financial concerns, allowing us to release our product into production by July. After moving backs to Toronto in October, we worked tirelessly to add features and address the actual demands of paying customers.
00:10:37.360
However, we soon encountered significant issues. We made poor architectural decisions, underestimated our domain's complexity, and faced separate deployments, introducing unnecessary overhead. We ended up spending an exorbitant amount of time fixing simple issues, resulting in a death by a thousand paper cuts.
00:11:30.960
To give you an idea, structuring large apps in Clojure can be less intuitive compared to Rails, which provides more hand-holding. While coding in Clojure is fun and often makes you feel clever, it can also result in code that is hard to read due to excessive expression and various subtleties of the language.
00:12:23.440
It also becomes challenging to pinpoint errors in stack traces when using anonymous functions, potentially leading to complicated debugging experiences. Documentation is often less accessible, and the JVM overhead complicates rapid development.
00:12:32.879
My application's slow boot time added more layers of frustration, as deep integration became necessary. The requirement of packaging all dependencies into one single object meant managing significant payloads, which created hurdles for minor fixes during deployments.
00:12:59.280
With all that being said, I have a natural fluency in Ruby that I unfortunately lacked in Clojure, leading to considerable frustration. A comment I found on Hacker News resonated with me, describing the Clojure ecosystem as user-hostile. It became clear that developer happiness is not a priority in that community.
00:13:46.960
Fast forward to May of this year; we recognized the need to refactor our application. After discussing it with friends, I was urged to consider a complete rewrite. This push became a pivotal moment, making it difficult to dismiss such an option any longer.
00:14:17.200
However, we knew that rewrites are fraught with risk. If successful, they often go unnoticed, but we were tired of fighting the ecosystem. We aimed to eliminate unnecessary components to streamline operations and refocus on what truly matters.
00:14:58.360
We built an API, and the process involved reviewing each step meticulously, resulting in complex workflows that ultimately made it impractical for our small team to manage efficiently. This led to unforeseen complications, with microservices creating a distributed system we hadn't intended, ultimately causing more work and less efficiency.
00:15:47.440
Dave McKinley, an early guide at Etsy, suggests focusing on a few hard things at once that truly drive a business. This lesson had a profound impact when realizing we were spending effort on building an object relational mapper for Datomic—something that offered little value to our clients.
00:16:31.880
Reducing risk is vital, especially in the startup world where every second and resource counts. As a small operation, we had the freedom to stall, which isn't possible for larger companies dealing with looming deadlines and customer pressures.
00:17:06.000
By mid-October this year, we had 8,300 lines of code in Clojure. The next day, after the rewrite, we had 8,700 lines of Ruby. The rewrite process took four months—a vast improvement when compared to the year and a half it took the first time when we were inexperienced.
00:17:29.080
While it was demoralizing to work on a system you cannot release, when the product finally shipped, it felt immensely relieving. Turning our application into one that leverages PostgreSQL's strengths made it more effective and performant. Generally, our application is now anywhere from 50% to 100% faster, a testament to the effort we put into the new data model.
00:18:29.600
Reflecting on the experience, I realized a sub-genre of technical blog posts yelling, 'Look, we rewrote our app from shiny new tech, and everything is faster!' If you didn't learn from your first attempt and improve performance, what was the purpose behind your efforts?
00:19:09.680
Returning to Ruby, I found that my approach had shifted dramatically during my time with Clojure. The immersion in functional programming crystallized thoughts I previously had about Ruby. The evolution in my understanding of Rails highlighted the importance of developing fat models and lean controllers.
00:19:44.560
I embraced the idea of creating value objects as well as applying immutability where possible. This practice of introducing value objects helps streamline the management of state within Ruby, ensuring that objects intended to represent states are immutable. By making this practice a habit, I found that my code became clearer and easier to manage.
00:20:57.519
Moreover, I've become far more aware of state flow through my applications. This also led me to create distinct manager objects that separate concerns, alleviating the role of controllers and models while maintaining clarity around object state.
00:21:43.680
To encapsulate my experience succinctly, I urge you all to dive into Clojure. While it may not be the most transcendental experience, the knowledge gained from the language can be beneficial. It's worth embracing its lessons even if my current focus veers toward Ruby.
00:22:20.039
When considering new tech, avoid building distributed systems as long as feasible. Embracing a monolithic architecture can simplify the development process significantly until your application demands scaling.
00:23:03.320
Before you reach for a new tool, exhaust use of those you already know well. These familiar tools can often save time and frustration when deadlines loom. Lastly, the experience I derive from writing code must be enjoyable and fulfilling, not solely the result of cleverness but genuinely rewarding.
00:23:43.629
In closing, cherish what you have within the Ruby ecosystem, and bring those values to any new communities you engage with. This is the journey and story of how we built a service that helps notify users about vulnerable software.
00:24:09.780
My name is Philip Mosa, and thank you for listening!
