00:00:18.619
Welcome! We're going to talk about Rails in an era of big tech.
00:00:23.820
First, I want to get started by talking about light bulbs. In the early 1920s, light bulb engineers in Europe and the U.S. made great technical advances; they were able to produce light bulbs that lasted over 2,500 hours.
00:00:29.640
However, the owners of these companies noticed that as their technology improved and the lifespan of their light bulbs increased, their sales began to decline. They made less money. This was counterintuitive because you would expect that technological innovations should lead to greater profits.
00:00:42.660
The owners, however, didn’t think that was how it should work. They were on the verge of inventing planned obsolescence. In 1925, they established the Phoebus cartel, an organization of the leading light bulb producers. They literally tested the lifespan of light bulbs and fined companies for exceeding a lifespan of 1,000 hours per bulb.
00:01:00.960
This limit was far below the average their companies had achieved before then. By enforcing this limit, they were able to double their sales. But how long do you think this cartel lasted? Only 14 years, until 1939. Initially, they had plans to sustain the cartel until 1955, but the war disrupted their collaboration.
00:01:15.119
You might be wondering what this has to do with Rails. In her keynote, Eileen spoke about how much purpose she found in making the framework better and giving back to the community. This is because the core of Rails is about enabling innovation and the success of others.
00:01:28.619
This stands in stark contrast to the light bulb cartel. Rails is particularly beneficial for smaller companies that are just starting out and for developers early in their careers. The Rails community provides societal benefits by organizing industry developers and companies to utilize innovations to improve the world.
00:01:39.660
Unlike the light bulb cartel of the 20th century, we do not collaborate to plan obsolescence or sabotage our competitors. Instead, we share knowledge and help each other succeed.
00:01:53.880
But can the same be said for the big tech giants? As consumers, we often overlook their free-use model with products like Google and Facebook. As developers, we also experience the tech patterns they popularize. In this talk, I will urge you to be cautious of these patterns and to consider a few key arguments.
00:02:10.879
The first argument is that big tech architecture can jeopardize a lot of companies, especially smaller and medium-sized ones. A corollary to this is that organizational tech fit truly matters. Often, we tend to adopt patterns based on their popularity or computational elegance, but this isn't always a good fit for our organizations.
00:02:27.900
Next, I will discuss some novel dangers associated with big tech's market power. While monopolies have existed for centuries with an antitrust legacy in the U.S. and around the world, in the past 20 years, there has been a new kind of market power that utilizes cybernetics and digital tools.
00:02:39.660
This is something that we, as an industry, need to be aware of, as we have been participating in it to some degree. Finally, I will argue that Rails patterns promote greater diversity because they are friendly toward smaller companies and enable smaller teams to compete in ways that would otherwise be impossible.
00:02:53.820
About me: I'm Jordan Trevino, the founder of Telos Labs, and I am a software developer. However, I wasn’t always a Rails developer; I previously worked as a strategy consultant for large pharmaceutical companies, launching cancer therapies. This experience taught me to take a broader view that encompasses business, operations, and culture.
00:03:06.000
I am also an enthusiast of democracy and organizational diversity. As you'll be able to tell, I’m from Mexico. While Mexico is not a resource-poor country, the welfare of its population is relatively poor due to crony capitalism and a legacy of colonialism.
00:03:19.260
I believe that the U.S. is at risk of becoming like Mexico, rather than the other way around.
00:03:23.820
Now, as a first-time RailsConf speaker, I am very excited.
00:03:30.300
Talking about diversity, we typically hear this term in the context of demographics. However, I want to expand the definition. I believe that diverse organizational structures defend against autocratic power and provide employees with choices regarding where to work and collaborate.
00:03:36.300
This diversity preserves essential human values, traits, and attributes. The image you are looking at displays a vibrant, lush ecosystem, which emphasizes that diversity is indeed a tremendous asset that helps organizations survive.
00:03:44.400
Monocultures, or having a single crop, are detrimental to resilience when faced with pests or ecological threats. Therefore, diversity is crucial for allowing people to find the right company for them.
00:03:56.400
Today, we are facing significant challenges such as climate change, inequality, and political polarization. In this big tech era, these issues seem to be worsening because businesses as usual offer no solutions. This business-as-usual approach could have catastrophic consequences for our society.
00:04:12.420
We need to nurture diverse attributes that big tech giants or traditional profit-seeking corporations may not cultivate effectively. Organizational alternatives could include non-profits, worker cooperatives, public utilities, community-funded organizations, or even platforms like Wikipedia, which operates differently in the digital space.
00:04:26.640
Let’s get underway with our discussion by first addressing the decisions we face daily when writing web software.
00:04:34.200
Typically, our careers start on the right-hand side, focusing on features. As we gain more experience and as projects become complex, we move to consider broader architectural strategies.
00:04:44.640
Rails encompasses all these areas. However, there are other popular practices in the community that we must discuss, especially microservices and single-page applications.
00:04:56.820
There are two approaches to these questions. The first is computational thinking, which developers are comfortable with; this is what is taught in computer science curriculums.
00:05:10.440
Computational thinking focuses narrowly on the task at hand, using linear and reductive problem-solving methods. While it works for computer-based problems, it is limited and reductive. It does not consider factors outside that narrow, linear perspective.
00:05:32.220
Therefore, it should be complemented by systems thinking. Systems thinking takes a more expansive, multidisciplinary approach, allowing us to address customer needs in diverse and effective ways.
00:05:45.480
Without systems thinking, we may fall into the trap of premature optimization or be enchanted by computational elegance that may not fit our organization or solve our issues.
00:06:01.380
Now, let's talk about tech patterns and the buzz around them and how we relate to them in this community. About two-thirds of developers work in small to mid-tech companies, and of course, we love Rails.
00:06:13.280
It is powerful for small to mid-tech firms, including larger companies like Shopify and GitHub. However, the big tech companies like Google, Facebook, etc., are categorically different.
00:06:27.600
These companies operate at a completely different scale and have different organizational characteristics. However, they are so powerful in society that they have tremendous influence in tech media, effectively defining patterns that become popular in our Rails community.
00:06:41.220
While we may use Rails, we do not always apply the Rails-first patterns. Instead, we may opt for trending practices based on the allure of their perceived benefits.
00:06:56.460
In a survey of the Rails community from 2022, we see that most Rails developers are still primarily working on monoliths.
00:07:07.020
About a third are experimenting with microservices or service hybrids. Now, I want a show of hands: who here is working in either a microservice or a microservice hybrid environment?
00:07:19.740
Yes, it seems to be maybe a third. On the other hand, single-page apps are much more popular. I think it might be controversial to consider them as big tech patterns.
00:07:31.020
We see that about 40 to 50 percent of Rails projects are using some sort of single-page app front end. In this regard, I found the usage of Stimulus and Hotwire to be surprisingly high.
00:07:43.300
Again, let’s have a show of hands: who here is using a single-page app front end for their main project? It looks like about 40 or 50 percent.
00:07:55.620
Now that we’ve talked about some common architectural choices and how they may influence Rails-first patterns, let's examine how these patterns impact our organizations.
00:08:07.500
In addition to tech patterns, we face business constraints like budget and resources. External market constraints also affect how quickly we must develop certain features.
00:08:21.520
These constraints are typically beyond our control as developers. However, one variable we can influence is the technology we choose in this environment.
00:08:34.380
Think of a metaphor: trying to reach escape velocity. The resources we have available are like fuel, while market constraints represent the Earth's gravitational pull.
00:08:44.520
Both the fuel and gravitational forces are out of our control, but we control how we assemble our technical team to boost our company’s chances of success.
00:08:59.859
A lack of resources compared to need can hinder success. The choices we make about technology significantly influence our resource needs.
00:09:14.640
We can visualize this with an abstraction, where the left side shows resource levels (high vs low) and the bottom axis indicates product development velocity.
00:09:26.520
When there’s a high need for product development velocity, it typically indicates the company is a startup aiming for product-market fit.
00:09:38.880
Conversely, established companies like Google can afford a more static product velocity. The key point I want to highlight is that big tech patterns require higher levels of resourcing, no matter what level of product development velocity you are at.
00:09:52.080
These patterns demand more people, skill sets, servers, and structured development teams (front-end or back-end), while Rails typically requires less overall resources.
00:10:07.500
The main advantage of Rails is tied to its doctrine of convention over configuration and integrated front-end patterns through a monolithic approach.
00:10:20.940
Throughout this presentation, I will present arguments about how single-page app approaches and microservices are big tech patterns that may not be as useful as the tech news would like you to believe.
00:10:33.060
Let’s consider what happens if your organization adopts big tech patterns but experiences an unexpected resource constraint. Suppose your company’s valuations or funding opportunities change.
00:10:45.780
If your organization has a required level of development velocity that is beyond what it can now produce, you may find yourself in what I call the death zone.
00:10:57.300
In this area, the technology you depended on is no longer sufficient. Let’s examine the differences in this scenario from a big tech perspective versus a Rails-based company.
00:11:10.440
This perspective assumes that Rails-first patterns genuinely require a lower level of resources. If this assumption holds true, then a Rails-based organization can achieve greater velocity within the same resource constraints.
00:11:24.300
This effectively reduces the size of the death zone, thereby increasing the chances of the company's success. However, it’s important to consider how often we evaluate these dynamics.
00:11:37.080
For example, do we consider how resourcing or market conditions may change when influencing tech decisions at our companies? This is typically outside the domain of developers who get caught up in computational thinking.
00:11:50.880
I argue that Rails already incorporates this approach, and we benefit when we sustain these considerations.
00:12:05.220
Now, let's look at a case study. At Teleslabs, we recently assisted a client that built its product with a Rails back end and a single-page app front end.
00:12:17.640
This company had raised hundreds of millions of dollars and had around 500 employees with a technology team of about 50 to 75 individuals.
00:12:30.840
After about six years, their tech team grew frustrated with their stack. They encountered issues with their monolith that they felt they couldn't or didn't want to address.
00:12:44.220
During this time, there was significant team turnover alongside the popularization of single-page apps and the notion of 'JavaScript everywhere,' which pushed the team to think that migrating away from Rails was the ideal solution.
00:12:57.840
They then decided to transition to a microservices architecture. In retrospect, we can name this approach a "distributed ball of mud." The Rails API was on one side, a single-page app at the top, and they ended up creating nearly 20 microservices running on Node.
00:13:09.840
The single-page app unfortunately comprised over 300,000 lines of code, which connected to these various services. They faced duplicative data transformation logic, resulting in many interdependencies across the services, which rendered it ineffective.
00:13:25.620
Their problems manifested in extremely slow development timelines, making it difficult to estimate features accurately. Despite estimates from the development team, it often took three times longer than anticipated.
00:13:39.240
Reliability issues arose due to the complex DevOps situation, including maintaining 20 microservices across multiple environments. Every outage was akin to a murder mystery, as they attempted to identify the malfunctioning service.
00:13:55.500
From a systems thinking perspective, experts in microservices had exited the company, yet they required significant product enhancements for growth. The company’s cash burn rate was high, and they were at risk of running out of funds.
00:14:10.020
Their customer base centered around enterprise clients numbering in the thousands—scaling to millions of users was unfeasible for them. So why did they choose this pattern?
00:14:23.220
The client was looking to accelerate development velocity, requiring us to build new processes that would allow for accurate feature estimates while also delivering a significant new reporting feature set in under six months. Thus, it was a complicated and urgent challenge.
00:14:37.620
Our first step was to conduct comprehensive learning sessions, working through technology audits and assessing requisite processes for the new feature. It was essential to engage with team leaders and consider the system perspective.
00:14:52.680
As we delved deeper, it became clear that the team wouldn't be able to deliver the new functionalities using the existing patterns due to their complexity. The experts who had built these patterns had left, making it challenging to navigate such a complicated domain.
00:15:09.900
It was evident to me that we needed to transform the organization's way of working, focusing on two main areas: first, how should we handle the complex single-page app that had been part of their structure from the beginning?
00:15:21.840
Second, how should we handle the data layer, which led to the options of either business as usual or a transition to a newer framework? Perhaps the challenge arose simply because they had chosen the wrong single-page app framework.
00:15:33.240
Conversely, they thought we might suggest revamping their Node services in line with their customer needs. Other teams were already working on overhauling those Node services, reflecting a crisis situation.
00:15:46.860
Referring back to the diagram from earlier, the green dot on the big tech pattern could represent where the company was performing. They had a specific level of high resources and lower product development velocity.
00:16:01.920
Now recognizing their need to improve velocity, they faced dwindling resources due to a high turnover rate and financial constraints. The only way forward, in my assessment, was to leap from the blue line to the red line.
00:16:15.840
If we could transition this client to a Rails-based solution, we could enable them to enhance their product velocity at a lower cost, highlighting Rails’s amazing power.
00:16:30.960
Rails not only yields more cost-effective outcomes than big tech patterns, but it can provide significantly higher product velocity for equivalent resources—at least that was my belief at the time.
00:16:47.520
Our recommendations centered on two primary strategies: transitioning to a server-rendered front end and restoring the monolithic structure, both of which were rather controversial from the client’s viewpoint.
00:17:02.280
The most contentious piece involved shifting to a server-side rendered front end. The previous diagram comes from Sam Stevenson’s presentation at RailsConf 2016 and illustrates the complexity inherent in a conventional single-page app configuration.
00:17:15.000
While it demonstrates two applications interacting—the initial load of an HTML page followed by a JavaScript request to a JSON API—multiple points of complexity arise.
00:17:28.620
In contrast, the multi-page app/server-side rendered pattern simplifies the structure, loading a straightforward HTML page initially before making requests to obtain new content.
00:17:41.520
I supported this shift, believing it would lead to simpler execution flows, making it easier for this company to utilize full-stack patterns instead of separating development teams.
00:17:57.000
This streamlining would minimize costs and resources by eliminating front-end state concerns while maintaining the swift user experience they desired.
00:18:11.760
Moreover, having strong domain models allowed for a clearer development approach. We initially thought Turbo was the best route; however, it didn’t have Internet Explorer support, which was a requirement for the client.
00:18:25.560
Thus, Turbo Links became our focus. The challenge was showing new content rendered from a monolith in a complex single-page application environment.
00:18:39.300
The eventual solution we implemented involved integrating an iframe to present the new server-side rendered front end into the single-page app. Upon clicking a link for the new feature, the content would render in a conventional Rails style.
00:18:53.820
This rendered content would be framed as a single-page app thanks to Turbo Links. Additionally, we could scope authentication to the user at the Rails side by passing their token as a query parameter.
00:19:09.540
However, authentication posed challenges, as did instances where the iframe had to communicate back to the parent single-page app.
00:19:22.680
To address this, we utilized postMessage communication alongside a library called Penpal, which facilitated message exchanges between the two environments.
00:19:35.460
Another technical hurdle involved the rapid provision of new reporting functionality, relying on data access from 12 different services. Comparing the previous single-page app with the new monolithic architecture illustrates this challenge.
00:19:49.920
On the left, we see a classic monolith with a single database structure. On the right, the new microservices pattern had multiple services, each managing its own databases.
00:20:02.520
As we contemplated our options, we debated whether to connect to the API layer or bypass it. The first option, while straightforward, complicated our Rails monolith.
00:20:15.180
The goal was to maintain the advantages of a classic Rails application while accessing various services. We couldn’t consolidate everything into a single database; instead, the solution involved integrating with multiple databases.
00:20:29.160
Restoring a traditional Rails monolith while managing multiple databases allowed us to tap into ActiveRecord's power.
00:20:42.420
A significant aspect of recreating a Rails monolith from a myriad of microservices is the chance to instill a domain-driven design, which can improve overall coherence in application construction.
00:20:55.740
A common challenge in microservices is generated complexity, especially if the services maintain bizarre naming conventions that don’t correlate with their utility.
00:21:09.060
By employing a Rails-driven methodology and creating understandable model names, we enhanced clarity not only for the current reporting feature but also for future feature development.
00:21:25.200
Ensuring that our approach allowed for easy access to the necessary domain information through ActiveRecord was key, despite managing multiple databases.
00:21:38.760
When comparing operational benefits of integrated patterns through full-stack development, we recognized that single-page applications commonly necessitate division among development teams, creating communication obstacles.
00:21:52.620
This fragmentation complicates resources management and is prone to errors. Full-stack development mitigates this complexity, ultimately improving project velocity.
00:22:06.420
This discovery, coupled with our previous efforts, afforded us the ability to create Rails teams, addressing the no-longer-feasible separate development team model.
00:22:20.460
Eventually, the company successfully went public. It was a remarkable instance showcasing the strengths offered by Rails.
00:22:34.020
Elaborating further, let’s tackle the concept of big tech and its implications. With concentration among significant players like Google or Meta, the enormity between them and a typical large Rails user, such as Shopify, is evident.
00:22:48.420
The disparity in lifetime company profits demonstrates the scale difference, where big tech giants operate with completely different operational characteristics.
00:23:02.580
These large companies have billions of users and hundreds of thousands of employees, with their sizeable cash reserves minimizing financial constraints. Instead, their main hurdles often relate to regulation, internal communications, or economic limitations.
00:23:16.260
Given their internal structures, managing operations efficiently becomes exceedingly challenging. Nevertheless, we analyze these companies and tend to assume that their success can be attributed to their adopted practices.
00:23:30.060
This assumption posits that if we utilize microservices or single-page apps, success is guaranteed. Contrarily, the earlier-mentioned resource constraints can jeopardize smaller companies.
00:23:44.460
Observing Google's growth trajectory illuminates that their embracing of microservices was a product of their existing success rather than a determinant of it.
00:23:58.740
In actuality, many Rails developers work in small to medium tech firms, typically comprising fewer than 50 employees, and even flourishing Rails companies tend to have far fewer than 100,000 staff.
00:24:11.940
It's crucial to understand market power's implications, as monopolies have existed for centuries, shaping economies and affecting supplier dynamics.
00:24:26.520
Yet, new dangers arise from these massive digital platforms, where the control over perception is significant. Social media giants may not provide identical content experiences for users, raising questions about trust.
00:24:38.520
Such technological practices primarily serve to modify consumer behavior for ad sales, but they also pose threats related to political control or economic dominance.
00:24:53.520
The danger extends into a realm termed techno-feudalism—where products are perceived as free, yet the underlying ownership dynamics mirror exploitative historical systems.
00:25:07.380
The primary take-home from this is that when engaging with big tech, companies must understand that these patterns may not fit their needs. Big tech patterns are inherently correlated with a scale that doesn’t always apply to smaller entities.
00:25:21.120
In closing, while the adoption of big tech patterns is tempting, embracing Rails-first practices is beneficial to your organization as a general approach.
00:25:35.280
Recognizing the tech architecture that you choose to implement must be aligned with your organization’s needs will be crucial for success.
00:25:49.620
This alignment is critical as you progress in your career, whether by leading teams or a technology group. As developers, focusing on computational thinking and trending tech can distract from the bigger picture.
00:26:03.000
Instead, prioritize understanding the organizational dynamics within which you operate, as Rails fosters the community's success. Thank you for helping make the Rails community such a fantastic place.