Services and Rails: The Shit They Don't Tell You

Talks

Services and Rails: The Shit They Don't Tell You

Brian Morton

@bmorton

#service-oriented-architecture

#cross-functional-teams

Services and Rails: The Shit They Don't Tell You

by Brian Morton

In this talk titled "Services and Rails: The Shit They Don't Tell You," Brian Morton from Yammer discusses the complexities of building and integrating services into Rails applications. He highlights that while smaller, encapsulated services can improve scalability and maintainability, they also introduce significant challenges. The key points from the discussion include:

Recognizing Early Complexity: Building services should be approached with caution, especially during the early stages of a startup, to avoid unnecessary complexity.
Service-Oriented Architecture: Morton advocates for a Service-Oriented Architecture (SOA) that allows for independent scaling and component reusability.
Examples from Yammer: He shares examples from Yammer’s experience, such as the search interface built using components like Flattery and Completey, illustrating how separation of components has facilitated easier integration in their infrastructure.
Development and Team Structure: Morton discusses the importance of a well-structured team to support service development, referencing Conway's Law, which states that the structure of a company influences the design of its systems. The shift to cross-functional teams has helped keep communication fluid and development agile.
Challenges and Trade-Offs: The transition to distributed services is filled with challenges, requiring teams to adapt continuously and manage technical debts. The need for robust monitoring, alerting, and deployment processes is emphasized, with tools like Dropwizard being cited as beneficial to streamline service management.
Conclusion: Morton stresses the importance of embracing complexity and being prepared for challenges in service development, advocating for a culture that supports learning and adaptation. Overall, the journey of migrating to service-oriented architectures is a balance of navigating new complexities while maintaining the agility of development.

00:00:20.480 My talk, as you said, is called "Services and Rails: The Shit They Don't Tell You." Today, we're going to discuss how Yammer builds services alongside Rails and explore some things that are often overlooked.

00:00:25.680 This is me—my name is Brian, and I work on the Rails team at Yammer. One of the things that the Rails team does is help extract chunks of functionality from our Rails codebase and integrate those into services.

00:00:32.640 I also love Zelda, music, Ruby, and, of course, Yammer. Now, maybe I can talk closer to the microphone; does that help?

00:00:57.280 Check... good? Awesome! So, first, I have a little pet peeve about the titles of talks, and I want to address that. I don't like when talks have a title that doesn't relate to the content, so I'm keen to ensure my talk includes the 'they don't tell you' gimmick.

00:01:09.760 As mentioned earlier, this is my first conference talk. Just a week ago, I was nervous but excited about presenting. While preparing, I thought back to the last time I had spoken in front of a class and recalled that one important aspect back then was choosing the right template for my talk.

00:01:32.720 Now, Yammer is part of Microsoft, which provides us with many templates. I found a slide in one of them that said, 'Full bleed photos can set a mood or evoke emotion, making for a more memorable presentation.' I thought this was valuable, so I included it in my presentation. This slide allowed me to connect my pet peeve about talk titles to our discussion.

00:01:57.360 It illustrates that every time there's a point we don't often discuss. This character will frequently be on screen to help highlight those moments.

00:02:30.800 Now, regarding this topic: This might not apply to you yet. However, if you are building a startup to determine viability, you might want to push some of this aside and focus on getting things done. Introducing a lot of complexity can be counterproductive at this stage.

00:02:41.280 Nonetheless, it doesn't mean you shouldn't write clean, well-designed code. It's just that, at this point, focusing on services might be a distraction. If you do have clean code, extracting it into services will be much easier.

00:02:52.959 As you begin to build services and scale, you must do some things that may feel uncomfortable.

00:03:04.720 At Yammer, we have a massive Rails application with over 300 models, some of which are substantial. Additionally, we have more than 200 controllers and significant content in our lib directory. Our system is backed up by over 20 JVM services we have built, with some of these services handling over a billion requests daily. Thus, we maintain a pretty substantial ecosystem.

00:03:21.599 However, we still deal with this massive Rails app, and the challenges intensify as we proceed. For a long time, this structure has worked for us, and we've managed to chip away at the problem by progressively building services, although it often feels painful.

00:03:39.520 However, tasks like sharding or updating Rails or Ruby turn into all-or-nothing projects, which can be quite difficult.

00:03:52.720 In this talk, we will discuss Service-Oriented Architectures: why we build services and what benefits they provide us.

00:04:03.439 One of our fundamental goals is to develop components that can scale independently. When you have small, focused services, they are more versatile and enable easier reusability.

00:04:16.799 Let’s talk more about reusability. We have our Rails app and several other services in our infrastructure. This example is based on some of our prior experience with search stacks. We were able to separate different components of our search stack.

00:04:28.560 In the middle is Flattery, which serves as a denormalized data store in Rails. We have hooks in Rails that publish data into Flattery, allowing us to store a denormalized representation. Dexi, depicted to the right, is our service that builds Lucid indexes from transaction streams sent from Flattery.

00:04:51.681 Ultimately, we built our search interface in this manner. When we wanted to incorporate a new search service into the stack, it was relatively easy for us. We developed an autocomplete service called Completey, which integrated directly into the existing pipeline.

00:05:14.560 We already had the necessary components in place and clean interfaces defined. The same principle applies to exporting data, where we have a service called Slurpee. Part of this pipeline already existed, and we didn't have to pull data from Rails again.

00:05:28.960 That's fantastic because now we have many independent components that can scale individually. This setup allows us to better understand each service's specific needs and performance patterns.

00:05:41.280 We can assess these metrics and avoid unnecessary resource allocation across the entire stack.

00:05:59.440 This flexible architecture is enabled by the loose coupling of components. We have encapsulated many of our concerns within smaller, more focused services.

00:06:10.160 We can independently push updates to each service without necessitating large-scale deployments. The Rails app still demands mass deployments; however, we can release smaller pieces of infrastructure in various ways.

00:06:27.360 Most importantly, we can swap out infrastructure components without needing to notify anyone, as long as we maintain consistent interface standards.

00:06:39.360 Currently, we are in the process of changing our files backend. It's been somewhat painful, requiring more adjustments than anticipated, likely due to some initial missteps.

00:06:52.080 Largely, we have the ability to replace this entire service with a new one.

00:07:10.560 The second goal of adopting service-oriented architectures is to create maintainable codebases across the organization. If you've worked with a monolithic application, you have probably encountered challenges like stepping on each other's toes or needing to retain vast amounts of application knowledge.

00:07:27.200 With services, you navigate through numerous 'black boxes' dedicated to specific tasks. You can learn about these services as required on a case-by-case basis.

00:07:45.440 This advancement gives us a concept of distributed execution enabled by the loose coupling of services.

00:08:00.320 This division isn't just in a computational sense but also applies to development. We split our codebases, which makes it easier to appoint a team to a specific service. That team can coordinate and establish how these components will interact with each other.

00:08:15.680 We often create dummy components at the initial stages, integrating some data on the Rails side while developing a service that accepts this data without yet performing any operations.

00:08:34.080 When the project progresses and approaches completion, we establish a full end-to-end test. This allows team members to unlock each other and incrementally develop the service.

00:08:51.440 Our discussion may seem complex; the journey of transitioning from a single codebase to distributed services is filled with challenges.

00:09:03.280 When you're an early-stage startup, managing this level of complexity can hinder your ability to ship products quickly. With a unified codebase, you have the freedom to implement changes swiftly.

00:09:17.760 You can interact with your data layers directly, share code easily, and evade some overhead associated with managing services. Although some challenges may resurface later, swift progress is vital.

00:09:32.560 We discovered at Yammer that moving to this distributed structure often requires organizational change.

00:09:46.560 At Yammer, we frequently discuss Conway's Law, which states that organizations designed to avoid bottlenecks will produce systems that reflect the communication structures of those organizations.

00:10:07.360 It's important to consider how our development teams are structured when contemplating services. Many organizations divide their departments vertically or horizontally, leading to silos.

00:10:19.920 This siloing can result in team members becoming overly attached to their responsibilities, which ultimately inhibits communication and decision-making.

00:10:34.080 In the early days at Yammer, our messaging team managed the service that handled all message feeds. The team was responsible for both the service and the Rails side of the implementation.

00:10:51.440 They decided on the interface and how to implement it, leading to siloed knowledge about the entire system—something that wasn’t necessarily the best use of their time.

00:11:07.200 As we considered scaling to accommodate future needs, we had to determine how to strategically manage feature teams.

00:11:24.320 To improve our organizational structure, we adopted a new model. This is what we have now: a Rails team and a Core Services team.

00:11:39.680 The Rails team is predominantly responsible for developments in the Rails app while the Core Services team handles service-oriented tasks. All codebases at Yammer are open, so being on the Rails team doesn't limit you to writing Ruby code.

00:11:56.320 While the Rails team has to understand the monolithic Rails application, moving elements into services helps reduce dependencies on that knowledge.

00:12:07.680 We also introduced the idea of cross-functional teams. When we begin developing a new service or feature, we assemble a cross-functional team with representatives from all relevant departments.

00:12:23.760 These groups usually comprise two to ten members from various functional teams, who work on a project together for two to ten weeks. This approach ensures that team members constantly work with new people on diverse projects.

00:12:40.800 This method resonates with Sarah's point about diversity in project teams. Although we don’t do consulting work, our domain remains dynamic, and we face numerous problems simultaneously.

00:12:56.960 From infrastructure projects to developer tools and core product features, we also handle backend tasks related to tech debt and service extraction.

00:13:10.880 Our analytics team builds tools for the data pipeline, and these tools and features sometimes circle back to product engineering.

00:13:26.240 While we have functional teams, we essentially have a pool of engineers who can work on a wide range of challenges.

00:13:39.120 An example of a cross-functional team would include two Rails engineers, a core services engineer, a mobile client engineer, and various other contributors, depending on the project needs.

00:13:52.320 This setup promotes a decentralized design process with autonomy, leading to well-crafted, isolated, and reusable systems.

00:14:05.120 These teams are ephemeral; they disband after completing their project and transition to new endeavors.

00:14:19.680 Another interesting feature is that any one of the team members can serve as the tech lead for a specific project.

00:14:31.200 Rather than a perpetual manager overseeing these teams, leadership roles shift with every project, allowing individuals to contribute code and gain diverse experience.

00:14:49.760 Once teams are assembled, they delve into their designated domains, leveraging distributed execution to accomplish their tasks. They coordinate on the API agreements between services and clients.

00:15:03.040 As a result, we regularly form service-oriented collaborations when these cross-functional teams convene.

00:15:20.160 It's important to note that while this model offers advantages, it also presents trade-offs. We have encountered some challenges but have generally managed them well.

00:15:39.920 One potential drawback of not having specialized experts across the application might seem negative, although some might argue it provides flexibility.

00:15:55.520 However, there are costs associated with having teams constantly learning new domains.

00:16:12.320 Another emerging issue we’re discussing more frequently is the risk of tightly coupling client API implementations, as our mobile clients demand increased customization of data.

00:16:27.760 It's crucial we remain vigilant regarding feature-oriented project designs to ensure we keep our systems decoupled.

00:16:42.080 Following project completion, we still need to support those features. Once teams disband, shipped products may encounter bugs, which we need to address.

00:16:59.520 We counter this through the formation of support engineering cross-functional teams, which regroup for two to ten weeks to tackle these as-needed issues.

00:17:14.720 This process can be a bit more challenging due to the unfamiliarity with the domain and codes written by the original teams. But it's a trade-off we accept.

00:17:29.280 There isn’t just one method to pursue this strategy; it may look different based on specific needs and circumstances.

00:17:44.320 It may still resemble the structure of a Rails project for the foreseeable future. We modified our organizational structure but continue to grapple with challenging problems.

00:18:01.680 A common temporary solution is routing traffic through Rails instead of allowing direct communication with services. This yields some practical benefits, but it also forces us to rely on Rails resources to access service functionality.

00:18:18.080 Eventually, we aim to permit browsers to communicate directly with services, but when pursuing this route, we must also consider issues like authentication.

00:18:34.320 Problems can arise when we try to implement this model, and navigating these challenges appropriately becomes critical.

00:18:51.680 Users often look for efficient authentication solutions when they wish to bypass Rails to access data stored behind it.

00:19:06.960 Ensuring seamless connection with databases is essential, although writing to those databases can create complications.

00:19:23.120 Active Record is a powerful tool with many conveniences, but we face complexities when we try to extract data.

00:19:30.080 Detangling data from services comes with its own unique obstacles, especially when we deal with callbacks, validations, and state machines.

00:19:46.720 We often find ourselves making decisions that involve consulting Active Record to access the original data.

00:20:01.120 One potential route is to minimize reliance on Active Record altogether. Some companies take this approach; however, at Yammer, we find it incredibly useful.

00:20:18.560 Often, we use our services as indexes, stored with IDs or relationships.

00:20:37.680 In practice, that means our services build the index structure, although we still need to fetch original data from Rails. Rails management is necessary to successfully retrieve and handle that data.

00:20:52.640 Recently, we've considered shifting more ownership of data to services, referring to these as "bodega services," where you rely on a focal point for certain types of data.

00:21:10.960 We aim to streamline operations by trying to ensure that these services perform as quickly as accessing data from memcache. This goal is inherently challenging.

00:21:30.240 When we talk about moving data, we often find ourselves duplicating information, which can present logistical challenges.

00:21:46.640 Chances are, if you're like us, you can't afford downtime when moving data. If a service doesn't perform as expected, we face difficulty in rolling back changes.

00:22:06.080 That's where having a backup plan becomes critical. We pivot towards 'double dispatching' to backfill all the data to services.

00:22:24.720 We simultaneously write to the database while posting data concurrently to the service. During this process, we monitor and profile the service's performance.

00:22:41.440 This allows us to incrementally move traffic to the new service while ensuring relevant safety measures are in place. We have an exit strategy available if necessary.

00:22:59.040 We will be introducing a significant amount of new data input to the service, which often can happen at a much faster rate than normal service operations.

00:23:15.200 This forward-thinking approach enables us to build capacity and anticipate how services will respond.

00:23:30.080 Once we initiate this process, we encounter another challenge: duplicated data.

00:23:45.440 We need to address how to manage and clean up this duplication swiftly to avoid confusing our developers.

00:24:01.440 At times, we fall short in cleaning up data promptly, leading to confusion—not an ideal situation.

00:24:15.760 Fundamentally, we must accept that we need to be willing to fully commit, which can be hard, as comfort with the old way might tempt us.

00:24:29.920 Staying in your comfort zone is not always an option; there needs to be readiness to confront new problems.

00:24:43.680 Understandably, you'd want to have a solid backup plan. Still, at some point, you need to choose to make the transition. The cost of maintaining a temporary strategy can drain resources.

00:25:00.160 Leaving your comfort zone and embracing the complexity of new systems is imperative.

00:25:14.560 For developers to succeed, they need a solid story regarding their development environment. If your environment is cumbersome, developers are likely to revert to comfort zones.

00:25:36.960 So, my big recommendation is to use Vagrant. It’s been a great asset in structuring our development environment.

00:25:50.640 We strive to keep the environment as similar to production as possible, running Ubuntu on both ends. This replica allows us to operate all services locally.

00:26:04.400 However, as our number of services grows, we are encountering issues due to resource limits, especially since our laptops typically support only 16GB of RAM.

00:26:18.160 We also need to keep up with rapidly changing services. Developers have to stay updated on these matters.

00:26:36.960 We built a tool, running inside Vagrant, called Soup Kitchen. It’s designed to help developers update services efficiently.

00:26:53.760 It allows us to manage service updates seamlessly, so the main focus remains on development quality rather than constant maintenance.

00:27:05.360 Furthermore, we have to consider the deployment of these services. It's crucial that we have a process in place for adding new services effectively.

00:27:20.400 We need a system that manages efficient deployments and provides stable, pre-released packages supported by ongoing development.

00:27:36.000 As we manage numerous applications, it’s essential to ensure every engineer can deploy their services effortlessly.

00:27:52.240 We created a one-click deployment tool named Diplomacy, which unfortunately is not yet open-sourced but allows engineers to add new services without complicated processes.

00:28:10.080 The other aspect we need to be acutely aware of is monitoring and alerting systems for our increasing number of services.

00:28:24.720 Planning capacity and monitoring service performance is vital for ensuring smooth operations.

00:28:39.200 We utilize several monitoring tools, including New Relic, alongside in-house tools to track performance.

00:28:56.960 With that said, you also need standardized tools across all services, which streamlines response formats, data protocols, monitoring interfaces, deployment strategies, and dependency management.

00:29:15.680 This approach eliminates the notion of unique or special cases that complicate service interaction.

00:29:34.480 At Yammer, we have adopted a toolkit known as Dropwizard. This tool, maintained by Coda Hale, helps us package necessary Java libraries for service development.

00:29:52.680 Dropwizard offers an efficient setup for a production-ready service with built-in monitoring, alerting, and metric reporting functionalities.

00:30:13.920 While we heavily lean on Java, this doesn't rule out the possibility of utilizing Ruby for building services. There are scenarios in which extracting services into Ruby first and refactoring existing code is a more valid approach.

00:30:30.720 Even if we identify a need for more performance later, we can always shift to another language as the situation dictates.

00:30:48.320 Service-oriented architectures come with trade-offs. They offer abundant benefits, yet also present complex new considerations that must be navigated meticulously.

00:31:05.440 For example, complex systems are inherently prone to failure. A robust strategy for managing service unavailability should be developed as part of the response plan.

00:31:23.440 You may face problems detecting issues within multiple service levels, as alerts could originate from locations that don't reflect where the actual failure occurred.

00:31:39.600 As you adopt service-oriented architectures, be aware that transactions aren't free. These additional services introduce new complexities that must be managed.

00:31:56.320 Adjustments may be required to streamline API updates, and coordinated service deployments must be effectively supported across multiple client versions.

00:32:11.360 To recap, it's vital to continuously assess the costs and benefits of your decisions regarding service architecture. Are you still aligned with the trade-offs you initially accepted?

00:32:28.320 Being aware of your organizational structure is important—it’s beneficial to build a culture that supports service development and flow.

00:32:45.200 As your systems grow more complex, staying efficient in service deployment becomes essential, especially as you strive to meet customer needs.

00:33:01.680 It's important not to allow difficulties in building services to hinder your progress. When pressure mounts, you might be tempted to revert to earlier, monolithic structures.

00:33:15.680 Continuing to innovate in service design is crucial, and many lessons await on this journey. In the same way that you are allowed to acknowledge when you are wrong, embracing the learning process comes into play.

00:33:31.520 Each time we rewrite elements of our codebase, we discover new facets of effective service-building. Don't assume past decisions will remain valid, as adjustments in circumstances and information can change.

00:33:49.680 With that said, my name is Brian, I work at Yammer, and I appreciate your attention today.

00:34:05.360 However, just to clarify, that wasn’t all I had to share! After preparing my presentation, some questions arose.

00:34:15.920 I thought it best to address them at the end of my talk since they interrupted my presentation flow.

00:34:26.960 One question that often comes up is: "What should I extract into a service?" My general answer is that it truly depends on your application.

00:34:42.720 However, we have identified some successes with less state-dependent features. These services are easier to extract when they haven't become tightly integrated with your Rails application.

00:34:59.680 When developing new services, it's always easier to build components that don’t yet exist.

00:35:16.960 However, bear in mind that not every feature should be extracted as a service. At Yammer, we utilize A/B testing frequently.

00:35:34.240 After confronting some performance issues, we considered building a service around our experiment framework, but ultimately decided otherwise.

00:35:50.560 The data we require resides more closely within our Rails app, compelling us to recalibrate our focus on existing solutions.

00:36:08.800 It may be tempting to start anew, but often enhancements within the framework you currently have can lead to better outcomes.

00:36:24.640 Even after extracting something into a service, it’s critical to remember that you're not absolved from traditional development challenges.

00:36:37.680 Service extraction requires continuing diligence in terms of technical debt and evolving requirements.

00:36:53.760 Thus, adapt adeptly to changing conditions, ensure the reliability of services, and retain awareness that comfort with existing solutions might invoke reevaluation.

00:37:09.440 Now that I may have points worth discussing, let's return to the last slide for closure.

00:37:23.360 The idea is to be prepared to embrace mistakes. Acknowledgment and recovery are vital in ensuring you remain resilient as a service builder.

00:37:45.600 Thank you all for your time!

Ruby on Ales 2013