00:00:20
Everybody, my name is Austin Story. I am an Engineering Director at Doximity.
00:00:26
I've been there for six and a half years, and the things that I'm going to be talking to you about today are a project that I was part of.
00:00:31
This project has gone from idea to implementation to deployment, and it's been in production for about a year now.
00:00:39
I do want to say, as I'm getting started here, thank you for taking the time to come to this talk.
00:00:45
I am both honored and humbled to be here.
00:00:51
First, I absolutely love Rails; this community is amazing.
00:00:58
It's the thing that keeps me programming and excited every day.
00:01:04
And, you know, I'm humbled that you are all choosing to be here.
00:01:11
I know it’s day three, and many of us are feeling pretty smug.
00:01:18
Some of you were out pretty late doing karaoke last night.
00:01:23
There are several other fantastic talks occurring right now, some of which I wanted to attend.
00:01:29
And finally, I know you’re all here even though there are chair massages available.
00:01:36
I will do my best to live up to that standard of being at least as good as a chair massage.
00:01:42
So, to get started, let's have a high-level overview of what we're going to be discussing.
00:01:55
This is a story about how Doximity, a Rails-based web application, used engines to deploy a complex system called GraphQL Federation.
00:02:01
We did this very intentionally, enabling integration without developers needing to get too involved with the system's details.
00:02:01
This is not a talk about GraphQL; I am not going to advocate for or against it.
00:02:08
GraphQL can be a controversial topic.
00:02:13
Instead, this talk focuses on the important considerations I keep in mind whenever I implement a system like this.
00:02:19
I aim to ensure that we deliver integrated solutions aligned with the Rails philosophy.
00:02:25
I want to address who this talk is applicable to.
00:02:31
There are two main groups of people.
00:02:38
Firstly, this is relevant for those getting started or further along in their careers.
00:02:43
I hope you walk away with a deeper understanding of intentional Rails knowledge.
00:02:48
I want you to know the built-in features of Rails that can help you integrate systems effectively.
00:02:56
And for the later-stage developers, I hope you glean insights about scaling your team.
00:03:03
Speeding up your team can often be accomplished using tools that integrate smoothly with existing Rails systems.
00:03:08
Now, let’s start with some definitions to make sure we're all on the same page.
00:03:13
The first one is something that everybody's probably familiar with: what is a Ruby gem?
00:03:14
To me, a Ruby gem is just some shared Ruby code that is quite useful.
00:03:20
You can add this to your project without worrying about creating database connections, thanks to ActiveRecord.
00:03:27
So what’s an engine, and why did I define gem before this?
00:03:33
An engine is really just another type of gem.
00:03:39
However, it's an enhanced gem that is more aware of Rails and its ecosystem.
00:03:45
It knows how to integrate with the Rails ecosystem, allowing for both useful code sharing and integration with web applications.
00:03:50
We'll discuss how we did this at Doximity with GraphQL.
00:03:56
So, what is GraphQL? It represents a way to query data from your servers.
00:04:03
GraphQL consists of three components.
00:04:09
First, there's a way to describe your data.
00:04:15
For example, we can define a project type with a name that is a string and a tagline that is a string.
00:04:22
Any language that can implement the specification can handle this, not just Ruby.
00:04:30
Libraries are available for Go, Node, and other languages.
00:04:37
The server implements the specifications, allowing clients to query what they need.
00:04:45
This creates a trade-off between benefits and costs.
00:04:50
GraphQL empowers mobile developers to retrieve exactly what they need.
00:04:57
For instance, in this example, we request the project and grab its tagline.
00:05:01
The response comes back predictably, which is crucial for mobile native applications.
00:05:06
Now, what is GraphQL Federation? It extends GraphQL further.
00:05:13
As a query language, GraphQL represents and responds to the requests for data.
00:05:19
Federation provides a unified API primitive.
00:05:26
In a microservices architecture, each microservice can respond to related requests.
00:05:32
Then, you combine these responses into a single endpoint.
00:05:38
This allows for requests to be routed efficiently to the appropriate microservice.
00:05:44
I have a video here that illustrates this concept—fingers crossed that the internet works!
00:05:52
In this example, we have a gateway that interacts with three microservices: a users microservice, a product microservice, and a review microservice.
00:06:05
On the left side, we see a query being sent.
00:06:11
The request includes the product ID and its associated reviews.
00:06:17
The gateway acts as a smart router, directing parts of the query to the appropriate microservices.
00:06:25
The responses are then collated and served back to the client.
00:06:31
To explain how we arrived at GraphQL and its Federation within Doximity, let me provide some context about our organization and the constraints we face.
00:06:43
Doximity is a 13-year-old Rails-based web application, and we’re a founding member of the Rails Foundation.
00:06:56
Rails is not only near and dear to our hearts, but it's something we want to support and continuously push forward in the community.
00:07:08
We are a professional medical network, which I like to describe to developers as a fusion between GitHub and LinkedIn, but exclusively for healthcare professionals.
00:07:16
We have 80% of US physicians on our network and provide an array of tools.
00:07:27
For instance, Telehealth capabilities enable voice and video calls.
00:07:35
Many of you have likely used a telemedicine service for video calls.
00:07:41
There’s a high chance that the system you utilized was powered by Doximity.
00:07:49
We facilitate hundreds of thousands of calls every day and prioritize a physician-first experience.
00:07:56
Thus, we structure our teams around distinct domains.
00:08:01
Telehealth, for example, is its own business unit with unique goals and scalability requirements.
00:08:08
As a result, we adopted a service-oriented architecture to allow each domain the flexibility needed for rapid change.
00:08:16
Physicians are busy; we aim to meet them on their terms and usually that's via mobile.
00:08:22
They have many appointments throughout the day, which is why we prioritize a mobile-native approach.
00:08:29
We support 30 application teams and have over 140 web developers.
00:08:36
Our system manages around 9,000 GraphQL requests per minute with 7,000 fields defined in our models.
00:08:44
Initially, we worked with a monolith.
00:08:51
Everything was fine until more domains like Telehealth emerged that required swift changes.
00:08:58
As this expansion occurred, we began facing resiliency issues.
00:09:03
We decided to start decoupling functionality into microservices.
00:09:10
Additionally, we aimed to address mobile native challenges by implementing GraphQL.
00:09:16
This transition began around 2017, which is about six years ago.
00:09:23
To visualize it, our monolith received requests from iOS, Android, and web clients.
00:09:30
If those requests were made to GraphQL, we routed them either to microservices or processed them internally.
00:09:37
Other teams also began implementing GraphQL on their services.
00:09:45
The perspectives of mobile and back-end teams differed significantly.
00:09:52
Mobile developers appreciated the API structure GraphQL provided.
00:09:59
It grants them easy access to documentation and ensures type responses.
00:10:06
This leads to fewer crashes caused by unexpected nil responses.
00:10:14
We stored every type of request made by mobile, allowing us to monitor changes.
00:10:20
Developers could identify if their changes would impact existing functionality.
00:10:28
On the other hand, the back-end perspective found GraphQL demanding.
00:10:35
It required more intentionality at the decision-making level.
00:10:43
Internal standards were challenging to implement since every team followed different conventions.
00:10:50
Moreover, service isolation was increasingly difficult.
00:10:55
For example, if our Newsfeed service relied on the monolith, outages could extend across the system.
00:11:02
Each service had to stand resilient, but dependent services tied too closely together hampered that.
00:11:09
The developer experience was considerably compromised.
00:11:16
Those working in the monolith were satisfied, while microservices developers struggled with multiple changes just to release one feature.
00:11:24
This led us to question whether maintaining GraphQL was worthwhile.
00:11:30
Ultimately, mobile developers confirmed its value.
00:11:35
We set out to streamline deployments in a way that Rails developers wouldn’t have to grapple with the complexity introduced by our decisions.
00:11:43
This led us to implement GraphQL Federation.
00:11:50
The changes in the flow presented here depict our transformation.
00:11:58
iOS, web, and Android clients now make requests via GraphQL Federation, functioning as an edge router.
00:12:04
Our microservices subsequently expose a GraphQL representation.
00:12:16
As for the benefits of this approach, we can now create internal standards.
00:12:23
Developers can still work independently within Rails whilst utilizing standard GraphQL features.
00:12:30
Each team can change their service without worrying about or needing to understand someone else’s work.
00:12:36
New changes mean better developer experiences.
00:12:42
We were able to deliver projects on time, as the complexity necessitated fewer overall changes.
00:12:49
To accomplish this focus, we had to mitigate the intricacies that developers must face.
00:12:54
The next steps meant integrating Rich functionalities with Rails.
00:13:01
I firmly believe in building integrated systems.
00:13:07
To achieve that, I turn to the Rails guides.
00:13:14
Even after a single day, I find rereading documentation beneficial.
00:13:19
It saves me countless hours on future projects.
00:13:26
Next, we must plan our approach, given the options to build a Rails engine.
00:13:34
Initially, our monolith will host GraphQL.
00:13:39
Our goal is to gradually migrate features into the new engine.
00:13:46
The dream is that we can simply drop the engine into a microservice, and everything will work.
00:13:53
The first thing we do is set up our engine.
00:13:58
Next, we implement GraphQL Playground and discuss key GraphQL concepts.
00:14:07
Afterward, I'll talk about how we integrated everything with Federation.
00:14:12
I’ll have a lot of code samples to share.
00:14:19
To initiate creating a Rails engine, check out the guides.
00:14:26
An engine is a mountable plug-in, requiring specific class names.
00:14:34
This generates code, but the most impactful aspect is the included test harness.
00:14:42
I emphasize having a dummy application that utilizes every feature from my engine.
00:14:51
This way, I can test changes in a live environment.
00:14:57
Having extensive request specs helps me avoid breaking functionality during updates.
00:15:04
Additionally, I focus on configuration.
00:15:11
Configuring how users can change runtime behavior is essential.
00:15:18
I generally use a standard approach to expose ways for users to alter configurations.
00:15:25
For instance, I might create an error based on invalid configurations.
00:15:32
These errors notify users of potential pitfalls in production.
00:15:38
Now, let's talk about GraphQL Playground and why it's beneficial.
00:15:44
GraphQL Playground is a full-featured HTML and JavaScript interface.
00:15:52
It exposes meaningful schema that describes what the server can do.
00:16:02
With this, we can provide crucial developer metrics.
00:16:09
This allows developers to see what data they can request without bugging the back-end team.
00:16:15
They gain a clear overview of how to interact with the API.
00:16:22
The live playground aspect also allows for autocomplete and querying in real time.
00:16:29
This delivery model vastly reduces developer friction when requesting information.
00:16:37
Ultimately, our goal is to create a tailored Rails application that simplifies interactions.
00:16:43
Everything should seamlessly integrate to simplify developers' tasks.
00:16:50
The Playground controller decides how much the engine interfaces with the application.
00:16:56
For our purposes, we aim for little influence on how it operates.
00:17:03
This helps maintain flexibility while allowing users to leverage our engine's capabilities.
00:17:10
Including default access tokens simplifies integration for developers.
00:17:18
We can’t allow developers to mess up access control, so we enforce strict checks.
00:17:25
Our focus rests on ensuring a seamless and intuitive experience.
00:17:33
Next, our routing logic is straightforward.
00:17:40
We configure the Playground controller to manage incoming requests.
00:17:46
We also ensure that this setup is flexible and can be modified if needed.
00:17:53
Following this, we update the configuration process.
00:18:00
This encompasses enabling or disabling the Playground as necessary.
00:18:07
The implementation is straightforward and concise.
00:18:13
This can entail simply providing options in the configuration process.
00:18:19
The final code here shows how to build a GraphQL playground quickly and seamlessly.
00:18:27
We want the integration for the developers to be as low friction as possible.
00:18:33
Once the engine is feature complete, we need to explore installation options.
00:18:39
Developers often struggle with installation if the barriers are too high.
00:18:46
What I often see is an overwhelming array of steps that ail developers.
00:18:52
First, they are told to install the engine, copy configurations, and create it correctly.
00:19:01
That said, we can do better.
00:19:07
Instead, step one can be to run a generator that automatically configures setup.
00:19:15
This promotes user engagement and benefits both parties.
00:19:22
The installation flow combines simplicity and efficiency.
00:19:29
We want developers to be excited to utilize our engine.
00:19:35
Moreover, generators are a significant part of Rails and engines.
00:19:42
They help scaffold out the necessary defaults.
00:19:48
This equips us to guide users without overwhelming them.
00:19:54
Creating a default initializer ensures that users have access to vital information.
00:20:01
We emphasize thorough documentation available right at their fingertips.
00:20:07
The generator simplifies handle initializations and provides clarity.
00:20:14
Ultimately, our goal is to deliver the functionality inherent in these conversations.
00:20:20
This brings us closer to our mission of seamless integration.
00:20:26
Next, we move on to extracting the GraphQL classes.
00:20:34
This involves managing features unique to Doximity.
00:20:40
After we establish the Playground, the second step includes extracting the relevant code.
00:20:46
By moving components over into the engine, we simplify standardization.
00:20:53
Additionally, security measures must be integrated.
00:21:01
Ensuring that security practices are maintained is a powerful feature of GraphQL.
00:21:08
We want to ensure that sensitive data is always safeguarded.
00:21:14
Following this, we incorporate RSpec for testing.
00:21:20
Integrating our personal testing practices is crucial.
00:21:27
We want a seamless experience for developers as they test GraphQL functionality.
00:21:34
Moving forward, we generate the types within GraphQL.
00:21:41
This is to keep the heavy lifting behind the scenes, enabling smooth updates.
00:21:48
We aim to reduce the complexity every developer must deal with.
00:21:55
They can focus on defining their schema without struggling with the underlying code.
00:22:02
The key is to facilitate greater abstraction within our system.
00:22:09
As we steer through this journey, we also prepare for integration with Federation.
00:22:15
This integration facilitates streamlined workflows among microservices.
00:22:22
We consider the requests collectively to maintain uniformity.
00:22:29
As we initiate requests, we can ensure they're processed uniformly.
00:22:36
Centralized routes clearly define how requests are structured.
00:22:42
Our goal is to manage incoming queries across all microservices.
00:22:49
For changes to propagate effectively, we need a clear schema.
00:22:56
This defines the structure and rules for interaction.
00:23:03
Lastly, we must define the deployment process as we push changes live.
00:23:10
Live changes require oversight to ensure compatibility.
00:23:16
Verification processes ensure accuracy within our ecosystem.
00:23:22
As we continue with Rake tasks, we're able to run validations throughout the process.
00:23:30
Each task is aligned with the goal of maintaining operational integrity.
00:23:38
It's crucial that we follow through properly to avoid potential issues.
00:23:46
Connecting these pipelines confirms efficient operation as we roll out changes.
00:23:53
In conclusion, we integrate systems and maintain a seamless experience in our development lifecycle.
00:24:01
Doximity now enjoys nine microservices that effectively utilize our new architecture.
00:24:09
Our work ensures simplicity in deploying and managing these changes.
00:24:16
This approach empowers developers by streamlining their tasks.
00:24:23
I truly appreciate your time today.
00:24:30
If you have any questions, please feel free to raise your hand.