Microservices

Summarized using AI

A Quick Guide to RPC Frameworks

Yulia Oletskaya • January 31, 2020 • Birmingham, AL

In her presentation titled "A Quick Guide to RPC Frameworks" at the Birmingham on Rails 2020 conference, Yulia Oletskaya discusses the importance and utility of Remote Procedure Call (RPC) frameworks in modern software architecture, particularly in the context of microservices. She starts by identifying RPC as a protocol allowing one program to call a service from another program across a network, which has gained popularity due to the rise of microservices and service-oriented architecture. Key points covered in her talk include:

  • Introduction to RPC: Oletskaya explains RPC and its role in enabling communication between various services, highlighting how it differs from traditional monolithic frameworks.

  • Diversity of RPC Frameworks: There exist numerous RPC frameworks in the market, each offering unique features, such as gRPC, Thrift, and Protocol Buffers, often backed by large corporations.

  • Use Case: Oletskaya shares her experience while working at a company with over 50 different services. She outlines the communication challenges they faced with their existing messaging systems, which led them to explore new solutions.

  • Benefits of gRPC: After considering several messaging systems, the team adopted gRPC. Oletskaya elaborates on the protocol’s advantages, including bi-directional streaming and efficient message serialization via Protocol Buffers, allowing for smaller message sizes compared to JSON.

  • Schema Management: gRPC's shared schema definition is highlighted as a crucial feature that helps prevent communication breakdowns between isolated teams, facilitating consistency across different services.

  • Challenges: The speaker notes challenges encountered during implementation, such as monitoring and configuration complexities with alternative frameworks like Kafka.

  • Comparison of RPC Frameworks: Oletskaya compares different RPC frameworks, their transports, and protocols, discussing their unique aspects and suitability for different use cases.

  • Conclusion: The talk concludes with Oletskaya emphasizing the evolving nature of RPC frameworks and the ongoing improvements to gRPC, pointing to a vibrant community and the availability of resources for troubleshooting.

Overall, the presentation provides valuable insights into the practical applications of RPC frameworks and offers guidance for choosing the right framework according to specific project needs.

A Quick Guide to RPC Frameworks
Yulia Oletskaya • January 31, 2020 • Birmingham, AL

Birmingham on Rails 2020 - A Quick Guide to RPC Frameworks by Yulia Oletskaya

Birmingham on Rails 2020

00:00:16.460 Today, I will be talking about RPC frameworks. Before we dive in, let me give you a quick summary of what I will cover. I have worked with one of the RPC frameworks, and I will explain why we decided to use it, what problems it helped us solve, and dive a bit into the protocol details. Additionally, I will compare the solution we used with other options since it turns out there are actually a lot of different RPC frameworks available.
00:00:32.160 Before I start, I should mention that I recently felt a bit sleepy. I hope I won’t fall asleep while speaking! So, are you ready? Yes? I also noticed that many of the foreign speakers often start their talks with pictures of their cities or maps. I’m no exception, so first, I would like to share a few things about Belarus, as I assume not many people here know much about my country.
00:01:06.930 Belarus is located in Eastern Europe. In some travel magazines, it has been mentioned as one of the top countries to visit in 2020. You might find it interesting that we produce some of the biggest trucks in the world, with the highest capacity, including the famous barrels. Belarus is also known for its exports of salt, ranking third in the world, with Russia being first and Canada second. Additionally, you may not know that we have a strong population of European bison, which were almost extinct but are now quite safe.
00:02:05.759 Another fun fact is that we consume the highest amount of potatoes in the world, averaging half a kilogram per person per day. Now, let’s get straight to the point. What is RPC? RPC stands for Remote Procedure Call. It is a protocol that allows one program to call a service from another program located somewhere in the network.
00:02:37.570 RPC has gained popularity among web developers, particularly due to the rise of microservices and service-oriented architecture, as it helps to solve communication problems between distributed services. An RPC framework is not a monolithic solution; rather, it is a collection of technologies that allows us to choose how we want to encode data and how we want to transfer it. There are usually different options to pick from within the same framework.
00:03:40.180 There are many RPC frameworks available, such as gRPC, Thrift, Cap’n Proto, and many others. Almost every one of them is backed by a large corporation. For instance, gRPC is developed by Google, while Thrift is a framework created by Facebook. These frameworks often serve as vital tools that large companies leverage to enhance their services. However, it can be challenging for open-source enthusiasts to engage with these solutions.
00:04:36.600 Let’s start with gRPC, which is one of the most recognized RPC frameworks. A key feature of gRPC is its support for bi-directional streaming and its extensive performance capabilities. gRPC also utilizes HTTP/2, which we found beneficial during our implementation.
00:05:14.700 I was working at a company with a significant system made up of more than 50 different services. To give you an idea of how it looked, we had an AWS Lambda function that would trigger each time something occurred in our S3 service, whether a file was uploaded, deleted, or an error had happened. This was a vital aspect of how we managed our system effectively.
00:05:53.478 We also had a subscription service where we were subscribed to various queues. For example, when packages or titles were processed, they would send messages to notify our management service, indicating that the processing was complete. At some point, we decided to change our setup. The main question we faced was, 'Why?' First and foremost, we encountered issues with non-streaming messages. I believe some of these issues have already been resolved.
00:06:46.330 The original service was not capable of handling a large volume of messages efficiently, even though our system was relatively small. We realized that some subscriptions began to stop receiving messages. After trying various solutions for a considerable time, we decided to change the way we handled message transmission. We evaluated different options: Kafka, RabbitMQ, and others, ultimately deciding that gRPC would be a suitable choice for our communication mechanism.
00:07:49.939 The reasoning behind this choice was primarily political; we had a sizable development department that was eager to experiment with new technologies. They were excited about trying something new and writing code in Go, possibly connected to gRPC. This experimentation turned out to be a great success for us. Now, we have a service that interacts with an RPC server, where each of our services implements an RPC client.
00:09:11.030 In summary, gRPC has proven to be a reliable system for us, allowing for effective communication across services. We began considering whether alternatives like Kafka could be more suitable, but we found gRPC to be effective for our needs. We appreciated the simplicity of use and efficiency it brought to our architecture.
00:09:52.310 Moreover, gRPC offers advantages in terms of message structure and logging. It enhances our ability to debug the system effectively as we can utilize logs from both clients and servers together.
00:10:10.200 This adds an additional layer of reliability to gRPC, especially in complex distributed systems.
00:10:19.080 Some implementations of gRPC include those across multiple languages like Python and C++. Google continuously updates gRPC with security fixes, ensuring that it remains a robust choice for developers. This was something we valued deeply during our migration.
00:11:03.700 gRPC provides us with flexibility; you can choose various transport methods, such as HTTP/2 or in-process communications for testing your messages without dealing with extensive infrastructure. gRPC makes it easy to handle structured data with its protocol buffers which are often more efficient compared to JSON in terms of payload size.
00:12:50.680 The way it compresses messages is beneficial, especially for large transactional data. In gRPC, information is serialized effectively, allowing significant savings on traffic. A growing community means you will likely find support for gRPC, including resources and documentation.
00:13:51.940 Given the advantages such as message size reduction and a vast ecosystem supporting various languages, gRPC stands out as a suitable option for RPC implementations in many contexts. The ability to share code across different systems while maintaining the same message schema significantly minimizes the risks of failure resulting from changes in services.
00:15:22.540 Now let’s discuss alternatives. For instance, we looked at Thrift, which is frequently used as well. Facebook created Thrift, and it is widely applied in many systems. However, some report issues with documentation and ease of use, especially when tailoring solutions for specific needs.
00:16:33.060 Next on our list is Cap’n Proto, which claims to be faster than traditional protocols. Despite this performance marketing, we found it challenging to gauge whether the improvements are significant due to different testing methodologies during evaluation.
00:17:20.520 Finally, we will discuss Flatter Buffers and the last RPC framework we will mention is Avro, which has gained traction in big data projects. It diverges from other frameworks as it uses dynamic typing and other unique features.
00:18:34.020 To summarize, each RPC framework we've discussed offers advantages and downsides, with choices based on specific project requirements. Whether one prioritizes performance, ease of use, or community support, several frameworks exist that can fit various development needs.
00:19:31.400 I encourage everyone to explore these frameworks based on your project. They offer unique strengths and weaknesses to consider, especially in the realms of language compatibility, performance scalability, and ease of integration.
00:20:07.800 In conclusion, RPC frameworks can help bridge various application components, allowing seamless communication across diverse software environments. Thank you for your attention, and I hope this discussion has sparked your interest in exploring RPC options that your projects might benefit from.
Explore all talks recorded at Birmingham on Rails 2020
+1