00:00:16.460
Today, I will be talking about RPC frameworks. Before we dive in, let me give you a quick summary of what I will cover. I have worked with one of the RPC frameworks, and I will explain why we decided to use it, what problems it helped us solve, and dive a bit into the protocol details. Additionally, I will compare the solution we used with other options since it turns out there are actually a lot of different RPC frameworks available.
00:00:32.160
Before I start, I should mention that I recently felt a bit sleepy. I hope I won’t fall asleep while speaking! So, are you ready? Yes? I also noticed that many of the foreign speakers often start their talks with pictures of their cities or maps. I’m no exception, so first, I would like to share a few things about Belarus, as I assume not many people here know much about my country.
00:01:06.930
Belarus is located in Eastern Europe. In some travel magazines, it has been mentioned as one of the top countries to visit in 2020. You might find it interesting that we produce some of the biggest trucks in the world, with the highest capacity, including the famous barrels. Belarus is also known for its exports of salt, ranking third in the world, with Russia being first and Canada second. Additionally, you may not know that we have a strong population of European bison, which were almost extinct but are now quite safe.
00:02:05.759
Another fun fact is that we consume the highest amount of potatoes in the world, averaging half a kilogram per person per day. Now, let’s get straight to the point. What is RPC? RPC stands for Remote Procedure Call. It is a protocol that allows one program to call a service from another program located somewhere in the network.
00:02:37.570
RPC has gained popularity among web developers, particularly due to the rise of microservices and service-oriented architecture, as it helps to solve communication problems between distributed services. An RPC framework is not a monolithic solution; rather, it is a collection of technologies that allows us to choose how we want to encode data and how we want to transfer it. There are usually different options to pick from within the same framework.
00:03:40.180
There are many RPC frameworks available, such as gRPC, Thrift, Cap’n Proto, and many others. Almost every one of them is backed by a large corporation. For instance, gRPC is developed by Google, while Thrift is a framework created by Facebook. These frameworks often serve as vital tools that large companies leverage to enhance their services. However, it can be challenging for open-source enthusiasts to engage with these solutions.
00:04:36.600
Let’s start with gRPC, which is one of the most recognized RPC frameworks. A key feature of gRPC is its support for bi-directional streaming and its extensive performance capabilities. gRPC also utilizes HTTP/2, which we found beneficial during our implementation.
00:05:14.700
I was working at a company with a significant system made up of more than 50 different services. To give you an idea of how it looked, we had an AWS Lambda function that would trigger each time something occurred in our S3 service, whether a file was uploaded, deleted, or an error had happened. This was a vital aspect of how we managed our system effectively.
00:05:53.478
We also had a subscription service where we were subscribed to various queues. For example, when packages or titles were processed, they would send messages to notify our management service, indicating that the processing was complete. At some point, we decided to change our setup. The main question we faced was, 'Why?' First and foremost, we encountered issues with non-streaming messages. I believe some of these issues have already been resolved.
00:06:46.330
The original service was not capable of handling a large volume of messages efficiently, even though our system was relatively small. We realized that some subscriptions began to stop receiving messages. After trying various solutions for a considerable time, we decided to change the way we handled message transmission. We evaluated different options: Kafka, RabbitMQ, and others, ultimately deciding that gRPC would be a suitable choice for our communication mechanism.
00:07:49.939
The reasoning behind this choice was primarily political; we had a sizable development department that was eager to experiment with new technologies. They were excited about trying something new and writing code in Go, possibly connected to gRPC. This experimentation turned out to be a great success for us. Now, we have a service that interacts with an RPC server, where each of our services implements an RPC client.
00:09:11.030
In summary, gRPC has proven to be a reliable system for us, allowing for effective communication across services. We began considering whether alternatives like Kafka could be more suitable, but we found gRPC to be effective for our needs. We appreciated the simplicity of use and efficiency it brought to our architecture.
00:09:52.310
Moreover, gRPC offers advantages in terms of message structure and logging. It enhances our ability to debug the system effectively as we can utilize logs from both clients and servers together.
00:10:10.200
This adds an additional layer of reliability to gRPC, especially in complex distributed systems.
00:10:19.080
Some implementations of gRPC include those across multiple languages like Python and C++. Google continuously updates gRPC with security fixes, ensuring that it remains a robust choice for developers. This was something we valued deeply during our migration.
00:11:03.700
gRPC provides us with flexibility; you can choose various transport methods, such as HTTP/2 or in-process communications for testing your messages without dealing with extensive infrastructure. gRPC makes it easy to handle structured data with its protocol buffers which are often more efficient compared to JSON in terms of payload size.
00:12:50.680
The way it compresses messages is beneficial, especially for large transactional data. In gRPC, information is serialized effectively, allowing significant savings on traffic. A growing community means you will likely find support for gRPC, including resources and documentation.
00:13:51.940
Given the advantages such as message size reduction and a vast ecosystem supporting various languages, gRPC stands out as a suitable option for RPC implementations in many contexts. The ability to share code across different systems while maintaining the same message schema significantly minimizes the risks of failure resulting from changes in services.
00:15:22.540
Now let’s discuss alternatives. For instance, we looked at Thrift, which is frequently used as well. Facebook created Thrift, and it is widely applied in many systems. However, some report issues with documentation and ease of use, especially when tailoring solutions for specific needs.
00:16:33.060
Next on our list is Cap’n Proto, which claims to be faster than traditional protocols. Despite this performance marketing, we found it challenging to gauge whether the improvements are significant due to different testing methodologies during evaluation.
00:17:20.520
Finally, we will discuss Flatter Buffers and the last RPC framework we will mention is Avro, which has gained traction in big data projects. It diverges from other frameworks as it uses dynamic typing and other unique features.
00:18:34.020
To summarize, each RPC framework we've discussed offers advantages and downsides, with choices based on specific project requirements. Whether one prioritizes performance, ease of use, or community support, several frameworks exist that can fit various development needs.
00:19:31.400
I encourage everyone to explore these frameworks based on your project. They offer unique strengths and weaknesses to consider, especially in the realms of language compatibility, performance scalability, and ease of integration.
00:20:07.800
In conclusion, RPC frameworks can help bridge various application components, allowing seamless communication across diverse software environments. Thank you for your attention, and I hope this discussion has sparked your interest in exploring RPC options that your projects might benefit from.