00:00:00.120
foreign
00:00:25.439
What Matt said resonates with me, and yes, I have been programming for quite some time. In fact, I've been programming since I was able to walk. My first experience with software was on a Commodore BASIC 4; it all started with a CBM 600, which was quite some time ago. Fast forward to 2012, I had my first contact with Ruby on Rails because I picked up a book when bookstores were still a thing in Hamburg, and it really hooked me on Ruby on Rails. Something about that book approached me in a way, igniting my passion for Ruby and Rails, and that devotion has lasted until today. I would like to talk about making software more transparent and will share a little story connected to my own journey.
00:01:18.780
Imagine you're joining a new team or company as the new guy. You're full of energy, enthusiastic, and ready to rock the stage while coding because you already know Rails; it's a Rails team. The framework's conventions make you feel at home right away. The team welcomes you, helping you familiarize yourself with the new software, and you're able to hit the ground running.
00:02:06.420
However, in a bigger company with multiple teams, you quickly realize you don’t have enough knowledge about the rest of the company. You’re unsure what every other team is working on, what stages they use, and what APIs are available. It can be quite chaotic and complex if you’re not accustomed to that kind of environment. Your focus primarily revolves around your team's project, as Ruby on Rails is excellent for the aspects that are part of the package, as it's "batteries included." Nevertheless, it doesn't provide answers for things outside your application. You cannot simply ask your app, "Hey, who are you communicating with?" or "What calls are being made by whom?" You can look at your logs while your applications are running to gain insight, but the critical missing element is context.
00:02:40.140
So, open telemetry comes to the rescue. While there are certainly other tools, open telemetry is the focus of this talk. I first stumbled upon it while working with Go when I performed research on building a data sync application, and I wanted to understand how latencies behaved over time. We have specific business hours where most customers are online, so understanding peak times in the morning and afternoon is crucial. Seeing how software performs during these times, and recognizing which other parts of the company’s services are calling your applications, can be eye-opening, especially when you realize others are using your software for their own purposes, which you might not have considered while developing it.
00:03:43.380
Open telemetry previously known as open tracing, acts as a helpful toolset. Open tracing was the predecessor to open telemetry and is the result of a merger between two major projects: Open Sensors and Open Tracing. They drew upon concepts from Google's Dapper research paper, which outlines performance monitoring techniques. The Dapper paper has inspired various open-source communities to adopt and adapt these concepts, making them more accessible. As for open telemetry, it facilitates tracing requests, enhancing context around your logs. Traditionally, logging without context leaves much to be desired. You usually log things using Rails logger functions, putting minimal context into your log messages. Open Telemetry moves the conversation to spans instead of logs, where each span serves as a unit of work encompassing specific parts, like method calls and database interactions. This method introduces clarity regarding how long a request took, when it was called, and in what order various calls were made.
00:06:47.040
Open Telemetry also provides metrics. However, many people may have other methodologies to extract metrics from their software stacks, such as StatsD for pushing runtime metrics to Grafana or Prometheus for gathering metrics from applications. Having various solutions can enhance how you monitor your software.
00:07:43.160
Now, let me illustrate this with a very simple example of a service running inside your data center. Many of you probably don't expose your Rails applications on the edge; instead, you would use load balancers and reverse proxies to forward requests to your application, which might connect to internal authentication services. As we have multiple applications that share a user base, we utilize a centralized authentication framework hidden behind the scenes, managing discussions with databases and arbitrary services, like a mail service triggered during specific actions.
00:08:57.000
This allows you to see the request flow through the system, creating a trace that outlines everything happening inside your infrastructure. While this visualization might seem dry, it's self-explanatory. Tracing encompasses all actions affecting your infrastructure, not including the browser, but instead containing components directly relevant to your context and domain.
00:10:00.600
This ability encapsulates what your application does internally, revealing interactions with other services. Often, newcomers may not be aware of which services their applications are communicating with, making it crucial to highlight those connections, particularly when it comes to maintenance windows or service dependencies. This understanding ensures effective communication when informing teams about potential impacts.
00:12:16.080
Open tracing operates using a sampling mechanism. The actual tracing documents actions occurring in memory, while sampling helps manage data storage, typically using head-based sampling as the default method, making it easier to determine in advance whether to collect traces. However, contextual errors can sometimes require tail-based sampling, which is a newer concept that adds operational complexity but allows fine-tuning of which traces are captured. You can make refined decisions about specific sampling for errors or general traffic.
00:14:15.570
To summarize, open telemetry proves its worth in complex environments requiring visibility and transparency in microservices; however, it’s equally relevant to monolithic applications interacting with external services, such as payment processors or open data sources. Monitoring latencies and ensuring instant response times for customers will necessitate understanding the flow of data across your infrastructure. While integrating open telemetry can simplify the process of identifying issues within your applications, it comes with operational complexity and budget considerations, particularly if you plan on using managed services.
00:19:30.000
Setting up open telemetry for local testing is relatively straightforward, as Jager enables quick infrastructure deployment through simple container setups. You can conduct experiments to understand how open telemetry can enhance your application’s monitoring capabilities. Today’s instrumentation is very user-friendly and allows for easy integration into existing applications, thus marking a significant step forward in effective performance monitoring.
00:21:17.640
Lastly, I'd like to show you the available instrumentation for open telemetry. There are familiar components in Ruby and Rails, as well as some that may not be widely recognized. You only need to add three gems into your Gemfile and include a minimal initializer to start utilizing open telemetry. The visibility gained through seamless integration enhances your application monitoring by clearly showcasing what your requests do under the hood. Thank you very much for your attention, and feel free to reach out with any questions!