Talks
Outside of the Web Box: Using Ruby for Other Protocols
Summarized using AI
Outside of the Web Box: Using Ruby for Other Protocols
by Danielle AdamsIn the video "Outside of the Web Box: Using Ruby for Other Protocols," presented at RubyConf AU 2018 by Danielle Adams, the focus is on leveraging Ruby for interfacing with hardware and other protocols beyond typical web applications.
Danielle begins her talk by sharing her experiences at Blue Apron, a company specializing in subscription meal kits where she works. She discusses the necessity of integrating Ruby with various hardware in the warehouse setting due to the constraints of their existing monolithic Rails application.
Key points covered in the presentation include:
- Introduction to Protocols: Danielle explains the distinction between HTTP requests, which operate at the application level, and TCP (Transmission Control Protocol), which is essential for reliable communication between devices at the transport layer.
- Routing Issues: The importance of effectively managing traffic through web servers and APIs to ensure proper response times and accuracy in operations.
- Ruby’s Capabilities: The decision to use Ruby for interfacing with warehouse hardware, despite suggestions to use Python for multi-threading. Key to this decision was Ruby’s built-in socket library that facilitates TCP connections, allowing for seamless interaction between machines.
- Implementing TCP Communication: Demonstrating how to create a basic TCP server and the role of middleware to simplify handling of data from various devices.
- Real-World Applications: Danielle provides examples of how they implemented solutions such as a shoe sorter that relies on precise timing and operational accuracy to prevent delivery errors, along with their approach to managing printing tasks through service workers.
- Challenges Faced: She discusses obstacles they faced like maintaining connections with hardware and the importance of rigorous testing due to unpredictable variables associated with physical devices.
Danielle concludes by emphasizing that Ruby is versatile and suitable for tasks beyond web development, urging the audience to embrace experimentation with different technologies. She reports a successful deployment of their solutions across multiple facilities within a year.
The main takeaway is that Ruby can handle various protocols effectively with the right approach and mindset, facilitating robust communication between software and hardware, leading to improved operational efficiency in warehouse environments.
00:00:00.030
Hey everyone, my name is Danielle Adams. I'm from New York, and I'm so grateful to be here.
00:00:07.109
First of all, thank you all for having me. It's my first time in Sydney, and it's been a really exciting week! Can we give a round of applause to the organizers?
00:00:20.550
I've had some great experiences, including meeting a kangaroo, which was just amazing! Usually, I would introduce my cat at this point, but this was a much more fitting story.
00:00:34.079
So as I mentioned, I'm Danielle. This is my Twitter handle and GitHub. I actually realized that the code for this presentation isn't on GitHub yet, but don't worry; I will get it up there once the presentation is over.
00:00:42.000
This is the world I come from, New York, which doesn't have nearly as much wildlife. We have sewer rats and pigeons, and that's about it! I work at a company called Blue Apron, which is known for its subscription meal kits.
00:01:05.970
Blue Apron is well-known worldwide for sponsoring podcasts. But they actually do a lot more than that, and that’s what I work on.
00:01:13.080
My role involves working behind the scenes of Blue Apron. We package prepared ingredients and ship them around the country for people to cook for their families. The original warehouse team that I worked with is shown here; it's a little bigger now, but this was the core team.
00:01:39.450
Today, I want to discuss how we expanded our service ecosystem to include interfacing with some of the hardware we introduced in the warehouse. Blue Apron started in New York and has been around for about five years. When I joined, the company had a few front-end applications and one monolithic Rails application.
00:02:08.100
That application handled all of the data management, but it wasn’t going to be enough as we grew. We realized that we needed to integrate with different hardware that our warehouse was using.
00:02:35.180
For example, we had scanners and printers that were crucial to our operations. When we encountered these technical challenges, one of the first questions we had was, 'In what language should we build our solutions?' Some members of the team believed we couldn't use Ruby.
00:02:58.170
My boss suggested using Python because he was convinced it was better suited for multi-threading. I didn’t quite understand where that came from, but I pushed hard for Ruby because the teams already had familiarity with it.
00:03:21.000
Eventually, we decided to write everything in Ruby. It turned out that Ruby could handle the various protocols we needed, and we discovered that there was an internal authentication library we had to use, which was a contributing factor to our decision.
00:03:54.810
This turned out to be a great decision because everyone was comfortable with Ruby. My talk today will cover protocols, interfacing with machines, and integrating with services in a Ruby environment.
00:04:31.610
At that time, we were scaling our hardware and software simultaneously, with multiple projects going on involving both Rails applications and Ruby.
00:04:51.020
First, let’s discuss protocols. For instance, what is an HTTP request? This is the primary request type we use mostly in Rails applications, Sinatra, or any Rack app.
00:05:04.639
So, how does a web server handle a request? A web browser generates a POST or a GET request from a computer. This request gets resolved to an IP address, and in between, we might have a caching layer and load balancer. If several instances of our application are running, we can divert the traffic to the appropriate instance.
00:05:41.220
In a typical Rack application, we have a web server such as Puma or Unicorn. Then, middleware processes the request before passing it to our web application.
00:06:18.830
Now, let's talk about TCP. This is a crucial protocol we relied on when interfacing with hardware.
00:06:29.620
TCP, or Transmission Control Protocol, is designed to be a highly reliable host-to-host protocol, crucial for packet-switched networking.
00:06:59.120
In 1981, TCP was introduced to support secure packet messaging.
00:07:04.520
To differentiate between HTTP and TCP, HTTP operates at the application level, whereas TCP operates at the transport layer. A few months later, we started prototyping with Ruby to see if it could effectively work with our scanners and printers.
00:07:44.300
We took our laptops into the warehouses, setting up everything. Picture us standing at the packing line with boxes to scan and interact with warehouse associates.
00:08:07.670
How did we accomplish this? Ruby has a built-in socket library that allows us to create TCP applications.
00:08:18.980
If you require the socket library in your program, you can easily create a TCP server right away. A socket effectively combines an IP address and a port.
00:08:50.490
For example, using localhost:3000 allows communication through the TCP socket library. The flow generally involves a TCP client (like a browser) talking to a server, which then updates a database with the desired flow.
00:09:24.460
In scenarios where our clients needed communication with non-HTTP servers, an open socket was necessary. The code to establish a TCP socket might look daunting at first, but it can be straightforward as well.
00:10:00.460
We merely require the socket library, create an instance, and write data to it before closing it. This allows us to build robust TCP servers.
00:10:38.040
Continuing, here’s a basic example of a Ruby script to create a local TCP server. Here we set a port, allow connections, and read incoming messages.
00:11:05.430
We also used various machines requiring TCP libraries, like USB scanners and networked printers connected via TCP sockets, as well as more advanced warehouse machinery.
00:11:44.020
Here’s what a TCP script might look like in action. We start the server, allowing it to recognize clients, then send a test message.
00:12:49.920
By the end of that year, we began transitioning these services within Rails applications and as standalone services using Sinatra as a proxy.
00:13:00.640
This shoe sorter integrates with our software, and I want to explain how we handled this large piece of machinery that communicates via TCP.
00:14:05.730
As you can see in this example, the shoe sorter has very specific timing and handling. If boxes are misrouted, they could ultimately end up on the wrong truck, causing significant delivery issues.
00:15:41.300
It is essential to manage this flow and keep connections alive to ensure the accuracy of our service and products.
00:16:25.890
It was vital for us to implement non-blocking integrations. We handled the requests from a client-side web app without blocking critical services on other processes.
00:17:45.130
In practice, suppose someone on the packing line clicks a button to print a label. We ensured that this process would not block the entire application instance.
00:18:04.220
The way we did that is by utilizing a service worker, which separated the printing task from the web application, allowing it to run independently.
00:18:58.920
This is an example of how our Rails controller interacts with a Sidekiq worker, sending a print command to the TCP socket.
00:19:51.030
Once the command is sent, the worker opens a socket, writes the necessary data to it, and then closes the connection. The data format varies depending on the required protocols.
00:20:53.780
Our barcode scanners also operate consistently on TCP connections. It is essential to effectively parse the incoming data since our database has its own processes for handling updates.
00:21:55.200
To ensure proper communication, we implemented middleware to abstract the complexity of the protocols being utilized.
00:22:32.720
So when we receive a message through a TCP socket, the middleware parses it accordingly and sends it off to the correct part of the software.
00:23:05.180
Now, let's discuss printing issues. As we tested our system, I encouraged participants to send texts to a specific number so I could demonstrate label printing.
00:24:03.550
Please keep in mind that printing fluidly from different devices and configurations might lead to inconsistencies based on how each device is set-up.
00:25:09.880
We printed various labels and fun symbols for demonstration purposes, showcasing how our systems interacted successfully.
00:26:05.500
Throughout this process, we did encounter challenges, especially when certain devices didn’t support multiple socket connections.
00:27:00.030
This led us to ensure that our hardware maintained connections accurately and was prepared for potential disconnections.
00:27:47.500
We ensured everything was on a secure private network to prevent unauthorized access to our devices.
00:28:32.500
Testing was often overlooked, which required us to pay more attention when dealing with physical devices that introduced unpredictable variables.
00:29:12.510
By constantly testing the hardware and deploying new features flexibly, we minimized potential failures thoroughly.
00:30:19.150
Today, I'm happy to report that we've been using Ruby in our production environment for over a year now.
00:31:07.800
We launched the shoe sorter in one facility, and that's now expanded to three different facilities, which has been highly rewarding!
00:31:32.750
My message to you is that if someone suggests using Python, consider Ruby. It's vital to know that Ruby is not just limited to being a web language.
00:32:14.120
Experimentation and exploration should be embraced wholeheartedly!
00:32:30.480
Thank you for your attention, and I would be happy to answer any questions!
00:32:54.790
It's always fun to experience unexpected situations, and I appreciate the opportunity for interaction through the discussion.
00:33:10.980
One question we often get involves handling different devices and the possible data issues due to their unique configuration.
00:34:03.709
In our case, we generally send strings as the default format and have protocols to translate between different equipment.
00:34:52.620
Establishing a fixed protocol is key, especially since we worked with older designs of devices which sometimes created conflicts.
00:35:43.180
Continuing to streamline these integrations makes the system as a whole much smoother based on the interactions of this hardware.
00:36:09.510
Testing the layout of label printing required careful attention to ensure proper placement of elements without overlap.
00:36:50.510
Our team came up with creative solutions, allowing us to adjust dynamically based on varying data lengths.
00:37:30.800
Despite facing obstacles, we successfully integrated new technology into our processes, which led to significant improvements.
00:38:32.560
We must always be prepared for the unexpected when it comes to software, particularly when interfacing with hardware.
00:39:24.290
While there may not always be clear solutions, each encounter allows us to build and refine our processes moving forward.
00:40:20.150
By maintaining open lines of communication with team members, we improve our synergy while also anticipating issues before they arise.
00:40:52.000
Having questions and answers about the data security measures we use throughout the protocols has ensured effective communication.
00:41:39.800
Everyone’s unique functionality plays an important part in overall success, so we will continue to update and refine these pages.
00:42:07.300
I appreciate your time and attention today. Enjoy exploring and have fun!
Explore all talks recorded at RubyConf AU 2018
+8