00:00:12.789
Hey everyone! My name is Danielle, and I'm a software engineer from New York. Thank you all for coming to see me speak.
00:00:18.160
This is my first time at RailsConf, so this is pretty exciting, though a little nerve-wracking.
00:00:25.330
Just about half an hour ago, I had to run back to my Airbnb to grab my labels because I completely forgot a pretty vital piece of my demo.
00:00:37.329
Hopefully, it can only go up from here! So, I'm here to talk about how we can integrate Rails with different types of protocols.
00:00:44.530
Let's get started! My Twitter handle is @AdamsDanielle, and my GitHub is DanielleAdams.
00:00:51.370
If you have any questions afterwards that maybe don't get answered, feel free to tweet me. There are going to be some code samples that you can find on my GitHub.
00:01:05.019
A little bit about me: I work at a company called Blue Apron. You might have seen our podcast sponsorships; that's usually how I hear about us.
00:01:10.860
I work on the warehouse management system team, which means that all of the software built within the warehouses for our employees is something my team helps build.
00:01:17.800
Another important fact about me is that I have a cat named Dennis. Like most cat owners, I spend about 90% of my free time taking pictures of him while he sleeps.
00:01:31.030
Some notable achievements include posting a tweet that went viral the day the internet went down. This was really exciting for me, as I’m not a big Twitter user. It had 400 engagements and ended up on Twitter moments.
00:01:50.500
Another notable achievement is that I was part of a team that introduced automation and hardware into Blue Apron's warehouse system last summer.
00:02:08.739
That’s what I’m here to talk to you about: how we can integrate Ruby on Rails and expand beyond web development to actually start interfacing with hardware.
00:02:21.380
So, first, I'm going to go back in time to spring of last year. Originally, Blue Apron decided we wanted to introduce a lot of automation into our software.
00:02:27.230
Up until that point, everything we were doing was web development based. We had many different web applications that we were interacting with, all built on Rails.
00:02:40.360
We started realizing that to help the company scale, we had to integrate with the printers and scanners we were using in the warehouses.
00:03:01.460
We needed to figure out what that integration would look like because we didn’t want to face latency issues or struggles with handling multiple connections.
00:03:12.470
We began asking ourselves if Rails and Ruby were the right tools for this job, or if we would need to switch to Python or do everything in multi-threading.
00:03:20.540
These considerations were vital because introducing new hardware was adding a layer of complexity to our system.
00:03:31.060
Here’s a high-level overview of what I’m going to talk about today: first, we will go over the different types of protocols we needed to use.
00:03:39.560
Then I will discuss how to integrate Ruby with machines, and finally, how to integrate hardware with a Rails system.
00:03:51.410
At the time, we were primarily focused on web development and had not done much software development involving hardware and automation.
00:04:00.560
All of our software was capable of speaking HTTP, but what we needed was a system that could communicate over TCP, which is the language the hardware spoke.
00:04:10.910
To understand how to make this leap, let’s first take a look at how an HTTP request works and how Rails handles it.
00:04:24.820
When a browser makes a request, the IP is resolved by DNS. In the middle, there might be a caching layer, and it might reach a load balancer.
00:04:47.180
Once it reaches our Rails application, it interacts with our application server, such as Puma or Unicorn, goes through Rack middleware, and finally reaches the Rails router and controllers.
00:05:05.810
So, what is TCP? TCP, or Transmission Control Protocol, was introduced in 1981 with the intent of serving the US Department of Defense.
00:05:25.880
It is a highly reliable host-to-host protocol used in packet-switched computer networks.
00:05:37.729
For those unfamiliar, RFC stands for 'Request for Comments,' which is a common method for introducing new ideas in computer and software development.
00:05:51.680
Now, how is TCP different from HTTP? Both are part of the same Internet Protocol Suite: HTTP operates at the application layer while TCP functions at the transport layer.
00:06:12.710
When we looked at different languages and frameworks for supporting this automation by summer, we decided that Ruby and Rails were the perfect fit.
00:06:34.219
We discovered that Ruby could indeed support lower-level connections, so we began figuring out how to integrate Ruby on Rails with the hardware.
00:06:46.520
In our warehouses, we used laptops running localhost to connect with different devices as we experimented to see what would work.
00:07:10.430
Now let's discuss Ruby and its integration with machines, highlighting how we discovered the interoperability with our Rails systems.
00:07:27.770
For those who may not know, Ruby has a socket library that comes with the standard library. It’s available as soon as you install Ruby.
00:07:36.020
A socket is simply a set of two values: an IP address and a port, which distinguish a unique endpoint in any computer network.
00:07:49.340
The Ruby socket library has a class called TCP Socket, which is built on the base Socket class. It provides all the necessary functionality for opening a socket to any device.
00:08:00.860
Typically, the flow of data looks like this: a client, such as a browser, makes a request to our Rails server, which processes updates to a database.
00:08:15.470
However, what we wanted was a client that could communicate directly with a printer.
00:08:20.539
To do this, we needed to establish a TCP socket.
00:08:30.930
In Ruby, establishing a connection might look like this: we import the socket library and create a socket object defined by an IP and a port number.
00:08:48.020
Once we write to the socket, we close it. On the server side, we can use the TCP Server class to handle incoming socket connections.
00:09:01.210
The code would establish a TCP server that listens for any client connections. Once a connection is made, it acknowledges the client, reads the input, and then closes the socket.
00:09:19.220
Examples of devices we can use with this library include data logic scanners and Zebra printers.
00:09:31.090
In a moment, I’ll do a demo showcasing what the client-server relationship looks like.
00:09:43.540
The server process will be listening, and I will show how it receives messages.
00:09:50.840
To illustrate, we can start the server on a chosen port, say 4000, to demonstrate the process.
00:10:06.340
On the client side, we would write a script that opens a socket, sends a message—say, 'Hello, Rails!'—and then closes it.
00:10:17.670
As we can see, once the client message is received, it outputs the message and closes the connection.
00:10:29.360
Fast forward to last winter, as we began integrating printers and scanners into the production environment.
00:10:47.000
We introduced a large machine into our warehouses, a shoe sorter, which directs packages on a conveyor belt according to scans.
00:11:02.660
This process is deeply latency-sensitive, so we had to be smart about integrating this with our Rails systems.
00:11:21.050
To ensure efficiency, we opted for asynchronous processes in our Rails application, avoiding any blocking behavior between tasks.
00:11:38.200
We didn't want our Rails application to be overwhelmed with tasks, leading to inefficiencies during operation.
00:11:54.120
Implementing non-blocking integration meant that we could handle connections to the hardware while concurrently managing database updates.
00:12:10.770
For example, when a browser requests printing, that request triggers a worker in the Rails controller, which can then communicate with the printer.
00:12:27.430
We're using Sidekiq to handle this process cleanly, managing print requests without blocking user experiences.
00:12:44.500
We define our labels and printers clearly in the database, enabling the system to work efficiently.
00:12:57.540
Another scenario involves a scanner that needs to update a database but speaks TCP, while the database requires its own protocol.
00:13:09.500
We employ a daemon process listening for scans, which triggers a corresponding worker to update our database accordingly.
00:13:25.920
This shows how different processes in our application can operate seamlessly in the background to keep data synchronized.
00:13:40.720
When integrating different devices, we encounter various protocols, creating complexity in message parsing.
00:13:54.620
For instance, Zebra printers speak GPL, a label definition and printer control language.
00:14:06.530
This language allows us to define the delivered messages in a precise format, just like an HTML string.
00:14:20.200
Formatting errors or miscommunications can easily result in print failures, so we must align expectations meticulously.
00:14:34.160
Once the message flows from the printer to the application, it often appears completely different and requires appropriate handling.
00:14:49.180
To manage this complexity, implementing shared middleware allows us to parse messages accurately, preserving context.
00:15:03.340
Now, let's see a demo showcasing the device queue and how the daemon and Rails application collaborate.
00:15:17.200
While demonstrating, I’ll show you the entire process with our setup, including the connection between devices.
00:15:32.060
Initially, there were no labels in our system. Let’s create some labels for demonstration.
00:16:01.140
We’ve created three labels which allow us to illustrate how the system processes them.
00:16:13.840
I’ll need two volunteers to provide their names and titles so we can use those values for our labels.
00:16:25.780
We will print these labels and then demonstrate how the scanning updates work by refreshing our label dashboard.
00:16:42.800
Now, let's review the dashboard after scanning one of the labels to see how the updates reflect in the database.
00:17:02.010
After scanning labels for Megan and Greg, both names received their updates in the database.
00:17:18.300
This was an example highlighting how Rails processes can run simultaneously with hardware interactions.
00:17:38.900
Now let's explore some obstacles that emerged along our journey.
00:17:55.720
Not all hardware can maintain multiple socket connections simultaneously, which can lead to complications during scaling.
00:18:10.420
When multiple devices need to communicate, we must ensure that each message routes correctly to prevent silent failures.
00:18:25.390
To ensure persistent sockets remain open, we must also be mindful of frameworks that may not support that adequately.
00:18:40.030
Security is also crucial; devices must remain on a private network to prevent unauthorized access.
00:18:59.410
Maintaining good practices with open sockets involves working with hosting services that support these security restrictions.
00:19:16.860
Finally, testing proves to be a larger challenge when integrating with hardware.
00:19:34.790
Development testing is simpler in a standard web app, but with hardware, the physicality of devices adds extra complexity.
00:19:50.460
Anticipating issues like unplugged devices or broken connections becomes crucial during manual testing.
00:20:03.560
Concluding my talk, I’m pleased to say that the software we developed is actively in use in Blue Apron warehouses across the country.
00:20:20.870
This integration ensures that boxes are being delivered efficiently, which is incredibly satisfying.
00:20:38.590
My message to all of you is to think beyond traditional web development and explore what Ruby can do with integration.
00:20:54.550
Thank you very much, everyone!
00:21:10.500
In terms of keeping printers on a private network, one possible solution could involve using localized servers.
00:21:48.740
Yet, maintaining your software on a local network requires managing its updates and stability.
00:22:02.230
When coordinating communication between devices that only accept one connection, it comes down to designing for simplicity.
00:22:19.520
In our warehouses, we accommodate each machine’s needs with separate processes handling those connections.
00:22:31.470
Addressing future proofing, we're exploring utilization of ERP systems for rendering Zebra printer commands.
00:22:45.530
Linking these devices to a single deployment pipeline was a pivotal part of our design.
00:23:01.040
We ensure no downtime during updates through Blue/Green deployments and remain diligent in maintaining these connections.
00:23:17.320
Feel free to reach out if you have any questions; I’m always passionate about discussing this integration.