00:01:36.900
Okay, welcome! So we're here to talk about a droid journey. This is our experimentation with mruby using it inside of Go.
00:01:42.390
My name is Chase McCarthy. You might know me on the internet as Code for Fun, but I've been having a lot of fun here in Japan recently.
00:01:55.680
Oh, and hi! I'm Terence. I go by 'Hozier' on Twitter. I work for Heroku, managing our apps on the platform.
00:02:06.750
You may have used code that I've written. We both come from Austin, Texas, which I like to say is the capital of awesome tacos. If you're ever visiting Austin, I'd be more than happy to take you out for some tacos and show you some delicious food!
00:02:18.480
I also run a conference called Keep Austin Weird. We're throwing one this fall, so if you're in town for that, please stop by!
00:02:39.570
This conference talk started around last summer, in August 2017, when a droid company called Sphero announced an R2-D2 droid. I was pretty excited by this because since I was a kid, I've been a big Star Wars fan, growing up watching a bunch of the old movies.
00:02:55.230
This R2-D2 droid, based on the trailers, seemed to be one of the most impressive replicas they'd ever made, with lights, sounds, and movement, including tripod and bipod functionalities that you'll be seeing in the talk today.
00:03:14.190
They also had a phone app that you used to control it. Historically, Sphero has enabled you to programmatically control the robots that you buy, and I was really interested in being able to do that using Ruby.
00:03:26.580
We wanted to write Ruby programs to control these robots in the language that we love. So, we started exploring what it would look like to program robots in general, unrelated to the Sphero stuff, and what was out there.
00:03:38.430
The most promising and prominent project we found was called R2 by the Hypergroup, which has some support for Sphero droids, but it only supports Bluetooth Classic. Unfortunately, all these Star Wars robots require Bluetooth Low Energy, also known as BLE.
00:03:56.500
Since September 2016, there hasn't been good BLE support in Ruby. Because of this, we haven't been able to make the BB-8 robot, which came out in 2015, work properly. So, we couldn't really get this to work well.
00:04:11.820
We started looking at a project called Gobot, which is also developed by the Hypergroup. It's a library for controlling robots using Go. One of the major differentiators for us is that it uses Gobot BLE, which is a Go implementation of BLE library that works on Linux and Mac OS. This allows us to control these robots.
00:04:40.170
Gobot actually has built-in support for controlling BB-8. If you visit their webpage, there's sample code available that demonstrates how to change the LED colors on the BB-8. To start, you need to instantiate a new robot, which takes the connection that you'll use to interface with the robot.
00:05:06.139
In our case, we establish a new BLE connection, and then pass that into the New Robot function. The next step is to instantiate a BB-8 driver, or whatever driver you're using to control the robot. Finally, you define the functions that you want to execute.
00:05:40.210
Once you call 'start', it executes the code you defined to connect to the robot and run it. I will demonstrate running this code shortly. Here’s the camera view—you can see it on the screen. The program picks random RGB values and sends those, lighting up the colors, showing an example of Gobot using BLE to control BB-8.
00:06:02.520
Unfortunately, while we had BB-8 support, there was no R2-D2 support, so we couldn't control the other two droids we have. The first challenge we faced was figuring out how to get information about what APIs are available.
00:06:17.890
As we mentioned, these APIs don't publicly share their specifications. We knew they were slightly different from Sphero's, and we did know that they operated using Bluetooth, since if you don’t have Bluetooth, the app doesn’t work.
00:06:37.830
Additionally, we know they use Bluetooth Low Energy since you don’t need to pair with them for them to be detected. A quick terminology note: BLE stands for Bluetooth Low Energy, which was introduced in the Bluetooth 4.0 standard.
00:06:53.129
The GATT, which I may refer to later, stands for Generic Attribute Profile, and it’s related to BLE. Attributes are essentially values that you can read, write, or do both with them. Generic attributes act as plugins for BLE.
00:07:08.550
To control these devices, we first need to identify what kind of services they offer. Bluetooth services encapsulate the behaviors of a device's component, almost like a collection of data and associated behaviors.
00:07:25.430
Services are higher level, akin to services on a web server, with multiple instances available. Each service will have its own API, which refers to the specific capabilities it provides, called characteristics.
00:07:46.020
To find these services, I'll quickly describe a series of attempts I made to figure things out. Some of them failed, but I mention them because if you try something similar, you may encounter failures—it’s common in the Bluetooth space.
00:08:02.840
The first attempt was to use BlueZ, a set of command-line utilities and libraries for Linux. The command-line utilities are not necessarily stable, but I found the 'hcitool lescan' to be reliable.
00:08:20.560
If you run this command right now in this room, it will show numerous devices. For example, we would see something like 'VD2' as R2-D2, with the MAC address on the left and the name on the right.
00:08:37.210
There's another tool called 'gatttool' which, based on blog posts from a few years ago, many thought would allow direct connection. You could run it, give it a MAC address, and an interactive shell would open up.
00:08:53.000
However, after connecting successfully, you'd receive an error after a short time, leading to a typical observation—this library has been in a broken state for the last few years; sadly, it doesn't work.
00:09:09.610
The second attempt involved Bluetooth sniffing. I had some different devices lying around, one costing about $25. While it could pick up traffic, it doesn’t decode encrypted traffic, which BLE devices utilize.
00:09:31.670
The third attempt was a man-in-the-middle approach. I thought about how the phone connects with BB-8, querying services and characteristics while logging the commands sent and responses received.
00:09:47.120
In this scenario, if a device is close enough, they can communicate directly with one another, complicating the packet manipulation process due to interference.
00:10:04.630
A more complicated approach that I got working was to separate the devices by distance and proxy them over the internet or LAN. It worked but seemed complicated.
00:10:20.510
Fortunately, there's a tool called 'bleah' that utilizes a JavaScript-based library called 'noble', which is the equivalent of the libraries we are using. I managed to get this working, albeit with quite a bit of setup.
00:10:39.780
However, it was much more work than anticipated, especially since I wanted to iterate on these concepts multiple times.
00:10:58.190
Luckily, if you have an Android device running 7 or later, enabling Bluetooth snooping allows your phone to record all Bluetooth traffic.
00:11:18.780
Once enabled, you can connect your phone to one of the devices, execute custom commands, and then in the developer settings, you can trigger a full bug report.
00:11:44.090
This report will include information about the traffic collected, which you can download and analyze.
00:11:54.480
You'll find information about service UUIDs and characteristic IDs, as well as the payload data sent over Bluetooth connections, which is critical for understanding what commands your device responds to.
00:12:12.700
By capturing data from multiple attempts and identifying patterns, I was able to learn how to structure packets effectively to communicate with the droids.
00:12:32.530
For example, commands involved incrementing a sequence number to prevent replay attacks, while checksums provided validation to ensure packet integrity.
00:12:54.390
Using Gobot, we wanted to go beyond pure Go, leveraging the capabilities of mruby to embed Ruby while maintaining control over robot functionality.
00:13:14.130
This enabled us to orchestrate higher-level functionalities like Go routines and channels for managing concurrent tasks.
00:13:30.500
We set out to define a Ruby interface so that users could program robots in Ruby without diving into the intricacies of Go.
00:13:49.470
By creating a simple API, users would seamlessly interact with robotic capabilities while maintaining extensibility, allowing for further development in Ruby.
00:14:05.620
To demonstrate, we instantiate an R2-D2 object while providing a name that identifies the Bluetooth connection.
00:14:25.590
In a continuous loop, we can call methods to sweep the droid's dome back and forth, showcasing control over these functionalities in real-time.
00:14:44.190
As you can see, the dome sweeps back and forth, demonstrating the effectiveness of our setup. This process is repeated for more complex behaviors.
00:15:09.860
As we reviewed the architecture of our solution for controlling the droids, we started multiple workers to manage the mruby virtual machines.
00:15:24.569
The goal was to allow for concurrent execution of multiple scripts while capturing input for controlling multiple robots seamlessly.
00:15:39.930
We utilize a wait group in Go to synchronize the completion of tasks that are initiated by the mruby VMs.
00:15:57.200
As the process unwinds, we ensure the robots receive commands only when they are ready, leveraging synchronization mechanisms inherent to Go.
00:16:14.750
When a command, say to move the dome, is executed, it passes through to the Go code, which handles crafting the appropriate packet to send.
00:16:32.180
An example of this can be seen when we run a simple Ruby script to control the R2-D2 droid, showing that our driver is functioning correctly.
00:16:49.440
By ensuring our functions are well-defined, we can trigger multiple actions within a single script, calling upon various behaviors of the robots.
00:17:06.170
Running everything together allows us to observe how actions can be coordinated effectively among various devices, demonstrating the possibilities with our setup.
00:17:22.620
At this point, we've showcased significant work, allowing the control of robots through a seamless integration of mruby and Go. The goal was to enable Ruby to be used to control robotic devices effectively.
00:17:40.590
GitHub repository for our project is available if you'd like to see the source code and explore the work we have done.