Search
Terence Lee
Controlling Droids™ with mruby & Go
Edit
#mruby
#go
#robotics
#bluetooth-low-energy
#concurrency
#domain-specific-language-dsl
Summarized using AI

Controlling Droids™ with mruby & Go

Terence Lee and Chase McCarthy • May 31, 2018 • Sendai, Miyagi, Japan

In the talk titled 'Controlling Droids™ with mruby & Go' presented at RubyKaigi 2018 by Terence Lee and Chase McCarthy, the focus is on utilizing the Ruby programming language, specifically mruby, in conjunction with Go to control robotic devices, particularly Sphero's R2-D2 droid. The speakers discuss the challenges they faced while integrating Ruby with robotics, a domain not typically associated with Ruby. They elaborate on how they used mruby to enhance Ruby's capabilities for real-time robot control through Go's event-driven model.

Key Points:

  • Introduction of the Speakers: Chase McCarthy and Terence Lee introduce themselves and their backgrounds, highlighting their passion for robotics and Ruby programming.
  • Interest in R2-D2: The interest sparked by Sphero's announcement of the R2-D2 droid leads them to explore programming it using Ruby.
  • Challenges with BLE: The presenters discuss limitations in existing Ruby libraries concerning Bluetooth Low Energy (BLE), which is essential for controlling modern droids.
  • Discovery of Gobot: They found Gobot, a Go framework that facilitated BLE communications, which allowed control over robots like BB-8.
  • API Investigation: They outline their efforts to understand the APIs available for Sphero droids and the challenges they faced due to the lack of published specifications.
  • Bluetooth Tools: Several attempts to identify services and commands using tools like BlueZ and gatttool are discussed, as well as issues encountered with connectivity.
  • Innovative Solutions: The use of Android phones for Bluetooth traffic debugging is highlighted as a novel method to gather necessary command information.
  • Integration of mruby and Go: The presentation details how mruby was embedded in Go to allow Ruby scripts to orchestrate robot functionalities and manage concurrent tasks efficiently.
  • Live Demonstration: A demonstration is provided showing the R2-D2 droid controlled through a simple Ruby script, illustrating how commands are sent and executed.
  • Conclusion and Repository Access: They conclude by emphasizing the potential for using Ruby alongside Go to control robotic systems effectively and offer access to their project via a GitHub repository.

Main Takeaways:

  • Mixing mruby with Go creates a powerful framework for controlling robotics using familiar Ruby syntax, opening possibilities for developers.
  • The project showcases innovative approaches to overcoming typical hurdles in integrating Ruby with hardware control, leveraging Go’s strengths while enabling high-level scripting capabilities.
  • The speakers encourage the audience to explore their source code and ideas for future developments in robot programming using Ruby.

Controlling Droids™ with mruby & Go
Terence Lee and Chase McCarthy • May 31, 2018 • Sendai, Miyagi, Japan

Ruby has never been at the forefront of dealing with robots, IoT, or other low level systems. What Ruby is great at is scripting and building DSLs. Using mruby we can leverage existing ecosystems while still using the language we love.

In this talk, we'll deep dive into how we can execute mruby handlers inside a Go event reactor to control a Sphero R2-D2. With surprisingly few lines of code, you can coordinate motors, lights, and sound concurrently. Come learn about mruby & robotics and see the Droids™ you're looking for in action.

RubyKaigi 2018 https://rubykaigi.org/2018/presentations/hone02

RubyKaigi 2018

00:01:36.900 Okay, welcome! So we're here to talk about a droid journey. This is our experimentation with mruby using it inside of Go.
00:01:42.390 My name is Chase McCarthy. You might know me on the internet as Code for Fun, but I've been having a lot of fun here in Japan recently.
00:01:55.680 Oh, and hi! I'm Terence. I go by 'Hozier' on Twitter. I work for Heroku, managing our apps on the platform.
00:02:06.750 You may have used code that I've written. We both come from Austin, Texas, which I like to say is the capital of awesome tacos. If you're ever visiting Austin, I'd be more than happy to take you out for some tacos and show you some delicious food!
00:02:18.480 I also run a conference called Keep Austin Weird. We're throwing one this fall, so if you're in town for that, please stop by!
00:02:39.570 This conference talk started around last summer, in August 2017, when a droid company called Sphero announced an R2-D2 droid. I was pretty excited by this because since I was a kid, I've been a big Star Wars fan, growing up watching a bunch of the old movies.
00:02:55.230 This R2-D2 droid, based on the trailers, seemed to be one of the most impressive replicas they'd ever made, with lights, sounds, and movement, including tripod and bipod functionalities that you'll be seeing in the talk today.
00:03:14.190 They also had a phone app that you used to control it. Historically, Sphero has enabled you to programmatically control the robots that you buy, and I was really interested in being able to do that using Ruby.
00:03:26.580 We wanted to write Ruby programs to control these robots in the language that we love. So, we started exploring what it would look like to program robots in general, unrelated to the Sphero stuff, and what was out there.
00:03:38.430 The most promising and prominent project we found was called R2 by the Hypergroup, which has some support for Sphero droids, but it only supports Bluetooth Classic. Unfortunately, all these Star Wars robots require Bluetooth Low Energy, also known as BLE.
00:03:56.500 Since September 2016, there hasn't been good BLE support in Ruby. Because of this, we haven't been able to make the BB-8 robot, which came out in 2015, work properly. So, we couldn't really get this to work well.
00:04:11.820 We started looking at a project called Gobot, which is also developed by the Hypergroup. It's a library for controlling robots using Go. One of the major differentiators for us is that it uses Gobot BLE, which is a Go implementation of BLE library that works on Linux and Mac OS. This allows us to control these robots.
00:04:40.170 Gobot actually has built-in support for controlling BB-8. If you visit their webpage, there's sample code available that demonstrates how to change the LED colors on the BB-8. To start, you need to instantiate a new robot, which takes the connection that you'll use to interface with the robot.
00:05:06.139 In our case, we establish a new BLE connection, and then pass that into the New Robot function. The next step is to instantiate a BB-8 driver, or whatever driver you're using to control the robot. Finally, you define the functions that you want to execute.
00:05:40.210 Once you call 'start', it executes the code you defined to connect to the robot and run it. I will demonstrate running this code shortly. Here’s the camera view—you can see it on the screen. The program picks random RGB values and sends those, lighting up the colors, showing an example of Gobot using BLE to control BB-8.
00:06:02.520 Unfortunately, while we had BB-8 support, there was no R2-D2 support, so we couldn't control the other two droids we have. The first challenge we faced was figuring out how to get information about what APIs are available.
00:06:17.890 As we mentioned, these APIs don't publicly share their specifications. We knew they were slightly different from Sphero's, and we did know that they operated using Bluetooth, since if you don’t have Bluetooth, the app doesn’t work.
00:06:37.830 Additionally, we know they use Bluetooth Low Energy since you don’t need to pair with them for them to be detected. A quick terminology note: BLE stands for Bluetooth Low Energy, which was introduced in the Bluetooth 4.0 standard.
00:06:53.129 The GATT, which I may refer to later, stands for Generic Attribute Profile, and it’s related to BLE. Attributes are essentially values that you can read, write, or do both with them. Generic attributes act as plugins for BLE.
00:07:08.550 To control these devices, we first need to identify what kind of services they offer. Bluetooth services encapsulate the behaviors of a device's component, almost like a collection of data and associated behaviors.
00:07:25.430 Services are higher level, akin to services on a web server, with multiple instances available. Each service will have its own API, which refers to the specific capabilities it provides, called characteristics.
00:07:46.020 To find these services, I'll quickly describe a series of attempts I made to figure things out. Some of them failed, but I mention them because if you try something similar, you may encounter failures—it’s common in the Bluetooth space.
00:08:02.840 The first attempt was to use BlueZ, a set of command-line utilities and libraries for Linux. The command-line utilities are not necessarily stable, but I found the 'hcitool lescan' to be reliable.
00:08:20.560 If you run this command right now in this room, it will show numerous devices. For example, we would see something like 'VD2' as R2-D2, with the MAC address on the left and the name on the right.
00:08:37.210 There's another tool called 'gatttool' which, based on blog posts from a few years ago, many thought would allow direct connection. You could run it, give it a MAC address, and an interactive shell would open up.
00:08:53.000 However, after connecting successfully, you'd receive an error after a short time, leading to a typical observation—this library has been in a broken state for the last few years; sadly, it doesn't work.
00:09:09.610 The second attempt involved Bluetooth sniffing. I had some different devices lying around, one costing about $25. While it could pick up traffic, it doesn’t decode encrypted traffic, which BLE devices utilize.
00:09:31.670 The third attempt was a man-in-the-middle approach. I thought about how the phone connects with BB-8, querying services and characteristics while logging the commands sent and responses received.
00:09:47.120 In this scenario, if a device is close enough, they can communicate directly with one another, complicating the packet manipulation process due to interference.
00:10:04.630 A more complicated approach that I got working was to separate the devices by distance and proxy them over the internet or LAN. It worked but seemed complicated.
00:10:20.510 Fortunately, there's a tool called 'bleah' that utilizes a JavaScript-based library called 'noble', which is the equivalent of the libraries we are using. I managed to get this working, albeit with quite a bit of setup.
00:10:39.780 However, it was much more work than anticipated, especially since I wanted to iterate on these concepts multiple times.
00:10:58.190 Luckily, if you have an Android device running 7 or later, enabling Bluetooth snooping allows your phone to record all Bluetooth traffic.
00:11:18.780 Once enabled, you can connect your phone to one of the devices, execute custom commands, and then in the developer settings, you can trigger a full bug report.
00:11:44.090 This report will include information about the traffic collected, which you can download and analyze.
00:11:54.480 You'll find information about service UUIDs and characteristic IDs, as well as the payload data sent over Bluetooth connections, which is critical for understanding what commands your device responds to.
00:12:12.700 By capturing data from multiple attempts and identifying patterns, I was able to learn how to structure packets effectively to communicate with the droids.
00:12:32.530 For example, commands involved incrementing a sequence number to prevent replay attacks, while checksums provided validation to ensure packet integrity.
00:12:54.390 Using Gobot, we wanted to go beyond pure Go, leveraging the capabilities of mruby to embed Ruby while maintaining control over robot functionality.
00:13:14.130 This enabled us to orchestrate higher-level functionalities like Go routines and channels for managing concurrent tasks.
00:13:30.500 We set out to define a Ruby interface so that users could program robots in Ruby without diving into the intricacies of Go.
00:13:49.470 By creating a simple API, users would seamlessly interact with robotic capabilities while maintaining extensibility, allowing for further development in Ruby.
00:14:05.620 To demonstrate, we instantiate an R2-D2 object while providing a name that identifies the Bluetooth connection.
00:14:25.590 In a continuous loop, we can call methods to sweep the droid's dome back and forth, showcasing control over these functionalities in real-time.
00:14:44.190 As you can see, the dome sweeps back and forth, demonstrating the effectiveness of our setup. This process is repeated for more complex behaviors.
00:15:09.860 As we reviewed the architecture of our solution for controlling the droids, we started multiple workers to manage the mruby virtual machines.
00:15:24.569 The goal was to allow for concurrent execution of multiple scripts while capturing input for controlling multiple robots seamlessly.
00:15:39.930 We utilize a wait group in Go to synchronize the completion of tasks that are initiated by the mruby VMs.
00:15:57.200 As the process unwinds, we ensure the robots receive commands only when they are ready, leveraging synchronization mechanisms inherent to Go.
00:16:14.750 When a command, say to move the dome, is executed, it passes through to the Go code, which handles crafting the appropriate packet to send.
00:16:32.180 An example of this can be seen when we run a simple Ruby script to control the R2-D2 droid, showing that our driver is functioning correctly.
00:16:49.440 By ensuring our functions are well-defined, we can trigger multiple actions within a single script, calling upon various behaviors of the robots.
00:17:06.170 Running everything together allows us to observe how actions can be coordinated effectively among various devices, demonstrating the possibilities with our setup.
00:17:22.620 At this point, we've showcased significant work, allowing the control of robots through a seamless integration of mruby and Go. The goal was to enable Ruby to be used to control robotic devices effectively.
00:17:40.590 GitHub repository for our project is available if you'd like to see the source code and explore the work we have done.
Terence Lee
@hone
Chase McCarthy
@code0100fun
Explore all talks recorded at RubyKaigi 2018
+62