00:00:06.710
Good afternoon, everybody.
00:00:12.240
Thank you very much for being here. There are so many other interesting talks, but you chose to come here.
00:00:17.760
Oh, I'm Shashank Daté.
00:00:23.279
This is my hobby, and I do it for fun and, unfortunately, some losses.
00:00:29.119
I have lost a couple of drones and never got them back.
00:00:35.040
Now, to get the energy flowing in this room, I want you all to participate.
00:00:41.440
On the count of three, I want you all to read the next slide as loudly as you can so that the energy in this room goes up.
00:00:48.320
Okay? Alright. One, two, three.
00:01:00.079
Thank you! Now, if you have blessed me, I'll show you why that blessing is required.
00:01:07.040
And here, pretty soon, these two should take off using my hello_edu.rb file.
00:01:13.439
I have to do one little thing that I forgot.
00:01:19.040
Give me just a second; I need to find the IP address of these two drones.
00:01:25.759
When they booted, they received different IP addresses.
00:01:31.439
The way I communicate with them is through a Wi-Fi router that I have set up here.
00:01:37.680
My laptop is connected to them through that, so I'm going to scan for these devices.
00:01:43.920
It shouldn't take more than a few seconds to detect their IP addresses.
00:01:49.520
The last three digits are 102 and 103.
00:01:56.320
If our energy is right, they're going to take off and land— not exactly in the same spot, but somewhere nearby.
00:02:02.719
If they start drifting, I have drone catchers in the audience who will catch them.
00:02:08.560
Okay? Let's see if it turns green. There you go.
00:02:24.239
That’s like my teenage daughter who doesn’t listen to me. She has a knife.
00:02:31.519
It will land in a few seconds; it has a default 15-second wait time.
00:02:44.560
They behaved much better today than they have in the past.
00:02:51.440
Okay, so now, these are a special class of drones called Tello drones.
00:02:57.840
Tello is the type of drone that I've been programming.
00:03:04.879
These are probably the cheapest programmable drones you can find at under $150 each.
00:03:10.959
The first few that I lost were during tests to see how high they could fly, but then the wind came up and they were gone.
00:03:19.040
Nonetheless, they are perfect for educational purposes.
00:03:25.760
They introduce beginners to programming.
00:03:31.920
For those of you who attended the previous talk, it mentioned a language called Logo, which taught kids to draw diagrams using turtles on a 2D plane.
00:03:38.159
These drones represent a 3D version of that. You can move not only horizontally but also vertically.
00:03:43.840
The purpose of these drones is to teach programming.
00:03:50.319
While there are many simple concepts to grasp, there are also deeper constructs that more advanced programmers can benefit from.
00:03:57.439
The type of drones I’m using are called Tello EDU type drones.
00:04:02.799
They have progressed from SDK 1.0 to 2.0, which includes the EDU capabilities we just saw.
00:04:09.680
This enables us to program swarms of drones.
00:04:15.519
I didn't have the budget to buy the full EDU kit that comes with 20 drones and costs about $5,000.
00:04:20.560
But I bought a couple and started experimenting.
00:04:26.400
These kits come with official support for Python, Scratch, and Swift.
00:04:32.880
I thought, 'Why not Ruby?' So, I took it on myself.
00:04:38.160
I got it working in Ruby as well.
00:04:44.080
These drones require concepts of network programming and concurrent communication.
00:04:49.280
They also involve video streaming and related tasks.
00:04:55.600
In terms of specs, these drones have an Intel Core i4 processor.
00:05:01.680
They are very lightweight and come equipped with a camera.
00:05:08.639
Believe it or not, they also have collision detection sensors.
00:05:14.320
There are two flight modes available in the EDU that you just saw.
00:05:19.680
There’s AP mode and Station mode.
00:05:24.880
AP mode is the access point mode that comes by default when you pull it out of the box.
00:05:30.000
In this mode, you can only control one drone directly from a single computer.
00:05:36.560
A single computer cannot control multiple drones.
00:05:41.680
I have never tried controlling multiple computers to manage one drone.
00:05:47.520
In this mode, video streaming from the camera is possible.
00:05:52.960
Then there is the swarm mode, also known as station mode, which is available in the second version of the Tello EDU drones.
00:05:58.960
To communicate with the drones in swarm mode, some manual setup is required.
00:06:04.639
With this mode, you use a router to communicate from your laptop to the drones.
00:06:11.039
In the AP mode, the drone itself acts as the access point.
00:06:16.080
It publishes its own SSID and password, which you must capture in your program.
00:06:22.160
This is why switching between those two modes is challenging.
00:06:27.840
These are the two modes of communication.
00:06:33.280
Now, let's talk about the programming prerequisites to control these drones.
00:06:39.120
First, you need UDP/IP communication between the drones.
00:06:44.800
Second, you should have some form of concurrency because you'll be sending commands in one thread while receiving responses in another.
00:06:51.120
You need to keep track of which command corresponds to which response.
00:06:57.120
Sending one command and waiting for the response in the same thread doesn’t work.
00:07:02.479
UDP is fast enough, and we'll see why.
00:07:09.759
Lastly, video streaming is optional; I couldn't get it to work for Ruby.
00:07:16.319
If you want to see real-time video, you need a suitable library in your programming language.
00:07:21.759
In Ruby, there's a UDP socket class available in the standard library.
00:07:28.840
I use that for communication.
00:07:34.240
Ruby 2.x has fibers and threads, and there are also reactors.
00:07:40.720
There are many ways to implement concurrency in Ruby.
00:07:47.680
When it comes to video streaming, I was excited when I discovered Ruby OpenCV.
00:07:54.720
I'll discuss this library a bit later.
00:08:00.319
So what is UDP/IP?
00:08:05.599
You must have heard of TCP/IP. UDP/IP is a sibling of TCP/IP.
00:08:10.639
It’s based on IP sockets and uses datagrams and port numbers for communication.
00:08:18.240
It is connectionless, meaning there is no need for handshaking.
00:08:24.639
You send a command and forget about it.
00:08:30.479
No buffering of datagrams occurs at either end.
00:08:36.479
While it is fast and meant for real-time communication, there is no guarantee of message delivery.
00:08:42.399
Also, commands might return in different orders.
00:08:47.600
Programmers need to keep this in mind when using this protocol.
00:08:53.200
You witnessed an example of this today.
00:08:59.600
One drone took the command to land while the other ignored it due to a packet loss.
00:09:05.200
That's the drawback of UDP; however, speed takes precedence.
00:09:11.839
In Ruby, if you wanted to see an example, this is a basic UDP socket implementation.
00:09:19.360
You require the socket library of Ruby, set up a host and a port,
00:09:26.080
and then prepare a message to be sent to that object.
00:09:31.920
You create a new socket, binding it to the host and port, then send the message.
00:09:37.600
Afterward, you’ll wait for the data to arrive.
00:09:44.320
In this example, it's not threaded; it's waiting for data to return.
00:09:51.040
Since it's essentially sending data to itself, it’s highly reliable.
00:09:56.880
Printing the data reveals the output received.
00:10:01.920
In summary, the first prerequisite for using UDP/IP is satisfied by Ruby.
00:10:07.760
In terms of concurrency, Ruby offers fibers.
00:10:13.600
Fibers handle cooperative concurrency, meaning the programmer must control the flow.
00:10:20.959
Threads provide preemptive concurrency, meaning the programmer has no control over thread switching.
00:10:26.800
Reactors come next, which are still in the experimental phase.
00:10:32.560
Ruby might soon include them as a core part of the language.
00:10:38.079
I wanted to highlight a library called Ruby OpenCV for video streaming.
00:10:44.880
This library wraps the OpenCV C++ library for video processing.
00:10:50.560
Unfortunately, I faced difficulties compiling it since it has been stagnant for eight years.
00:10:56.400
Indeed, I couldn't get OpenCV running properly on my Mac.
00:11:03.680
Because of this, we won't see a video streaming demo today.
00:11:10.560
Now, concerning mruby, by default it has only fibers; there are no threads.
00:11:15.600
To enable socket communication, you need to compile mruby with the socket gem.
00:11:21.600
This core gem comes with the mruby installation.
00:11:27.760
If it's not present in Ruby, it's not likely to be in mruby either.
00:11:34.800
As of now, I haven't seen attempts to integrate video processing or streaming into mruby.
00:11:40.240
For those unfamiliar with mruby, it’s a lightweight implementation of Ruby.
00:11:47.200
The 'm' stands for Embedded Ruby, Modular Ruby, or Minimalistic Ruby.
00:11:53.360
It's a proper subset of Ruby; anything you can do in mruby can also be done in Ruby, but the reverse isn't true.
00:12:00.000
In terms of deployment architectures, mruby can be deployed in more spots and environments than Ruby.
00:12:06.800
A key advantage of mruby lies in its ability to compile your code into a single executable.
00:12:13.440
This reduces dependencies on the operating system library.
00:12:20.079
Typical use cases include bare metal programming.
00:12:26.639
There might not even be an operating system or file system available.
00:12:31.760
For memory-constrained devices, I previously attempted to program point-of-sale systems using mruby.
00:12:38.240
However, it's likely still bulkier than what those devices actually require.
00:12:45.040
With progress made since my last attempt, it may now be more feasible.
00:12:51.920
The latest stable version of mruby (3.0) can launch an app within 100 KB of RAM.
00:12:58.080
The devices I had back then required less than 64 KB of RAM, so I'm skeptical.
00:13:05.600
Nevertheless, progress is ongoing, and this is one of Mats's favorite projects.
00:13:12.480
You'll see regular commits coming into mruby.
00:13:20.079
Game engines and other applications could also see enhancements.
00:13:26.639
Now, moving towards the implementation, you notice one line that is different: there is no require.
00:13:32.560
Why no require? It's because mruby does not assume the existence of a file system.
00:13:40.000
The require statement requires files to exist from a specified path.
00:13:45.440
So, you'll have to recompile the interpreter with the core gem.
00:13:51.440
There's a build configuration script in mruby.
00:13:58.400
If you add an extra line in the build configuration file to compile with mruby, make sure to clean up previous builds.
00:14:05.600
Once you clean the previous version, you can compile it with the socket layer.
00:14:10.240
Once compiled, you’ll be able to run the example.
00:14:16.880
What if you want to convert this program into a single executable?
00:14:23.360
This process is slightly convoluted, so let's break it down.
00:14:30.080
You can compile your example.rb code into a bytecode using the mrbc command.
00:14:43.920
This bytecode can then be distributed to anyone who has mruby installed.
00:14:50.959
They won’t see the original source code, just the bytecode.
00:14:57.679
If you want to go further and ensure the person doesn't need mruby installed,
00:15:04.480
you’ll need some C programming.
00:15:11.200
It’s a two-step process.
00:15:16.000
First, use mrbc to generate a symbol table, which is a data structure for the bytecode.
00:15:23.360
You can assign a name to that table for easy reference.
00:15:30.280
This is important for structuring your code.
00:15:36.320
In the next step, you'll write boilerplate code and include the emitted file.
00:15:44.320
Then you open the VM (virtual machine), which is standard for compiling any mrbc code.
00:15:50.959
Use the generated symbol table in the C file to execute your code.
00:15:56.160
You can compile it using your regular C compiler, and this creates a usable binary file.
00:16:02.079
That binary file can then be distributed without needing mruby installed on the user's machine.
00:16:09.200
So, this is the process to create a native binary.
00:16:15.360
For this talk, my three focus areas were Tello drones, actors and fibers, and mruby.
00:16:23.040
I opted to go deeper into mruby since that's my favorite.
00:16:29.840
Now, how do we get these drones to fly? Are they still on?
00:16:36.240
I should have turned them off earlier.
00:16:41.440
One observation I've made regarding the battery life is that the reported flight time is around 15 minutes.
00:16:47.040
However, I've never experienced a full 15 minutes on a single charge.
00:16:53.440
I usually only get about five minutes of flight time.
00:16:59.919
It’s vital to manage your battery effectively.
00:17:05.440
So, let's see how much time we have left.
00:17:11.600
I have a project on GitHub for Tello, where I’ve written the same code in Ruby, mruby, and Crystal.
00:17:17.760
A friend contributed code in Elixir as well.
00:17:23.760
Remember, those two concurrency prerequisites are essential to control these drones.
00:17:30.480
If you have a favorite programming language not included in the project, please submit a pull request.
00:17:37.440
Today, we're going to focus on the GitHub project where I've curated Ruby files for this conference.
00:17:44.400
The code we saw running was hello_edu.rb, which we'll review.
00:17:50.560
This all started because of a gentleman named Tom Black.
00:17:56.160
In a previous RubyConf, he conducted a five-minute lightning talk showing off these drones.
00:18:02.400
I got excited by his demonstration. Though he used an earlier version without swarm capabilities.
00:18:08.880
I collaborated with him to create the second version of the gem.
00:18:15.760
My pull request is still pending, but I am confident he will merge it once he sees the quality of the code.
00:18:22.640
If you visit that GitHub repo, you'll find a gem that allows you to set up a local Tello server.
00:18:30.320
This way, you don’t even need an actual Tello to test your client code.
00:18:37.680
Tom has put a lot of effort into this gem, and I'm grateful to him.
00:18:42.720
Now, let’s focus on the code we will be modifying today.
00:18:49.640
Since I'm not on the public Wi-Fi, I'm on a Wi-Fi that isn’t connected to the internet.
00:18:54.960
So I won’t be able to drive the drones right now.
00:19:02.560
I'll show you the code, which is not production-quality code; it's more like a hello world example.
00:19:08.280
I used global variables, which isn't a good practice.
00:19:13.920
This code is specifically designed for two drones.
00:19:19.440
So, to communicate with the two drones, I’ll use methods for sending commands.
00:19:26.160
All my work was done on a Raspberry Pi equipped with a camera.
00:19:32.320
As soon as the drones take off, the camera starts recording.
00:19:40.000
If something goes wrong, I will at least have the video evidence.
00:19:44.640
Thankfully, everything has worked so far.
00:19:51.680
Here’s the command structure I used.
00:19:56.880
This if statement will handle the response from the drones.
00:20:03.520
I send commands to set the Tello into command mode.
00:20:09.520
Now I will check the battery level since certain maneuvers are only possible with higher battery values.
00:20:15.840
For take-off and landing, you don't need much power.
00:20:22.080
Now, I will attempt to take off.
00:20:29.440
We’re going to do some live testing with the drones.
00:20:35.360
Let’s hope there’s enough battery for the drones to perform the tasks.
00:20:42.560
While they boot up, I'll try to get them to flip.
00:20:50.080
Let’s see how that works out!
00:20:55.360
One of these drones will flip while the other will shift left.
00:21:00.960
Let’s give that a shot.
00:21:05.760
I’m hoping everything goes as planned.
00:21:11.200
They should be up and flying now!
00:21:24.080
Alright, I have about five more minutes. Should we keep going?
00:21:30.399
Let’s try to do two flips this time!
00:21:35.919
They should perform their flips now.
00:21:42.080
If there aren’t any errors in the console, they will return together.
00:21:48.000
Okay, here we go.
00:21:54.000
Now they should perform their flips!
00:22:00.480
And they're off!
00:22:12.720
I just want to do one more thing, and then we can switch to Q&A.
00:22:22.080
This time, one drone will rotate clockwise while the other will rotate counterclockwise.
00:22:28.560
Let’s see if that works.
00:22:35.680
Okay, let me clear up some space here.
00:22:42.080
Can you all read the size of the text?
00:22:49.360
Let’s get ready!
00:22:57.440
The drones should begin their rotations!
00:23:05.600
Here we go!
00:23:11.680
If everything goes well, they should execute flawlessly.
00:23:19.200
There you go! Thank you very much!
00:23:30.399
That’s it, everyone! They’ve been on their best behavior today.
00:23:35.679
I have about three minutes for questions.
00:23:39.680
As you can see, it’s really straightforward.
00:23:45.920
Please submit pull requests (PRs) for the code in the GitHub repository.
00:23:51.920
If you do submit a PR, please ensure it follows the model created by my friend.
00:23:58.400
I’m going to switch over to the public internet now.
00:24:05.920
In my codebase, I have contributions in other languages like Crystal, Elixir, and more.
00:24:11.920
Just last week, a friend submitted a PR that included detailed instructions.
00:24:18.000
He provided examples of how to install and use it, which is great!
00:24:24.600
I would love if PRs maintained a similar level of detail.
00:24:28.600
Thank you very much for your attention!
00:24:34.000
Any further questions? Let's give that a try!
00:24:39.679
I have only one minute left.
00:24:45.679
So let's proceed to try that!
00:24:52.560
It is possible, let’s explore this option.
00:24:59.679
I’m going to navigate to my Tello gem folder.
00:25:07.360
I believe rake builds it.
00:25:13.600
I might have the Tello gem and the EDU gem installed.
00:25:19.560
So let’s see if that works.
00:25:26.560
This is like an interactive mode, similar to IRB.
00:25:34.560
Unfortunately, I’m encountering a permission error.
00:25:39.520
The Tello gem allows for interactive commands, but I need to remember how to start.
00:25:45.920
You must first issue a command to initiate the Tello, followed by others like takeoff.
00:25:52.560
Alright, I’m almost out of time.
00:25:59.200
If you want, we can try this outside and capture live interactions.
00:26:06.560
You’re welcome to join and test things out!
00:26:13.279
Thank you once again!