00:00:11.040
Hi, welcome to "Hello, Computer: Writing Ruby with Voice Recognition." Before I begin, I'd like to give a small content warning; there will be some medical information in this presentation.
00:00:18.160
I'll indicate when I'm about to reach the part with a diagram. There will be one slide featuring a diagram of the anatomy of a hand. It's not super graphic, but I understand it might be uncomfortable for some people. I won't be offended if you prefer to step out and return later.
00:00:30.160
Most of the medical content is at the beginning of the presentation. I will get to the coding demo part of it around five to eight minutes in. So anyway, hi, my name is Nat Budin. I work at AKBlue Technical Services, and we are hiring. We are a remote-first organization, but our main office is in the Boston area. Feel free to chat with me about that if you’re interested. If you'd like to follow me on Twitter, my handle is @nat_budin.
00:01:10.560
Before I dive into the main presentation, I want to give some context. I don’t code by voice full-time anymore; I did so for a few months while recovering from an injury. As such, I'm not really an expert in this software. I haven't developed the level of expertise that someone who uses it regularly would have. There are people who are much better at this than I am.
00:01:37.680
I consider myself fortunate in terms of being a voice coder. I speak in what is often referred to as Middle American English, which definitely helps with the voice recognition software's understanding. I don’t have the kind of medical issues that would require me to use voice recognition full-time, so I feel very lucky in that regard.
00:01:59.280
Now, speaking of my hand injury, here’s how I got into voice coding. The story begins in August 2020. This is a vacation photo from when I was camping with my partner in Central Washington. In the background is Sun Lakes Dry Fall State Park, which used to house the largest waterfall in the world during the ice age. It's since dried up, but you can still see the remnants. You can camp right in the basin of the cliff, and I highly recommend it if you're ever in the area.
00:02:23.600
While we were there, I started to feel some pain in my right index finger. Initially, I couldn’t identify the exact movements causing the pain, as it didn’t occur in the same spot every time. I tried to ignore it, thinking, ‘I’m just getting older.’ I figured I’d be fine, but the pain did not improve.
00:02:45.360
Eventually, like anyone would do, I self-diagnosed on Google. I read about various medical conditions, one of which was De Quervain's tenosynovitis. Spoiler alert: that wasn't the issue, but I thought it was for quite a while. The next step after self-diagnosing was, of course, self-treating with Amazon.
00:03:02.720
I purchased a finger splint that I wore 24/7 for about ten days straight, attempting to work with it on. It was supposed to help by preventing my finger from bending to relieve any pressure causing the problem. However, at the end of those ten days, my condition had worsened.
00:03:25.760
I discovered that my work's health plan included a telemedicine option, which was critical during the peak pandemic period. I was reluctant to see a doctor in person, particularly before vaccines were available. Finally, I consulted with someone through telemedicine, and she advised me to stop wearing the splint, stating it was causing more harm than good. I was told to only wear it at night, if at all. She also recommended seeing a hand specialist.
00:03:51.360
Here’s a diagram illustrating what was going on with my fingers. If you're uncomfortable with it, feel free to step out now. The hand specialist confirmed that I had a trigger finger. There's a tendon running through each finger and down to the arm, known as a flexor tendon, which is depicted in this diagram. When you bend your finger, it acts like a rope moving through a pulley, which, in my case, had become inflamed.
00:04:35.200
The inflammation prevented my tendon from gliding smoothly, causing pain that could manifest anywhere in the finger. The hand specialist provided an injection of cortisone directly into the affected area, which worked almost immediately. I began to feel significantly better within a few days.
00:05:14.120
However, this was a gradual healing process that took months. During that time, I still wanted to code, so I tried various methods, including coding with one hand, which was incredibly frustrating, or trying to avoid using my index finger altogether, which only made things worse.
00:05:51.680
I remembered a talk from the Perl Conference in 2019 that my friend Amy Newell had told me about but I had never watched. The speaker, Emily Shea, uses voice recognition full-time due to severe arthritis, making keyboard use nearly impossible. In her talk, she demonstrated a piece of software called Talon.
00:06:17.920
Talon is a customizable system for desktop voice recognition. Unlike traditional voice recognition software, Talon not only transcribes speech but recognizes commands. It supports several voice recognition engines, including its own optimized model called Conformer, which is incredibly effective.
00:06:43.920
Talon was developed for programmers by a programmer, and its community repository contains various scripts and commands written in scripting languages. While the software isn’t open source, users can support its development via Patreon.
00:07:05.840
Now, I'd like to demonstrate how Talon works. I'll run a few commands for you. Let me open my editor.
00:07:45.440
Talon, wake. Say "now Talon is active." Enter. Go up. Go word right. Clear word right. Space. Say Talon. Go down.
00:08:03.760
As you can see, there are plenty of commands available for editing text. While it doesn't always recognize my commands correctly at first, it's actually quite efficient most of the time.
00:08:27.120
You might think that while text editing is useful, as a programmer, I need to type things differently, like writing in Ruby or JavaScript, which have specific naming conventions. For example, the commands include prefixes such as 'snake' for underscores or 'camel' for CamelCase, enabling different outputs as needed.
00:09:04.000
This feature can be especially useful for someone like me who often switches between Ruby and JavaScript in my work. I'm constantly adjusting to different variable naming conventions across different languages.
00:10:07.680
After getting the hang of it, I found that Talon significantly elevated my programming capabilities and allowed for precision that keyboard typing alone might not offer. This makes it an exciting prospect for anyone dealing with repetitive stress injuries or those simply looking for an innovative way to code.
00:10:56.560
Talon has additional prefixes available when I’m in Ruby mode. For example, saying 'instance loaded records' automatically generates the instance variable format, which is incredibly helpful when working in Rails.
00:11:42.240
Talon adapts to various programming languages as well, detecting which language you're using based on the active window title. This means if you open a Ruby file, it tailors the commands to fit Ruby programming, making it versatile.
00:12:36.400
Talon also includes a phonetic alphabet for those moments when it doesn't recognize certain words. It’s user-friendly by using one-syllable words to represent letters, making it easier to spell out commands.
00:13:20.480
You can use the Talon online help resource to get assistance with commands or debug issues. It features a command history window, allowing you to see what commands have been executed.
00:14:15.680
Now, let me copy some actual Ruby code to demonstrate how Talon works in practice. This is code I have from an open-source project, not proprietary code.
00:15:00.160
Talon, wake. Snake, rescue from space.
00:15:05.440
Hammer pundit, colon, twice. Hammer "not authorized error".
00:15:20.960
State do, arg pipe, say "error". Slap.
00:15:36.560
Hammer rails, dot, dotted logger, warn, paren.
00:16:05.040
Many of the commands involve going left and right to make precise adjustments. Talon can identify where my cursor is and make the necessary changes.
00:16:22.960
While I’m not entirely practiced with the software at the moment, I’m hoping this demonstration shows you how possible it is to program using Talon, honing in on voice commands and editing features.
00:16:59.520
Over time, people using this software full-time can become as efficient with it as they were with a keyboard within a few months.
00:17:40.480
However, for beginners, the first couple of weeks can be challenging, but it gradually becomes more intuitive, almost like solving a puzzle.
00:18:17.600
If you plan to start using Talon, remember some tips to ease that process. For one, don’t attempt to do everything at once, initially, use both your voice commands and manual input with a trackpad or mouse.
00:19:14.720
Listen to your body; if something hurts, it’s crucial to modify the way you attain your tasks. Take breaks, adjust your approach, and don't push yourself into discomfort.
00:19:53.360
Excessive gaming during my injury recovery period played a part in how I developed it in the first place.
00:20:42.640
Finally, if you’re keen to learn more about Talon or get started, the resources provided will be immensely useful.
00:21:24.200
Thank you for joining my talk, and I’ll be available on Discord for further questions.
