Sweaters as a Service

by Amy Wibowo

In her talk at RubyConf AU 2015, Amy Wibowo, a software engineer at Airbnb, explores the intersection of technology and knitting, inspired by a long-forgotten Nintendo knitting add-on. The presentation emphasizes the themes of problem-solving, hardware hacking, and creativity in engineering. Key points and takeaways from her journey include:

Inspiration: Wibowo's interest was sparked by an article about unproduced Nintendo peripherals, particularly a knitting add-on that could transform computer designs into real knitted garments.
Acquisition of Technology: After researching, she obtained a 1980s electronic knitting machine, which was complex and missing parts, embodying the challenge of retro technology.
Hacking the Machine: With help from her enthusiastic team at Airbnb, they approached the project as a hackathon goal, learning about the intricacies of the knitting machine, which felt similar to operating a complex device like a ‘control station of a Starship’.
Engineering Challenges: They encountered and solved multiple problems—including preparing the yarn correctly for the machine—using inventive solutions such as constructing a yarn spooling device from a coffee cup.
Programming Integration: A crucial step was developing software to convert images into a format readable by the knitting machine, including the creation of a web interface to make it user-friendly for anyone wanting to knit custom patterns.
Team Dynamics: The project highlighted the value of collaborative problem-solving and the absence of ego, which allowed the team to push through challenges with enthusiasm.
Personal Growth: Wibowo reflects on how the experience deepened her respect for knitters and emphasized the fun and humility in learning something new as a team.
Future Possibilities: She speculates on further advancements in knitting machine technology, such as automating the knitting process with robotics and integrating software with hardware for enhanced control.

In conclusion, Wibowo's journey illustrates not just a unique intersection of knitting and technology, but also underscores the importance of collaboration, creativity, and adaptability in engineering. She encourages the audience to explore and combine their hobbies with technical skills, resulting in exciting projects and personal development.

00:00:00.599 Hi, my name is Amy Wibowo, and I'm a software engineer at Airbnb. Today, I'm going to talk to you about knitting. Before you get up to leave because you think you might be at the wrong conference, I want to reassure you that this talk is also about problem solving, hardware hacking, and even contains some code. You'll just have to wait to see how.

00:00:21.880 All stories start with inspiration, and here is a somewhat unexpected source of inspiration for this journey. A few years ago, a video review site called Kotaku published an article about Super Nintendo add-ons that didn't quite make the cut. You've probably heard of some of the outlandish Nintendo peripherals that did make it, like a baseball bat for playing baseball video games, a golf club for golf games, and Super Scope wireless guns for shooting games. However, you might not have known that Nintendo was seriously contemplating making a knitting add-on.

00:00:56.600 Here is the proposed advertisement for the knitting add-on, complete with horrible text. If you can read it, you'll see that on the bottom right corner, it literally says, 'Why needle the competition?' The interface would resemble Mario Paint, but instead of pictures as output, you'd get your designs on actual scarves or mittens. Most of the comments on the Kotaku article were along the lines of, 'We can totally see why this never made it to production,' or 'What were they even thinking?' However, my reaction to hearing about this add-on was quite different. I was very disappointed and thought, 'I would totally use one of these all the time if I had one.' I kept trying to think about how to make this a reality.

00:01:32.000 So, I did some research. Did Nintendo make any prototypes that I could get my hands on? Did any other company create electronic knitting machines for home use? After a lot of research, it turned out that, in the 1980s, there was a whole line of electronic knitting machines that were made but subsequently discontinued. That's when my dream was born: to acquire one of these machines and make something like the Nintendo knitting machine add-on that never was. As fate would have it, I found an old knitting machine for sale on Craigslist, about an hour and a half away. It was old, broken, and missing parts, but it was the start of something meaningful—the beginning of my dream.

00:02:10.560 To reiterate, the dream was to have a knitting machine that could take images from a computer and turn them into sweaters—essentially creating a networked yarn printer. Now that I had gotten hold of one of these machines, the first thing I needed to do was learn how to use it. Figuring out how to use the basic built-in functionality of the knitting machine was daunting because it was quite complex.

00:02:37.200 This is what a knitting machine looks like: there are 200 knitting needles that move in and out, and you thread however many needles as you want your knitting to be wide. There are mechanisms to thread up to two different colors of yarn through the needle, and alternating the two colors creates patterns. Operating the machine felt overwhelming; there were countless buttons and moving parts, and the machine came with a thick manual full of unfamiliar terms. It felt like operating the control station of a Starship, and I was worried that if I pressed the wrong button, something disastrous would happen, or at the very least, I might break some needles and not know how to fix it.

00:03:05.080 Luckily, I had a team of fearless and inventive people at Airbnb who helped me face the daunting knitting machine. This is my team of engineers who assisted in hacking the knitting machine during last year's Airbnb hackathon. Airbnb hosts internal hackathons twice a year for the entire product team, including both designers and engineers. And when you ask, 'What does knitting have to do with traveling?' I asked the hackathon organizer if working on this project was valid, and they told me that the spirit of the hackathon and our engineering culture is about creating useful things for people. They felt this project was both awesome and useful, so I was approved to work on hacking a knitting machine.

00:04:08.320 I had no idea if anyone else would want to join me on this ridiculous project, but when I pitched it, I was surprised by the amount of interest. People approached me, excited about being on my hackathon team, and they were just as enthusiastic, if not more so, than I was. There were different types of engineers who came to talk to me, including an iOS engineer, an SRE, a performance engineer, a data infrastructure engineer, and a front-end engineer. This just goes to show that we were a group of engineers with no experience in knitting or knitting machines, but my teammates were all amazing and showed great enthusiasm. It turned out to be one of the best and most productive teams I've ever worked with.

00:04:51.360 I believe the success was partly due to the fact that we had no ego attached to machine knitting and none of us knew how to knit, so it felt like a wild adventure that we were all embarking on together. The need for fearlessness arose because the knitting machine was daunting, and we were faced with problems we had no idea how to solve, and we had to get inventive. The technology we were using was from the 1980s, and the machine was missing some parts. Thankfully, as engineers, we could apply our problem-solving skills to anything, including knitting.

00:06:12.800 One example of a problem we encountered was in preparing the yarn for the machine. When you buy yarn at a store, it’s optimized for hand knitting, not machine knitting. To feed the yarn into the machine, it had to be wound into a spool that would unwind neatly and feed into the machine at an even rate. Our machine had come with a contraption to turn balls of yarn into spools, but it was missing some parts, so we had to MacGyver a solution using a coffee cup. This is my teammate Dave spooling yarn using our MacGyver coffee cup solution.

00:08:05.760 After we learned to set up the knitting machine and successfully knit rows of plain colors, the next step was to learn how to use the machine to knit built-in patterns. Since you can feed up to two different spools of yarn into the machine, it has some patterns built into its memory. The machine comes with a book listing all the patterns and the numbers to input for each. Although this was existing functionality, we found it to be a daunting step, as it meant dealing with two yarn colors, which created twice as many opportunities for tangling. We had to read an additional 20 pages of the manual to tackle this next level of machine knitting.

00:09:10.000 To illustrate how non-trivial this was, our first attempt to knit a built-in pattern ended up looking quite ugly due to pressing some wrong buttons, using incorrect yarn settings, and accidentally skipping some needles. A few hours later, however, we successfully knitted the polka dots, which you can see on the right. I wish there was a photo to show just how elated we were when we finally succeeded in using the machine's built-in patterns.

00:09:44.000 Another example of engineering problem solving necessary for knitting arose when we noticed that the yarn would get stuck more often when we knitted patterns compared to solid colors. We tried different yarns, adjusted settings, kept track of results, and formed hypotheses about what thicknesses we should try next. Eventually, we discovered the optimal thickness of yarn to use and observed that animal yarns possessed natural oils that helped prevent tangling. However, after we figured out how to knit the machine's built-in patterns, we still hadn’t achieved our dream, as the intended user of these knitting machines seemed to be a grandmother from the 1980s, as evidenced by the patterns of bows and teddy bears.

00:10:51.440 While I would prefer a sweater with Grumpy Cat on it rather than a teddy bear, we still wanted to find a way to send our own images to the knitting machine. This was step three of our goal, which we would break down into several parts. Because the machine was from the 1980s, it takes additional patterns via floppy drive, and you could buy additional pattern packs on floppy discs. We wanted to emulate a floppy drive in software, and we needed to create a custom serial cable to connect our emulated floppy drive to the knitting machine. Thankfully, we found documentation on the knitting machine's pinout and also utilized a library that emulated a floppy drive.

00:12:29.530 Next, we needed to convert our input images into the LRA's 1-bit bitmap format—the file format that the knitting machine accepts. There were several methods to do this, but we wanted to include a conversion algorithm that supported dithering. We were fortunate to find a library for this, so we could have the knitting machine read data from our computer as if it were a floppy drive. Here is one of our first custom-made patterns noting the dithering; it’s hard to read, but it says, 'Wow, oh my God! Such Fair Isle,' which refers to a type of knitting stitch. Additionally, here’s a knitted output of a Grumpy Cat pattern.

00:13:49.920 We were having an absolute blast with this project, but we didn't want the ability to send pictures to the knitting machine to be limited just to the people who hacked on it and studied the manual. We aimed to make it accessible for anyone to print memes from yarn. Thus, the next step was to develop the knitting machine image upload feature on the web. It would be a simple interface where users could choose a photo to upload, send it to the machine, and receive instructions about which buttons to press on the machine to begin knitting their pattern.

00:15:01.220 As part of our demonstration, here’s a video showing the whole process in action. [Clip plays with music] Now, here’s a close-up of the input image and the knitted output in case it was too small to see in the video. Now, let's discuss the code that made this possible. It's written in Ruby and Haml and is straightforward. We required the NRA and Haml libraries, as well as wrapped libraries that convert images into 1-bit bitmaps and the floppy disk drive emulator library we dubbed 'Pattern Dis Drive.'

00:16:13.680 When a user presses the upload button, we save their photo to a file, then use the image processing library to convert the image to a 1-bit bitmap while emulating the floppy disk drive. In the web browser, we display instructions for the user on what buttons to press on the knitting machine. Some lessons we learned from this adventure: first, we gained immense respect for grandmothers and anyone who knits since it's hard. I recall many moments where my teammates would say things like, 'I thought my job was hard, but optimizing database queries isn’t comparable to this!' Additionally, we noticed that teams work well when everyone operates without ego.

00:17:23.900 Moreover, it's incredibly rewarding to apply Ruby to any of your hobbies, no matter how unrelated they may seem. I challenge you to find a hobby that seems less related to Ruby than knitting, but even in that case, it was feasible. Exploring the union of technology and your hobbies reveals how strangely similar they can be. I would never have guessed that I would be making off-by-one errors while knitting! Lastly, it feels fantastic to dive into something completely unfamiliar. My teammates shared that it was one of the most enjoyable and challenging projects they had worked on all year. Learning something completely new and different brings humility and refreshes your perspective, reminding you what it's like to be a beginner—a crucial benefit for developing empathy for new programmers you might work with or mentor.

00:18:47.360 There is still much to explore within the realms of computing and knitting. The exciting aspect of sharing this talk at conferences is that I’ve encountered individuals who have additional ideas for furthering knitting machine hacking. For instance, it would be fantastic to create an interface where you can draw an image and send it to the knitting machine without needing to rely on existing images. It could provide previews of what your design might look like using different colored yarns. One goal I've had for a while after hacking the knitting machine was to learn to use it to knit entire garments such as socks and sweaters. At last year’s Madison Ruby, a fellow programmer and knitter named Liz Abinante told me about a knitting machine academy in Seattle that teaches how to use these old machines to create full garments.

00:20:45.190 In terms of hardware hacking, it would be amazing to devise a way to automate more of the knitting process, possibly using a robotic arm that moves back and forth, which would save me manual effort. I've also been reading about hacks that involve replacing the entire logic board of the knitting machine with an Arduino. This would be an exciting endeavor, as it would allow more control over the knitting machine's operation, such as showing users how much progress they’ve made and how much remains to be knitted.

RubyConf AU 2015