Sweaters as a Servive

Hardware Hacking

Amy Wibowo

5 talks

Sweaters as a Servive

by Amy Wibowo

In the presentation titled "Sweaters as a Service" by Amy Wibowo at RubyConf 2014, she explores the intersection of hardware hacking and knitting through a unique project that pays homage to an unproduced Nintendo NES knitting add-on from the 1980s. Wibowo describes how her disappointment over the failure of this knitting add-on led to the creation of a networked yarn printer by hacking a vintage knitting machine. The project involved a collective effort among Airbnb engineers who, despite having no prior experience with knitting technology, embraced the challenge with enthusiasm. Key points discussed include:

Inspiration and Initial Discovery: The project was inspired by an article highlighting Nintendo’s potential knitting peripheral, leading Wibowo to seek out a vintage knitting machine.
Team Formation and Spirit: During an Airbnb hackathon, a diverse team of engineers united for this endeavor, marking it as one of their most enjoyable projects due to their collaborative spirit and lack of egos.
Technical Challenges: The team faced difficulties in learning to operate the knitting machine, requiring problem-solving and creativity, especially in converting yarn formats for the machine. They developed a workaround using coffee cups for yarn feeding.
Success with Patterns: After overcoming various issues, they managed to knit patterns, eventually customizing their own designs by interfacing a laptop with the knitting machine, emulating a floppy drive to upload one-bit bitmap images.
Web Interface Development: They created a web interface that allowed users to upload images and receive knitting instructions.
Future Exploration and Learnings: Wibowo highlighted the joy of tackling new challenges, the importance of empathy in programming, and potential future directions, including knitting entire garments and automating processes.

The main takeaway from Wibowo's presentation is that combining different fields—like art and code—can be both fun and insightful, prompting engineers to step outside their comfort zones and enjoy the process of learning new skills. She encourages the audience to explore new hobbies and pursue the cross-pollination of ideas in their work, anticipating further developments in her knitting machine journey that she hopes to share in the future.

00:00:18.080 Hi everyone. I'm Amy Wibowo, and I'm a software engineer at Airbnb, focusing on international growth. Today, I'm going to talk to you about knitting. Now, before you leave the room thinking you're at the wrong conference, I'm going to reassure you that this presentation is also about hardware hacking and problem-solving, and it even includes some code. You're just going to have to wait to see how.

00:00:45.920 All stories start with an inspiration, and here is the somewhat unexpected inspiration for this particular journey. A couple of years ago, a video game review site called Kotaku published an article about Super Nintendo add-ons that didn't quite make the cut. You've probably heard of some of the more outlandish Nintendo peripherals that did make it, like a baseball bat for playing baseball video games, a golf club for golfing games, and a Super Scope wireless gun for shooting games. But you might not have known that Nintendo was seriously contemplating making a knitting add-on.

00:01:02.239 Here is the proposed advertisement for this knitting add-on, complete with horrible puns. I’m not sure if you can see, but in the bottom left corner it says, 'Why needle the competition?' The interface would be like Mario Paint, but instead of just pictures as output, you'd get your design on a scarf or mittens. Most comments left on the Kotaku article were along the lines of, 'We can totally see why this didn't get made,' or 'What were they even thinking?' But I had a totally different reaction to this article, and I was incredibly disappointed.

00:01:34.160 I was sad that this product was never made because I imagined having one and using it every day. I had never wanted something to exist so badly. So, I did some research. I wanted to know if Nintendo had ever made any prototypes that I could get my hands on, or if anyone else manufactured home knitting machines that were programmable. After a lot of research, I found out that in the 1980s a few companies made home knitting machines. There was a whole line of electronic knitting machines by Brother, but they were discontinued because they lost popularity.

00:02:10.959 That’s when my dream was born: to get my hands on one of these machines and make something like the Nintendo knitting add-on that never was. As fate would have it, there was one on sale on Craigslist, an hour and a half drive away from where I live. It was old and missing a lot of parts, but it was a start—the start of my dream. This dream is illustrated in cartoon form: to start with images, send them to a knitting machine, and receive sweaters with those images on them at the end. Essentially, to create a networked yarn printer.

00:02:46.879 Now that I had my knitting machine, the first step was to learn how to operate it. This is what a knitting machine looks like: It has a row of 200 needles and to produce a piece, you first start by individually threading which needles you want threaded, depending on how wide you want your piece to be. The needles move in and out, allowing you to thread up to two different colors of yarn. If this is what a knitting machine looks like, this is what using one feels like—there are so many buttons and moving parts.

00:03:10.080 It comes with a thick manual with super unfamiliar terms, and I worried that if I pressed the wrong thing, something bad could happen, like breaking some of the needles without knowing how to fix it. Luckily, I had a team of people with me who were both fearless and inventive while learning how to use the knitting machine. This is the team of Airbnb engineers that helped me at last year’s December hackathon to hack the knitting machine.

00:04:13.599 Airbnb hosts two internal hackathons a year for the entire product team, including designers and engineers. You may wonder what this has to do with traveling. I asked the hackathon organizer if the knitting machine was a valid hackathon project, and he said, 'The spirit of our hackathon is about making awesome things, and so is the spirit of our engineering culture. So sure, go ahead.' I had no idea if anyone else would be as excited about this project as I was.

00:04:42.000 But when I pitched the idea to my co-workers, I was surprised by the enthusiasm. People who came up to me wanting to help were just as excited as I was, and there were many different kinds of engineers who signed up—an iOS engineer, an SRE engineer, a performance engineer, a data infrastructure engineer, and a front-end engineer. So, we had a team who had no experience with knitting or machine knitting, but were all eager to work on this project.

00:05:01.680 This became one of the best teams I've ever been a part of, largely because we all had zero ego; none of us considered ourselves knitting machine experts or needed to ensure our ideas were used over others. It became a wild adventure we were all in together. The reason we had to be fearless was that the knitting machine posed a really daunting problem, and we had no idea how to use it or get it to print the images we wanted.

00:05:26.320 We had to be inventive too since we were dealing with 1980s technology, and some parts were missing. We needed to MacGyver some replacement parts. The great thing about being an engineer is that you can apply your problem-solving skills to any problem, including knitting. For example, when you buy yarn at a store, it's usually in a ball or twisted into a skein. However, to use the machine, you want it to feed evenly and smoothly, more like a spool.

00:06:29.919 The knitting machine comes with a contraption to convert a ball of yarn into a spool, but it was broken and missing parts. So, my co-workers and I engineered a fix using coffee cups. After we set up the machine and successfully knitted a single solid row of color, the next step was to learn how to knit the machine’s built-in patterns. You can thread up to two different colors into the machine to create patterns, and it has some of those patterns built into its memory.

00:07:36.000 Though this was part of the machine's existing functionality, it felt like the next level in machine knitting because it required pressing a bunch more buttons and reading a lot more of the manual. To illustrate the challenge of this step, on the left is our first attempt to knit one of the built-in patterns—a polka dot pattern. By the end of the day, we finally figured out how to successfully knit a pattern of polka dots on the right, and I wish I had a picture of how happy we were when we got it to work.

00:08:45.760 While knitting patterns, the machine would get stuck more often than when we were just knitting rows of solid color. We experimented with different kinds of yarn and documented our findings to adjust our hypotheses. Eventually, we settled on using only animal wools because they made the machine jam less, likely due to the natural oils in the fibers. We also used a thicker yarn for the background and a thinner yarn for the detail.

00:09:06.360 After successfully knitting the built-in patterns on the machine, we still hadn’t achieved our dream yet. The intended audience for this knitting machine seemed to be 1980s grandmothers, as all the patterns were things like bows and teddy bears. I don't know about you, but I would rather have Grumpy Cat on a sweater than a teddy bear. Therefore, we had more work to do!

00:09:53.600 Step three of our plan was to send the knitting machine our own patterns. This objective was split into several parts. Since it was a knitting machine from the 80s, it could accept additional patterns via floppy drive. Back then, you could buy a floppy drive add-on and additional pattern packs on floppy disks. We could send our own images to the knitting machine by putting them on a fake emulated floppy drive.

00:10:16.160 We emulated the floppy drive on one of our laptops and needed to build a custom cable. The cable that would have gone from the floppy drive to the knitting machine would now be a USB from our laptop to the knitting machine's serial port. We had to make this custom because it turned out the knitting machine has a weird pinout. Luckily, we found documentation on this strange pinout and a library that emulated a floppy drive, which helped us move forward.

00:10:44.640 Next, we needed to convert our images to the proper knitting machine pattern, which was a one-bit bitmap. We also found a library that could convert images—JPEG, PNG, or GIF—to one-bit bitmaps. This meant we could have the knitting machine read from our computer as if it were a floppy drive. Here’s one of our first custom patterns, featuring their shiba inu with some dithering effects.

00:11:16.000 Note the dithering on the Shiba Inu! For many of the one-bit bitmap algorithms we tried, this specific one looked the best. If it’s hard to read, it says, 'Wow, oh my God, such Fair Isle,' which refers to a type of knitting stitch. And here’s Grumpy Cat! We spent a day enjoying ourselves uploading lots and lots of photos. However, we didn’t want this to be limited to just us who had hacked it and studied the manual. We wanted anyone to be able to print memes from yarn.

00:12:53.680 So, step four was to make the knitting machine image upload available on the web. It was a super simple interface: you choose a photo to upload, send the image to the knitting machine, serve it up on the fake floppy drive, and the web browser shows you instructions for which buttons to press on the knitting machine to start knitting your pattern.

00:13:01.680 Here is a video of the whole thing in action.

00:15:12.160 In case you didn’t see it in the video, here’s a close-up of the knitted image and the original input. I brought a bunch of other knitting samples with me, so if you find me after the class, you can see Grumpy Cat in person. There’s also one featuring some bunnies.

00:15:39.040 Now it’s coding time! Here’s the code for the knitting server. We included a wrapper for the library we used to convert images to one-bit bitmaps, as well as a wrapper for the disk drive emulator. First, you save the user-uploaded photo to a file, then you use the library to convert that image to a one-bit bitmap. After that, you start emulating your floppy disk drive with that image on it, and then the server shows knitting instructions for which buttons you should press.

00:16:16.000 From this adventure, we have learned some key lessons. First, we gained mad respect for grandmothers and anyone who knits because that is really hard work. Many moments were filled with my teammates saying things like, 'I thought my job was hard, but optimizing database queries ain’t got nothing on this!' Another important takeaway is that teams work exceptionally well when everyone operates with zero ego. It’s incredibly enjoyable to apply programming to any of your hobbies, no matter how unrelated they seem; especially hobbies related to art and creating.

00:17:53.000 I challenge you to find a hobby that is more unrelated to Ruby than knitting! Even with knitting, it is possible to make it work. Lastly, it feels great to take up something new that you don't know anything about. So many of my teammates said this was the most fun and challenging thing they had worked on all year. Learning something brand new and different from your daily routine is humbling and challenging, reminding you what it's like to be a beginner, and that kind of empathy is vital when working with or mentoring new programmers.

00:19:09.640 There’s still a lot left to explore in this space. The amazing thing about presenting this talk at various conferences is that attendees have approached me afterward with ideas on how to take this further. Currently, everything I've shown you was just a swatch of knitting, but it would be amazing to use the machine to knit whole garments, like socks and sweaters. I signed up for a machine knitting academy class in Seattle in January to learn how to make actual sweaters.

00:20:01.360 The next time I give this talk, I hope to be wearing a Grumpy Cat sweater. On the web interface side, it would be incredible to have an interface where you could draw an image and send it to the knitting machine without needing to use an existing image. Previews showing what your design would look like with different colors would be a fantastic addition as well.

00:20:07.679 On the hardware hacking aspect, it would be amazing to figure out how to automate more of the process since a person has to manually move the carriage back and forth. After some talks, others have shared ideas for how to make that happen, and I would love to explore those ideas. Additionally, I’d love to hear from anyone who has thoughts on hardware hacking, knitting, or combining art with code. You can reach me on Twitter at @sailorhg. Thank you so much for listening, and thank you for having me.

RubyConf 2014