Let's Pretend

Talks

Let's Pretend

Sarah Allen

@ultrasaurus

#testing

#minimum-viable-product-mvp

Let's Pretend

by Sarah Allen

In the video titled "Let's Pretend," Sarah Allen discusses the journey of creating Mightyverse, a platform aimed at preserving language and culture through the sharing of phrase videos. She emphasizes the challenges of transitioning from concept to execution and the importance of learning from both failures and successes. Key points covered include:

Introduction to Mightyverse: Sarah shares her vision for Mightyverse, focusing on a global network for language and culture that allows users to create and share videos of phrases in their native languages.
Early Challenges: She recalls how the initial versions of Mightyverse faced technical difficulties and failed to engage real users, leading to a re-evaluation of their product approach.
Testing and Iteration: Inspired by Eric Ries's learn-build-measure method, she highlights the significance of testing early and often, using automated tests to define the project's direction and functionality.
Traffic Growth and SEO: Sarah explains how the proper encoding of content for search engines led to significant increases in traffic, despite initial missteps in visibility and engagement.
Crowdfunding as Validation: The video outlines the decision to pursue crowdfunding, detailing strategies for campaign success and the need for fun to drive user engagement in language learning.
Game Design for Language Learning: Sarah discusses the development of a card game designed to enhance language acquisition while being enjoyable, evolving through community feedback during playtests.
Community Engagement: Collaborative efforts with the community have played a vital role in refining their project and designing the new game. Sarah invites the audience to participate in learning phrases and engaging with Mightyverse.
Conclusion: She emphasizes the importance of understanding user needs beyond coding practices, encouraging an agile mindset that focuses on real-world impacts and benefits of software development.

Overall, the session promotes a culture of experimentation and continuous improvement within software development, particularly in the context of language learning.

00:00:26.960 Good morning! That was a lovely way to start the day.

00:00:32.239 I didn't even mean to rhyme! I'm so excited to be here talking about what has been a labor of love for many, many years.

00:00:39.920 Before working on Mightyverse, I've worked on software projects that have reached hundreds of millions of people, such as Shockwave and Flash video.

00:00:46.960 People have suggested that my work creating, or helping to create, the first version of After Effects has changed the industry.

00:00:53.600 But with each of those projects, there was a time at the beginning when we labored to create a new tool or experience for an audience of dozens, and it was in those times that the magic happened.

00:01:05.680 So today, I'm going to tell you about Mightyverse. I'm going to share moments of painful failure and joyful success. I don't know where this is all going, but I know that we're already reaching our audience of dozens and more than dozens.

00:01:19.520 I hope that some of the techniques I share with you today will inspire some work from you and help you fail quickly and move on to the next success.

00:01:38.159 So, Mightyverse — our big vision is to create a network for sharing language and culture. The idea is for people to create and benefit from a global corpus of human spoken language.

00:01:56.079 At its heart, it's thousands, someday millions, of tiny phrase videos where people record a phrase in their native language, and it's cross-translated into other languages.

00:02:08.160 When we started, the idea was that it would be a marketplace and that language experts would share their knowledge with others. We would capture the language that you can't find in a dictionary and things that are too colloquial or niche to respond well in Google Translate.

00:02:28.160 We also have a social mission where we're aiming to help the languages that are disappearing in the world. Almost 50% of the world's languages are at risk, and the people who decide whether a language lives or dies are three years old.

00:02:40.239 Our hope is that Mightyverse can start to highlight the value of diversity in language and culture so that young people will choose to speak the language of their parents, thus preserving and revitalizing the diversity of culture in our world.

00:03:06.480 The beginning of Mightyverse actually started a few years before I came on board. Mightyverse was a web app written in Rails 2.1, using the Globalized plug-in. Many of you may remember that before Rails had internationalization, and we had something else before Devise.

00:03:55.760 It also had a mobile app that was a Flash app. In 2008, before the iPhone, that seemed like a good technical choice. It wasn't released because at that time you had to have deals with carriers. We also had a desktop app for mass recording of these small phrase videos, and we still use that app today.

00:04:19.759 The web app was my challenge when I came on board. I was set to add a recording feature so that people could record on the web, which would help fulfill this crowdsourcing idea, as well as fixing a few bugs. I needed to learn Rails and Ruby. I got into the codebase and realized it was not just a few bugs. More often than not, when you start development without it being in front of real users, there are all sorts of implementations made just before the next demo.

00:05:18.479 What I really needed to do was finish application development. So we reset our sights on exactly what the first release would be. At the same time, I was paying attention to Eric Ries, who was blogging about startup lessons learned and popularizing the idea of an MVP, defined as the smallest thing you can learn from.

00:05:59.280 He defined a learn-build-measure-learn cycle, which I actually like to think should be learn-measure-build. But I'll get to that in a minute. We defined our minimally viable product and decided to launch something that may not be useful for real people right away but would be something we could learn from.

00:06:12.080 In getting into one of the exciting things I wanted to learn with Ruby was test-first development. I found that tests were really helpful in defining the project and the product. I was new to this codebase and it had some tests, but they didn't cover the functionality fully. This was part of the reason we encountered mysterious features that were actually just static text on the page.

00:07:02.400 So we built a number of tests to help us understand what the product did and define it better. We used automated testing extensively and went through an epic upgrade to Rails 2.2, launching the site in June 2009.

00:07:21.280 The first version allowed access to our then 22,000 phrases with search and highlighted episodic phrase lists. We engaged with real people later that year by packaging specific phrase lists for specific occasions—one for Japan, one for Italy, and two for Russia. These images are from Tai Roberts, who used a phrase list to find coffee in Japan. He successfully engaged with people who didn't speak any English using his broken Japanese, which was delightful.

00:08:28.400 What we didn't publicize was that one of the early investors of Mightyverse was the person who used it in a shop in rural Italy where the storekeeper couldn’t speak any English. He played a phrase to the storekeeper, who got so excited he talked for five minutes in Italian.

00:09:11.519 When he returned to the United States, he said, 'I sure hope I'm not your target audience.' So we learned from this painful experimentation and, even with the public failure, learned that we were really interested in intermediate language learners—people increasing their fluency.

00:09:32.080 We found that this is a tremendous number of people, yet there are very few resources for them once they surpass the beginner stage. This cycle we went through was something my consulting company Blazing Cloud rearticulated of Eric Ries's cycle.

00:10:29.519 People believe that the majority of people in the world now speak multiple languages. Some suggest a third of people do. However, in America, it’s not as common to speak multiple languages as in other countries. But interestingly, this is changing. Fifty percent of all current college students have an active passport, and thirty percent have used it in the last year.

00:11:45.920 With this swath of people learning or already speaking another language, we needed to find our target audience. We reframed the question as, 'How can our target audience find us?' We realized we had this asset of 22,000 phrases collected over a few years.

00:12:37.280 We looked at Google Trends and learned that a lot of people just type things into Google like 'How do you say this?' or 'How can you say this in this language?' This gave us the insight that we had this long-tail content suited for organic search.

00:12:55.440 However, we inadvertently hid all the content from Google. The episodic phrase lists meant that only phrases on the front page were visible while everything else was behind a search box. This was sobering and a little embarrassing, but it had a fairly easy fix.

00:13:29.760 I created a page for each phrase using a single template in Rails, indexed them through a new navigation via speakers. When doing search engine optimization, I always try to think about humans as well as search engines. By doing something for humans, the search engines will likely adapt their algorithms to match your website.

00:14:21.760 So I made this page with all the speakers and linked to all the phrases. In two and a half months, our traffic increased by a factor of eight, which was incredible and rewarding.

00:14:46.000 This growth continued into early January the following year until something weird happened. The first half of the graph of our traffic looked excellent. Yet, shortly thereafter, something altered.

00:15:02.400 I checked my commit logs to figure out what was happening around that time. I had performed a Rails 3 upgrade which possibly impacted our performance as we also received some press at that time that only mentioned our name without linking to our site.

00:15:55.680 Then early February, I broke caching on the site; when a site is slow, Google doesn't favor it. I failed to notice this for six weeks. Fortunately, when I fixed the caching issue, our traffic began to climb again.

00:16:31.200 This lag in my response was really frustrating. With search engine optimization, it takes a while for your traffic to recover after fixing an issue because you appear to be a flaky site. During that time, my tests were green, which seemed like a problem.

00:17:16.400 Maybe I could have gone back and written a test for that specific bug. Yet the next thing impacting Google traffic could be anything; though now I'm kind of an SEO expert, I certainly wasn't back then.

00:17:59.440 So I went to the Google Analytics console and found these, you know, custom alerts.

00:18:00.960 I set a low watermark for my traffic, indicating that if traffic fell below that point, I wanted an alert. Every six months to a year, I would increase that number, and mostly I'd forget about it. Then, a couple of years later, I received an alert that the traffic had significantly decreased.

00:18:47.600 I would have found it much later without the alert, which would have had much more significant effects. This is my first significant lesson: look for tests that exist outside your automated tests, focusing on achieving real effects in the world.

00:19:57.680 How do we measure what we want? Automated tests are merely your first line of defense. I believe every web application should have an immune system, ideally a self-correcting one, much like continuous deployment.

00:20:56.480 However, for now, I have only a couple of alerts. All my tests are green, and I'm excited that my traffic grows year after year, but that begs the question: is my software really working?

00:21:41.440 As it happens, no. I didn't aim to create what Mightyverse has on the site right now, which effectively is a visual phrase dictionary. All our incoming traffic grows year over year, but it’s similar to a sieve; it just exits because there’s really not much there.

00:22:48.720 We still believed in the concept of a language exchange, a social game where individuals could exchange phrases and learn from one another, feeding the content creation feedback loop for endangered and major languages.

00:23:28.400 However, we lacked the crowdsourcing feature that made this viable. We faced the reality that without a rewarding feedback loop, the crowdsourcing of phrases wouldn't take off. We decided to pursue crowdfunding for validation.

00:24:19.919 Crowdfunding behaves like a lean startup landing page technique, creating an advertisement for your product. The goal is to collect money for pre-orders of your product. If the raised amount is sufficient, you go ahead and build it.

00:25:11.200 We identified four drivers of successful crowdfunding campaigns. Firstly, make a video; we had that down. Secondly, have multiple founders; we were set there too. Third, ask for less than 10k. This was a big success factor for others, but 10k isn’t enough for me.

00:26:31.360 Fourth, have experience running a crowdfunding campaign. We lacked that, but we resolved to proceed anyway, going back to the drawing board.

00:27:05.760 We sketched out the desired app and defined our phrasing lifecycle, focusing on leveraging our community of bilinguals and language learners. We aspired to make this experience fun, believing that language learning should be intrinsically enjoyable.

00:27:42.640 We did research on game design because I've designed game platforms but never a game in practice. We sought to incorporate key elements of meaning, mastery, and autonomy. Fun, as Ralph Koster elegantly framed it, is another word for learning under optimal conditions.

00:28:39.760 He recommended creating a process to capture these conditions. You must know your users and create paper prototypes, playtest, and iterate. You don't know what fun is until users start to enjoy it.

00:29:46.400 We took our app design to a cafe equipped with index cards and markers and devised a game for our first test. We asked ourselves if we had fun. Although we aren't the target audience, we indeed found much fun in the game we created.

00:30:18.720 However, upon reflecting, we realized our game had encoded the worst practices of language learning — namely, translation. So our next test was to see if we were moving toward our goal.

00:30:50.560 We worked together to invent a new game involving one native speaker helping others guess phrases without using English. Remarkably, this new game helped propel us toward our goal and only needed one bilingual player.

00:31:47.360 It evolved into a card game for our crowdfunding campaign. This was a hard decision, as we feared it could signal a pivot to becoming a card game company, which wasn't our intention. Yet we realized the need for playtesting the card game for several months.

00:32:39.440 We found an excellent venue at SF Babel meetups, where people speak different languages and get matched up. We played our game with various groups and learned through observation and testing that the game became enjoyable after initial struggles with rule comprehension.

00:33:31.760 We then launched our crowdfunding campaign, aiming to raise 5,500 but ended up collecting 7,300. It was thrilling, and we're currently producing this game in Spanish and a Japanese version instigated by the crowdfunding campaign.

00:34:12.800 During our crowdfunding video, a viewer alerted us to a typo. I reached out to him for assistance with translations, and we collaborated on refining our project. It's been fantastic working with the community.

00:34:33.240 In closing, I want to encourage you all to experiment with me! In the next few weeks, we'll have our new game in stores. If you're interested in receiving a copy of our game or the upcoming Japanese version, please tweet at Mightyverse.

00:35:06.560 To participate, share a phrase you want to learn how to say in Spanish or Japanese with the appropriate hashtag. We will reward creativity while engaging with our community and the learning journey.

00:35:40.560 Let me finish with a quote from Kent Beck: 'Agile is what works.' I challenge everyone to think beyond just coding practices toward understanding how our users will benefit from our software. Thank you!

Sarah Allen

@ultrasaurus

Rocky Mountain Ruby 2014