Ruby Video
Talks
How Not to Build a Service
Mike Perham
2 talks
#failure-analysis
#startups
#ruby-on-rails
#business-strategy
#technical-debt
#agile-development
#software-development
Summarized using AI

How Not to Build a Service

by Mike Perham

In the talk "How Not to Build a Service," Mike Perham shares valuable lessons learned from his experiences at FiveRuns, a startup focused on providing monitoring and performance tools for Rails developers. The presentation is organized into three parts: the nature of failure in software development, the specific mistakes made during the development of their Manage 1.0 product, and the lessons learned to improve future endeavors.

Key Points Discussed:

  • Understanding Failure: Perham explains that failure in startups often results from a cumulative series of mistakes rather than a single misstep. Key considerations for avoiding failure include hiring experienced and smart people, and ensuring adequate financial resources to correct mistakes.
  • Types of Mistakes: He categorizes mistakes into three distinct areas:
    • Business Mistakes: Key decisions should prioritize understanding the customer. FiveRuns initially misidentified its target market, focusing on the Enterprise systems management space rather than the Rails development community, leading to wasted resources.
    • Social Mistakes: Internal team-related errors such as lack of Rails experience led to inefficient use of tools and practices. Not utilizing a testing framework made it challenging to refactor code effectively.
    • Technical Mistakes: While these are generally less critical, they can still cause significant issues. For example, FiveRuns developed unnecessary features tailored for Enterprise needs that weren't valued by Rails developers.
  • Significant Realizations:
    • Recognizing the need for trials to attract customers.
    • Understanding that support needs differ significantly between the Enterprise and Rails communities.
    • Realizing the importance of using their own software to understand its value and to improve trial conversions.

Conclusion and Takeaways:

The overarching lesson is that successful service building requires a deep understanding of customer needs, continuous iteration based on feedback, and the necessity of using one's own products effectively. Perham emphasizes that while technical mistakes can often be resolved quickly, customer-oriented decisions hold far greater weight and can lead to long-term consequences if not approached thoughtfully. The importance of forming a well-structured business plan and maintaining awareness of the ramifications of decisions is stressed as essential to minimizing mistakes. Ultimately, learning from experience and continuously improving practices are key to success in software development.

00:00:06.359 Video equipment rental cost paid for by PeepCode.
00:00:18.840 Screencasts. My name is Mike Perham and I'm from FiveRuns here in Austin. I'm going to be talking today about how not to build a service.
00:00:31.199 So, who am I? I'm probably like a lot of you guys. I'm an ex-Java developer; I did Java for the last decade and started playing with Ruby when I began working with Rails a couple of years ago. I loved it.
00:00:38.440 I'm probably best known in the Ruby community due to the Data Fabric gem that I wrote, which adds charting to Active Record. That is now Rails Envy approved since Jason from Rails Envy this morning said it was an innovative technology of the year. Thanks to him for that compliment.
00:00:50.640 FiveRuns is my sixth startup since 1999, so I've seen a little bit of failure and a little bit of success.
00:01:10.439 So, who is FiveRuns? We're focused on building tools for Rails developers, specifically in the monitoring and performance realms. We have three products right now: Install, which is a free stack that integrates Apache, MySQL, and Rails; TuneUp, a developer plugin for your Rails app that puts a little bar at the top of your web pages showing you how long it took for your page to render and the various components' load times; and our flagship product, Manage.
00:01:25.000 Manage is a full-stack monitoring solution that monitors Linux, Apache, MySQL, Rails, Memcached, PostgreSQL, basically any major part that a Rails app requires. This talk is about the mistakes we made in building the 1.0 product and how we addressed them in 2.0.
00:01:43.760 So, what not to expect? This is not a technical talk. You're not going to see any Ruby code here, and I won't be discussing web services, so no WSDL, SOAP, or XML. I hope you're not too disappointed.
00:02:00.080 Another caveat I should mention is that I was not a member of the Manage 1.0 development team, so when I talk about the mistakes they made, there's a lot of speculation and hindsight on my part. Just keep that in mind.
00:02:39.560 I divided this talk into three parts. First, I will generically discuss failure in the software world and what it means. Then, I will talk about the mistakes we made, and finally, I will wrap up by figuring out some generic lessons we can learn from the mistakes FiveRuns made.
00:02:58.239 Why do startups fail? I can't provide specific reasons for every startup in the world, but more generally, failure is due to the accumulation of a series of mistakes. You make one mistake, then another, then another, and that adds up to failure. Think about something like New Coke or Microsoft Bob; they didn't fail for a single reason but due to a number of mistaken decisions.
00:03:34.799 This implies that there are more ways to fail than to succeed because you make hundreds and thousands of decisions as part of a project or company, and any of those could lead to mistakes that accumulate to failure. The question then becomes: how do we reduce our chances of failure? Well, in the software world, there are a couple of very simple rules of thumb. You need to hire experienced and smart people.
00:04:01.720 When you hire experienced people, what you're really getting is the benefit of their past failures. You're paying for their lessons learned, which means you won't have to pay for those failures yourself. As for smart people, I'm not so sure that smarter individuals fail less than those who are not as smart. Perhaps they are just quicker to gather data and draw analogies from previous situations.
00:04:30.759 Money is the third consideration. Having enough money allows you to make more mistakes because you can afford the time necessary to correct them. To summarize, how do we avoid failure? You must think carefully about the decisions you make when writing software or starting a company and understand that each mistake has a cost. Avoid making costly mistakes, or ensure you have enough resources to correct them; if you do, you won’t fail.
00:05:15.400 Now let’s discuss the types of mistakes you can make. You should prioritize your decisions to determine which ones are most likely to lead to failure. When I considered the mistakes FiveRuns made, I identified three categories: business, social, and technical, with business decisions being the most critical as they can have wide-ranging effects.
00:05:57.520 Business mistakes are those that involve the entirety of a company. These mistakes can take years to fix. One of the most fundamental mistakes we made was not knowing who our customer was and what we were building. Initially, we targeted the Enterprise systems management space, using Rails simply because it was a hot technology and we wanted to bootstrap quickly.
00:06:43.000 It turns out that the Enterprise systems management space has wildly different needs and expectations for their software than the Rails development community. After spending half a year building our software for Enterprise, we realized Rails developers didn’t value much of what we had built.
00:07:24.560 As a result, a lot of what we rewrote in Manage 2.0 was refocused to better serve the Rails community. Once we recognized that we needed to focus on Rails developers, the question became: what do we know about these users? We, of course, are Rails developers ourselves.
00:08:06.360 We realized the fundamental decision we had to make was to adopt our own Rails development practices because that’s how Rails developers work. All the support mechanisms we had catered to the Enterprise market—like a 24-hour 800 number—made no sense for Rails developers. They needed email support, Campfire support, and forum software.
00:08:51.040 Pricing also varies dramatically between these two markets, as does marketing. For the Rails community, magazine ads are ineffective; instead, we should focus on conferences, open source projects, and blogging. Another crucial realization was that trials are fundamental for acquiring customers. The trial is where you begin and must continuously improve because every person who tries your software without converting to a customer represents lost revenue.
00:09:31.920 You need to understand the value they want from your software and demonstrate that value as soon as possible. Getting potential customers to click that download button relies on solid marketing strategies. However, once they download, they must be able to install and quickly start using it—that's where you need robust development practices.
00:10:24.080 Now let’s discuss social mistakes. These typically involve a single department within your organization. Business mistakes impact the entire company, while social mistakes are more localized. One social mistake we made was starting off our system in Rails when our team had no Rails experience, which led to numerous fundamental problems.
00:10:42.160 For example, we invented the wheel on various technologies and didn’t use Capistrano for deployment; instead, we utilized a shell script. It's clear that experience isn't solely about knowing how to write Ruby; it also entails knowing the available tools. We misused certain technologies and ran demons continually, using the full stack of Rails unnecessarily, which was memory intensive.
00:11:18.280 This occurred because we didn’t know how to spawn a Ruby daemon or use Active Record independently. Consequently, we employed a full Rails stack for background processes. Moreover, our Java developers were writing Ruby code, leading to code that resembled Java instead of Ruby.
00:12:07.360 Ruby is different from Java, and it typically takes time to adapt to writing proper Ruby. Another social mistake we made was having no tests at all. Initially, it seemed like to remain agile, the team thought they couldn't have tests breaking.
00:12:50.440 However, everyone understands the necessity of tests. Not having a testing framework made it impossible to refactor the Manage 1.0 system. The code often resembled a shark attack, and without tests, we found it easier to rewrite significant portions than to refactor the existing code.
00:13:39.680 Today, we're much better at writing code with solid unit, functional, and integration tests. Although we still don’t have automated browser tests, it seems there are diminishing returns after a certain point with those types of tests.
00:14:10.760 The last social mistake we made was not eating our own dog food. Since we're Rails developers selling tools to Rails developers, we needed to use our products internally to demonstrate their value effectively. We were advising clients on monitoring their systems but were not utilizing our solution.
00:14:59.120 The result was that we sold a product more suited for Enterprise systems management than for Rails developers, which led to poor trial conversions.
00:15:44.560 Technical mistakes are generally the least critical. As developers, we typically know how to fix our mistakes fairly quickly. However, sometimes, a technical error can take a long time to resolve. For instance, the original client for our Manage service was written in Java and C. This was likely due to our focus on the Enterprise market and concerns over source code obfuscation.
00:16:36.640 Our new client is written in Ruby and is much simpler; it's more memory-efficient and significantly shorter than the original 70,000 lines of code we had before.
00:17:20.320 However, we still encounter permission issues due to our client running as a FiveRuns user, while the Rails app might be installed under another user. We’re actively working on resolving these client issues.
00:18:02.640 Additionally, because we had an Enterprise focus, we built many unnecessary features not needed in the Rails world. For instance, we didn't implement SSL, as we mistakenly thought it would be blocked by firewalls, and instead we created our own custom HTTP encryption.
00:19:05.520 We also developed a custom proxy for clients behind firewalls, but in the Rails world, this is a minor concern; we only had one customer requesting this feature. Therefore, it’s not cost-effective for us to maintain such a complex proxy when it’s unnecessary for the vast majority.
00:19:56.960 Lastly, we failed to provide integrated billing. After going to the trouble of getting customers to the site, getting them to download the trial, and providing value, we placed an additional hurdle when they wanted to purchase, asking them to fax their credit card information.
00:20:33.520 This additional step created further obstacles, risking the loss of customers. We’ve now upgraded to an online buying system, allowing customers to purchase without needing to talk to our sales team.
00:21:06.160 So, what can we learn from this? FiveRuns is no different from any other software company. You're going to make business mistakes, social mistakes, and technical mistakes. You must recognize the importance of each decision, anticipating what could lead to a mistake.
00:21:55.360 In general, the more knowledge and context you have, the better your chances are of making the right decision. This relates to the need for a well-structured business plan and awareness of the ramifications of the choices you make.
00:22:30.760 It's crucial to understand that the longer it takes to recognize and fix a mistake, the more costly it becomes. This notion often recurs in software engineering, where design decisions can lead to more severe repercussions than implementation bugs.
00:23:09.840 Raising your level of awareness about decisions by enriching your knowledge and context acts as insurance against future mistakes. That brings me to two other aspects that can lead a software development group to make excessive mistakes.
00:23:56.960 One is groupthink, which often happens when a developer who is seen as an expert voices their opinion. People tend to agree with them without considering alternative perspectives, especially when their experiences are contextually different.
00:24:38.920 The second issue is optimism. As developers, we often misjudge time estimates, thinking about the ideal conditions rather than the complete picture. Fred Brooks points out this tendency to estimate based on wishful thinking rather than realistic assessments.
00:25:20.560 What has FiveRuns learned? To recap, we discovered that changing markets is expensive. The underlying question affecting all business decisions revolves around understanding your customer and how to effectively interact with them.
00:26:02.320 The trial experience is incredibly important, representing a user's first impression of your service. It's essential to outperform initial expectations. After deployment, you must iterate based on customer feedback, continuously improving your offerings.
00:26:47.200 Lastly, it's imperative that you use your own software. If you are selling development tools as a developer, you must personally utilize these tools to affirm their value, which, in turn, assures that your customers will find that value as well.
00:27:32.120 What can you take from this talk? While our experiences at FiveRuns may not directly translate to your situations, the broader lessons hold. Larger decisions require you to gather adequate knowledge and context, which serves as insurance against making costly mistakes.
00:28:00.320 Technical issues are often less critical since developers can typically fix such problems within a week. It's the customer-oriented decisions that can be far more detrimental. We tend to focus on minutiae rather than the overall picture, and it's essential to periodically step back and assess if we're on the right track.
00:29:08.160 As developers, ongoing learning and experience are vital. The most effective way to achieve this is through making mistakes and learning from them. I'm not going to dwell on the less favorable decisions we've made, but you're welcome to ask about those afterwards.
00:29:43.360 That's my contact information, and that’s all I’ve got. Thank you.
00:29:59.880 Video equipment rental costs paid for by PeepCode screencasts.
Edit
Explore all talks recorded at LoneStarRuby Conf 2008
+18