Open Source Maintenance

Talks

Eric Hodel

@drbrain

#open-source

#rubygems

#software-maintenance

#community

#contributing

#developer-experience

#commit-messages

Open Source Maintenance

by Eric Hodel

Summary of 'Open Source Maintenance' by Eric Hodel

Eric Hodel's talk at Ruby on Ales 2014 explores the joys and challenges of maintaining open source projects, drawing on his six years of experience with RubyGems, RDoc, and related libraries. He highlights how open source development can be both fulfilling and demanding, sharing his personal journey as a paid open source developer.

Key Points Discussed:
- The Nature of Open Source Work: Hodel describes his role at AT&T, focusing on flexibility with his release schedules and project priorities, and the unique privilege of being paid to contribute to open source.
- Community Engagement: He emphasizes the importance of community in open source, noting that despite disagreements, contributors work from a place of care and shared values. Engaging with users through documentation and guidance is framed as key to being a good steward of projects.
- The Significance of Good Commit Practices: Hodel advocates for atomic commits—small, focused changes accompanied by descriptive commit messages. He explains how this assists in maintaining comprehensible project history and simplifies debugging and collaboration.
- Tools and Strategies for Maintenance: He mentions using GitHub's issue tracker effectively, labeling, and managing milestones to track progress and enhance organization. This approach facilitates better communication with contributors and users.
- Handling Bugs and Features: Hodel shares insights into addressing bug reports and feature requests thoughtfully, suggesting that new features should be approached with caution to ensure they don't complicate existing maintenance burdens.
- Contributing to Open Source: He encourages viewers to start contributing to projects they already use and enjoy, stressing that even small changes can be impactful. This not only fosters personal interest but also supports community growth.

Hodel concludes by reiterating the importance of simplicity in code, stating that complexity should be minimized to enhance maintainability. He invites questions, emphasizing his openness to further discussions about open source maintenance and best practices.

00:00:16.800 Everyone, please welcome Eric.

00:00:24.720 I want to start out by saying thank you to the organizers for accepting my special request. Kobe is my boss, and he shields me from just about all of the bureaucracies, except for training and expense reports.

00:00:32.079 Every time I try to pay for a conference, I get yelled at because either I pay for it on my personal card, and they don’t like that, or I pay for it on the travel card, and they don’t like that either. They say I have to get this third card, and I'm like, 'What third card is that?' But they're like, 'Never tell me.' So I just keep getting yelled at.

00:00:43.360 I almost did not make it here. Way back in January, I got a jury duty summons, and I was like, 'Yes, this is exciting!' It was two weeks before the conference, so I thought, 'Oh, it'll be fine.' Then I was put on a two-week trial.

00:00:49.760 I sat through the whole thing, and then they dismissed me as an alternate. Since the verdict hasn't come in, I can't tell you what happened or anything like that, but I can tell you this: it wasn't about the trial.

00:01:01.840 So we did puzzles. This is a bunch of dogs dancing at New Year's. You can't really see it very well, but there's this really awkward couple over there; something strange was happening with those two dogs. We did this parrot puzzle; it was much easier because the pieces were bigger, but there were a couple missing. That really wasn't a problem.

00:01:14.080 The first day, we got to sit in these uncomfortable office chairs, which I don’t ever do. I was all fidgeting around trying to get it to feel comfortable at all. Before lunch, the judge asked, 'Is there something wrong with your chair?' I replied, 'I don’t like these. Can I sit on a yoga ball?' She was kind of like, 'Okay.' So, I went to Target during lunch and got a yoga ball and then when they dismissed me, I took my ball and went home.

00:01:44.400 I think I was probably the only person the judge has ever seen sit on a yoga ball in the jury box. That was much better than those chairs.

00:01:50.159 So, open source is my job. I work for AT&T with Kobe, and our team in general is building cloud stuff. I'm not really sure what all we're doing anymore because I don’t pay attention, but we use lots of open-source software. Kobe wants to contribute back, so he gives me a paycheck to do that, and I don't know many other people who get paid to do that, other than Aaron and Ryan.

00:02:22.480 The main benefit of this is that I get to set my own release schedule, set my own priorities, and work on what I want. However, if someone asks Kobe, 'What's this guy doing?' then Kobe has to be able to say, 'All this great stuff.' So, I still have to go and do valuable things.

00:02:51.760 The projects I pick include RubyGems, where I've been integrating Bundler features into RubyGems so we can merge the two projects. I also work a lot on RDoc, net-http-persistent, and other HTTP-related projects. I have a whole bunch of other projects I work on occasionally when I get bored or need to change things up.

00:03:03.920 My main responsibilities are to maintain this software for other people. When people find bugs or want new features, I want to be a good steward of the project. If somebody wants to implement a feature request, I’ll guide them on how to go about that—how to write tests, what makes good documentation, and also teach people how to use the software I write properly.

00:03:40.159 So, what is it about open source that makes it work so well, and why do I work on what I do? One of my friends, a wise developer, tweeted before he disappeared: 'Programming is rather thankless. You see your work become replaced by superior works in a year, unable to run it all in a few more. It takes a lot of effort to keep your software updated and running amidst changing ideas and disagreements with others about how to solve similar problems.'

00:04:19.360 From another viewpoint, Gary Bernhardt said, 'Here’s what becoming e-famous made me realize: everyone is sincere and doing it because they care.' I was very surprised to learn this. Even though there may be disagreements, the people who work on open source are almost always motivated by something they really care about. If you take the time to listen and understand their needs and concerns, you can usually work something out.

00:04:55.920 This idea is amplified by Lindsey Beerta, who said, 'I get paid to do something I love, something I’ve done in my spare time since I was a kid, and that’s awesome.' This is how I feel. I care about open source and software, and I want to share that with everyone. I care about writing useful and easy-to-use software, making it flexible, and continually improving it over time.

00:05:19.520 Working in open source gives me some pleasant surprises. I had a bug file for net-http-persistent where a user reported, 'Oh, I had this error message.' When I looked into it, it was about a connection failure, and this guy had put through 750,000 requests via API calls. I was amazed that my software could handle that well.

00:05:38.000 Another surprise was when someone used the RDoc code coverage feature, which shows how much of your methods are documented. He had all the documentation, but RDoc said it was broken, so he wasn’t at 100%. He thought it was essential because he had a contract requiring 100% coverage and was using that tool to validate it. I had no idea that someone would depend on that feature for a contract.

00:06:15.600 There are many people who care about open source, and they share common values, forming a community that sustains and fosters the growth of the software. A good community helps you collaborate positively by providing good bug reports, feature requests, and other contributions, and I want to maintain those kinds of things.

00:06:41.280 The essence of open source is sharing and promoting reuse—not just the software itself but also the components inside of it, breaking them out into libraries. For example, Rails depends on many different gems, some developed within Rails and some from outside. All these things help form that community. Ryan of GitHub wrote an article titled, 'Your Team Should Work Like an Open Source Project.' You can find it at this link.

00:07:50.400 There’s also a great talk he gave on how GitHub operates, showing how you can apply these open source values to proprietary development within your companies. He listed unique constraints of open source software: they use electronic communication, like GitHub issues, emails, IRC, or mailing lists for support.

00:08:14.399 Open source projects have visible work. You can look at the repository, check the issue tracker, see recent changes, explore why changes were made, go back in time, and understand past decisions. The work is usually asynchronous, allowing you to leave comments on commits or update tickets, and then they can come back to it when they have time.

00:09:00.679 Also, it’s usually lock-free. You don’t need to worry about what someone else is working on; you can continue your own tasks until it's time to merge those changes. What I like about Ryan's post is that he describes the benefits GitHub gains from following the open source model, which is applicable to writing proprietary software.

00:09:46.560 Next, I want to talk about how I like to make maintenance easier by changing how I commit my software. The changes I make are one of the easiest ways I've found to improve my life, at least in terms of software maintenance. The most basic thing a commit does is create history. The commits you make over the lifetime of a project show how you began, how you grew the software, what problems you had, and how you overcame them, including some big problems and successes.

00:10:52.160 Through the comments you put on your commits, you should be telling a story about your software. For example, when you add a new method, explain why you added it, what use it serves, how it interacts with the rest of the project, and if there are any issues you may need to revisit later. Good commit messages can turn boring history into a narrative, allowing you to easily refer back when problems arise.

00:11:27.360 To make history easier to read, commit small changes. Smaller commits make your history clearer, particularly with blame views in your files. When you need to go back to understand why something is the way it is, they provide a clearer snapshot. If you find a bug and want to revert, a small commit means there's much less change to undo. It's easier to backport small commits if you maintain multiple branches.

00:12:14.320 I also enjoy bragging about my commits on GitHub; right now I have about 2500, which is a decent number. However, I know there are people with two to four times more than I do. In Sandy's talk, she had a graph displaying complexity across various stages. While she had around 10 different stages, I would probably create 20 or 30 commits for each minor change.

00:12:36.880 You can put those changes on a branch and squash them later if you like, but I prefer to keep them as-is. This way, when I revisit them, I remember what each change relates to, which helps in debugging. This idea of committing small changes is known as the atomic commit convention. You can find more about it on Wikipedia, as well as articles describing problems caused by larger commits.

00:13:43.760 What does a small commit look like? If you’re fixing a bug, just fix that one bug and include only that change. If you're adding new features, just add a method or two. Clean up whitespace with a separate commit, or do a typo correction or documentation improvement as their own distinct commits. If a feature touches many files, work on a branch, so you don't have a lot of work-in-progress commits on your master.

00:14:31.760 It’s important to remember that small commits alone aren’t enough to tell a story; your commit messages must be descriptive. Tim Pope tweeted a commit comment demonstrating the opposite: 'Not sure where I was going with this.' This exemplifies a state of mind that isn't useful for the next person reading it - they’ll be left wondering what Tim was thinking.

00:15:56.480 In response, he later stated: 'One thing closure has taught me is that good commit messages are a luxury of people that know what they are doing.' I believe that if you strive for this, you too can have the benefit of good commit messages, even if you are unsure of your work.

00:16:30.000 Your commit message should include what you’re doing and why. If you're fixing a bug, reference the bug number in your tracker. If the bug was introduced by another commit or feature request, mention those too, so others can see if there’s a deeper issue. Taking time to trace these things will allow you to identify potential problem areas.

00:16:58.560 The older the software you're working on, the better this practice become, because it’s hard enough to remember what you did last month, let alone six months or a year ago.

00:17:32.000 When I add features and encounter existing behavior I want to integrate with, I typically run into situations where tests fail. It’s critical to see why we wrote something the way we did, and with good commit messages, it's easier because I'm reminded by both the code and the messages about the original thought process.

00:18:23.840 Git has built-in guidelines for writing commit messages. When you run 'git commit help,' it suggests beginning with a brief, single line summarizing the change. Here are some examples of short summaries I’ve written.

00:19:08.240 The first one is: 'The URI argument to gem request new must be a URI.' This tells me I probably added an argument check or removed some form of argument conversion.

00:19:28.160 The second one is: 'Only display relevant release notes upon update.' This indicates that I added a new feature or fixed a regression. In this case, it came from RubyGems, so 'gem uninstall' would likely show a related ticket reference.

00:19:51.840 The last example I will mention is an unfortunate commit for 'gem uninstall food-dash-hall,' which indicates I added a new feature or potentially a bug fix for a previously broken feature.

00:20:24.320 These brief summaries don’t need to be particularly useful to someone unfamiliar with your project; the full message should be. They serve more as reminders for me when I glance through my history.

00:21:09.600 Following the summary, write a more thorough description that fits your thought process. For the URI argument fix, I mentioned that this was correcting a bug introduced in the tests.

00:21:30.720 In this case, I was passing a string into request new instead of a URI, and the tests were the only place this happened. I eventually ran this on Ruby 1.8, and it wouldn’t accept the string as a URI.

00:22:03.360 As such, I simplified the argument conversion. In the description, I aim to explain the old behavior, why it should change, and what the new behavior is.

00:22:41.120 It's good practice to reference issue numbers and avoid assigning blame unless it's warranted. I am completely fine with blaming myself if that's the case.

00:23:19.600 Apart from commits, bug reports, feature requests, and pull requests usually come through your issue tracker, which is how I interact with others on my projects. Having a good process in place helps keep people updated and assists in managing maintenance.

00:24:26.240 Being organized is really important—not only so that you know the status of your tracker and project but also to allow others to understand the progress of their bug fixes, feature requests, or pull requests.

00:25:13.040 I like to utilize issue labels on GitHub. I usually have two or three, at least a status label and maybe a type and category. My status labels are accepted, rejected, feedback, or duplicate.

00:25:47.040 I use 'duplicate' to keep an eye on frequently reported issues, indicating something significant that may require urgent attention. I label bug reports or feature requests primarily by type.

00:26:05.600 I also use milestones to track when issues are resolved and set to release. I usually keep three: one for future tasks, one for the current milestone (say, the next RubyGems release), and one for things I never want to do.

00:26:38.720 I rarely create milestones for bug fixes, as I'm usually ready to discard those right away. Staying organized allows for the quick handling of issues.

00:27:23.040 Overall, keeping an open mind is beneficial when investigating problem sources. While I may think I've identified an issue, it's helpful to remain flexible.

00:27:44.720 For example, with Rubygems issue 510, I fixed a permissions problem while creating the Rubygems credentials file. I received an issue about how the permissions should be included in the publishing guide.

00:28:27.200 Initially, I was confused because I thought the issue was already resolved. However, I learned that users had used curl commands to download things and lacked correct credentials. It taught me to read the documentation more carefully.

00:29:05.920 Politeness is crucial when mistakes occur. By double-checking assumptions, you build mutual understanding with your collaborators. When problems arise, it's easy to ask for clarification if you're polite.

00:30:10.440 Sometimes, I receive bug reports stating that a feature is difficult to use or is broken. Although I remember it working before, I always double-check to see if something can be improved.

00:30:53.520 When considering new features, I approach them cautiously because they can complicate maintenance. I want to fully understand what the user is attempting to achieve and see if there's a more manageable solution.

00:31:28.960 Kathy Sierra suggested that instead of solving a hard problem, you should ask if there is a way to make the hard problem irrelevant. The answer is often yes. Therefore, when addressing complex issues, find ways to hide unnecessary details.

00:32:26.760 For instance, in my Cap API, I chose to enable users to get a capture without dealing with complexities.

00:32:55.760 When writing features, it's crucial to keep the code simple. Eleanor McHugh wisely stated that the worst part of clever code is not being clever enough to understand it. I’ve often made things unnecessarily complicated only to return later and ask myself, 'What was I doing?'

00:33:42.560 To ensure ease of maintenance, always aim to simplify any features you add, and ensure your commits tell a coherent story. Whenever possible, keep your features simple, create atomic commits, and implement extension points for complex requests. Document the reasons when you can't simplify as desired.

00:35:35.760 In closing, I often get asked how one should contribute to open source and what projects to choose. My advice is to start with the gems you use. There are many open source software available to work on, so contributing to things that interest you makes sense.

00:36:15.200 You could find bugs, improve documentation, or fill in missing examples. Contributions don’t have to be massive; even simple fixes count. Look through issues for projects you’re interested in to see if you can contribute.

00:36:48.960 Overall, start small, gain familiarity with the code, and build a rapport with maintainers. Open source is about addressing personal interests while supporting the community.

00:37:02.480 Thanks for listening, and I'm happy to take any questions.

Ruby on Ales 2014