Testing

Summarized using AI

Tales of Interest

Jon Rowe • February 19, 2014 • Earth

In his talk titled "Tales of Interest" at RubyConf AU 2014, Jon Rowe, a maintainer of the RSpec Ruby project, shares his experiences and insights into the world of open source contribution, maintenance challenges, and community involvement. Rowe emphasizes the importance of giving back to the open source community and provides a look at the difficulties faced while managing an open source project, particularly when it comes to supporting multiple environments and dealing with unintended usage by contributors.

Key Points:

  • Open Source Involvement: Rowe's journey into open source contributions began through a platform called Code Judge, which helped him discover relevant issues to work on within RSpec. This community engagement was crucial in his growth as a contributor.
  • Challenges of Maintenance: He highlights that significant projects will inevitably face maintenance challenges, especially as user contributions come from various environments and setups.
  • Diversity of Environments: Differences in how Ruby is used and installed (e.g., RVM, RBM, or direct installation) can lead to unique bugs and issues, complicating the maintenance of projects. Rowe discusses the range of Ruby versions (e.g., MRI, JRuby, Rubinius) developers may encounter, leading to inconsistencies.
  • Unintended Usage: A significant source of issues arises from users utilizing features in ways that the maintainers did not anticipate, often due to undocumented assumptions.
  • Cross-Platform Considerations: Rowe shares the difficulties of supporting diverse systems, particularly Windows, where issues arise that are hard to replicate in a Mac-centric development environment. He recalls an incident where RSpec's latest version failed to display colors correctly in a Windows environment, showcasing the constraints of testing across platforms.
  • Encoding Issues: Encoding presents another layer of complexity when dealing with different languages and character sets. Problems can occur when handling test files with varied encodings, leading to compatibility issues that may break functionality.

In conclusion, Rowe encourages viewers to engage with open source projects while highlighting the responsibility that comes with maintaining such contributions. He stresses the need for patience, collaboration, and a willingness to help others navigate issues they encounter, ultimately fostering a supportive community to drive open source projects forward.

Tales of Interest
Jon Rowe • February 19, 2014 • Earth

RubyConf AU 2014: http://www.rubyconf.org.au

So you want to be an open source contributor? Think you can handle supporting multiple rubies, tracking down weird bugs, stomaching all that the wider developer eco system can throw at you? Well good reader, I can tell you stories that will turn your stomach, and blind your eyes.
Let me take you on a journey deep into the depths of open source triage and follow the kind of madness that comes from tracking down bugs across differing versions of Ruby and VMs.
Finally let me entertain you with stories of the perils of dogfooding your testing framework, and throw in some stories about the kind of hackery you can commit with a bit of mischievousness.

RubyConf AU 2014

00:00:09.440 Hey everyone.
00:00:11.340 Um, so, as Mark kindly said, I'm Jon. The main reason I'm standing in front of you all here today is that I currently have the pleasure of being one of the maintainers of a little Ruby project called RSpec.
00:00:16.320 So today's talk, "Tales of Interest," stems from my experience working on RSpec. However, it might need a little bit more explanation than that. The blurb was quite intriguing, and we talked about the problems and challenges faced in working with open source.
00:00:29.760 So, you're probably asking yourself, "What have I let myself in for here? Is this going to be a massively technical talk? Is this talk just going to be about RSpec?" Pretend I’m not here if you're just going to lecture us on RSpec all day because I really hate that sort of thing. But I am going to talk about RSpec because that's kind of why I'm here. But that's not the main point of this talk.
00:00:44.219 The point of this talk is actually about open source. It is about the experiences involved in helping our community build a maintainable tool that helps others build maintainable tools to develop better software. And although it is used the world over, if you're a MiniTest fan, you might want to go and watch Mario's magic because, you know, I'm an RSpec guy.
00:01:06.540 So in a way, this talk is about every one of you. If you've ever used RSpec or any open source project, this talk has a little bit to do with you. I wouldn't be here without the wider Ruby community, because I would have no job, because I would have nothing to do. I kind of wanted to give back to the community that has given a lot to me. I wanted to give back to open source, which is why I started contributing to projects like RSpec.
00:01:40.380 It was this time at RubyConf last year that I encountered a site called Code Judge. Have any of you seen Code Judge before? Well, a couple of hands. That's good. Code Judge is a site dedicated to encouraging people to give back to open source. You sign up, and you can follow some repositories and look at issues that are meaningful to you. It gives you an email prompting you to look at issues every day. That was how I started getting involved with RSpec.
00:02:05.880 I'd always done little bits and pieces, but actually having that stream of issues sent to my inbox encouraged me to write more code, submit more pull requests, and give more feedback, which eventually led to me and a few other people joining the RSpec core team. We went from one to five people working on RSpec to keep it up and running, partly because we've been working on a new version, which I may get into in a minute, but mostly because we wanted to give back to the community.
00:02:32.820 So, what is this talk? It is about giving back to the community, but it's also about maintenance. Any open source project of significant size, or any commercial projects of significant size, are eventually going to run into maintenance issues. Some of you are probably thinking, "Maintenance? My code's perfect. I don't need to do maintenance; I only work on awesome code. This talk is not for me." Well, again, Mario's downstairs with his magic tricks.
00:03:01.740 Any project big enough or widespread enough is going to encounter differences in the way that the software is put together. Differences that you probably won't have encountered on your machine; differences that are probably not going to be encountered on your co-worker's machine, which, if you're in any sort of trendy startup, are probably all identical MacBook Pros.
00:03:38.940 Not everyone is going to be using those folks, so you're going to run into environment issues where you've got different Ruby tools—people using RVM, RBM, True Ruby, or maybe they installed Ruby from the package manager if they're a Linux user, or they use the Rails installer if they're on Windows. All of these factors add up to differences in your environment, and you might find you run into interpreter bugs. Because you're suddenly realizing, "Hmm, there are what, six versions of MRI?" Sort of rough usage; there's JRuby, there's Rubinius. So differing Rubies are going to mean you encounter different problems.
00:04:45.780 And the biggest maintenance issue you're going to encounter is unintended usage. I originally titled this slide 'user error,' but I thought that was a bit unfair because, actually, it's not just users; it's people using things in a way you didn't intend.
00:05:20.460 It can be your own fault for not documenting something. You can make assumptions about how something works, and someone has a different assumption. In the case of RSpec, some people seem to find that they use the should syntax for assertions outside of tests and expect that to work. I haven't quite understood why that one happens, but it does, and it's something that we need to account for.
00:06:08.460 So if we go back to the environment—not the environment that Tony Abbott is trying to destroy—but back to your development environment. At some point, your open source project is going to encounter someone who has a different idea of how things are set up.
00:06:20.640 You can have different forms of Ruby tools, you can have different operating systems, and your code is going to break. The worst thing you can say to these people is, "It works on my machine." Everyone's heard this phrase, but it's going to be a killer for contributions to an open source project. No user is going to help you if you just go, "It works on my machine. Works on CI." You might be a little more politically correct and say, "I can't reproduce it," but as a maintainer, it's your job to help people reproduce problems.
00:06:43.440 It can be tough, but it's something you've got to work on—you've got to get better contributions from people by being kind and teasing that information out of them. I know I'm not necessarily always the best person to do that. If anyone's ever interacted with me, but I do try to gently seek out the information. The worst question related to this is: "Does it work on Windows?" It's kind of the stuff of nightmares for you as a Ruby developer, and you're all on Macs. When someone asks that question, you go, "I have no idea." There's five people developing on the RSpec core now, and none of us use Windows.
00:07:55.200 We occasionally get people filing issues in Windows, which proves that it is being used by Windows users, but we struggle to help them because we have no environment in which to test it ourselves. I've discovered recently, in fact, when I was running a workshop, someone did use Windows. I discovered that the latest beta of RSpec doesn't output colors properly, so half the text is invisible. That was rather embarrassing for me.
00:08:55.440 But we do this in some hope on that front. I have heard a rumor that Travis CI may be supporting Windows in the near future, which would replace that VMware argument, because obviously, Travis is better than VMware.
00:09:11.880 But even if it's not running on Windows, we are still going to have subtle differences between the way that OS X and Linux handle things, especially when it comes to file systems and case-sensitive vs. case-insensitive file systems. Not every Ruby developer is going to be aware that if you spell a file name differently with uppercase on a case-insensitive file system, that file will be fine, but then stick it on a Linux machine with a case-sensitive file system and...yeah.
00:09:56.160 So you've got to be aware of the differences in your environments. You've got to be aware of the differences in the emergency phase. One of the other inconsistencies you're going to come across in your environment is encoding. Not everyone is going to have problems with that.
00:10:13.680 If you're just a web developer, and you're outputting to a browser, a lot of things are taken care of for you. But encoding has always been a major pain if you're dealing with console output, because you've got to constantly think about the encoding of your files.
00:10:46.440 And I don't mean that in the case of, well, you know, you're sticking 'coding: utf-8' at the top, right? But how that interacts with other people's code is going to affect you. This is especially true of a tool like RSpec, where we actually load things from a developer's file system, run them, and mix them with our encoding systems and output strings.
00:11:29.160 There are plenty of people using RSpec who don't speak English, and they write their specs in Korean, or Russian, or French, or any weird character set, which does happen. We're also going to be aware that people, because it's a testing tool, are actually going to be doing stuff and testing changes in encoding. I've had people who have been stubbing out 'Encoding.default_internal' and then expecting encoding to work correctly.
00:12:10.920 And we have to be mindful of how this interacts with output encoding because, as I’ll speak as a console tool, it interacts with a console that may be a different encoding than our internals. This means that if you're going to get mixed output displayed, in the worst-case scenario, we will discover that RSpec doesn't work because it tries to print to the console; the encoding mismatches, and it blows up.
00:12:53.160 How does it work? There's also a little issue with literal encoding. Now, most of you will be familiar with string literals, and the default string literal encoding, of course, defaults to Encoding.default_internal, which for most systems is going to be UTF-8. But there are other things that are encoded; regular expressions default to US-ASCII.
00:13:59.100 Who knew? We didn't know until we tried to run a diff against this and tried to actually substitute parts of strings encoded in UTF-8 or UTF-16LE, which aren't compatible with ASCII. That blows up and produces strange errors for people, and you look at it and you go, "Well, obviously, let's just use the wrong string encoding." Until you start digging deep and go, "No, something's going wrong here. Oh no, literals!"
Explore all talks recorded at RubyConf AU 2014
+17