Security is hard, but we can't go shopping

Talks

André Arko

#security-best-practices

#ruby-community

#rails-security

Security is hard, but we can't go shopping

by André Arko

In his talk at RubyConf AU 2016, André Arko discusses the complexities and importance of security in Ruby development. He emphasizes that while security may often seem challenging, especially with the increase in vulnerabilities, it is a necessity that developers cannot ignore. Arko begins by introducing himself, highlighting his background in Ruby and open-source development. He notes the significant shift in Ruby's approach to security releases over the years, with the frequency of updates increasing drastically, which has left many developers feeling overwhelmed.

Key points of Arko's discussion include:

Understanding CVEs: Arko explains the concept of Common Vulnerabilities and Exposures (CVE) and how it serves as a standardized way to identify and communicate about security issues.
Rails as a Security Leader: The Rails framework has become pivotal in addressing security concerns within the Ruby community due to its early engagement with security researchers, leading to a series of high-impact vulnerabilities and subsequent updates.
The Importance of Updates: Arko compares applying updates to having insurance, suggesting that it involves a small ongoing effort that can prevent significant future issues and protecting against hacks.
Responding to Vulnerabilities: When vulnerabilities arise, following responsible disclosure is crucial. This involves private communication with developers and clear paths for reporting issues rather than public forums.
Community Collaboration: He stresses the need for empathy and ongoing communication within the development community to effectively handle security vulnerabilities, underscoring that all developers share the responsibility for maintaining security.

Arko shares an example involving the famous ruby gems website, which had to undergo intense scrutiny and repair after a breach, illustrating the dire consequences of neglecting security. He also highlights the frameworks that exist within the InfoSec community to facilitate responsible disclosures and effective communication.

In conclusion, Arko urges developers to prioritize security updates actively and maintain clear communication channels regarding vulnerabilities to foster a secure and thriving Ruby ecosystem. His call to action encourages Rubyists to collaborate and carry forward best practices in security, ultimately contributing to the integrity of the coding community.

Additionally, Arko introduces Ruby Together, a nonprofit organization he founded, which aims to enhance the infrastructure around Ruby development tools, emphasizing their collective security improvement efforts.

00:00:00.589 Hello! My talk is about how security is hard. But before I talk about security, I guess I should introduce myself. I’m André Arko, and that’s my avatar photo if you recognize it from something on the internet. As he just mentioned, I work at Cloud City Development in San Francisco. We focus on app development and web app development, and I also do training, joining teams, working with them on architecture, and helping people out when they're confused about writing code.

00:00:11.010 That's what your team could use; that's the core of what we do. I wrote a book called "The Ruby Way." I guess I didn’t really write it; I rewrote it. The second edition was great but only covered Ruby 1.9. The third edition focuses on 2.3, and it’s pretty awesome. It’s basically a collection of ways to use Ruby to accomplish various tasks, which is kind of the opposite of the usual Ruby book where it lists every single thing that Ruby can do. Instead, it emphasizes what you’d want to do and how to achieve those goals using Ruby.

00:00:39.750 I work on Butler, and my experience with open source has turned out to be really valuable, even though it can be quite difficult. It’s a wonderful learning opportunity about Ruby, the Ruby community, and what people actually do with Ruby. To be honest, before I worked on Butler, I had little idea of what others did with Ruby, aside from the companies I had worked for. Recently, we’ve been trying to make it easier for people to get started with contributing to open source. If that's something you’re interested in, feel free to email or tweet at me!

00:01:30.449 Now, getting back to security, I feel like a typical response to things being hard is often to give up. Unfortunately, in the case of security, that's not really an option. It’s very easy to feel discouraged when dealing with security issues, but we need to discuss why that occurs and what you can do about it. You may not know this, especially if you’ve been a Rubyist for a while, but there was a time when Ruby only had security releases maybe once a year or every other year. It wasn’t a frequent concern.

00:02:03.329 More recently, however, that has changed significantly. There was a time when Ruby had releases every single month for three consecutive months. The release management team felt pretty embarrassed about it, with their communication often being along the lines of, "I'm sorry, here's the new version." It became a situation where there were just too many security releases happening far too quickly, which left a lot of Ruby developers feeling overwhelmed.

00:02:42.110 Even recently, there were six Rails security releases in one day! As I start discussing security issues with people, I often hear them ask, "What's a CVE?" Many seem aware that a CVE implies a security problem, which is true, but there’s much more nuance to it. CVE stands for Common Vulnerabilities and Exposures, which means that each CVE is just a number issued by The MITRE Corporation. They created a numbering scheme for security vulnerabilities, and it has been adopted by everyone from the U.S. government to major corporations that produce software.

00:03:10.250 MITRE has distributed blocks of numbers to what they call numbering authorities, which include companies like Oracle, Apple, Microsoft, and Red Hat. If you’re a significant player in software, you likely receive numbers directly from MITRE and assign them to actual security vulnerabilities. The Red Hat security team does extensive work related to Ruby, providing CVEs for Ruby and Ruby gems using their block of numbers. Users can look up every single CVE that has ever been assigned on both the National Institute of Standards and Technology’s website, as well as MITRE’s site, which keeps track of all CVEs.

00:04:13.049 This is valuable because it allows you and anyone else to ensure that you’re discussing the exact same issue. After all, you can’t simply say, "Hey, remember that bug that came out in Rails and was a security problem last Wednesday?" because there could have been several over that time. This system helps maintain clarity about what went wrong, what you can do about it, and what steps you can take to access an updated version or address the issue—sometimes by just feeling defeated and going home to cry.

00:05:23.145 Rails bugs, similar to Ruby bugs, have increased remarkably in pace over the past couple of years. Historically, the Ruby community was known for its kindness, with phrases like "Matz is nice, so we are nice" circulating. Unfortunately, this perception shifted when individuals realized how easy it was to write code in Ruby and began creating automated exploits for its security vulnerabilities. Security researchers became aware of this and eventually realized how significant these vulnerabilities were. With most of Ruby’s history approaching it as if everything was fine, this is now a growing concern.

00:06:05.060 Rails has become the leading force within the Ruby community tackling security problems because it was the first and most significant project to engage with security researchers. Ruby could have posed problems without significant attention, but many sites depended on Rails. If Rails has a vulnerability, it becomes the Rails security team's problem. Thus, the security team now embodies the most concentrated experience with Ruby-related security issues.

00:07:02.810 In discussing Rails 3, it’s important to note that it isn’t new anymore, but it introduced structural changes aimed at creating a more modular Rails framework. Rails 3 aimed to create smaller components, meaning that each component now has separate chances for security issues. Unfortunately, an eight-week period saw significant vulnerabilities across numerous components, including New Relic, JSON, XML, Rack, Action Pack, Active Record, Active Support, and even the documentation tool, RDoc.

00:07:43.010 The frequency of these updates was overwhelming for developers. Therefore, what can you do about these problems? Unfortunately, the answer is constant vigilance. It's essential to outsource that vigilance if possible, rather than attempt to manage everything yourself. We must all remain attentive because when security vulnerabilities occur, they can have severe consequences. While keeping everything updated may feel burdensome, there’s a constant tension within software development.

00:08:31.140 There’s a strong cultural consensus about the importance of applying security updates, with phrases routinely emphasizing why developers should prioritize these. But unfortunately, it’s often not explicitly explained why it’s essential. On the other hand, business individuals often push for immediate feature implementation and demand to know why security updates have been prioritized over new features.

00:09:40.380 The best tool I've found for explaining this to non-developers, particularly business stakeholders, is to compare it to insurance. Updating equates to insurance against breaches; you exert a relatively small effort over time to mitigate the risk of being hacked. This pricing structure aligns with insurance in everyday life, where you pay a nominal amount to protect against potential catastrophic events. Despite hackers needing to target you among potentially many vulnerable systems, having no insurance can lead to catastrophic consequences.

00:10:26.250 For instance, cleaning up after a security breach can take down your website for an unpredictable duration. Consider the incident where rubygems.org got hacked; we had to shut the site down and manually check every gem to ensure the integrity of our system. The process was tedious and resource-intensive, requiring a lot of engineering time and consequently involving lawyer interactions due to the mishandling of private data. Following different laws and obligations after a breach varies depending on your operating state, creating an immense setup of potential legal liabilities.

00:11:06.590 All told, this is a perfect example of why proactive updates vastly outweigh the reactive chaos of cleaning up after a compromise. In conclusion, while updating can be tedious and time-consuming, it is worth the effort. Continuous low-level maintenance greatly reduces the chances of facing a high-cost, high-stress, catastrophic issue. Make sure to perform your updates so that you can sleep well at night and explain to your boss why they are necessary.

00:12:00.900 Now, having discussed why security is crucial and how to effectively communicate that importance, let’s address what you should do when you discover issues. How do we approach each other about security concerns? Fortunately, the Infosec community has existed far longer than the Ruby community and has developed established frameworks for addressing these issues, one of which is the practice of responsible disclosure.

00:12:47.900 This concept arose as the best approach after years of trial and error; despite often resulting in dissatisfaction on all sides, it minimizes the risk for parties involved—everyone remains informed without being severely impacted. The disclosure aspect is something companies typically hate; they do not wish to admit they’ve been hacked or face associations with such vulnerabilities. Consequently, it's a highly damaging situation for corporate reputation and results in companies going to extreme lengths, including copyright claims over vulnerabilities.

00:13:44.120 The responsible aspect ensures that those discovering vulnerabilities cannot share that information until it has been remedied, with a defined timeframe for responses. There are established practices that provide developers with a reasonable amount of time to act before the vulnerability is made public. It ultimately allows open communication regarding the vulnerability after it is appropriately fixed.

00:14:51.710 Moreover, many companies have taken to offering rewards as an incentive for responsible disclosure. Google, for instance, runs an application security program that compensates individuals who find issues, scaling pay based on severity. This encourages responsible collaboration that benefits everyone involved. Several other organizations, like Facebook and GitHub, have similarly adopted rewarding schemes; while some, like Engine Yard, make clear they don't offer such incentives. However, the emphasis remains on making the overall process positive.

00:15:35.310 When it comes to practical applications, if you discover a bug within software, especially if that bug pertains to security, you must prepare to navigate that responsibly. First, ask yourself if the bug allows inappropriate access to users' data or disables functionality for others. If you can answer yes to either question, it’s essential to follow the responsible disclosure protocol. You should contact the developers of the software directly rather than disclosing the information on public forums, letting them patch the issue privately.

00:16:34.910 Ideally, the affected teams will have security policies outlined that detail how to report problems. If they don’t, search for contact information on their website or their GitHub profile. Ultimately, exercising empathy toward other developers is crucial—they’re often not striving to create security problems. If all else fails, and you're still dependent on a library with security concerns, consider fixing it directly—forking the code and posting your solution.

00:17:27.660 Moving on to the topic of writing your Ruby gems, I’d love to see a show of hands: How many have created a Ruby gem? Wow, that’s quite a few of you! The ease with which Ruby allows developers to release gems often comes with responsibilities that any gem maintainer must uphold. Let’s examine scenarios involving possible conflicts and how to address them responsibly. It’s always critical to communicate openly with any error that surfaces.

00:18:38.670 For instance, if your coworker finds an issue or if a sympathetic voice raises concerns, facilitating correction within the code can be straightforward. However, it becomes more complex when vulnerabilities are identified after a product has already shipped. In this case, it’s crucial not to shy away from announcing issues publicly and instead focus on fixing them in the meantime. If scrutiny arises from researchers eager to demonstrate their prowess with security flaws in your software, you must follow the responsible disclosure guidelines carefully.

00:19:56.400 Respond quickly and set clear communication expectations about the timeline for coming to a solution. Researchers often become vocal if they feel ignored for too long; thus, try to keep communication tight. Once a remedy has been devised and the issue has been fixed, transparency regarding the bug's existence helps foster a community confidence in your solutions.

00:21:12.830 If you personally maintain a Ruby gem, clearly provide email contact information in your gem specifications. If you're part of a development team, establish a security address for inquiries, perhaps even a disclosure policy posted on your website. To aid the broader Ruby ecosystem, I have initiated a Ruby Security Announcement mailing list, gathering information about Ruby gems with security concerns.

00:22:02.080 In summary, utilizing existing frameworks and fostering open communication within the community is vital for maintaining security. We should carry the right values and practices forward by working together rather than alone, ensuring that Ruby development continues to thrive while addressing vulnerabilities comprehensively.

00:22:39.600 And actually, now that I've concluded, I realize I need to add a little more. I've brought a second talk about Ruby infrastructure. Ruby infrastructure plays an essential role in your development toolkit.

00:22:48.280 RubyGems.org and Bundler are critical tools used by Rubyists. Ruby Together is a nonprofit I founded dedicated to ensuring that these infrastructures function appropriately, which is paramount for all developers who rely on these tools to perform their tasks efficiently. The foundation emphasizes inclusive access by offering critical resources—both economically viable and collectively owned.

00:23:53.350 Ruby Together has a roadmap that outlines how we plan to enhance these tools to benefit the entire community. While we've paid for the development of Bundler infrastructure and security patches, it's also crucial to ensure that we alleviate burdens on developers to maintain the quality and integrity of the Ruby ecosystem. Ultimately, ensuring collective security and ongoing enhancements strengthens our global community.

RubyConf AU 2016