Suggest modification to this talk

Title

Description

Title: Polyglot Paralellism: A Case Study in Using Erlang and Ruby at Rackspace
Presented by: Phil Toland

Two years ago Rackspace had a problem: how do we backup 20K network devices, in 8 datacenters, across 3 continents, with less than a 1% failure rate -- every single day? Many solutions were tried and found wanting: a pure Perl solution, a vendor solution and then one in Ruby, none worked well enough. They not fast enough or they were not reliable enough, or they were not transparent enough when things went wrong. Now we all love Ruby but good Rubyists know that it is not always the best tool for the job. After re-examining the problem we decided to rewrite the application in a mixture of Erlang and Ruby. By exploiting the strengths of both -- Erlang's astonishing support for parallelism and Ruby's strengths in web development -- the problem was solved. In this talk we'll get down and dirty with the details: the problems we faced and how we solved them. We'll cover the application architecture, how Ruby and Erlang work together, and the Erlang approach to asynchronous operations (hint: it does not involve callbacks). So come on by and find out how you can get these two great languages to work together.

Help us caption & translate this video!

http://amara.org/v/FG9l/

Date

Summarized using AI?

If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.

Show "Summarized using AI" badge on summary page

Summary

Markdown supported

In the presentation titled "Polyglot Paralellism: A Case Study in Using Erlang and Ruby at Rackspace," Phil Toland discusses how Rackspace addressed significant operational challenges in managing and backing up 20,000 network devices across nine data centers on three continents. The core challenge was to create a robust solution with minimal failure rates in a high-throughput environment where SSH access to the devices was complicated by latency and communications bottlenecks.

Key points covered in the presentation include: 
- **Initial Challenges**: Rackspace struggled with previous solutions using Perl and Ruby that lacked reliability and efficiency due to slow I/O operations and inefficiencies in handling vast datasets. Early attempts resulted in a 260-gigabyte MySQL database that suffered from slow queries and a rigid schema that hindered adaptability to new devices.
- **Adopted Technologies**: Rackspace adopted a polyglot approach, leveraging Ruby for its front-end Rails application and Erlang for backend processes due to its superior support for concurrency and fault tolerance. MongoDB was integrated for its flexibility compared to traditional RDBMS.
- **Operational Benefits**: The new system increased scalability and reliability while improving maintenance through simplified code structures. Moreover, Erlang facilitated the independent operation of worker processes, allowing the system to handle failures gracefully without service interruptions.
- **RESTful API Implementation**: Utilizing the Web Machine framework in Erlang, the team developed a REST API to efficiently handle network device communication and data management.
- **Key Learnings**: The team emphasized understanding and respecting Erlang’s programming paradigms, including its functional nature and concurrency model, which greatly enhanced application efficiency.

In conclusion, the mix of Erlang and Ruby provided a successful solution to Rackspace's complex device management needs, showcasing the power of polyglot programming and the importance of choosing the right tool for the job. The presentation highlights not just the technical solutions but also the crucial lessons learned during the development process.

Cancel