Talks
Speakers
Events
Topics
Sign in
Home
Talks
Speakers
Events
Topics
Leaderboard
Use
Analytics
Sign in
Suggest modification to this talk
Title
Description
After years of tackling background job complexities with Resque and Redis at 37signals, the team finally decided to build an out-of-the-box solution. Enter #SolidQueue, a default now in Rails 8. Rosa Gutiérrez presented Solid Queue at Rails World and shared the journey and the challenges they faced to get it live. Note: Rosa lost her voice the morning of this presentation but put on her game face and delivered the talk anyway. #Rails #Rails8 #solidqueue #resque #redis Thank you Shopify for sponsoring the editing and post-production of these videos. Check out insights from the Engineering team at: https://shopify.engineering/ Stay tuned: all 2024 Rails World videos will be subtitled in Japanese and Brazilian Portuguese soon thanks to our sponsor Happy Scribe, a transcription service built on Rails. https://www.happyscribe.com/
Date
Summarized using AI?
If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.
Show "Summarized using AI" badge on summary page
Summary
Markdown supported
Rosa Gutiérrez's presentation at Rails World 2024 focused on Solid Queue, a new backend introduced in Rails 8 designed for Active Job processing. The talk discusses the evolution, development, and challenges faced in creating this system, especially after prior experiences with Resque and Redis. Key points covered during the presentation include: - **Background**: Rosa outlined how Solid Queue emerged as a response to the inefficiencies encountered with Resque, emphasizing her experience at the Rails World conference where it was announced. She humorously noted the motivation induced by public promises regarding project delivery. - **Need for Solid Queue**: Several existing components were lacking adequate functionality, prompting the team to develop internal gems to enhance job handling, including mechanisms to prevent job loss and manage overlapping jobs effectively. - **Core Components**: Rosa discussed the architecture of Solid Queue, featuring multiple internal gems and forks that facilitate job execution, scheduling, and management, particularly the challenges faced with Redis memory limits when handling millions of scheduled jobs. - **Design Innovations**: The presentation covered the implementation of advanced database techniques such as ‘select for update skip locked’ to optimize job claiming processes and mitigate contention issues seen in earlier systems. - **Efficient Polling**: The introduction of a hot table for efficient job polling, alongside a method to keep track of workers’ heartbeats to manage job claims responsibly, were highlighted as significant design features. - **Performance Metrics**: The architecture achieved impressive performance metrics, processing about 20 million jobs daily across 74 virtual machines with 800 workers, while also allowing for bulk job processing. - **Community Involvement**: Rosa expressed gratitude for the Rails community’s collaboration in addressing challenges and bugs encountered, underscoring the importance of community support in software development. - **Future Directions**: The talk concluded with an invitation for further discussion and collaboration to enhance Solid Queue’s functionality as the team continues to test and refine the system based on real-world usage. Overall, the presentation encapsulated the journey of Solid Queue from conception to deployment, emphasizing design decisions tailored for efficiency, community engagement, and a focus on performance improvements, marking a significant advancement for Rails developers handling background jobs.
Suggest modifications
Cancel
