The Littlest ORM

by Noah Gibbs

In the talk titled 'The Littlest ORM,' Noah Gibbs, a seasoned Rubyist and author, presents an in-depth guide to building a mini Object-Relational Mapping (ORM) system using Ruby. The video aims to demystify ORMs by illustrating how tools like Sequel, ActiveRecord, and DataMapper function while providing attendees with a practical example they can follow along with. Key points covered in the session include:

Purpose of Building a Custom ORM: Gideon discusses why one might opt to create a custom ORM, highlighting potential limitations of existing ORMs and the necessity for tailored solutions in specific applications.
Starting with the Basics: The process begins with the setup of a gem and integrating the sqlite3 gem to establish a working environment. Emphasis is placed on getting comfortable with the SQLite database as a beginner-friendly solution.
Defining Class Structure: The example focuses on a mini-blog application where class definitions correspond to specific database tables, illustrating how object names correspond to their respective database structures using conventions.
CRUD Operations: Gibbs walks the audience through the implementation of Create, Read, Update, and Delete operations while stressing the importance of SQL string construction and security against SQL injection through proper mapping techniques.
Pared Down Code Complexity: The talk emphasizes that a fully functional ORM can be created with just 81 lines of code, encouraging developers not to feel intimidated by building their frameworks.
Testing and Resources: Attendees are encouraged to check out supplementary materials on GitHub, complete with tests and additional chapters from Gibbs's book, 'Rebuilding Rails.'
Discussion on Performance and Design: In a Q&A segment, Gibbs discusses the importance of efficient ORM design to avoid unnecessary database requests. He also touches on considerations for transitioning between different database systems like MySQL and PostgreSQL.

The main takeaway from the session encourages developers to explore ORM customization to meet their unique needs while highlighting that with proper understanding, writing mappers and frameworks in Ruby is manageable and rewarding. The video concludes by underscoring the advantages of using ORMs in terms of reducing programming time and enhancing maintenance efficiency.

00:00:20.720 Okay, our next talk is by Noah Gibbs. He is the lead Rubyist at OnLive and the author of 'Rebuilding Rails.' He's a Bay Area local, coming to us from Fremont, and he's been doing Ruby for seven or eight years now. So, he's pretty much an old-timer now. Noah will be talking to us about building the littlest ORM.

00:00:52.640 Good to see folks. Somebody yell, 'Slow the hell down!' every so often. I tend to go quicker when I get nervous. So, when you get to the point where I’m too fast for you to hear me, that'd be great.

00:01:12.640 Why build an ORM? First off, a commercial announcement from the folks I work with: like everyone else hiring Rubyists in the Bay Area, we're desperately trying to hire Rubyists. For those of you looking, remember that we're on the list of places that would love to hire you.

00:01:20.479 I'm also told that in addition to cloud gaming, we're redefining how applications are consumed on the internet. It’s possible that the future's so bright, you have to wear shades! A lot of the problems we work on are really interesting. This talk mostly covers Chapter Seven from my book, so that’s where I draw much of this material from.

00:01:48.000 Why build an ORM? It’s not like there’s a shortage of ORMs out there—Active Record is pretty good. You wouldn’t want to write that many lines of code yourself. The reason for this is all the mappers work in a fairly similar way. One day, you may end up using Cassandra, MongoDB, or Redis, and something’s going to be wrong with the mapper.

00:02:09.039 I don’t mean that you should have to create a big open-source project. If something goes wrong with the mapper, I hope you will seriously consider rewriting a better version for your specific use case. You can choose whether to put in a pull request, which would be great, but at some point, something will indeed go wrong, and you'll want to write a mapper.

00:02:56.400 I’m not claiming this slide deck will prepare you to debug Active Record, but each ORM has its own flavor, and for each one of these, you'll find multiple mappers. The reason for that is each mapper gets one thing right, and you may need a couple of these right at some point.

00:03:12.400 This means you may not just be working with Redis directly; you may want to add some kind of structure or migrations to your ORM. I won’t speculate on how to keep people from going down to the raw low-level API, but sometimes, that’s a very good idea to do so.

00:03:34.880 For those of you who want to follow the code directly as we go along, really you don’t have to. Here’s the GitHub URL for everything we’ll go through. Everything we cover will be the full code for a working ORM. If you type it all out really fast as we go along, you’ll end up with a working ORM by the end of this talk.

00:04:01.360 To begin, you start this like any gem. You say 'bundle gem [gem name].' It doesn’t have to be Git, but pick a source control system, add the sqlite3 gem to your gemspec, and now you’ve got an empty gem with that setup.

00:04:20.000 This is important, as by the end you can have a working ORM. Remember that, if you don’t clone from GitHub, we’re using SQLite because it’s the ultimate web-scale database; it’s built into your browser—unless you’re using Firefox.

00:04:54.960 You’ll also need a table for this implementation. I won’t cover the whole migration system, but you could just use one from Padrino, as it does a great job integrating several of them. SQLite will cheerfully create a new blank database, and I’m going to assume you’ve already created a table to work with. This mini-blog application can have a body that is only 128 characters long because I feel that Twitter poorly manages the 140-character limit.

00:05:30.320 You need to determine the table name. When I say 'name,' it means 'name.downcase,' which uses the class name. If you inherit a Posts class from this model class, it means the table name will be 'posts.' Your class Posts will inherit from 'ORMModel' instead of 'ActiveRecord::Base.' The code should not look intimidating, and my goal is that when we look at the code at the end, you should be able to say 'That’s an ORM?' If you’re thinking it looks simple, I’ll be a happy man.

00:06:06.080 We need to consider the table info call, which returns the schema and allows us to iterate through it. It’s basically a array of hashes where we grab the name and the type. We just put them in a hash saying that 'body goes to varchar128,' 'title goes to varchar32,' etc.

00:06:32.160 What we have to do is take this relational database and do something with it in Ruby. Once you get past the fact that there's a string at the top being interpolated into the SQL query, you’ll see it’s simply taking your keys from your table where ID equals your ID. You'll be building a long string and passing it off to SQL.

00:06:57.440 After covering the incidental stuff and the CRUD operations—create, read, update, delete—what you need to do is create a row. Looking at this, it operates just like the previous SQL string building; you’re inserting into your database with a set of keys and values.

00:07:26.480 The subtlety you’ll see is that I’m mapping a bunch of question marks. I hope you won’t have to do much of this with your various NoSQL interfaces because that just means we’re doing SQL escaping. It’s essential to avoid SQL injection attacks where someone can pass in malicious SQL code.

00:07:57.680 In this way, you can look up fields from your hash and assign object attributes. Here’s how you can delete an object: 'DELETE FROM table WHERE ID = ...' The 'to_i' function is there for security reasons to keep malicious code out.

00:08:15.680 To update the database row, you build the SQL string by setting fields where ID equals . This SQL string construction is simply how SQL API works. In case you didn’t realize, you’ll be building a pretty large SQL string, as that’s the core of interacting with the database.

00:08:45.680 If you want to create a new object, you’ll insert a row into a table, find the object, and that’s what we’re looking at here. After inserting values, you’ll want to retrieve your new ID, which requires a select from your inserted row.

00:09:02.480 Now we can take a breath. This feels like a lot of code as you sit through multiple densely packed slides. But when you see that 81 lines of code create a functional ORM, none of those lines are complicated.

00:09:21.520 I’ve been discussing NoSQL throughout, but all of this is relevant to NoSQL. You could build an ORM specifically for NoSQL and you should! However, I want you to take away that the lines of code in Ruby frameworks are tiny. They’re easier to read and write and don’t take a long time to put together.

00:09:44.880 Remember, don't be intimidated by writing frameworks or mappers in Ruby. Once you know what you're doing, there really isn’t much to it. In my book, 'Rebuilding Rails,' I guide you to build a Rails-like framework from nothing. Each chapter contains pieces of Ruby magic that shouldn’t intimidate you.

00:10:08.480 As for tests, there's enough code on these slides, but if you clone the GitHub repository, yes, there are tests included. This URL provides links to the slides and the GitHub repository, which also contains two chapters on ORMs from my book.

00:10:32.800 To summarize, check out that URL again. My team at OnLive is hiring, specifically, you'd be working for me. Now that you've heard me speak, you can decide how that might feel. The comic artwork throughout is from Earthworld, which I really appreciate since I’m a big fan of webcomics. All the artwork is Creative Commons, making it easy to share.

00:11:08.160 Thank you very much! Alright, we generally don’t do Q&A at the end of sessions, but Noah blasted through that material. Do you want to do a quick rewind?

00:11:34.720 Given that we have extra time, let's open it up to questions. For those I haven't met today, Thomas here from Van Harten—it’s my experience that a lot of ORMs are slowing down as they often make excessive requests.

00:12:00.000 Would you recommend creating more ORMs? This might sound odd at a Ruby conference: if you're truly worried about absolute maximum performance, something may be wrong. Your ORM may never match the fastest low-level API calls, but as programmer costs rise and machine costs decrease, a good ORM can save significantly on programming time.

00:12:35.680 If your ORM enforces useful constraints and helps streamline processes, it will ultimately save you expensive programming resources.

00:12:56.080 On the topic of requesting unnecessary data from your database, if you write your own ORM framework, consider how its design impacts performance. Aim for a solution that avoids creating futile requests. As a rule of thumb, be careful about overload when employing the database.

00:13:54.720 For those looking to transition their ORM from MySQL to PostgreSQL, there’s likely some part of the ORM that becomes obsolete. ActiveRecord itself has a large structure with separate database-specific adapters and type mappings. You can learn how to build your ORM by studying how ActiveRecord manages these databases through its adapter pattern.

00:14:15.760 The spearhead for building ORMs with multiple backends is often hard. If you are creating your own ORM, do so specifically for your use case with minimal backends for the best results. The best advice I can give is to investigate the adapter implementation in ActiveRecord as it handles database-specific implementations.

00:14:36.240 So, that's all for this session. We’re going to give you a slightly longer break; make sure to be back.