Rails 5 Features You Haven't Heard About

Talks

Sean Griffin

Rails 5 Features You Haven't Heard About

by Sean Griffin

In the talk "Rails 5 Features You Haven't Heard About," Sean Griffin presents a detailed exploration of lesser-known features introduced in Rails 5, focusing on quality-of-life improvements that enhance the developer experience.

Key Points Discussed:

Emphasis on Quality of Life Features:
- Sean highlights that while major features like Action Cable and Turbolinks often grab attention, small improvements are foundational for enhancing developer workflows.
Type Attributes API:
- One significant feature is the type attributes API, which was partially implemented in Rails 4 but expanded in Rails 5. It allows developers to define attributes in a manner that aligns more closely with their underlying database structures, particularly benefiting those who need data encryption.
Left Pad Method:
- A humorous highlight was the inclusion of a left pad method, which had been requested many times by the community, showcasing the importance of incorporating even the most trivial requests for user convenience.
Active Support's 'or' Method:
- The talk introduces a long-awaited feature allowing 'or' expressions within Active Record queries, making it easier for developers to combine conditions dynamically. Sean elaborates on the careful consideration that went into its implementation, emphasizing the need for clarity and usability in the API.
Metadata Table for Safety:
- Sean also discusses a new metadata table that prevents the accidental dropping of production databases by tracking migration states and alerting developers to risks when performing destructive actions.
Enhanced Migrations:
- New features in migrations ensure they are tracked by their Rails version, offering better control over legacy code execution and improving backward compatibility.
Accessed Fields in Active Record:
- The new accessed_fields feature allows developers to track which fields have been used on Active Record models, helping optimize database interactions in applications.
Boolean Field Handling:
- Updates to the handling of boolean fields aim to clarify truthy and falsy values, thereby avoiding pitfalls most developers encounter due to misunderstandings in earlier versions.

Conclusion and Takeaways:

Sean's talk serves as both an informative session on new Rails 5 features and a narrative on the journey of developing them, emphasizing collaboration, safety, and ease of use in Rails.
It strikes a balance between explaining technical features and sharing personal insights from the development process, ultimately reinforcing the community-driven nature of open-source software development.
The overall takeaway is that while significant updates are vital, the many minor refinements are what truly impact developers' day-to-day coding experiences, making Rails a continually evolving tool for web development.

00:00:10.180 Hi everyone, let's go ahead and get started. This talk is about the features in Rails 5 that you haven't heard about.

00:00:16.730 My alternate title was 'Sean dramatically reads the changelog', but I don’t have a great dramatic voice.

00:00:26.360 So instead, this will be 'Sean undramatically reads the changelog'. Actually, the main focus of this talk is less about the features themselves and more about the stories that went into implementing them.

00:00:39.500 We’re going to focus on the smaller quality of life changes. Those are the things that excite me every time Rails updates.

00:00:47.899 Once I get started, I’ll be fine. I’ll just be pressing buttons on the keyboard anyway.

00:00:58.820 What excites me with every version of Rails is the smaller quality of life features. These are things that aren't going to fundamentally change how I build applications but will make my life a little bit easier.

00:01:06.710 These features include smaller tools, new protections, and performance improvements. Many of them have very interesting stories behind their addition or, in some cases, why they were not added.

00:01:12.260 That’s what we’re going to talk about today. My name is Sean Griffin, and I work full-time on open-source. I’m a committer on Ruby on Rails, and Shopify sponsors me to work on open-source full time.

00:01:24.080 So, thank you, Shopify. As I mentioned, every feature has a story about when it was implemented, why it was added, and in cases like relations, why they weren't added sooner.

00:01:30.110 The first feature I’d like to talk about is one I’ve discussed a lot in the past: the type attributes API. This one is particularly interesting to include in a discussion about Rails 5 because most of the work that went into implementing this feature was actually done in Rails 4.

00:01:42.590 In Rails 4, we revamped the entirety of how we handle type coercion in Active Record in preparation for this public API. The API was 90% implemented in version 4.2 with a few edge cases, and I’ve talked at length about this before.

00:01:58.009 However, I don’t think I’ve ever told the story of why this feature was added. There were many contributing factors, but one project really stood out as the reason we needed this change.

00:02:12.650 This was something I worked on while at Thoughtbot, where the project required all data to be encrypted at rest, including database credentials. So, if you had access to the database, you still needed to be unable to read anything. We initially used a gem called 'attr_encrypted'.

00:02:39.820 This is what using that gem looks like: you call 'attr_encrypted' and provide the name of an attribute, and it will assume that the database column is called that attribute name plus '_encrypted'. It defines a reader and a writer for the unencrypted form and performs the encryption in Ruby.

00:03:06.590 However, the problem with this approach is that it doesn't work with methods like 'where' or 'find_by'. There was a feature at the time that has since been removed, which provided an escape hatch, but it was defined in a way that we don't write Rails applications anymore.

00:03:18.830 Using this gem would mean that our entire application had to adjust to accommodate this encrypted field. We ended up using gems like Ransack for complex search queries and had to write a significant hack using the PG Crypto extension.

00:03:40.430 This hack involved traversing the entire Arel AST looking for any binary nodes where the left side was an encrypted attribute and replacing it with the corresponding encryption or decryption methods. It turned out to be some of the dirtiest code I've ever written.

00:04:04.100 Around that time, Ernie Miller was giving a talk at RailsConf about some of the intricacies of Active Record, and he mentioned that it would be really nice if Active Record defined the accessor methods to match the database schema by calling some public API automatically.

00:04:21.620 I agreed, as it resembled what I had been looking to do for quite some time. Fast forward eight months, I got into Rails to start adding this feature and vastly underestimated the amount of work involved.

00:04:36.510 We quickly put together a working implementation of the API, but I was bothered that type casting in Rails 4.1 happened by grabbing the column object associated with that field and calling a typecast method.

00:04:50.930 In Rails 4, types were basically represented as symbols when loading from the database. We looked at the SQL string representing the type in the database and created a simplified type from that, which was ultimately represented as a symbol.

00:05:08.240 The initial implementation got rid of the case statement, replacing it with objects that we could inject. However, typecasting continued to utilize columns in 4.2, and in the earliest implementations, when you called the attributes API, we created a new column object that was stored in the columns hash.

00:05:30.020 It was particularly concerning when used for something not backed by a database field because there wouldn’t be a column involved. Much of the confusion stemmed from conflating columns and attributes. The initial creation of the API was somewhat straightforward, but it took another eight months to arrive at an implementation I was truly satisfied with.

00:05:49.580 This required rewriting nearly every part of Active Record, aside from the association code, that was touched by this change. We finally reached a point with a clean and simple implementation.

00:06:08.840 The final API looks like this: you call the class macro 'attribute', provide the name of the attribute, and then give it a type object. I like to provide examples passing an explicit type object, especially when using constructor arguments.

00:06:20.850 This was important for what I humorously refer to as my overly-engineered, extremely complicated replacement for 'attr_accessor.' I aimed to work with objects, as this clarifies the semantics of things like dependency injection.

00:06:37.740 This API is universal, ensuring that we can retrieve type information from a column when needed, which is somewhat how it worked in Rails 4.1. I aimed for the code to be structured to prevent users from mistakenly using the column object.

00:06:57.600 The big benefit of the attribute API is its compatibility with methods like 'where'. For instance, if you have a 'price' column represented as a money object, you can use 'where' with it seamlessly.

00:07:14.370 The schema inference code looks like what we described: we loop through all our columns and call the public API to define attributes. The 'define_attribute' method functions similarly, but it is stricter than the 'attribute' method, which is lazy as it waits for the column definitions.

00:07:34.480 The lengthy development time for this feature occurred because I didn’t want to ship it until I was content with the implementation. Specifically, I felt that modifying the column objects was technically visible through public API.

00:07:53.420 Now, we are in Rails 5, and this feature is finally available. This was instrumental in my journey into open source, reaffirming the need for another full-time contributor to Rails. It holds a special place in my heart.

00:08:09.710 I've discussed this topic extensively in the past, and I don't want to dwell on it too long. Now, let’s move on to the single most frequently requested feature since Rails 3.

00:08:21.020 I’ve seen it proposed countless times, and I'm sincerely sorry it took this long to get introduced: Active Support’s left pad method!

00:08:35.210 Finally, we provide a way to add padding to the left side of a string. We feel this feature is so important that we are shipping it as a separate gem independent of Rails.

00:08:45.260 We genuinely hope to have everyone in the Ruby ecosystem depend on this great new feature... sorry, I might be getting ahead of myself!

00:09:00.250 But, seriously, this is probably the most requested feature—in relation to Active Record—finally here: after six years since relations were introduced, you can now add an 'or' expression to your where clause.

00:09:14.920 With such a public response to this feature, it begs the question: what took so long? Why didn’t we do this sooner?

00:09:24.750 Many reasons contributed to this delay, but the most significant is that the API is considerably less obvious than one might think.

00:09:41.030 One thing people often forget when dealing with open source projects is that it's extremely difficult to change or remove an API once it has been introduced.

00:09:57.130 We can't just implement something ‘good enough’ as a temporary patch unless we are confident that it won't become an issue later on.

00:10:12.460 We needed to be sure that we were shipping the right solution. The API we released takes a relation and is a method on that relation, taking another relation as the argument.

00:10:29.480 This is the intended use case: if we have two named scopes, one called 'recent' and another called 'pinned', and we want anything that is either recent or pinned to appear on the front page of our blog.

00:10:41.080 You would then create another scope using 'recent or pinned'. This specific design was made to optimize for named scopes.

00:10:54.950 We wanted to allow you to reuse named scopes since, if something appears as half of an 'or', it is likely to be used independently of the 'or' as well.

00:11:07.240 We really aimed to optimize for compositional abstraction, offering developers a tool that allows writing reusable, easily changeable code.

00:11:22.060 However, there were several other proposals for this API, and this was likely the most common suggestion: simply using 'or' with a hash that behaves exactly like 'where'.

00:11:40.950 Yet, this introduced significant issues. For example, 'or' does not imply 'where' at all, and it's not the only place in SQL where it can appear.

00:11:55.370 It could also appear inside a 'having' clause. Since the relation exists in the realm of set theory, it would be reasonable to think that 'or' meant a union query.

00:12:07.900 Another common proposal was to make it 'where da or'. We ultimately decided against this as it wouldn’t meet our goals for the API, specifically not allowing easy reuse of scopes.

00:12:19.460 There are drawbacks to our approach, particularly outside of named scopes. If you’re constructing the 'or' ad hoc in a controller, you have to repeat the class name.

00:12:29.830 This gets even trickier when you're not using a named scope and need to perform a one-off query.

00:12:39.220 Ultimately, we encourage developers to avoid ad hoc one-off queries and instead define named scopes for better code reuse.

00:12:51.430 Now, let’s discuss a different topic: how many of you have ever set your database URL to your production database and then accidentally dropped it?

00:13:07.320 This happens all the time, usually when using tools to pull down environment variables from your production server, forgetting to unset the database URL before running tests.

00:13:26.750 If you're running tests with Capybara and using a server that runs in another thread, running database cleaner can lead to the accidental deletion of your production database.

00:13:40.270 This was happening frequently enough that we decided it was time to take action against this common mistake. Our first thought was to simply remove the database URL.

00:13:57.630 We suggested running 'rails db:migrate', but then we’d break workflows on platforms like Heroku. Richard Neiman championed this feature and led the initiative to rethink this issue.

00:14:18.850 We convened to brainstorm alternative solutions that wouldn’t disrupt container-based infrastructure, and we ended up with a more complex implementation.

00:14:34.390 In addition to schema migrations—automatically generated for you—we now have a general-purpose metadata table with two columns: key and value.

00:14:55.540 Currently, the only entry in this table will be the name of the environment that was last run against, which may not be visible during typical operations.

00:15:12.830 If you attempt to perform a destructive action before running a migration, you will receive an error stating that we don’t know the state of the database.

00:15:28.530 This new feature protects against the catastrophic mistake of dropping a production database, and although it may seem minor, it's very significant.

00:15:51.160 It allows Rails to silently protect you from this common danger if we've done our job right. Again, I’d like to thank Richard for his work on this feature.

00:16:06.880 Now, let's discuss something different regarding migrations. A question that isn’t often asked is whether you can still run migrations from two years ago.

00:16:24.350 Can new developers run all migrations without running 'db:setup' first? Unfortunately, this has been a problem.

00:16:39.980 Weak migrations are static code that isn't tested. If a new Rails version introduces a breaking change, you're unlikely to discover that issue upfront.

00:16:56.350 Additionally, we’ve seen a trend in user requirements where Rails apps are deployed independently, meaning multiple production environments.

00:17:11.750 In cases like Discourse or ManagedIQ, where many production instances exist, the ability to run old migrations can lead to trouble.

00:17:25.800 An upgrade might skip a version, resulting in executing migrations aligned with Rails 5, even if they were created for Rails 4. This can cause breakage.

00:17:42.270 Still, we want to keep adding new features, changes, and improvements. For instance, we have added foreign key support and improved indexes.

00:17:55.830 However, we cannot change the behavior of existing API methods like 'references' or 'null: false' due to backwards compatibility.

00:18:09.800 In Rails 5, migrations now display the version of Rails they were generated against as square brackets after the class name.

00:18:21.740 If a migration doesn’t have this version number, we will assume it was for Rails 4.2 or older. Migrations written today will work for the foreseeable future.

00:18:38.270 This change is not trivial, as maintaining every version of the migrations API until the end of time adds complexity.

00:18:55.640 While this means bug fixes will become more challenging, I still believe it is advantageous to maintain functionality.

00:19:05.680 Another feature we added is 'accessed_fields' in Active Record. This gives you an array of all the fields accessed on an Active Record object.

00:19:22.300 For example, when you create a new user model and access no fields, 'accessed_fields' returns an empty array.

00:19:37.470 However, after accessing a field, such as 'name', it would return that field's name in the array.

00:19:54.520 The pattern I noticed is that developers rarely call 'select' unless performing specific calculations.

00:20:08.400 In views displaying user names, we frequently select all columns, even though a user table may have nearly 100 columns, which incurs a performance cost.

00:20:24.340 So, to simplify this process, I hoped to facilitate copying the list from 'accessed_fields' directly into 'select'.

00:20:40.560 If a field was accessed that wasn’t selected, it would throw an exception, prompting you to update your 'select' statement.

00:20:55.360 The idea was to create a background job that would warn you if less than 50% of selected fields were accessed, indicating significant potential for performance improvement.

00:21:11.700 This feature was trivial to implement, and I was happy to see that it worked well within the refactored Active Record reached after typecasting changes.

00:21:25.760 Next, let’s discuss using boolean fields in Rails. Let's think about how you might use regex with a boolean field.

00:21:40.340 Imagine you have a regex matching a string and want to assign the result to a boolean field. You might expect the field to reflect whether a match occurred.

00:21:56.480 In your head, it's clear that the output would be true; however, in Rails, the implementation has nuances that can yield unexpected results.

00:22:12.700 For example, in Rails 4.1 and earlier, this boolean type handled constants that included strings such as 'T'.

00:22:25.480 Match data objects are not included in the array of 'true' values, leading to confusion. Using the squiggly operator instead might yield misleading results.

00:22:41.520 To mitigate these issues, we adjusted how we handle boolean evaluations, establishing a clearer distinction between truthy and falsy values.

00:22:57.320 This aimed to fix the complexities arising from earlier implementations while making our approach closer to how Ruby operates.

00:23:14.200 The goal was to eliminate confusing scenarios and ensure that our boolean handling works more distinctly and coherently.

00:23:28.640 Those are some of the stories I hoped to share with you today. While I have you all as a captive audience, I’d like to promote my new ORM written in Rust called Diesel.

00:23:43.750 If you're interested, I have stickers! Thank you to Shopify for sponsoring me to work on open-source full-time. I appreciate them allowing me to attend conferences and share knowledge.

00:23:58.850 If you'd like to collaborate with Raphael and me at Shopify, please let us know. Thank you all very much! I will now take any questions.

00:24:12.380 The first question was about chaining 'or'. Yes, you can chain 'or' expressions, and you can also combine them with 'and'. The precedence will match what you pass along.

00:24:20.500 The next question was about adding an explicit 'and' to the API. I think it’s an interesting idea, but we haven’t had a specific discussion about it.

00:24:28.300 Yes, null does not represent a faulty value. If your column is not null, attempts to insert null will cause an error.

00:24:40.100 The question was about access to fields in Active Record relations. Currently, we only track accessed fields for individual models.

00:24:56.490 Accessing fields for models in a relation would present implementation challenges, as I'd need to know that every model would be used the same way.

00:25:10.720 The following question was about the metadata table mentioned earlier. It’s named something like '__active_record_metadata__'. I don’t expect much collision with it.

00:25:24.450 Furthermore, to ensure uniqueness, we opted for a double underscore prefix. The next question discussed pulling down the production database locally.

00:25:36.830 Typically, this should be smoothly handled. If you perform destructive actions without running a migration, you’ll see an error.

00:25:49.870 The question about dropping outdated tables will also result in errors unless you’ve executed a migration.

00:26:02.400 The square brackets in migrations were implemented using the square bracket method on the class object to return a new anonymous class.

00:26:17.210 The question was about future ideas for Rails 6, particularly regarding connection adapters. We're considering making third-party connection adapters more stable through a public API.

00:26:35.310 This will necessitate breaking changes, but ultimately, we want adapters to function well long-term. I'm also contemplating moving the attributes API to Active Model.

00:27:02.660 This migration will involve breaking changes pertained to dirty-checking functionality, but likely wouldn’t be too disruptive overall.

00:27:18.480 Thank you all very much! I will remain available for any additional questions you may have.

RailsConf 2016