Building an offline experience with a Rails-powered PWA

Talks

Alicia Rojas

@aliciaorojas

#progressive-web-apps-pwa

Building an offline experience with a Rails-powered PWA

by Alicia Rojas

The video, titled "Building an offline experience with a Rails-powered PWA," presented by Alicia Rojas at RailsConf 2023, delves into the creation of Progressive Web Applications (PWAs) using the Rails framework. The session begins with an explanation of what a PWA is, emphasizing its ability to blend features of both web and native applications for a richer user experience. Key points discussed include:

What is a PWA?: PWAs are characterized by their capacity to provide offline capabilities, enhanced performance, and usability akin to native applications, through the use of service workers and application manifests.
Case Study Overview: Rojas shares a practical implementation case—developing a PWA for the Chilean government to assist farmers in diagnosing field conditions. The app was designed to function reliably in areas with poor network access, focusing on user-friendliness given the audience's limited tech literacy.
Turning a Rails App into a PWA: Key steps include setting up service workers and manifests, which play crucial roles in enabling offline functionality and app installation on user devices. Rojas details practical coding techniques and tools, highlighting the need for a service worker to manage network requests and caching efficiently.
Caching and Offline Functionality: The presentation discusses techniques for intercepting network requests and providing cached responses. Tools like Workbox are recommended to simplify service worker setup and management. Rojas also addresses the importance of creating an offline fallback page that maintains user experience when connectivity is lost.
Data Handling while Offline: Rojas covers how to manage data during offline periods using IndexedDB for storage and Background Sync for deferring sync tasks until a connection is available. This allows users to create and store records locally, which can then be uploaded once the app goes online.
User Interface without Internet: Considerations for how to allow users to interact with locally stored data and ways to manage updates to this data, ensuring a seamless experience that mimics server-rendered behavior.
Conclusion and Opportunities: Rojas concludes with the potential for creating streamlined tools to automate PWA development in Rails, thus simplifying the process for developers in future projects.

Overall, the video emphasizes the significance of PWAs in providing robust online and offline experiences, particularly for unconventional users, and showcases the effectiveness of using Rails combined with modern front-end tools to achieve these goals.

00:00:19.939 Welcome to this session! I'm really excited to be sharing this with you.

00:00:25.320 I hope you enjoy this talk as much as I enjoyed preparing it.

00:00:31.199 Today, we're going to talk about how to build an offline experience with a Rails-powered PWA.

00:00:36.719 My name is Alicia Rojas, and I'm from Chile. I work as a software developer at Telus Labs.

00:00:45.120 Before becoming a software developer, I worked as a natural resource engineer and studied environmental science.

00:00:51.180 This background has fueled my passion for technology and sustainability.

00:00:58.140 I am especially interested in how technology can improve sustainability efforts, particularly in sustainable agriculture, which is the focus of my case study.

00:01:09.720 As a quick overview, we will cover what a PWA is and why I think they are great.

00:01:15.600 Then, I'll explain the case study, introduce you to the challenges we faced, and share the solution we developed.

00:01:21.659 After that, I will explain how to set up the main tools to turn your regular Rails application into a PWA.

00:01:27.360 We will then discuss caching and how adding an offline fallback works in a Rails PWA.

00:01:34.799 Finally, we will talk about some key takeaways and further opportunities related to this topic.

00:01:42.180 So, first of all, what is a PWA?

00:01:48.600 PWA stands for Progressive Web App. As the name suggests, these are regular web applications enhanced with service workers, manifests, and other web platform features.

00:01:56.699 These enhancements allow the application to behave similarly to native applications, with capabilities like improved page performance, push notifications, and offline support.

00:02:05.380 I like to think of PWAs as a mix of the best features of both native applications and web apps.

00:02:13.440 They can reach a wide audience like regular web apps, while also offering more complex capabilities like native apps.

00:02:19.860 I've seen charts that place PWAs and native applications at the same level, but I believe this is misleading.

00:02:26.700 Native apps have platform specificity, giving them more complex capabilities, yet PWAs can reach many of these functionalities very effectively.

00:02:38.400 Moreover, PWAs can be indexed by search engines, unlike native apps that require installation through app stores.

00:02:43.700 PWAs can be installed but also accessed directly through a standard web browser, making them very accessible.

00:02:52.580 Now, let’s talk about our case study. This application was requested by the Chilean government.

00:03:04.440 It was designed for farmers and farm technicians to perform diagnostics on their fields.

00:03:11.099 It was crucial for this application to function in areas with unreliable network conditions or without any connection at all, given our target audience of non-tech-savvy users.

00:03:19.680 Therefore, the application had to be very user-friendly and accessible on mobile devices.

00:03:28.740 The application's main feature was a comprehensive survey, allowing technicians to fill out extensive forms and provide results to farmers.

00:03:36.300 The farmers could see how sustainable their farms were based on the results.

00:03:45.120 As I mentioned earlier, this application had to function in areas where internet availability was limited.

00:03:49.620 Our solution was to build an application using the Rails framework, a platform we were most experienced with.

00:03:57.000 This allowed us to deliver value to the client quickly while progressively enhancing the application with offline support features.

00:04:05.520 We considered developing a native app, but that would require building a range of API endpoints and native code.

00:04:13.680 This approach would complicate the web experience further, requiring an additional web front-end.

00:04:21.600 Our goal was to create something quickly and efficiently that would also be easy to maintain.

00:04:29.880 Therefore, we chose to build a Rails application and enhance it with PWA features.

00:04:40.920 With our Rails application set up and the form working, the next step was to turn it into a PWA.

00:04:46.920 To do this, we needed two key ingredients.

00:04:53.040 The first is the service worker. Service workers are specialized JavaScript assets that act as proxies between the web browser and the web server.

00:05:02.820 Their purpose is to improve reliability by providing offline access, boosting page performance, and adding other capabilities.

00:05:09.960 You can think of service workers as a small application running parallel to your main web application.

00:05:15.000 Users do not interact with the service worker directly, but it performs critical tasks in the background.

00:05:21.999 It's important to mention that service workers enhance functionality; your application should work even if service workers are not supported.

00:05:30.000 The second essential ingredient is the app manifest. This is a JSON file that informs your browser how the PWA should behave on the user's operating system.

00:05:37.680 The app manifest is crucial for making your app installable and enabling it to appear and behave like a native application.

00:05:45.180 For example, if you visited railsconf.org and received a prompt to install the app or add the icon to your home screen, that's because this application has an app manifest.

00:05:53.700 It's important that this manifest file is accessible at the correct scope, as this allows the service worker to manage the routes below that path.

00:06:01.260 Now, let’s explore how this works within the Rails application framework.

00:06:09.110 There are several methods we can use, and we encountered many gems, although some did not meet our needs because they relied on the old Rails asset pipeline.

00:06:15.900 These gems often required managing assets through Webpacker, which made the processes cumbersome.

00:06:23.900 Ultimately, we settled on an approach that was not originally developed by me. Instead, it was revitalized by Mike Rogers, who presented on this topic at RailsConf a few years ago.

00:06:31.620 A huge thank you to Mike Rogers if you're watching this!

00:06:39.480 The first step in this approach is to create a service worker controller, which behaves like a regular controller inheriting from the application controller.

00:06:44.820 We defined two methods for our main assets. Adding a line at the top is especially useful if you're using Devise for authentication.

00:06:52.440 This ensures that the route is accessible even outside of authentication.

00:07:05.700 Next, we need to configure the routes so that our application aligns with the assets we provide later.

00:07:12.060 Here’s how the custom routes look.

00:07:19.620 Finally, we need to add the service worker and the manifest views.

00:07:25.620 Since we created the service worker controller and added the routes, we can serve these files as regular views from the service worker directory.

00:07:34.620 Now, let’s look at an example manifest.

00:07:40.920 It’s a standard JSON file. I added the ERB extension to utilize the image path helper for the icon displayed on the user’s home screen.

00:07:48.720 You can customize the splash screen background color and more, and specify how the display will appear in the browser.

00:07:55.799 In this example, I chose the 'standalone' display mode since I want this application to appear like a native app.

00:08:04.920 After all of this, we need to include the manifest in the application layout, requiring the file properly.

00:08:11.700 This is how a service worker example looks.

00:08:18.300 It's written in JavaScript and consists of callbacks triggered whenever the browser listens for specific events.

00:08:25.640 Install, activate and fetch events are the main service worker lifecycle events that occur.

00:08:31.920 I won't go into detail about the service worker lifecycle; you can research that on your own.

00:08:39.160 After writing the service worker, we need to connect it to our Rails application.

00:08:45.480 As I mentioned before, the service worker runs parallel to your main application.

00:08:52.200 In the companion.js file, we create a bridge between the two.

00:08:58.920 This file tells the application when and where to load the service worker.

00:09:06.460 In companion.js, I first check for service worker support.

00:09:12.840 You can also add conditional statements to manage situations where support is absent.

00:09:20.240 Then, I register it and include logging if you want to verify successful registration.

00:09:27.060 Handling updates is crucial due to the way the lifecycle operates by default.

00:09:35.400 We need to ensure we correctly register the service worker and follow all conditions.

00:09:41.760 We need to rail this file by pinning it in the import map and importing it into application.js.

00:09:49.680 This is the new way of handling assets without needing transpiling or bundling, introduced in Rails 7.

00:09:57.480 Once we've completed those steps, we can start our server, go to the browser, and open the console to see our logs.

00:10:05.720 With everything set, we can proceed to cache assets and add an offline fallback.

00:10:12.580 In a regular web application, we constantly fetch resources from the server through requests.

00:10:20.000 However, in a PWA, these requests are intercepted by the service worker, allowing us to handle them differently based on network availability.

00:10:27.120 For instance, we could program the service worker to respond to an image request with a cached response, boosting page performance.

00:10:35.160 This means we do not need to repeatedly fetch the image from the server if it's already cached.

00:10:42.480 To enable this functionality, we must ensure we cache the images in advance.

00:10:48.600 So, while we're building a PWA, it turns out we’re effectively writing a lot of JavaScript code.

00:10:54.960 Fortunately, we don’t need to code these callbacks from scratch, as modules exist to streamline the process.

00:11:04.040 Workbox is one such module I highly recommend, as it simplifies the writing of service workers.

00:11:10.960 It was developed by Google and offers easy-use solutions that reduce the effort required to write service workers.

00:11:17.240 You can configure common service worker routing and caching by following established patterns.

00:11:25.960 The easiest way to use Workbox is through a CDN. I added the CDN link to the top of my service worker.

00:11:33.720 Next, I utilized work strategies—common patterns dictating how a service worker generates a response.

00:11:39.240 For example, one strategy is 'Network First, Fallback to Cache.' Here’s how that works.

00:11:46.600 In this case, if a user requests data while offline, the service worker checks if there is a cached response.

00:11:54.600 If a cache match is found, it retrieves the response to serve to the user.

00:12:01.440 There are also similar strategies, such as cash-first, followed by fallback to network.

00:12:09.600 But what if we are offline and the service worker intercepts a request without a matching cache?

00:12:15.720 In a standard scenario, the user would see a default error image, but we do not want that.

00:12:21.120 We want our users to have a smooth offline experience.

00:12:27.480 This is where the offline fallback comes into play—providing a friendly notification when users are not connected.

00:12:34.560 We need to cache this offline notification in advance to guarantee its availability.

00:12:41.400 Workbox offers an offline fallback recipe, but I created my own version tailored to my needs.

00:12:47.520 The first step involves updating our service worker controller to include additional routes and create views for each.

00:12:55.680 This is similar to what we previously did when adding the service worker and the manifest.

00:13:02.700 Next, we set up a catch handler, which tells the service worker what to display whenever a response fails.

00:13:09.600 Think of this as how a 'begin-rescue' block functions in Ruby.

00:13:17.400 This custom implementation starts with warming up the runtime cache.

00:13:24.780 We cache the list of URLs we want to store in our runtime cache.

00:13:30.300 We define the strategy used for this catch handler—from our set routes, sometimes nothing will match.

00:13:36.780 Using modules like Workbox, we don't have to reinvent the wheel.

00:13:44.460 This gives us an exceptionally reliable network experience.

00:13:50.700 Next, we'll explore how to create records offline and synchronize them later.

00:13:58.920 To accomplish this, we utilize IndexedDB and Background Sync APIs available in most modern browsers.

00:14:06.540 Initially, we considered alternatives like cookies or session storage, but I favored IndexedDB for its greater complexity manageable.

00:14:12.420 IndexedDB allows storage of complex objects—from JSON objects and strings to blobs such as images and files.

00:14:20.040 It also offers persistence across sessions and tabs, allowing users to access data seamlessly across multiple instances.

00:14:29.520 It’s a JavaScript API that relies heavily on promises. It's advisable to have at least basic understanding of promises.

00:14:38.720 Even if you are familiar with the syntax, I recommend using a wrapper for a more programmer-friendly experience.

00:14:45.540 I used Dexie.js, which is one of many available wrappers.

00:14:54.060 The second API we used is Background Sync, which allows web apps to defer synchronization to a later time when the device is online.

00:15:02.200 In this scenario, when a user creates a record and attempts to make a POST request without internet access, it fails.

00:15:11.700 At this point, we can store that record in IndexedDB.

00:15:17.320 Once the service worker detects a network connection, it will trigger the Background Sync API to handle the synchronization.

00:15:25.320 Now, let’s look closer at this process.

00:15:30.840 In broader terms, we will store records in IndexedDB using Stimulus.

00:15:39.240 The first step is to verify the network status.

00:15:48.720 We found that polling—fetching a one-pixel image at regular intervals—was the most reliable way to check connectivity.

00:15:54.780 When a successful response occurs, we store a boolean in local storage indicating the network's availability.

00:16:02.280 The next step is to declare or find our IndexedDB database.

00:16:08.280 These actions—checking network status and managing IndexedDB—were common across several Stimulus controllers, so we placed them in mixins.

00:16:15.240 When submitting a form, the associated Stimulus controller detects network availability before proceeding.

00:16:22.560 If the network is unavailable, the object is stored in IndexedDB instead.

00:16:30.000 Let’s now review how our Background Sync setup looks within the service worker.

00:16:38.400 It’s important to check for background sync support since not all browsers, like certain versions of Safari, have this feature available.

00:16:46.080 Once support is confirmed, we can proceed to implement background synchronization.

00:16:53.400 This code, included in our service worker, first adds an event listener for a sync event, defining a callback for when this sync occurs.

00:17:01.200 The callback begins by finding or creating the IndexedDB database using the chosen syntax.

00:17:08.880 It checks for any stored services in IndexedDB, organizes them into an array, and builds a service IDs remove array.

00:17:15.240 Then, our worker iterates over the array and makes requests for each record.

00:17:22.680 If a request is successful, the record ID is pushed to the removal array, and unsuccessful requests are handled accordingly.

00:17:31.260 After processing all records, the worker deletes them from IndexedDB.

00:17:38.520 That covers our background sync operations.

00:17:46.800 Now, let's discuss manual synchronization.

00:17:54.240 Users may want to synchronize forms manually—this can occur for various reasons.

00:18:03.600 Failures in the background sync process, lack of support, or wanting more control over the synchronization process.

00:18:10.320 To achieve this, we can again employ Stimulus.

00:18:17.760 Adding a button connected to a Stimulus controller allows users to manually trigger synchronization.

00:18:23.880 This approach utilizes the same methods defined in our service worker.

00:18:31.680 We also depend on mixins to verify network status and manage the database, and ultimately call AJAX requests.

00:18:39.600 The next step involves deleting records from IndexedDB after they've been successfully uploaded.

00:18:45.060 With this, we’ve addressed the create action for CRUD operations.

00:18:52.320 Now, we'll explore how to read and update records in IndexedDB.

00:19:00.600 To read data from IndexedDB, we can implement more Stimulus controllers.

00:19:07.800 This controller would allow users to view and render saved records using injected data.

00:19:16.320 HTML templates are beneficial here, as we can utilize the 'template' tag.

00:19:23.640 This tag stores content that will remain hidden until we want to display it through JavaScript.

00:19:30.840 Highlighting mustache variables helps with populating the template with IndexedDB data.

00:19:38.000 We combine Stimulus with mustache.js to render the data dynamically in the view.

00:19:45.600 Here’s an example function that grabs the template from the view and uses mustache.render.

00:19:52.240 We pass an object containing the values we want to populate and extract them from the IndexedDB record.

00:19:58.800 The user interface can showcase which records are saved in the database and which are not.

00:20:06.600 We've introduced features allowing users more control over the app behavior.

00:20:13.200 Consequently, enabling network checks lets us guide users on using the application.

00:20:20.040 If they are online yet prefer offline behavior, they have options in the interface.

00:20:27.480 The update process mirrors the create process—using a template to populate values.

00:20:34.680 We're utilizing mustache in conjunction with Stimulus to achieve this dynamically.

00:20:41.040 Furthermore, we can reuse our controllers to save those changes back to IndexedDB.

00:20:48.600 While implementing this solution, it's crucial to ensure validation is consistent across front-end and back-end.

00:20:56.280 This ensures that background synchronization and manual sync do not fail due to mismatched validation.

00:21:02.520 Another important aspect is understanding your audience and assessing the need for browser compatibility.

00:21:09.720 There are nuances to consider, particularly with background synchronization.

00:21:15.840 Some key takeaways from this journey include that PWA's features can supercharge your application.

00:21:35.400 You can create apps suitable for all users, expand reach to unconventional audiences.

00:21:43.920 For instance, we successfully reached farmers and farm technicians in isolated areas using Rails.

00:21:54.000 Thirdly, utilizing Stimulus proves to be a powerful tool for enhancing offline capabilities.

00:22:01.020 It can emulate single-page application behaviors without relying on complex front-end frameworks.

00:22:09.480 You can harness server markup and server-side rendering without complications.

00:22:16.680 This permits efficient usage of views and controllers across different parts of the application.

00:22:25.680 I see further opportunities to build upon these concepts.

00:22:32.760 For instance, it would be valuable if we could run a command like 'rails generate pwa:install' to automatically install all necessary assets.

00:22:40.440 It would also be beneficial to generate offline versions of our views with a single command, streamlining the developers’ tasks.

00:22:49.080 This approach would introduce maintenance considerations, as we'd then have two views for different states, but this logic could be consolidated.

00:22:55.920 I welcome any ideas on how to realize these plans!

00:23:03.360 These are some of the resources I utilized. I’ve written a couple of blog posts covering the content shared today.

00:23:12.120 The first two parts cover most of what I've discussed with you today; I’m still working on the third.

00:23:19.680 I’ve also linked Workbox, Dexie, and Mustache, which are the JavaScript libraries I used.

00:23:26.880 That’s it! Thank you very much!

RailsConf 2023