00:00:12.480
Thanks for coming out late Thursday, everyone. It's been a long couple of days. I'm Nick Merwin, a founder of Coveralls. I hope you guys have seen some of our badges around town—GitHub, ReadMe, etc. We're kind of all over the place, hopefully right now.
00:00:28.400
Today, we're going to discuss how I approached getting Coveralls, our Software as a Service (SaaS) app, into the hands of customers who wanted to run it within their corporate networks. This was pretty uncharted territory for me, but I hope this general overview, along with a basic prototype that we'll see later, will help you gain some confidence to also take your app to the next level by offering a private version to a customer base that may have seemed impossible before. This discussion will partly revolve around business models and general sysadmin topics, all viewed through the lens of Rails, since Coveralls is a Rails app.
00:01:10.600
So, why would you want to set your app free into the outside world, beyond your secure deployment environment? Let's say your app is currently hosted on Heroku or a VPS, like Coveralls is on Digital Ocean. If you have a subscription-based service where users are paying monthly, and maybe there are a few tiers and usage-based add-ons, and it’s humming along, accruing users, it might seem like your potential customer base is completely covered.
00:01:40.159
However, you might be neglecting an even more lucrative and preferable customer base—enterprise clients. These could be large companies where development teams face difficulties in getting upper management to embrace cloud-based tools, or perhaps the issue is that setting up a $5 monthly service fee necessitates an extensive amount of vendor approval and paperwork.
00:02:05.119
So, let's chat about how offering a hosted version of your app, in addition to your cloud service, can make sense. I came across three key reasons for wanting to pursue this: first is security. It's clear that we all worry about using cloud apps since we essentially give up control over our data. We hope that the developers on the other side use bcrypt for our passwords, but we never really know for sure and just hope there will never be a real data breach.
00:02:36.120
Beyond passwords, any sort of data that a company’s security department has forbidden from leaving their networks effectively precludes the possibility of them ever using your app. For instance, a medical company might want to use your data mining tool but cannot upload anything to the cloud due to HIPAA compliance issues. Or perhaps it's a development team that wants to use your source code analysis tools but cannot risk sharing or leaking proprietary code.
00:03:00.360
Additionally, there are general business apps like CRMs, internal task management, chatting, and scheduling that contain plenty of information those companies wouldn't want to share with competitors. By running a hosted version of your app within their networks, they can be more confident that they won’t ever have their accounts hacked or their data compromised on the open internet.
00:03:42.799
The second reason is reliability. This is a worst-case scenario consideration. If the app you're offering is a DevOps tool that's part of the build or deployment pipeline, teams won’t want any downtime or unplanned maintenance interruptions. For example, with services like GitHub or Travis CI, if something goes down, you do not have control over it, which could disrupt your deployment process. By managing it internally, your development teams can coordinate to ensure everything runs smoothly during crunch time.
00:04:30.360
Lastly, we have cost. The traditional subscription model of simply paying monthly doesn't work once you transition your app to a customer. Often, these potential customers are less price-sensitive and might be willing to pay a premium for your hosted service, along with some extra white-glove support. However, once you deliver the app to them, you must assume it won't be able to 'phone home' to check the subscription status.
00:05:12.520
Instead of a time-based subscription like a monthly fee, you’ll likely need to implement some sort of seat-based usage or another model beyond just monthly or yearly billing. When considering seating, you can create seat packs as tiers based on the number of active users accessing the app within their network. Additionally, you might consider building a self-destruct or shutdown mechanism that forces the customer to purchase or renew the license after a set period, such as six months or a year, to ensure they keep a valid license.
00:05:43.519
With that said, there is considerable opportunity to address business model challenges beyond the traditional SaaS model. Before diving into how we figured this out, let me briefly discuss what Coveralls does and why we aimed for the enterprise market. Coveralls is a tool for tracking code coverage and providing notifications. It annotates pull requests with success or failure statuses based on changes in coverage percentage and sends messages to chat or email with updates.
00:06:19.060
This functionality helps development teams ensure they do not deliver untested code, especially in production. In the on-site version, you can see line-by-line coverage of the codebase. The cloud version for open-source projects is free, but we charge for private repositories.
00:06:49.600
The challenge arises because while we don’t store source code on our servers, we do store private scoped OAuth tokens for GitHub, and for some companies, that's a complete dealbreaker. They cannot allow us to have access to their private repositories. Additionally, some companies are utilizing GitHub Enterprise and are not using cloud GitHub to begin with, making interoperability a challenge.
00:07:19.760
To explore this market, we gathered interest from our potential customer base over several months. It became apparent that there was sufficient interest to pursue this initiative, which ultimately brought us to today.
00:07:48.200
Now, let's discuss the significant hurdles I encountered in converting a general cloud Rails app to one that operates within corporate firewalls, often referred to as 'the fog'. The first hurdle we need to tackle is delivery and installation.
00:08:04.680
The best user experience for your customers would be to minimize the amount of work required to get your app running in their environment. I believe that the most consolidated method to achieve this involves three downloads: the virtual machine that the app will run on, the packaged application files themselves, and a license file specific to each customer.
00:09:18.680
The virtual machine could just be a Linux box, typically between 800 MB and 1 GB in size. The application package can vary considerably, sometimes between 50 MB to 100 MB or more, depending on how many gems need to be bundled and vended into the package.
00:09:46.399
Lastly, the license file would be generated uniquely for each customer. From those three files, customers will probably only need to download the virtual machine once in a while, such as when you implement significant upgrades or new dependencies. The application package is something they would download whenever updates occur, and license files would be relatively quick to download during trials or when converting to active status.
00:10:17.960
Furthermore, let’s talk about those incremental updates. Normally, when your code is hosted on Heroku, deploying features and fixes is simple. However, when your app is running in the wild with no access to it, the app packages come into play.
00:10:51.160
These packages contain only the most up-to-date bug fixes, features, etc. This smaller download is much quicker than having to redownload and reconfigure an entire virtual machine. Regarding the license files, it makes sense to build a secondary app in addition to your main cloud app to service those customers and manage subscription fees, trial accounts, and so forth.
00:11:24.360
Installation-wise, the first thing customers will interact with is the networking configuration. Since you wouldn't want them to log into the VM themselves, the optimal solution is to create a tiny, menu-driven Ruby app presented on boot.
00:11:42.360
This way, customers won’t need to know the specifics about Linux networking. Also, assume no external access will be possible. If they're purchasing the hosted version, it is for the confidence of security behind it; they wouldn’t want any external access to it, ensuring it remains entirely sandboxed within their network.
00:12:12.040
Next up is process management. You need a strategy to manage background jobs and ensure that the server remains alive and starts up on boot.
00:12:43.480
Another hurdle relates to product support. Typically, if a user encounters a 500 error, you immediately know about it via email or monitoring services. However, in this scenario, if something goes wrong, you have no visibility. You will only receive an email from the customer reporting an issue.
00:13:12.760
So, you must provide them a way to send you details about the exception, storing those exceptions in a way that can be sent over, either in an email attachment or another method to keep track of them.
00:13:45.760
Log management is another crucial aspect. If users experience issues, you want to be informed about any unusual parameters passing through the routes, especially if they aren’t resulting in exceptions.
00:14:14.080
Lastly, you must consider resource management. If it seems like performance is lacking, you will need a way to address that. This is more of a system admin topic, so I won’t delve any further here, but it's definitely something to keep in mind.
00:14:56.240
Now, let's address the intellectual property concerns. This is a big one because you're providing them a VM, which they can unpack, mount the disk, and review everything included. Although it comes packaged into one file, it is extractable, and users can investigate it as if they were a root user.
00:15:21.399
Understanding that database access is difficult to secure, it’s wise to assume that companies will have access to the database. Furthermore, there’s no thorough method to protect your source code once it is running in somebody else’s environment. You can write a code obfuscator as a deterrent—but ultimately, there’s no foolproof way.
00:15:45.720
Instead, consider covering yourself with your licensing agreement. State in the license that the software is for use only, and modifications and redistribution are not permitted. It all comes down to legal concerns being your last line of defense. If worries about this keep you awake at night, remember that your customers should primarily be interested in receiving updates, bug fixes, new features, and support rather than attempting to breach your source code.
00:16:39.600
Now, let’s delve into the nitty-gritty of the architecture we use. This diagram, while complex, shows the general structure of how the app runs within our VM, which is Ubuntu. We chose Ubuntu due to its widespread usage and ease of configuration. Key components within the VM include a network config app that customers see when they boot it up.
00:17:27.720
This app allows users to set static networking or DHCP, manage name servers, and perform actions like rebooting or shutting down the VM. Behind this web server, we'll explain our choice of Passenger, housing a pre-installed app designed to handle package file extraction containing the actual application and its updates.
00:18:13.080
Within this architecture, the Rails app will also be running. The license file reader will load and cache the license upon the Rails app booting and check it on every page load. Actions like enabling and disabling features or locking down the app can be achieved based on this check.
00:18:41.360
Data import and export is another significant feature. When customers need to retrieve data from the old virtual machine to the new, the app requires a way to facilitate this transfer. This includes dumping the database and potentially aggregating user-uploaded files. Lastly, we need a support package generation and download functionality.
00:19:24.000
We can use Rails' rescue feature to collect exceptions occurring in controllers, archiving them into a temporary directory. Admins can generate an archived file when they need support that can be encrypted and emailed or Dropboxed to us.
00:20:05.520
Setting up the environment during development is crucial. I recommend using VirtualBox, as it is free and straightforward. Exporting an appliance from your machine as an OVA file makes it easy to link to your CDN.
00:20:51.960
Given Ubuntu's reliable built-in security settings and standards, we can trust it grants our customers that confidence from the outset. When provisioning for your Rails app, it’s essential to use the minimum dependencies necessary to keep the download size small.
00:21:19.160
While many installations quickly balloon to 2 GB or more, you can keep it down as small as 1 GB by avoiding unnecessary installations. In the development phase, it also helps to have two separate versions of the virtual machine: one for packaging the app and the other for local development that can assist with platform-specific issues.
00:22:10.120
Next, let's discuss the networking configuration. A customer interacts with a simple menu-driven Ruby app upon their first load of the VM, assisting them in gathering crucial information from system calls. This module permits user interaction without requiring complete blocking.
00:22:50.520
I chose Passenger for managing the server because it allows your app just by placing it in a predefined directory and starting to serve it. The setup page assists users in uploading the packaged and license files, which will then be extracted for future integration.
00:23:30.440
This installation package handles placing both the apps in their designated folders and preparing them for future access by the user. The encryption library utilized to decode data is effectively an added layer of protection over the package.
00:24:33.440
The user experience is meant to be simple, assisting customers in the transition from the old version to the updated package without excessive complexity. If the package and license files are present, they will be decrypted, cached as the app operates, and applied instantly.
00:25:28.160
We also discussed how to handle the license file within our framework. The license file contains unique data for each customer and checks against every page load. Any new information related to Rails secrets should likewise be included within the license file for every boot, preventing cross-customer interference.
00:26:19.760
A straightforward licensing module demonstrates how to read and write the license file, using the same encryption key as earlier indicated. This module is loaded at the initialization stage, ensuring seamless access throughout the app, a crucial aspect of compliance and security management.
00:27:09.480
Support package management allows us to track user interactions of exceptions via an encrypted file for easier access when customer support is needed. It’s vital to securely track backtrace and error reports that facilitate prompt customer service responses.
00:27:51.920
Data import and export management comprises critical functions of PG restore and PG dump commands. These functions enable full database backups and integration with the client’s updated packages, representing important features of maintaining operational transparency and support.
00:28:26.440
When it comes to background job management, I won’t delve too deeply, but we use 'foreman' and 'export' to maintain running processes tied to Ubuntu, increasing the efficiency of server operations. The next logical step is creating a dashboard within the app to provide users with real-time updates improving their overall operational awareness.
00:29:31.600
When all these pieces are in place, finding the best way to make them distributable is key. Streamlining to a single script for deployment could simplify user experiences significantly. This strategy entails linking updates to a CDN, indicating a new version for customers to download—essentially making updates user-friendly.
00:30:25.760
The process of creating this streamlined version is straightforward: execute a rake task that examines the version, runs 'bundle vendor' to prepare vendor directory packages, and compresses relevant files while excluding unneeded components. The task then seamlessly encrypts this structured update with the shared key before uploading it to the CDN and notifying users.
00:31:31.920
By managing each download's version effectively, your users will have a seamless way of identifying their updates and making the transition a stress-free process.
00:32:01.680
Upon successfully landing in customers' hands, if they are already using services like AWS, they might want to see it as an Amazon image. This conversion can typically be done quite easily through available command-line tools, making the process accessible for all clients.
00:32:42.920
As your package evolves, consider including further enhancements like resource management for admins to gauge VM performance. Introducing clustering may improve performance for clients using multiple VMs simultaneously. The need for a specified mail server will also arise, as traditional cloud sources won't be applicable under the VM setup.
00:33:15.640
Discuss incremental VM updates that require bash scripts for package evolution within this framework. All changes should consider the lack of internet access in the VM.
00:34:00.440
While I mentioned the enterprise side as somewhat of a joke, it's essential to recognize the scope for uncharted territory in that market.
00:34:40.080
As we approach the conclusion of this session, let's use the remaining time for any questions you may have! The question was raised about our approach toward AISC, which remains a work in progress as we navigate using a Ruby C extension similar to GitHub’s compiled Ruby.
00:35:19.360
In replying to how we aim to minimize the updates required to ship, while it is complex, it boils down to customer relations management. Bug fixes will certainly need to be bundled and delivered swiftly where necessary.
00:35:59.960
If bugs are non-critical, they can be saved for a major point release. If it critically affects the business, a new version needs to get released immediately so that everyone impacted is updated accordingly.
00:36:41.720
When discussing scalability, our support team hasn’t yet needed to scale significantly due to a manageable customer load. We anticipate that a review system will also need adjustment when supporting various package versions and handling incoming inquiries, particularly those requiring detailed responses.
00:37:53.920
Ultimately, we’re here to support our clients effectively and efficiently. There are two prototype apps to demonstrate our features—the first, an elementary blog app, utilizes license reading—while the second is our app loader.
00:38:40.760
Thank you all for attending today, and I hope this session has provided you with valuable insights and ideas for deploying your own applications securely behind corporate firewalls.
