00:00:12.010
The next speaker unfortunately couldn't be here with us, so we arranged her talk over video. It's going to last roughly twenty minutes, and there will be time for Q&A. Bust is going to be here on stage, translating your questions to Brittany over the interwebs.
00:00:22.249
Brittany works as the lead web developer for the Pittsburgh Cultural Trust, a nonprofit organization. She is the host of the 5x5 Ruby on Rails podcast, and her alter ego, Norma Skates, plays and referees roller derby. Although she couldn't be with us in person, we are glad she was able to join us virtually. Let’s give her a round of applause as we enjoy her talk.
00:00:58.940
Today, Brittany's talk is titled 'Rails Against the Machine.' She will tell us how we can use Rails and AWS to manage bad actors and protect our online businesses. Enjoy Ruby on Ice! My name is Brittany Martin, and thank you for tuning in as I kick off my talk, 'Rails Against the Machine.'
00:01:23.240
First, I want to thank the organizers for making it possible for me to participate in this conference, even though I had a scheduling conflict. This conference features wonderful speakers, snowy activities, and I imagine a great group of attendees. I am super jealous and hope you’re having an absolutely wonderful time! Fortunately, the organizers have worked with me to host a live Q&A after this talk.
00:01:55.250
Throughout the talk, you'll see my Twitter handle at the bottom footer, @brittjmartin. If you have any questions or comments about the talk, please tweet at me, as I am eagerly reading those tweets back in the States. So, who am I? I'm the lead web developer for the Pittsburgh Cultural Trust, an organization that supports the arts in Pittsburgh. We aim to enhance the cultural and economic development of the Cultural District, a 14-square-block area of downtown Pittsburgh. I work on a Ruby on Rails e-commerce site, which serves as the ticketing platform for ten different arts organizations, including our very own ballet, opera, and symphony.
00:02:40.750
The ideas around this talk are a culmination of all the experiences I've had, which span various roles including working with a hosting provider and supporting a call center. As I mentioned earlier, I also play roller derby at night. I am Norma Skates, a play on Psycho's famous mother, Norma Bates. I play for the Little Steel Derby Girls in Youngstown, Ohio, and the Pittsburgh East Roller Villains in Greensburg, Pennsylvania. I started playing about two years ago, and I didn't know that derby would become one of the most significant and enjoyable things I would do in my life.
00:03:18.200
Between programming and derby, these activities have become pillars of my self-confidence. If you have ever had an interest in roller derby, I highly recommend reaching out to your local league to see how you can get involved. It's a wonderful sport that teaches you about athleticism and teamwork. Lastly, I’m the host of the 5x5 Ruby on Rails podcast, which you can subscribe to through various platforms, including iTunes. I have enjoyed chatting one-on-one with many of my Ruby heroes, including several of the speakers at this very conference. Please talk to me about it, and remember to tweet me if you have feedback or guest suggestions.
00:04:01.400
But enough about me; let’s move on to why I’m appearing before you today. In an ideal world, our customers would visit our websites and apps and happily purchase the products we create for them. However, in reality, that is not always the case. There are villains who visit your website. Who are these villains? They are the people accessing your website with the intent to game the system, to cheat, and to manipulate your business for their benefit.
00:04:54.810
Throughout this talk, I will refer to these users as 'bad users,' because they are indeed using your site, even if you want to ignore their existence. These users cheat, harass, and seek to manipulate you and your business, aiming to claim what they want. For the purposes of this talk, I will focus on bad users who disguise themselves as regular customers, a crucial distinction. I am not addressing BOTS or hackers who attack via security vulnerabilities on your website; rather, I'm focusing on the behavioral bad users. I do want to emphasize that those who engage in theft are generally not doing so to intentionally cause harm; their actions often stem from underlying needs.
00:05:37.920
As an example, imagine if I threw an American football reference into this talk! We're not focusing on a streaker wandering onto a field with the intent to disrupt. Instead, we are concentrating on learning the tactics to deal with players who've been rostered onto the team but commit penalties to gain an advantage. The first step in protecting yourself is identifying which users on your site may be attempting to break the rules. Luckily, we have a variety of avenues to achieve this.
00:06:46.460
Ask questions, review your user personas with your design team, and consult with your customer support team to identify negative patterns. Your customer support and sales teams generally have the most considerable one-on-one interaction with your customers. Lastly, review your application logs to find patterns of malicious users—not necessarily users trying to hack your system, but users trying to exploit coupons or seek excessively favorable deals.
00:07:04.350
So, what do these bad users look like? I'd like to present some examples of bad users I've encountered in the wild. One main type of bad user I deal with daily is ticket resellers. They buy tickets in large quantities and resell them for profit. Many ticket resellers use URLs similar to legitimate box office websites and often spend significant amounts on Google AdWords to promote their sites above legitimate ones. They imply that they are the official ticket source without clearly stating the real prices of the tickets they're selling compared to what they paid for them.
00:08:06.650
Another example is serial returners, who might switch the UPC code on a $600 product for a lower-cost code that rings up at $50. They buy the expensive item, replace the fake UPC tag with the original higher-priced code, and return the item without a receipt to receive store credit for $600, which they can later sell online. Additionally, I want to address upvote rings, a practice where users form rings to upvote items on community platforms like Hacker News, avoiding the need to pay for advertising to promote their content.
00:08:56.240
What are the consequences of these bad users? When they come between you and your ideal customers, it can lead to lost profits, damaged relationships, and inflated prices you need to charge. It is up to your company to decide on a strategy: you can either openly kick the bad customers off your site or quietly degrade their experience. In my experience, I find that quietly degrading the experience is more effective. Word spreads quickly among bad users; if you establish strict rules, they will work harder to circumvent them.
00:09:53.480
In this talk, I will show you the tools at your disposal to execute the second strategy. Regarding identifying and executing a strategy against bad users, I recommend automating the process as much as possible to reduce resource demands and maintain fairness in identifying bad users. As developers, we are fortunate to have a wealth of tools ready for use, particularly in the Ruby on Rails community, which is why we are all here today.
00:10:40.960
Once you have identified bad users, create a model that categorizes these users throughout their journey. For instance, our model is called 'reseller.' We’ve determined that users purchasing tickets within our tri-state area are considered safe. When assessing whether a user is permitted to print their tickets at home, we perform a check on their billing address against a list of safe states.
00:11:09.780
We share a CRM between our box office, call center, and website, which is critical for ensuring we have accurate customer records. This allows us to track when user interactions suggest a potential reseller. When we identify a user as a reseller, we mark them in our CRM, which we use as the definitive source for making future decisions about that user.
00:12:01.250
As users navigate the purchasing process, we check their session to determine if they are a reseller. We conduct these checks as they perform typical actions on our site, such as logging in and creating accounts. These checks incur costs since they require calling our CRM API. Therefore, we utilize background processing to manage this workflow, specifically using Sidekiq, a popular library in the Ruby community.
00:12:48.350
Whenever we detect a new ticket reseller logging onto the site, we push a new worker task into the queue. As we chose the quieter degradation strategy, we allow them to continue making purchases for now while we track their activity. When we find a reseller on our site, we don't block them immediately. Instead, we trigger another worker that sends a request to a Slack channel, allowing us to monitor how many resellers are being flagged.
00:13:35.860
In our Slack notifications, we create a JSON string that includes the user’s information, indicated by a skull emoji to denote a reseller. This alert system allows us to monitor reseller activity while keeping the identification details minimal. We can look up their ID in our CRM if further investigation is necessary without sharing personal details in our Slack channel.
00:14:54.280
As we implement these systems, we rely on Rails' cache fetch method to efficiently manage low-level caching. Each user is assigned a unique session when they log in, and we only need to check if the user is a reseller every 30 minutes. To optimize performance, we cache the results of these checks, improving our application's efficiency.
00:15:56.080
We should assess how often we need to gather user information to streamline our processes. The more data we have, the better. Knowing which user is currently accessing your site is essential. I recommend encouraging all users to log in as early as possible. For our site, we offer exclusive discounts to subscribers who log in, which incentivizes early registration.
00:17:04.090
As for anonymous users, they are still considered innocent until proven guilty. They cannot buy from us without logging in. With the future of third-party cookies becoming uncertain due to rising privacy concerns, we use cookies as a secondary set of data. When a known ticket reseller logs into our site and creates a new account, we insert a small cookie into their session. This information allows us to tag that account as belonging to a ticket reseller in our CRM.
00:18:15.520
Once we've identified a bad user on the site, it's time to execute your plan. Collaborate with your team to determine which features of your website users love and which ones might be used to exploit the system. For us, we found that three out of four attendees want to choose their own seats for events, a feature we allow for regular users but remove for ticket resellers.
00:19:23.540
Additionally, we offer print-at-home tickets, which are convenient for legitimate users but not available to ticket resellers. When they attempt to purchase tickets, they are unable to select the print-at-home option, effectively degrading their experience. We have witnessed this system work in practice—during a pre-sale event, the non-legitimate users did not secure any tickets, while genuine customers had access to the seats they desired.
00:20:03.620
As you develop features for your site, consider what the experience is like for both your ideal patrons and any ticket resellers. For instance, I can simulate different user experiences while developing features to understand how they interact with the site. This insight helps in building a more user-centric application.
00:20:57.769
Your hosting provider can also assist you in protecting your application. As a certified AWS developer, I've utilized their tools to enhance our automation process. Of all hosting providers, Amazon boasts the largest market share and offers a comprehensive suite of features for applications like these. One such tool is the Web Application Firewall (WAF), which protects your applications from common exploits that could compromise security or hinder performance.
00:21:22.319
With WAF, you can tailor your traffic rules to allow or block specific requests. In our recent analysis, we identified referral sites where ticket resellers congregate, and we now use WAF to automatically block them from accessing our site, effectively making it seem that tickets are sold out. Additionally, Amazon GuardDuty continuously monitors for malicious activity, such as unusual API calls, and provides another level of insight into potential threats.
00:22:30.230
To wrap up this talk, I want you to remember all the tools you have available. We are fortunate to have a wealth of resources at our disposal within the Ruby and Rails community. Fight tirelessly for your users, as they will certainly appreciate your efforts. Thank you so much for listening to my talk today. You can follow me on Twitter @brittjmartin or on GitHub @Wonderwoman13. I couldn't have done this without the wonderful organizers of Ruby on Ice, for which I am forever grateful.
00:23:26.210
Now, after today's talk, we have arranged a special live Q&A. I will be joining you shortly. Thank you again for joining me.