Building DEF CON CTF with Ruby

Big Ruby 2014

by Vito Genovese

In this video titled 'Building DEF CON CTF with Ruby,' speaker Vito Genovese discusses the intricacies of building and running the Capture the Flag (CTF) competition at DEF CON, one of the largest hacker conferences in the world. Genovese outlines various aspects including team building, the qualification rounds, and the finals of the event.

The key points discussed include:

Background on DEF CON and CTF: DEF CON is a prominent hacker conference known for its relaxed atmosphere and massive attendance, often likened to a 'frat party' for tech enthusiasts. CTF competitions consist of two formats: Jeopardy-style and attack-defense, where teams solve problems and hack into each other's servers.
Team Formation: Genovese shares the process of forming a team to host the CTF, emphasizing collaboration among a diverse group of experts, including software vulnerability specialists and network engineers. He highlights the importance of functional contributions over rigid adherence to roles.
Qualification Rounds: The qualifying rounds involved a 48-hour Jeopardy-style game with various problems across categories such as web exploits and reverse engineering. Genovese mentions that the top team was 'Plaid Parliament of Peoning' who maintained a lead throughout.
Technical Setup: The scoreboard for the competition was built using Ruby on Rails and hosted on Heroku, with discussions on challenges faced regarding server performance and costs. The team utilized different programming languages and frameworks to address the competition challenges.
Event Technicalities: During the finals, Genovese narrates the intense atmosphere and the technical hurdles faced. Teams connected to their servers via SSH while managing both offensive and defensive strategies within a time-constrained environment.
Lessons Learned: Genovese emphasizes the importance of communication, adaptability, and documentation throughout the competition process. He reflects on the need for continuous improvement and the learning that stemmed from operational challenges and technical setup.

In conclusion, the video not only illustrates the technical and team dynamics involved in hosting a large-scale CTF competition but also illustrates the camaraderie and learning experienced throughout the event. Genovese stresses the future opportunities for growth stemming from past experiences within the DEF CON environment.

00:00:10.820 Thank you.

00:00:19.340 All right, thanks everybody. Good afternoon. I'm Vito Genovese.

00:00:22.140 When I’m talking about DEF CON CTF, a lot of you will know me from other scenarios with different names. My Twitter handle is VitoLBS and my email is [email protected]. Today, I'm going to discuss what DEF CON and Capture the Flag (CTF) are, how we built our team, how we ran the qualification round, and then how we ran the finals.

00:00:39.540 DEF CON is one of the oldest continuously running hacker conferences and also one of the largest. Last year, I think it had about 20,000 attendees. It’s relatively inexpensive compared to other conferences like Black Hat, which is also hosted by the same founder. For a university event, your boss might pay for you to go to that one, but DEF CON feels like a frat party where you stay over the weekend on your own dime.

00:01:07.980 This is the entrance hallway where everybody funnels through to buy badges. People line up a day ahead of time because they sell 20,000 badges exclusively for cash. This is a tamper-resistant contest where someone might be delaminating a physical Bitcoin to steal the public or private key from the middle and then laminate it back together. Long story short, don't buy physical Bitcoins for cash.

00:01:16.200 Capture the Flag is the standard computer security game. If you’ve seen the documentary 'The Social Network,' you’ll remember four guys trying to break into a server to win an internship at early Facebook. There’s the Jeopardy-style Capture the Flag where there’s a grid or a list of problems. You pick one, get a clue, solve it, and turn in the answer for points. On the other side, there’s an attack-defense Capture the Flag where each team has servers. You hack into their services, steal secrets, and submit them for points while simultaneously having to patch your own services.

00:01:38.640 So how do you compete in Capture the Flag? You build a team and become exceptionally good at assembly. When presented with a .c file, you compile it and run it through IDA Pro to generate an ugly flowchart. You also get really proficient at web attacks like SQL injection, cross-site scripting, session fixation, and cryptography. During the game, you solve problems. The first time you probably won't do very well, but you take all the problems you didn’t solve, write up solutions, and next time you will improve.

00:02:53.400 This is a website called ctftime.org, which has a list of all the Capture the Flag competitions and serves as a fantastic resource. There are several dozen competitions scheduled between now and when we host qualifications again in May. Here are two sample challenges from last year's game, titled 'Hype Man' and 'Worst Medicine.' These are web challenges, and I believe both are in Sinatra. The best teams solved these in under 30 minutes each.

00:03:14.640 You've been working all weekend on these CTF challenges. Most games or events last 48 hours, and you end up with interesting scenarios. For example, this is a Dunkin' Donuts box of coffee, which is actually just a foil bag of coffee inside a box that has been at room temperature for about 10 hours.

00:03:34.200 Now, how do you build a DEF CON CTF team? At the end of 2012, my friend Gynophage informed me that Dark Tangent, the founder of DEF CON, was looking for a new team to run Capture the Flag. He asked us to help write a proposal. We had about eight people involved in this initiative, including a few software vulnerability experts, several talented hardware specialists, and a fantastic network engineer who types routing rules into routers faster than I can type here.

00:04:01.680 As a web app generalist, I’ve worked with Rails for quite a long time and have experience with NoSQL databases. I also love SQL and can create websites hosted on Heroku—this is my first shoutout to Heroku, for which I haven’t been compensated! We gathered a team of skilled yet opinionated individuals who needed to collaborate on a software project or hardware project with dozens of moving parts on stage in a public setting. The lack of control over deadlines made it a recipe for disaster.

00:04:40.680 What we found worked best was that everybody just did what they could. If you needed help, you asked for it. If you were unsure about something, you either kept quiet or demonstrated and built something better. It was far better to have something functional in the end than to be the person who says, 'I told you so.' For qualifications, we submitted our proposal at the end of February, and by the end of March, we learned we would win. This gave us a timeline from the end of March until the beginning of August to prepare for the qualifications.

00:05:30.042 The qualifications consisted of a Jeopardy-style game with self-contained questions that anyone with a computer could enter. We ran it for 48 hours over a weekend, and we built around 25 problems, though we actually created 24 across a variety of categories—web, shellcode, binary challenges, ACM style programming challenges, and reverse engineering. Some of these questions already had solutions, which the small hearts indicate, while those marked with flames indicate problems yet to be unlocked.

00:05:51.639 The Plaid Parliament of Peoning was at the top of the scoreboard throughout both the qualifications and finals, and they even appeared on CNBC. I built the scoreboard in Rails using unicorn and hosted on Heroku, ensuring that it could handle the demand. Most PostgreSQL requests came in under five milliseconds, and with our six one-gigabyte dynos, we maintained a 50-millisecond request time.

00:06:58.680 About a day later, I thought we could save some money by scaling down the number of dynos. However, reducing them caused some requests to pile up in a queue, leading to slower loading times. In the end, we couldn’t save enough money, as it turned out we spent more on whiskey than on computing resources! Besides the scoreboard, we had 24 challenges that functioned correctly, with a variety of languages and frameworks, including Sinatra, x86, GoLang, ARM, PHP, and Python. We leveraged Heroku for most of our Sinatra apps, and they were all successful. We used EC2 for the ARM challenges but quickly learned that micro instances were unreliable in production due to performance issues.

00:08:37.799 The ARM challenges relied on three Raspberry Pi units housed in a university data center. These model B units came with built-in USB hubs and Ethernet ports. Despite facing challenges, we pushed through. There were instances where I needed to sleep, requiring us to edit some questions and update files on S3. However, no one else on the team had control over the scoreboard, so they had to create a new S3 bucket, upload files, and communicate extensively on Twitter and IRC to inform participants about updates.

00:09:26.940 During the event, some of the challenges were not ready until halfway through, like 'Worst Medicine,' which I had the idea for in a dream. It was innovative but also had an unanticipated vulnerability, resulting in two viable solutions that made it easier than intended. Another problem we never quite managed to fix satisfactorily was replaced with the tradition 'Hack the Planet.' One of my favorite bugs happened when one of the team members accidentally submitted a challenge twice due to a missing PostgreSQL index, resulting in double points without needing to redeploy the app.

00:11:11.940 Having IRC and Twitter for communication was invaluable for coordinating with participants. However, we faced issues with a lack of reset password functionality, leading to me manually resetting participants’ passwords via these communication channels. When we launched the game, you had to be logged in to view the scoreboard. Many people wanted to link their scores to family and friends. Halfway through, we created another Heroku app linked to the same database that publicized the scores for everyone.

00:12:46.700 Qualifications were straightforward as we could use cloud services while being home. However, the finals had a different dynamic. My DEF CON history started in college when I took a computer security class, and the professor was enthusiastic about the event. Walking into DEF CON, the CTF room is centrally located. You walk through it on the way to talks, always showing distraction videos. There are numerous competitors dedicated to their laptops, and some teams brought large server arrays.

00:13:14.680 The atmosphere is intense, filled with emotion, and the most elite individuals create an exciting environment. The day before DEF CON, while I was driving a rented Prius back with banners I designed, I realized: it’s really happening. This is what the finals room looks like. The systems like PPP were pivotal in the competition, with broadcasting systems emitting constant feedback. Initially, the room was empty, but as it filled with over 30 computers, it became an electrifying sight.

00:14:29.519 Hosting an attack-defense-style game meant that teams connected to their servers via SSH, analyzed binaries, and patched vulnerabilities while attacking others. We started the game with 50,000 flags, with rounds lasting five minutes in which each team's server received fresh token deposits. If a team’s servers went down or if they stole tokens indiscriminately, they would lose points. In this model, various strategies emerged for obtaining flags without ever exposing oneself to risks.

00:16:44.040 The process of scoring its associated complexity. The first edition I attended had extensive server racks, and I had only brought a Mac Mini to run the game, which sufficed certain checks. Using client storage for authentication was beneficial since it alleviated password concerns. We would arrive at the conference at 8 A.M. in Vegas and burn 20 CDs. It was necessary, albeit tedious, as the traffic could be strictly monitored.

00:19:25.880 We utilized Kibana and Elasticsearch for log processing, making it easy to identify when teams tried to score old flags. Logging everything proved incredibly useful, granting insights into uptime states. We rapidly realized the potential criticisms we would face regarding our systems and processes throughout the contest.

00:20:55.560 Two weeks before the event’s commencement, one of our scripts required an overhaul regarding uptime checks. Midway during the game, we encountered teams that struggled with service outages. I implemented an alarm system call to mitigate the number of concurrent processes and ensure optimal performance. Ultimately, we focused our improvement efforts on the recognition of inherent pitfalls while maintaining communication across various channels as the competition unfolded.

00:23:02.340 There was an intriguing situation during the contest where teams could inadvertently score on themselves. The goal was to restrict other teams from obtaining their flags for points. We could have distributed points once flags had been scored, but a misconfiguration on my part led to a misunderstanding after the event ended, creating discrepancies in how many flags were potentially scored.

00:23:51.200 Despite that, we learned a substantial amount of crucial insights about API documentation and team coordination. The importance of addressing technical challenges directly enhanced our adaptability. Moving forward, I’m more enthused by the prospects of growth from past experiences.

00:24:04.440 Thank you.

00:24:06.620 Goodbye.

See Slides on speakerdeck.com

Big Ruby 2014