00:00:24.860
Good morning! Whoo! Yes, good morning. I'm very excited to be here.
00:00:31.680
A couple weeks ago, I went to Ruby on Ales and caught a cold. I think I got it from Mike, and I feel like this conference is 'Ruby for what ails you,' which is a cold.
00:00:41.370
My name is Aaron Patterson, and you can find me on the Internet as 'tender love.' I am on the Ruby core team and the Rails core team. This doesn't mean that I know what I’m talking about; it just means that I'm really bad at saying no.
00:01:07.860
I’m actually from Utah, I grew up here in Salt Lake City. People always ask me the same question when they find out that I'm from Utah, and I want to preemptively answer it: the answer is no, I don’t know how to ski.
00:01:19.050
I grew up in Rose Park but moved to Seattle about 14 years ago, so don’t ask me for any food recommendations. I want to see how many of you live here in Utah now and how many of you are from out of town.
00:01:47.280
Okay, so about half and half. Half of you will already know some of the things I’ll talk about regarding Utah.
00:02:04.049
This is the Utah State cooking pot: a Dutch oven. I don't know why we did this, but I remember there was a hubbub back in 1997 when it was passed as a law. People were like, 'Why are we passing a law about cooking pots?'
00:02:14.579
Anyway, I love my Dutch oven! You should check out Stonecutter; they have scones that are different from anywhere else. I’ve never been to Stonecutter, but you should also go to Crown Burger because they have really good pastrami sandwiches.
00:02:27.090
Now, did you know that the state snack is green jello? This is totally true! So if you go somewhere in Utah and order food, try to get green jello. For authenticity points, it will probably have carrots in it.
00:02:36.180
Also, you should try fry sauce. It’s a Utah thing. You dip fries in it, and I guarantee that anywhere you go to eat where they have fries, you can ask for fry sauce, and they will have it.
00:02:42.840
The last thing I want to mention for those of you who are new here is that Utah has some weird stuff. We have a pyramid; look it up online, it’s quite interesting. Also, there’s a Sphinx of Joseph Smith. You should check it out at Gilgal Park, which is close by and within walking distance.
00:03:05.220
Gilgal Park has a lot of stonework and was once closed to the public as it was privately owned. Back when I was in high school, I would sneak into this park to take pictures. I’ve been kicked out of this park many times, but eventually, the city bought it and turned it into a public park.
00:03:20.340
So yes, you should check this park out. Today, I brought two very special guests with me—my parents. Nobody embarrass me, they are down here!
00:03:48.390
I invited them because I want them to see what I do. I’m not sure they believe that I actually earn a paycheck. But, to clarify, I’m not getting money from loan sharks or anything like that!
00:04:01.320
I thought it would be nice for them to see what I actually do for a living. Unfortunately, this talk has absolutely nothing to do with my job, but hopefully, they’ll learn something.
00:04:24.000
They taught me a very important lesson: if you learn things, you can do things. If you look at my grades from school, you’ll see that I didn’t learn this lesson for quite a while.
00:04:40.320
Unfortunately, the usefulness of this talk is about zero. We're going to discuss some related stuff, but let's get into it. This talk is called "Oh Ho Ho! It's Magic!" I could talk a lot about Magic: The Gathering.
00:04:53.360
Yes, I play Magic: The Gathering. Anyone else here play? A few people. Recently, I started playing again. For those who don’t know, Magic: The Gathering is a collectible card game.
00:05:09.240
You collect cards and play against people. I played this game about ten years ago and bought a whole bunch of cards. I was having a good time until I went to a tournament and got destroyed by a 13-year-old.
00:05:27.240
He insulted my parents and I quit. It felt like playing Call of Duty, except in real life. Recently, a friend of mine asked if I wanted to play Magic again, and I thought, 'Sure, I have thousands of cards in my closet!'
00:05:49.500
So I pulled the cards out, but I realized I had no idea what I was doing. I literally have thousands of cards—like five or six thousand—and didn’t know if any were good or how much they were worth.
00:06:12.540
It seemed like a job for computers. This task is repetitive and something computers should be able to handle. So, I put on my robe and wizard hat and put together a system to identify these Magic cards.
00:06:31.650
Today, I will talk about how I identify these cards using a computer. The primary idea is to lessen the work for myself, although you’ll see it actually entails more work. But it’s all about the journey!
00:07:25.380
So, let’s discuss my hardware. I have a laptop and a webcam, along with a light box that’s a bit washed out. The webcam points inside this box, and here's a demo of the final system.
00:07:51.500
The upper right is a live stream from my webcam, and in the upper left, you can see the program identifying the cards. It shows me what it thinks is the right card, and I verify if it’s correct. If so, it saves that card.
00:08:20.370
The high-level process includes taking a photo of the card, extracting the card from that photo, identifying it, and then saving the data. We repeat this loop continuously.
00:08:53.200
To identify cards, I start with a corpus of known images and their corresponding data. The program makes a guess based on this corpus. If it guesses correctly, I confirm, and if it’s wrong, I teach it the correct answer. Every time I teach it, it saves the data to improve future identification.
00:09:52.059
For the actual code involved, I utilize library functions for card matching, OpenCV for recognition and cropping, and SQLite for data storage. I’m going to go over how I gather the information.
00:10:13.360
I got it all from the Wizards of the Coast website. I need to gather information such as the card name, image, set, and rating, because a particular card can be reissued multiple times, affecting its price.
00:11:01.720
Thinking about this, I modeled the data. An image has one card, and a card has one image; this seemed straightforward. But I soon found that one image could correspond to two cards. Everyone who's written a web application knows how frustrating it is when your data changes!
00:11:42.430
I fixed that issue by adjusting my model, ending up with the realization that an image can have many cards. Next, I want to explain how I downloaded this data using promises in Ruby.
00:12:09.890
A promise in Ruby allows you to execute a chunk of code later and access the resulting value at that time. The API enables me to ask for the status of a promise and cancel it if needed.
00:12:55.600
That’s the entire implementation of the promise system in Ruby. The downside is you can create an unlimited number of promises, which can be inefficient, especially if your machine has limited CPUs.
00:13:57.560
Thus, I implemented an executor pool—basically a thread pool that lets me run a limited number of promises in parallel. This allows me to queue up a bunch of work, limiting the number of simultaneous connections.
00:14:35.800
I downloaded 1.6 gigabytes of data in about 40 minutes, which was awesome! However, I later discovered a website called MTG Jace Hong that has all the data I downloaded in JSON format. The moral of the story: Google first, as it can save a lot of time.
00:15:42.210
Next, I use perceptual hashing to identify images. For example, I have two images: one is my reference and the other is my scanned image. Each image has a key that we can compare using Hamming distance, which tells us how similar the images are. A lower Hamming distance means the images are more similar.
00:16:47.550
I needed to sort this data because Wizards has produced about 25,000 unique cards. To do this efficiently, I extended SQLite to add a Hamming distance function, allowing me to sort by distance in the database.
00:17:08.480
Then, I had to calculate the hashes for all the images stored in the database. This process took about an hour, and I had to get smarter about utilizing CPU resources effectively.
00:17:28.890
Now, let’s move on to recognition and cropping. What I am trying to solve here is how to crop images of Magic cards correctly so they are nicely formatted. I capture a photo with my webcam, and it needs to be processed accordingly.
00:17:54.640
The first step in pre-processing is to change the photo to grayscale, which allows us to detect edges better using the Canny edge detector. Once the edges are detected, I can find contours from the image.
00:18:21.720
After that, we loop through the contours to find the largest one. The largest contour should correspond to the card itself. Once we identify this contour, we convert it into a polygon for cropping.
00:18:41.900
I utilize OpenCV to obtain a polygon from the contour, which can then be warped into a rectangular shape. This enables accurate identification of the card.
00:19:03.120
After adjusting the image, we can use the Hamming distance to identify the card in the database, leading us to save this information. Once saved, the system becomes smarter and can easily identify cards based on previous inputs.
00:19:27.760
Now that I have implemented this system, I can finally answer questions like what cards I have and their respective ratings. Even though I haven’t integrated pricing systems yet, I can see how well each card performs.
00:19:49.660
Some challenges remain, such as the webcam's focus time. It takes forever to focus, which is not ideal when trying to identify thousands of cards. The other issue is total failure when the system cannot identify a card.
00:20:10.180
When the system fails, I need to teach it manually which card it is, but the good news is that once I've taught it, it remembers the information for the future. There are also problems with similar artwork where different cards look nearly identical.
00:20:54.010
In summary, this has been a glimpse into the system I built for identifying and managing my Magic card collection. I hope you’ve gained insights into using these technological methods in unique and interesting projects. Thank you very much!
00:28:31.990
Cheers!
00:28:45.180
You.
