Networking
Keynote: Learning DNS
Summarized using AI
Keynote: Learning DNS
by Julia EvansIn her keynote "Learning DNS in 10 Years" at RubyConf 2022, Julia Evans reflects on her personal journey of understanding the Domain Name System (DNS) and offers insights on learning complex topics. Although the talk focuses on DNS, it emphasizes broader strategies for tackling challenging subjects in programming and tech.
Key Points Discussed:
Introduction to DNS:
- DNS, or Domain Name System, translates domain names into IP addresses essential for accessing websites.
- Evans highlights its significance as an invisible system crucial to the internet's functionality, established by stable technologies dating back to 1987.
Personal Learning Journey:
- Evans acknowledges that her journey of learning DNS took 16 years, which she considers normal in a fast-paced industry.
- She encourages prioritizing topics that spark interest, despite challenges in learning.
Learning Strategies:
- Using DNS as a case study, she shares her favorite methods for learning:
- Deep Dives: Engaging in short periods of intense study interspersed with breaks for reflection.
- Observation: Utilizing tools like 'dig' to execute DNS queries and analyze responses, making the learning process engaging.
- Recognition of Confusion: Understanding when confusion arises as a prompt to delve deeper into the topic.
- Experimentation: Building a safe environment to conduct DNS experiments, exemplified by her website 'Mess with DNS'.
- Implementation: Writing a simplified version of a DNS query in Ruby boosts her confidence in understanding DNS mechanics.
Case Study: Mysterious Caching:
- Evans recounts her experience with caching issues in DNS, illustrating how misunderstandings can lead to frustration but ultimately deepen knowledge and understanding, specifically focusing on negative caching.
Reading Specifications:
- Understanding DNS protocols through foundational documents such as RFC 1034 and 1035, which provide clarity and confidence in technical discussions.
Conclusion:
Evans concludes by reiterating that achieving a solid understanding of complex systems like DNS is a gradual process. By employing careful observation, acknowledging confusion, and experimenting with hands-on projects, learners can demystify complicated subjects, ultimately fostering greater confidence and knowledge. The empowering nature of creating and understanding one's flawed version of a system is a significant takeaway from her talk.
00:00:11.280
Years ago, I started my professional career as a software developer. I remember going to a lunch-and-learn where we would eat lunch and watch a tech video.
00:00:17.279
One of the first videos I watched was actually by Julia Evans. I don’t remember what she talked about, but I do remember how I felt.
00:00:23.880
I thought to myself, 'One day I'm going to be up there; I'm going to give a conference talk.' So I was really excited to hear that Julia Evans was keynoting at RubyConf Mini.
00:00:31.880
It's really special because this is her first talk in three years. So, let's give it up for Julia!
00:01:11.760
I'm Julia. This talk is called 'Learning DNS in 10 Years.' You might have seen the zine I wrote about how DNS works, thanks to RubyConf for printing them.
00:01:18.119
However, this talk is not really about DNS. This is a Ruby conference, right? So this talk is really about learning hard things, using DNS as an example.
00:01:25.080
Learning DNS was hard for me, and it took me maybe 16 years from the first time I bought a domain name and set it up with my DNS record.
00:01:31.560
When I really felt like the system worked! One thing I want to mention at the beginning of this talk is that I think taking 16 years to learn something is kind of normal.
00:01:37.979
The idea that 'Oh, I should understand this by now' seems a bit silly. For most of us, in our careers, we are often doing other things.
00:01:43.740
So this talk is not about how you should know about any particular thing; it's really about prioritizing something you think is cool to learn, even when it's hard.
00:01:49.740
We'll discuss strategies for learning that concept. We're going to talk about learning through a series of tiny deep dives.
00:01:57.240
My favorite way of learning is to do nothing most of the time. That's why it takes 10 years, guys!
00:02:02.880
For about six months, I’ll do nothing, then I’ll furiously learn something for maybe 30 minutes, three hours, or an afternoon.
00:02:11.760
After that, I feel like I’ve leveled up my knowledge this much and go back to doing nothing for months.
00:02:19.140
I find this method works really well for me. I want to discuss a few strategies for these little deep dives.
00:02:25.020
First, we are going to briefly talk about what DNS is because that is the example throughout this talk.
00:02:31.080
Then, we'll discuss spying on it, being confused—which is my main mode, I think—reading documentation, doing experiments, and implementing our own terrible version of DNS in Ruby.
00:02:36.660
So, what is DNS? Very briefly, DNS stands for Domain Name System. Every time you go to a website, like www.example.com, your browser needs to look up that website's IP address.
00:02:45.239
DNS translates domain names into IP addresses. It also translates domain names into a few other things, but mostly we will talk about IP addresses in this talk.
00:02:50.459
I want to briefly explain why I think DNS is cool because we'll talk about it a lot. There are two things I think are cool about DNS.
00:02:57.420
One is that it's this invisible system that controls the entire internet. Anytime your computer or phone is using Google Maps, it needs to know where maps.google.com is.
00:03:01.920
If you’re visiting something like reddit.com, your computer needs to know the IP address. If we didn't have this system, the entire internet would collapse.
00:03:07.800
It's fun to learn how this behind-the-scenes stuff works.
00:03:15.840
The other interesting thing about DNS is just how old it is. I meant to write 1987 here.
00:03:21.299
There is a document that defines how DNS works that was created in 1987. If you write a program based on that document, your program will work.
00:03:26.759
I find that kind of wild and very cool. Stable technologies like this are fascinating to learn because if you're going to take 16 years to learn it, you want it to be something stable.
00:03:32.040
The good thing about DNS is that everything is still the same as it was 16 years ago, or even 35 years ago! The basics haven't changed since before I was born.
00:03:40.560
So, we can afford to be a little bit slow in our learning, and I like that.
00:03:44.280
That's all about what DNS is! Now, I want to talk about observing DNS, which is one of my favorite ways to learn.
00:03:51.060
I’m going to talk about two spy tools for DNS as examples. We’ll talk about dig, which is a tool for making DNS queries.
00:03:56.340
We just discussed how your browser finds maps.google.com; we can do that in dig.
00:04:02.100
Let’s look at maps.google.com for a bit, and if you’re having trouble connecting to the network, don’t worry!
00:04:09.780
Um, okay, come on.
00:04:11.760
Oh yeah, we're also going to learn if the internet is working.
00:04:17.040
So, if we look at maps.google.com, we get some information back.
00:04:25.560
We have five pieces of information here. I've used example.com in this example.
00:04:30.840
First up, we have the domain name, which is no big deal.
00:04:36.780
The time to live (TTL) tells you how long to cache that record for.
00:04:43.260
In this case, it's one day, which means if you have that information, it's valid for the next day.
00:04:49.080
The record type 'A' stands for address, because this is an IP address.
00:04:55.680
I think the funniest detail about DNS is the 'IN' in the middle which stands for Internet.
00:05:02.760
They probably used it back in 1983 or 1987 when they assumed there might be many networks.
00:05:08.040
In reality, we’re all on the internet, yet every DNS query has this little detail.
00:05:15.479
I just find it funny; every time you make a DNS query, it brings me so much joy.
00:05:20.820
Another interesting point about DNS is that it allows you to poke around the internet.
00:05:27.600
For instance, we talked about A records.
00:05:34.560
If we look up the IP address for example.com, there are also other kinds of records, like text records.
00:05:40.740
Let’s look at a text record really quick on twitter.com.
00:05:46.560
Text records are typically used for domain verification. For example, if you want to prove you own twitter.com.
00:05:53.159
You can set a DNS record called 'Google Site Verification h6dji' on your site.
00:05:59.160
Google will then verify you own it. It's fun to poke around in DNS and see what tools companies are using.
00:06:05.040
It allows you to feel like you're getting an inside peek at what companies are doing.
00:06:11.760
Regarding dig, by default it looks complicated. I personally find it overwhelming.
00:06:18.840
However, dig has a configuration file for streamlining responses to make them more user-friendly.
00:06:25.680
Another tool I want to share is Wireshark, my favorite networking tool.
00:06:30.660
Wireshark allows us to spy on all things related to computer networks, including DNS queries.
00:06:38.780
If we do a DNS query on Wireshark, we see a lot of data that can seem intimidating.
00:06:44.559
When I see something that looks complex, I start by looking at things I understand.
00:06:51.120
For example, I know that example.com is a domain name.
00:06:56.840
It's notable that DNS queries actually aren't as complicated as they might seem.
00:07:02.560
In fact, there are only three fields involved: the domain name, the type, and class, which is always the same.
00:07:08.999
You essentially only need to specify the domain name and tell it you want the IP address, which feels exciting to discover!
00:07:15.180
Despite the complex appearance of DNS queries, it’s often straightforward once you drill down into it.
00:07:21.299
One caveat with using encrypted DNS: if it's encrypted, this won't work, but there's plenty of non-encrypted DNS available.
00:07:29.460
The second thing I want to mention for learning is noticing when you're confused.
00:07:36.900
I want to share a story called 'The Case of the Mysterious Caching' that occurred while learning about DNS.
00:07:44.460
To understand this, I need to explain a bit more about how DNS works.
00:07:52.200
When your browser makes a DNS query, it asks a server called a resolver.
00:07:58.260
You only need to know that the resolver is cached, which is often problematic in computer science.
00:08:03.840
The resolver checks with the source of truth, which has the real answers, but its job is to cache.
00:08:10.620
I had a mental model about how I thought DNS would work.
00:08:17.340
For example, when I set something to be cached for a short time like five minutes, I believed I wouldn’t have to wait more than that.
00:08:24.480
But then, one day, I set up a new subdomain for a project and it didn’t work.
00:08:31.620
According to my model, I should have only had to wait five minutes. But I waited and it still didn’t work.
00:08:38.460
So my mental model of how caching worked was broken.
00:08:45.600
Usually, when weird things happen with a computer, people tend to let it go. They think to themselves, 'I guess I have to wait longer.'
00:08:52.200
But sometimes I get a little mad at my computer and think, 'I will find out what’s going on.'
00:08:58.500
So I Googled my issue furiously and discovered something called negative caching.
00:09:05.220
What was happening was that I set up a new record, but I had visited the domain before creating it.
00:09:13.260
The resolver remembered that it didn’t exist, and thus, cached that information.
00:09:20.760
As a result, I couldn’t connect to the new record because the negative response was cached.
00:09:27.060
Ultimately, I learned about the TTL of negative records that can require a wait.
00:09:34.560
This brings me to another useful technique for learning: reading specifications.
00:09:41.760
When I often go looking for answers about networking, it usually boils down to a simple question.
00:09:48.599
In this case, I wanted to know how long I had to wait for the domain to become accessible.
00:09:55.740
Typically, in networking, there's always a detailed technical document called an RFC or request for comments.
00:10:02.739
For DNS, a main RFC is 1034 and 1035, both from 1987.
00:10:09.300
I think reading these RFCs gives me confidence and clarity when discussing networking or DNS.
00:10:16.560
After finding the information I needed, I understood how long I had to wait for the domain record to become live.
00:10:24.060
In many instances, when my conceptual models break, I also often have uncertain feelings.
00:10:31.920
The key takeaway is that there are often just a few critical pieces of information that fill in the gaps.
00:10:39.540
After this experience, I understood how DNS caching works, and I became more confident in my knowledge.
00:10:46.440
I learned to only visit domains after I'd set up their DNS to avoid negative caching.
00:10:53.520
Next, I want to talk about doing experiments. I believe in experimenting more than others do.
00:11:01.920
Sometimes I’ll pair program, and my colleague is nervous about experimenting. I usually say I'll just try it!
00:11:07.200
I don't know if I recommend this approach, but it usually turns out fine.
00:11:13.620
Due to my interest in DNS, I built a small website called 'Mess with DNS.'
00:11:19.500
On this site, you can perform DNS experiments safely; it's my responsibility, not yours.
00:11:26.400
The site enables people to create DNS records and test things without worries.
00:11:33.600
Let’s do some DNS experimentation! Here’s a subdomain we can use.
00:11:40.440
We will expect that if we make a query, it’ll ask the source of truth which we control.
00:11:46.920
We should only see one query because it should be cached.
00:11:53.040
Let’s set an IP address to a new subdomain. What should I set it to?
00:12:00.060
Perfect! I've set the IP address.
00:12:06.840
Now I’m going to query it multiple times.
00:12:13.200
I’ll look for how many responses I get. Here we go! 7.7.7.7!
00:12:19.560
Amazing! I just queried for it several times.
00:12:25.380
Checking how many queries came back... wow! I see eight queries returned.
00:12:31.680
This is interesting because I expected to see just one, given that it should be cached.
00:12:37.620
The result suggests that the resolver has more than one independent cache.
00:12:44.040
In Montreal, my home, the experiment behaved differently.
00:12:51.000
This complicates the notion of caching, as there might be multiple caches at play.
00:12:57.600
Also, we could ask other DNS services, like Google or Cloudflare, what they say about the domain.
00:13:04.920
One suggestion is to test negative caching, which could be fun, but I'm going to skip that demo.
00:13:12.000
With so many different caches, it becomes complex!
00:13:19.440
Now, I want to discuss my favorite strategy for learning anything—implementing a simple version!
00:13:25.920
Creating a very bad version of something gives me an incredible amount of confidence.
00:13:32.760
You might think building DNS software is complicated, but I find it's surprisingly easy.
00:13:40.440
The first thing we need to do is establish a network connection.
00:13:47.040
We typically connect to a DNS server; one you might know is 8.8.8.8, which is Google's DNS server.
00:13:53.760
We just have to specify the port number, which is 53.
00:14:00.300
This requires just four lines of Ruby!
00:14:05.520
After establishing the connection, we now send a DNS query.
00:14:11.040
If you're unsure how to write a DNS query, we can copy the structure from tools like Wireshark.
00:14:17.520
I just right-click on a query and select 'copy as hex stream' to retrieve it.
00:14:23.280
Now, I can convert it into the byte format required to send it through Ruby.
00:14:30.360
Let's check Wireshark to see if it registers our DNS query.
00:14:37.260
Just look for my UDP packets, and let’s get started.
00:14:44.000
We will see our query transmitted successfully!
00:14:50.880
Now, in response, we see a successful conclusion; it’s authentic!
00:14:57.720
I assure you, it's valid and working. Let's explore the structure of the response!
00:15:04.739
Within the response, we need to separate the header and question.
00:15:10.760
The first part is the header, and the second part holds the question consisting of the domain name.
00:15:17.460
For a domain like example.com, it gets represented through its segments by length.
00:15:23.220
For instance, example has length 7, and com has length 3.
00:15:29.070
Hence, example.com translates into 7, example, 3, com, 0 to signify the end of the name.
00:15:37.000
For the remaining information, we specify type and class—where class denotes the internet.
00:15:44.520
With all these elements combined, we can create our query.
00:15:52.320
While sending queries is straightforward, decoding responses does involve more work. However, I want to demonstrate how it works.
00:16:00.000
Let's use rubyconfmini.com as our test domain.
00:16:06.240
Hooray! That works!
00:16:11.520
This implementation is approximately 120 lines of code; it's a reasonable limit!
00:16:18.600
If you want to know how to decode the response, you can check out my blog post—it provides a detailed explanation.
00:16:25.320
Now, let's recap what we've discussed.
00:16:32.399
We covered several methods for learning about DNS.
00:16:39.840
First, we discussed observing and understanding things I'm curious about, which simplifies the complex.
00:16:45.240
Next, I emphasized the importance of noticing confusion; it’s something we often overlook.
00:16:51.420
Reading specifications like RFCs offers real confidence when participating in technical discussions.
00:16:58.680
Doing hands-on experiments with DNS shed light on how things function in practice.
00:17:04.800
And finally, implementing your own simple version can lead to unusually high levels of confidence!
00:17:11.100
Overall, it’s empowering to create and understand your flawed version of something; it enhances your grasp of concepts.
00:17:17.520
Thank you for listening!
Explore all talks recorded at RubyConf 2022 Mini
+29