Ruby Development

Summarized using AI

Packet-Fu with Ruby

Tod Beardsley • September 04, 2008 • Earth

The video titled "Packet-Fu with Ruby" features Tod Beardsley, a security researcher, who discusses mid-level packet manipulation using the Ruby programming language. The presentation took place at the LoneStarRuby Conf 2008 and introduces the concepts of crafting and manipulating network packets. Beardsley explains his motivation for working with Ruby to test network equipment, particularly security devices such as Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS).

Key Points Discussed:
- Introduction to Packet Manipulation: Beardsley explains why one might want to manipulate packets instead of relying on standard OS stacks, highlighting scenarios such as testing device resilience against unusual packet sizes or compositions.
- The Need for a Ruby-based Solution: He mentions the lack of packet manipulation libraries available in Ruby compared to Python's Scapy or Perl's Raw IP. This gap led him to create a Ruby library named Pakao, designed for packet manipulation, starting from his experience with Scrooby, a port of Scapy in Python.
- Tools and Libraries: The presenter outlines various libraries he used, including PcapRub for leveraging libpcap functionality essential for working with network packets and BinData for creating structured binary data types in Ruby.
- Packet Structure and Types: Beardsley discusses the types of packets that have been implemented within Pakao, such as TCP, UDP, ICMP, ARP, and IPv6, describing how packets are structured as Ruby objects and treated within the framework.
- Upcoming Innovations: He shares his vision for future improvements in Pakao, including integration into the Metasploit framework, enhanced packet response capabilities, and the development of a Honeypot Demaster to deceive attackers.

Examples: A Demonstration of "Who Am I" Functionality:

- Beardsley gives a practical demonstration of the "Who Am I" utility, which sends out a packet to identify the present machine's network details.

Conclusions and Takeaways:

- The presentation emphasizes the importance and challenge of packet manipulation in Ruby, underscoring Beardsley's commitment to developing tools that facilitate network security research and testing. He encourages viewers to engage with his work, promising ongoing updates and improvements in Pakao.

Packet-Fu with Ruby
Tod Beardsley • September 04, 2008 • Earth

Packet -Fu with Ruby by: Tod Beardsley

LoneStarRuby Conf 2008

00:00:06.359 Video equipment rental costs paid for by Peep Code.
00:00:19.880 Screencasts. All right, this is Packet Fu ninja style, mid-level packet manipulation for Ruby.
00:00:25.960 Hello, I'm Tod Beardsley. I'm a security researcher most of the time.
00:00:34.000 This means I do a lot of attack and defense research and application work.
00:00:40.760 At Breaking Point, we create test equipment mainly for routers and security devices,
00:00:49.160 such as IDS (Intrusion Detection Systems) and IPS (Intrusion Prevention Systems).
00:00:54.680 How many people here have ever administered an IDS device? A couple of you. IDS is an intrusion detection system,
00:01:01.719 which warns you when you've been compromised. IPS is intrusion prevention,
00:01:08.119 it's the system that tells you that you almost got owned.
00:01:14.119 I work a lot in testing these devices, and I'm a Ruby newbie. I've been using Ruby for less than a year.
00:01:20.400 I started using Ruby because it seemed like all the cool kids were doing it.
00:01:25.680 As soon as I started, I noticed that there was no reasonable way to perform packet manipulation in Ruby! Network packets.
00:01:33.439 You may be asking yourselves, "Why forge packets when you have perfectly wonderful OS stacks that handle all your packet processing?"
00:01:39.000 Well, like I said, I test network equipment, so very often I have to create strange packets that would never be found in the real world.
00:01:46.000 I do this to observe how security devices and routers interact with such packets.
00:01:52.680 Do they fail if you send them packets that are ridiculously large or small, or a great number of them?
00:01:59.240 It's the same with security equipment, which is supposed to prevent attacks.
00:02:05.039 If I can perform weird TCP reordering, will that affect your IDS?
00:02:11.160 You may also want to capture packets and sanitize them.
00:02:16.280 For example, you might massively rewrite IP addresses or MAC addresses for sharing.
00:02:21.480 You want to share data but not reveal your identity, which is common in the security community.
00:02:26.640 By the way, this is my first language conference and definitely my first Ruby conference.
00:02:32.800 It's very different from security conferences.
00:02:38.040 I see many open notebooks here. You don't see that much at security conferences because people are paranoid.
00:02:45.120 They're concerned about being on untrusted networks surrounded by strangers who may hijack their devices.
00:02:50.319 Apparently, you guys don't mind. Our badges are definitely less cool than those at Defcon.
00:02:57.000 They are much easier to forge.
00:03:02.400 So why would you want to sniff packets on the wire instead of relying on your applications for that?
00:03:08.760 You may have connectivity issues and want to see what's happening at the packet level.
00:03:15.159 One of my favorite things to do with packets at a lower level is device fingerprinting.
00:03:20.640 I want to know how many MAC addresses are in the room, or what your web server really is.
00:03:27.599 You may have an IIS banner, or you may have an Apache banner, but you could be dishonest, and the packets will reveal the truth.
00:03:33.920 In almost all cases.
00:03:41.200 And lastly, for application reversing, if I'm dealing with an application that may not have an RFC,
00:03:49.959 like everything from Oracle, I want to understand how it works at the wire level for reimplementation.
00:03:55.840 This is largely what I do when I'm not engaged in security work, so I need to examine packets as they appear on the wire.
00:04:03.040 So why Ruby? Like I mentioned earlier, I just picked up Ruby, and it's super rad.
00:04:10.640 All the usual reasons apply: Ruby is object-oriented, and so forth.
00:04:16.239 Threading might not be the greatest, but it is super easy.
00:04:23.120 Ruby is cross-platform; it works great on all the environments I care about, which are Linux and Windows.
00:04:30.360 Sorry, Mac users, but of course, Ruby is cross-platform on Macs as well.
00:04:37.120 But there's nothing available in Ruby for this kind of packet manipulation,
00:04:44.960 which is a bummer because Python has Scapy.
00:04:50.039 Has anyone here used Scapy in Python? No one?
00:04:55.840 Perl has Raw IP, which is somewhat similar.
00:05:02.960 C has Libnet, which is considered the standard for packet forging.
00:05:09.400 But there's nothing for Ruby. But wait, there is!
00:05:14.759 Did I just reinvent the wheel over the last couple of months?
00:05:20.919 There is Scrooby.
00:05:26.800 Scrooby was released in April of 2007; it's a port of Scapy for Python.
00:05:34.120 So if you love Scapy, you may like Scrooby. Its syntax is similar, albeit rather unusual.
00:05:40.120 There's Racket, which was released this past March.
00:05:46.240 The folks at Matano Security use RIT for their protocol debugger.
00:05:53.080 They utilize a tool called Ramble that’s built on Racket, and it's quite new and impressive.
00:05:59.840 Racket is a bit tricky for some applications, as its syntax can be verbose.
00:06:06.000 However, it’s quite strong.
00:06:11.520 There are literally thousands of different implementations of similar concepts in Ruby.
00:06:18.360 Thus, I created Pakao, which I started in September of 2008.
00:06:26.520 Just yesterday, I even made some check-ins for bug fixes.
00:06:32.680 I began looking into this back in March, when I posted a question to the local Ruby users group.
00:06:40.520 I asked if Scrooby was truly the state-of-the-art for packet manipulation in Ruby.
00:06:46.880 There was a bit of silence on the mailing list.
00:06:53.520 Then, replying a couple of days later, I thought, "Well, that's great.
00:06:59.879 I can now write this and learn Ruby at the same time!"
00:07:06.360 Pakao is based on BinData. Has anyone here used BitStruct or BinData?
00:07:12.800 Yes? BitStruct has a great interface for some things.
00:07:20.759 However, BinData is more robust for what I need to accomplish, which is why I chose it.
00:07:26.520 So let me briefly discuss its design and core functionalities.
00:07:34.679 I utilize PcapRub because it employs libpcap, which is quite ubiquitous.
00:07:40.800 If you're doing this type of development, libpcap drives popular tools like Wireshark, TCPDump, and TCPReplay.
00:07:46.240 I definitely did not use Ruby Pcap. Don't ever rely on Ruby Pcap if you want to copy or reimplement anything related to packets.
00:07:54.199 You will search for Ruby LibPcap and find it on the first couple of pages of Google.
00:07:59.720 However, it hasn't been updated since 2001, so it isn't utilizing LibPcap's packet injection features.
00:08:06.720 The threading model employed is blocking, which is not ideal.
00:08:12.000 PcapRub is the best-kept secret in Ruby packet manipulation; you won't find it on the first couple of pages of Google.
00:08:18.960 It's actively maintained as part of the Metasploit project now.
00:08:25.680 When I began building binary structs in Pakao, as many developers might do,
00:08:30.360 I created my own factory methods, which I called Packa Factory.
00:08:36.479 I also recently purchased a book on Ruby object-oriented design. That was a huge mistake!
00:08:42.039 I quickly abandoned this idea around April.
00:08:47.760 After that, I switched to BitStruct, as it's widely used and has a nice interface.
00:08:53.160 Unfortunately, when working with things like variable-length data or optional headers and data, BitStruct can be complicating.
00:09:00.800 However, BinData handles all that natively. It has an array type and supports reading until the end of a file.
00:09:06.000 This makes it very feature-rich. It's a structured language and is actively maintained at Ruby Forge.
00:09:12.600 The maintainer is responsive and quickly accepts patches, which is excellent.
00:09:18.360 This is somewhat what BinData headers look like. These examples come from the Pakao documentation.
00:09:22.680 In this case, we can see the TCP header as defined by BinData.
00:09:28.360 For instance, this may show things like TCP source address, which might be a 16-bit value.
00:09:35.040 These are all pretty standard data types for network packets.
00:09:44.400 For example, four bits for the field and eight bits for another field.
00:09:50.600 The ability to work with bit widths is significantly useful when dealing with networks.
00:09:56.120 You can also define custom ones in BinData.
00:10:02.680 For instance, TCP Flags can be defined as one bit each. With BinData, you can create your own data type.
00:10:10.560 This makes implementing such applications much easier. I strongly urge you to explore BinData if you're working with binary libraries.
00:10:17.160 Pakao creates packet headers using BinData, collecting them into packets that function like regular Ruby objects.
00:10:23.959 This is the core magic behind how Pakao operates. There is a wealth of intelligent design regarding various packet types.
00:10:29.160 I have several basic packets currently implemented, and I plan to add many more soon.
00:10:35.280 I noticed that not many hands went up when I asked about using IDS.
00:10:41.080 However, has anyone here used Wireshark recently? About half of you.
00:10:48.320 I won't bore you with the OSI layer model; besides, I'm not sure it's entirely accurate for me.
00:10:54.239 This is a quick diagram of how Pakao treats packets. Picture all these as objects.
00:11:01.279 This is the Ethernet header, containing elements such as source address, protocol, and payload.
00:11:08.240 There's a blank spot in the payload.
00:11:13.360 Within the payload, we have IP headers, which also include several elements like version and header length.
00:11:20.800 And a TCP header that has attributes such as source port, sequence numbers, flags, and more.
00:11:26.760 That tiny box represents the body of the TCP header, where things like SSH headers and data reside.
00:11:34.080 I generally don't go deep into the application-level protocols; I'm mainly interested in the first three layers.
00:11:41.840 You can work with application protocols in Pakao, but that's a topic for another time.
00:11:48.720 All of this is encapsulated within a packet, which serves as the outermost object in this design.
00:11:55.680 Packets consist of headers that store elements in an array.
00:12:01.279 Header zero is the Ethernet header, header one is the IP header, and header two is the TCP header.
00:12:07.680 You can access all of these headers and read and write data.
00:12:13.159 Packets also contain payloads, primarily the body of the TCP header. The payload is typically what you're looking for.
00:12:21.240 So that's built in. Now I have to sift through everything.
00:12:27.640 I see you have TCP headers there. Can you also work with UDP packets? Absolutely!
00:12:33.960 Right now, I've implemented TCP, UDP, ICMP, ARP, and IPv6 to some extent.
00:12:40.760 I believe that's pretty much what you're going to encounter online.
00:12:45.360 One important point to mention is this star packet. This packet's identity is determined by the innermost header.
00:12:53.680 So, when you construct packets in Pakao, it derives its type from the data.
00:13:00.480 If I have a file with ICMP packets, I simply store all the binary data into an array.
00:13:06.680 This is how packets are structured, but it's not very useful by itself.
00:13:14.360 If I establish an ICMP packet variable, it can equal an unknown packet.
00:13:20.480 This allows you to inspect the unknown packet class and determine its attributes.
00:13:28.560 The class will analyze the binary data and identify components, such as Ethernet or IP.
00:13:37.240 It will determine the type of packet being examined, such as ICMP.
00:13:43.840 This process allows for converting raw binary data into useful packets for practical applications.
00:13:50.760 Moving on, when attempting to talk to the network, you go through PcapRub.
00:13:56.760 You can transmit and receive through a network interface, requiring root privileges, of course.
00:14:06.320 When handling libraries, you can also read and write files using a standard libpcap format.
00:14:12.720 This means you can direct packets to a specific location: the default is /tmp/out.
00:14:18.720 I presume you’re using Linux, but you can easily override that with the target file name.
00:14:27.920 I have a couple of utility functions that are Singleton methods.
00:14:34.720 I favor Singletons and don't care what specific critics say about them!
00:14:42.560 One function is called Who Am I? When I began developing this, I struggled to obtain my IP address or MAC address.
00:14:49.679 I found myself doing shell commands and looking through files.
00:14:56.079 So, what the Who Am I function does is confirm who I am by sending out a packet.
00:15:03.359 Let's see if I can demo that functionality.
00:15:09.879 I prefer to avoid revealing sensitive information this early.
00:15:14.920 Let’s just check if it works.
00:15:19.920 Oh, right! I need root access.
00:15:25.440 Oh wait, the password will be concealed.
00:15:32.920 Great! I'm connected.
00:15:41.880 This is Wireshark for anyone who hasn't seen it.
00:15:48.520 So, in the context of Pakao, the Who Am I utility sends out a packet to check who I am.
00:15:54.760 Then, I will capture that packet and read the response.
00:16:02.160 After capturing, I'll get back the matching data, confirming my identity.
00:16:09.120 Unluckily, this requires root privileges.
00:16:16.320 I also have ARP functionality because I don't want to crawl through usual ARP tools.
00:16:24.760 I've created a function to generate ARP packets.
00:16:31.920 You can, for example, generate an ARP table using my utility.
00:16:39.480 I employed various mnemonic functions within the processes.
00:16:46.560 You send packets through PcapRub, or you can utilize the send method.
00:16:54.720 If W is your default interface, that's also easy to incorporate.
00:17:02.160 In Pakao, the 2f method saves packets to a file, which we mentioned earlier.
00:17:09.680 This framework simplifies packet creation.
00:17:16.960 For example, you can create a new UDP packet by recycling an old TCP packet payload.
00:17:22.720 This helps in modifying your data to appear UDP-ified.
00:17:29.280 Still applicable, I also support varied packet flavors.
00:17:37.679 I haven't fleshed out this feature yet, but I aim to improve it soon.
00:17:45.520 Have any of you used Nmap or POF? My knowledge suggests that such functionality is very evolving.
00:17:52.960 Like device impersonation, for example, you can create realistic packet headers.
00:17:59.440 You can impersonate other operating systems by establishing packet flavors.
00:18:06.160 This could be beneficial for things like asset management on managed networks.
00:18:11.760 Once you understand your devices, you might wish to query how many Apple devices are accessible.
00:18:17.720 I will be introducing additional features as they develop.
00:18:24.680 Currently, the version is 0.03, and improvements will be added down the line.
00:18:31.440 I invite you to check out the utility at the specified location.
00:18:38.440 I encourage everyone to download it, but please avoid attacking anyone!
00:18:45.720 You can manipulate various headers and packet flavors.
00:18:54.440 Such as deploying distributed scanning techniques, which can scan for a variety of device fingerprints.
00:19:01.440 I have a friend crafting a distributed scanner using the concepts I discussed.
00:19:10.160 Wait, let's circle back and discuss upcoming innovations for Pakao.
00:19:17.360 I see a future where this tool may find incorporation into Metasploit.
00:19:22.760 Both HD and I aren't particularly keen on Scrooby's implementation, given its quirks.
00:19:29.200 I intend for Pakao to eventually replace it, so if you use the Metasploit framework, Pakao will be included.
00:19:40.000 In addition to this concept, I plan to develop streamlined packet response code.
00:19:45.440 This will allow users to receive responses in real-time.
00:19:54.960 As it stands, a comprehensive TCP reassembly process remains on my radar.
00:20:00.360 I feel this generally falls under the application developer's responsibility.
00:20:09.240 However, I presume I'll have to tackle this issue eventually!
00:20:15.840 The Honeypot Demaster is another concept I wanted to showcase.
00:20:22.920 I believe this functions as a charming decoy for unsuspecting attackers.
00:20:29.560 Much like the classic Phantom of the Carnival scenario—only a digital version!
00:20:37.200 By crafting and generating packets tailored to mislead them, perhaps they may think they are probing an actual service.
00:20:44.160 These measures are critical for asset management in networks where user permissions may not be guaranteed.
00:20:51.080 This ability has potential usability in terms of inquiries about device counts.
00:20:58.440 I can zip around stealthily while gathering relevant information.
00:21:05.760 It's necessary for me to refactor any remaining kinks in the application.
00:21:12.960 A series of rational test cases need to be constructed as well.
00:21:20.160 Since I primarily operate within IRB, my functional tests typically only confirm success.
00:21:28.440 That's the game plan post-RubyConf.
00:21:36.720 Once again, you can find the tool at coded.google.com.
00:21:42.890 It's not up on Ruby Forge yet. This is just a trial to increase visibility.
00:21:49.480 I'm open to any questions or comments regarding the presentation.
00:21:56.640 Thank you for your attention!
00:22:03.440 Any additional inquiries before we conclude?
00:22:08.440 I'll definitely make myself available.
00:22:15.240 I’m also keen to learn about various perspectives.
00:22:20.240 And if there are no further questions, I will step away for now.
00:22:27.280 Once again, thank you everyone, and I hope you enjoy the rest of the conference.
00:22:35.400 Take care!
00:22:41.920 Video equipment rental costs were covered by Peep Code. Screencasts.
Explore all talks recorded at LoneStarRuby Conf 2008
+18