Summarized using AI

Unix For Rubyists

Nick Marden • October 12, 2017 • Selangor, Malaysia • Talk

In the video titled Unix For Rubyists, speaks at RubyConf MY 2017, Nick Marden introduces the fundamental concepts of UNIX and how they can be beneficial for Ruby developers. He sets a lively tone for the talk, referencing a playful competition to access a fictional 'bank' controlled by his colleague Jimmy, where attendees learn to leverage UNIX tools and principles to solve problems. Throughout the presentation, Marden emphasizes the importance of UNIX in the development process, especially for web developers. Key points covered include:

  • Introduction to UNIX: UNIX, created at Bell Labs in 1969, has evolved over nearly 50 years, yet its core concepts remain relevant today.
  • UNIX Philosophy: Encouraging developers to create small programs that do specific tasks well, leading to a more modular and efficient coding environment.
  • Understanding Processes: Marden explains processes in UNIX, focusing on the fork and exec commands, which are essential for creating new processes and executing different programs.
  • Inter-Process Communication (IPC): He delineates how processes communicate in UNIX using pipes and sockets.
  • UNIX System Calls: Marden demonstrates important system calls like ps to monitor processes and kill to send signals to them.
  • Security Considerations: The importance of understanding permissions, especially around SSH agents and their potential exposure if not properly managed.
  • Practical Demonstrations: Marden gives insights into using tools like strace to diagnose running processes and using gdb for debugging.
  • Conclusion: He concludes that mastering UNIX tools can greatly enhance a developer's ability to solve problems efficiently, especially in production environments, while also cautioning against the risks of SSH agent forwarding.
  • Contest Winners: Marden wraps up by announcing contest winners who participated in the challenge related to the bank vault.

Unix For Rubyists
Nick Marden • October 12, 2017 • Selangor, Malaysia • Talk

Speaker: Nick Marden

Website: http://rubyconf.my

Produced by Engineers.SG

RubyConf MY 2017

00:00:06.359 hi everyone I know it's been a fairly long day and to the end here so I'll try to keep
00:00:11.709 things lively and quick I wanted to start by reiterating what's been said
00:00:18.039 before which is I like to thank Jimmy and everyone who's worked to to organize
00:00:24.550 this conference this is much bigger than in the past and it's really awesome to
00:00:29.649 see so many people turning out and to see how vibrant the community is hearing in KL and Malaysia so thank you very
00:00:36.399 much Jimmy for all your hard work so
00:00:42.070 yeah my name is NIC I'm an American I'm one of the founders of Rapid River software which is based in Kuala Lumpur
00:00:48.850 or a consultancy and I am on github I'm
00:00:55.660 Nick Martin I'm some garden us I've been a web developer since 1993 which makes me pretty much almost as old as every
00:01:02.050 web developer that could be and when I'm not writing software my favorite thing
00:01:08.110 to do is show other people things that I've learned about writing software that's my son Lucas we're building a
00:01:13.619 robot here and when I thought about the
00:01:18.670 topic of UNIX and and what Rubeus can use it for I thought what better way to
00:01:26.009 to show these ideas than stealing Jimmy's money because you know every
00:01:31.060 Jimmy is rolling in cash from this in this conference and you know I think
00:01:36.429 maybe we should take a little back but Jimmy's got other ideas he's he's written a ruby script and he's put it on
00:01:43.869 a super secure server on a private computer that we can't get to from the
00:01:49.599 outside world it's called the vault is the name of the host and and in that script is his money and so I've opened
00:01:56.319 up a car I did actually open up a competition and I don't know I think about a dozen of you participated and I
00:02:02.770 gave people access to this Bastion host and the job was to get Jimmy's money so that's our agenda for today and we're
00:02:09.970 gonna use Unix so if you want to follow along the actual repo for the code that we'll be
00:02:16.300 discussing is Nick Mardon steal Jimmy's money and normally I'd say like I don't
00:02:21.850 like to see everyone's laptop open while I'm talking please open your laptop and take a look this talk itself in case you want to
00:02:27.970 refer to it in the future is the UNIX for Rubeus so Jimmy's bank account okay
00:02:34.300 so what we're gonna see is we don't have access to Jimmy's bank account and it's
00:02:39.940 got an in-memory combination that's combinatoric lis extremely difficult ooh
00:02:45.040 guess there's brute-force protection built into his bank so we don't it's not
00:02:50.890 easy to get the money out but we have Unix so we're gonna steal some things here this is steely from Rick and Morty
00:02:58.150 he's gonna be our guy before we get started I'd like to talk a little bit
00:03:03.190 about Unix because I think for some of you this is old hat and you're gonna know everything I talk about in this
00:03:08.410 talk and some of you UNIX just happens to be the place where your stuff runs in production and you
00:03:14.050 don't really know that much about it because you kind of live in your IDE but UNIX is an operating system has been
00:03:20.290 around since 1969 it was created at Bell Labs in New Jersey and also at Bell Labs
00:03:27.040 in New Jersey is where the first ever field effect transistor was created an interesting thing to think about is that
00:03:35.430 appertain and m bar d n-- and Shockley created that transistor in 1947 and UNIX
00:03:43.060 was created 22 years later UNIX is almost 50 years old which means the time
00:03:50.350 from the first transistor which is a bit janky looking and doesn't scale down very well to today only one third of
00:03:57.970 that time happened before UNIX was created and UNIX is still with us today as an aside by the way is my favorite
00:04:05.380 tweet of all time it's true your code is not as happy as
00:04:12.980 you think because a CPU is actually just a rock that's been tricked into thinking
00:04:18.400 the old UNIX that was hand labelled by Richie and mailed out to universities
00:04:25.150 AT&T sis V is what we think of when we think of old UNIX and it got a big
00:04:30.980 modern facelift in the early 1990s it's got that awesome David Hasselhoff look
00:04:36.110 now when Linux was created and FreeBSD followed shortly thereafter and the bsd
00:04:43.810 family later included darwin which is probably on all your laptops so unix as
00:04:49.970 a concept has gone through these two versions two major sort of phases but
00:04:56.330 it's conceptually the same thing that it was back in the day i like to think of
00:05:02.570 unix as being like Yoda right maths pointed out in the talk he gave it at red ruby this year that very few
00:05:09.200 projects survive 20 years UNIX has been around for almost 50 years in in some
00:05:14.240 incarnation another another and it's sort of like Yoda in in the Star Wars
00:05:20.390 movies at the beginning you think oh that's kind of an old and scraggly looking guy and it doesn't look like he
00:05:27.110 can do that much but you find out later that actually he's super smart and he can kick your ass so that's actually
00:05:33.440 what UNIX is so anyways let's talk about how we're gonna try and pull off this
00:05:39.380 hack so we're gonna talk about some UNIX ideas and some UNIX tools that everyone should know about we're gonna crack
00:05:46.690 jimmies bank dot our B script locally we're gonna figure out the exploits we're gonna get access to the vault and
00:05:53.330 then we're going to execute the solution on the vault now it's not gonna be a live demo because I'm save you guys a
00:05:59.120 hassle but we're going through all the steps of how you do it so if you got the script open we'll look
00:06:05.990 at the very first part of the Bank Darby asks how much money you want to deposit
00:06:12.350 that's actually the key to the puzzle you have to know what number Jimmy typed in if you get that answer you crack
00:06:19.150 the bank and in Ruby idiom I use a range
00:06:26.110 you know zero to fifty I pick five numbers that are at random it's like it's like the draw of the lottery and
00:06:32.309 that's my combination and I register a thing called a signal handler and the
00:06:38.889 signal handlers reaction to the user one signal should be to print out the combination so we're in need the
00:06:45.189 combination unlocked the bank and the script is willing to give it to us but the thing is that Jimmy started up his
00:06:51.849 bank script and put it in the background and left the server so we don't have his
00:06:58.809 terminal so printing out the combination isn't gonna help us this is a problem your smart guy Jimmy protecting your
00:07:05.499 money the next part of the script sets the combination pin so there's a bank
00:07:11.680 right so there's easy you have a lock with tumbler pins and you've got to get them in the right order so the way I the
00:07:19.749 way Jimmy chose to implement this is it just creates 50 well 51 51 if I pose
00:07:32.199 they're just UNIX sockets they're files on disk that or they appear to be files on disk but they can be written or read
00:07:40.059 from by the bank process and I actually I fork off all these processes and each one of them owns one of these photos and
00:07:46.659 so the way that you press a pin you should just write something to that FIFO there's 50 51 files you just pick one
00:07:53.319 then you write to it and that says you press that pin now the structure of this code you read it and I'm using each with
00:08:01.689 object which is the most awesome thing I am creating a hash I start with an empty
00:08:06.849 hash and 51 times I create a temporary
00:08:11.889 file and get rid of it so that just gets a cheap way to generate a temporary file name and I call fork and fork is an
00:08:19.149 important unix concept that we're going to get into in a little bit but i'm just creating a new child process it belongs
00:08:24.279 to its parent is the bank script and I tell that child process to print out
00:08:33.099 in the PS in the process listing to print out which pin it is we're going to need that later I print out a little bit
00:08:40.269 of helpful text to tell us what the name of the temp file is that we can write to
00:08:45.550 to activate that that combination and then I just start listening around the FIFO right anything at all to the FIFO
00:08:52.120 and now we'll the process will exit
00:08:58.839 because the break inside of the loop statement ends the processing of the loop the fork block exits and under the
00:09:07.320 behaviors of Ruby Fork that means the child process will exit okay the last
00:09:15.070 thing I do is in UNIX fork when you do a fork you get two processes the parent
00:09:22.029 and the child in the parent process the return value of fork is the process ID of the child so I store that in a hash
00:09:28.959 and I'm gonna need that hash later to test the combinations that's pretty
00:09:34.420 simple what does it look like in real life so Ruby thank Ord Jimmy wow so much
00:09:41.320 she's got four hundred fifty six units of whatever currency it is we're working with and you type that in and it prints
00:09:48.730 out the combination and it says but that's okay if you ever forget this you can just send Sunday user one signal to
00:09:54.250 this process and then it starts listing out the files that represent the different pins and so if I I see the
00:10:03.579 first number in the combination is six and if I echo foo which is the only
00:10:08.980 thing you should ever echo when you need to echo something if I echo that word into that file the bank script tells me
00:10:17.100 that I've correctly correct the press the first pin if I echo foo into the
00:10:22.990 file that represents pin number 0 well 0 isn't yet it's not time to press 0 yeah
00:10:28.870 but my next one should be 44 so 0 is wrong and so I get an error
00:10:33.990 the bank crashes and swallows Jimmy's money and he's kept it safe so that's
00:10:40.589 the idea and this is what the implementation of that looks like at the end of the script this is where this is
00:10:46.589 the whole script we're done I just do a loop and I call wait pit which is what
00:10:52.860 the the parent is waiting for the children to exit children will exit when you press the pin and I call processes
00:10:59.820 dot delete and everyone knows the fast way to get rid of something in a hash and find out the value is to take the
00:11:05.339 return value of delete all in one line and if you press the right thing I tell
00:11:11.459 you you press the right thing if you press the wrong thing I raise an error and we swallow Jimmy we swallow the money so that's the whole script now I
00:11:20.390 actually open this up as a competition to people and some people came and they
00:11:25.649 actually solved it so they'll be prizes at the end but a lot of people struggled
00:11:31.470 with various conceptual parts and I want to step back and explain the ideas in
00:11:36.690 UNIX and how you apply them to get the solution of the problem so a fundamental idea of UNIX architecture is kernel
00:11:43.110 space versus user space I'm sure most people are familiar with this so in user space where your process runs you might
00:11:49.830 do things like ask what your pit is well that requires a call to the kernel and
00:11:55.350 so we have what's called a system call get pit if you want to mess with a file
00:12:01.380 you have to open it that requires a call to open also assist call and there are
00:12:06.540 parts of the kernel that deal with implementing on the kernel side all of this behavior system calls do almost
00:12:14.310 everything interesting in a program that you would care about so it's really important to think about if you've got a
00:12:20.130 problem you need to solve you think what system calls are happening because if they aren't the ones I expect that's probably the problem okay one of the
00:12:29.400 thing about documentation I use this all over the place in these I keep putting numbers after things again most people
00:12:35.070 probably know this these are man pages man page manual pages as they are known
00:12:40.320 have sections and if it's got a one it's a program that you could call something you type to the command line if it's got
00:12:46.410 a to its assist call that you could invoke and there are other sections and you can generally
00:12:52.929 like a program like PS that does a process listing you don't have to say man 1 PS because there are no other PS s
00:13:00.069 but stop for example there's a stat program you can run it at the command line that's that one there's a stat sis
00:13:07.720 call which tells you about the details of a file and returns it in the C code that's stat - so to be unambiguous you
00:13:13.629 could say man 2 stat you're saying I want the I want the SIS call not but not the program so I'm gonna use this
00:13:19.059 terminology as I go through the other slides and as a user of Unix you should
00:13:24.159 be thinking about man pages anytime you get stuck just type man PS or whatever
00:13:31.419 okay so one other important concept is UNIX philosophy this is was summarized
00:13:38.889 really well in this quote sort of the core idea of UNIX is you should have a
00:13:45.549 bunch of really small programs that do one thing well that's kind of probably
00:13:50.829 an applet of monolithic rails app developers but you do one thing well and
00:13:56.049 you write them to work together and at the time when UNIX was created the
00:14:02.079 universal interface interface of data from one program to another was text streams so programs like grep and awk
00:14:08.679 and said and all that all they do is they take in data they do something to it and they spit it back out a
00:14:14.619 functional programmer would love that right because the stream is just a thing
00:14:21.279 that gets passed through every function so in keeping with that concept UNIX has
00:14:27.909 the idea of standard file handles there are three of them they're numbered 0 1 & 2 every program has its standard input
00:14:35.879 its file handles 0 it has its a standard output which means things you would want
00:14:42.369 to get out of the program if things are working well and every program has its standard error which is things you'd
00:14:48.099 want to get out of the program if things weren't working well in this example I run my program and I
00:14:56.069 take its standard error file handle - and I send it to Devon all I don't want
00:15:01.350 to know about it and then with the pipe symbol I take that standard output and I
00:15:06.569 make it the standard input of grep so that becomes file handle zero of grep I
00:15:15.319 get rid of grep - V means don't show me the lines that match this thing I graph
00:15:20.579 out the useless junk and then its output its file handle one pipe becomes file
00:15:26.339 handle zero of the T program and T is named T because it's like a plumbers T
00:15:31.589 it takes the output and it puts it in a file and it also prints it to the terminal literally plumbing so those are
00:15:40.079 the concepts I wanted to review now in Ruby what does that look like well if you ask Ruby I've created a little
00:15:47.040 function here called fin foe it's the file info if I ask Ruby for the fin foe about standard in standard out standard
00:15:53.459 error and sure enough it tells me that the file hell numbers are 0 1 & 2 Ruby is very heavily influenced by UNIX
00:15:59.220 philosophy so this makes sense but if I open other types of files for example if
00:16:04.529 I use the open your eye module and open up the FML subreddit then that's a temp
00:16:12.000 file it still got a file handle number it's number 9 but it's a temp file ansi
00:16:18.600 passwords a regular file and then interesting thing that we're gonna use later on if i close a file and open a
00:16:25.079 new one i reuse the file handle number and that's an interesting trick that we use later so it's a there's a list and
00:16:32.730 when you close it leaves a hole the next time an open happens it uses that empty number in the list
00:16:40.160 Ruby's sort of standard files are named standard and standard out standard error
00:16:45.449 you can also use the dollar sign version of it they're immutable you can reassign them this is a pretty awesome trick you
00:16:51.540 can actually just say yeah I want standard out to actually be this file over here so this example shows that standard
00:16:59.800 output to one file standard error to another lastly there's processes now
00:17:06.100 processes do everything in UNIX the kernel is there to offer its help but the processes do everything every
00:17:11.560 process has a numeric ID you can find out about it from process stop it or just dollar sign dollar sign which is
00:17:16.630 part of the great legacy of Perl magic variables that always send you to a man
00:17:22.780 page to figure out what they are and every process has a parent the only
00:17:27.790 process works that's not true is this one process on your system called system D or an it it's got process ID one
00:17:33.580 because it's the one that gets started by the kernel and it starts everything else okay so we've I've just reviewed a bunch
00:17:40.840 of basic concepts probably all you guys all know that stuff step number one let's get the combination we're gonna
00:17:46.930 use PS and we're gonna use kill PS is gonna tell us where the bank process is
00:17:54.040 and kill it's gonna help us get the combination now kills funny name because when you type when you say kill a
00:17:59.590 process you think I'm gonna kill it well that's not actually the only thing it does you can give any signal name to kill and it will just send that signal
00:18:05.620 to the process up to the process what it wants to do so we're gonna send a user one signal to the bank we did a little
00:18:13.660 piece here we use some pipes we find out that the pit is six nine six seven and we send a kill user one to it and
00:18:20.010 nothing happens right because we don't own the terminal where this process is
00:18:26.740 running so it got printed somewhere but we can't see the answer so that was actually pretty useless okay
00:18:33.460 hmm system call well turns out there's just really awesome tool that every
00:18:38.529 single developer should know inside and out it's called s trace s trace list
00:18:43.600 system calls as they occur in a process you can say s trace foo and s trace will
00:18:50.649 start up foo and tell you what it's doing or you can say s trace - P and
00:18:55.960 give it a pit and it will attach to a running process and tell you what that process is doing and you confer it gives
00:19:02.500 you a lot of noise so you can filter it further on system calls l trace does the
00:19:07.539 equivalent thing for user library calls but the thing I said earlier is true
00:19:12.820 when a programs not doing what you expect it's probably a system call that you wouldn't expect and you want to go
00:19:19.000 find out what that is so let's attach to Jimmy's program so
00:19:25.020 six nine six seven is the pin so s trace - P and I tell it I only want to see
00:19:30.220 times when the write function is called right is the system call whose job is to write to output file handles or disk
00:19:37.330 files or whatever so here we see that after we sent the cig user 1 that file
00:19:44.559 handle 1 was told to write these characters well that's it got our combination we just used s trace now
00:19:50.980 this is obviously a completely contrived example but I think you can all now think of cases where if you ever stuck
00:19:57.039 on something on a production server this would have been a great way to find out what was going on if you have no other output ok so that was step number one
00:20:05.950 step number two is to enter the combination into the program
00:20:11.020 not PS again okay so we'd TS grep for the word bank these are all our bank
00:20:16.840 pins that's awesome we know the process IDs hmm here's an interesting program
00:20:23.670 list of open files else off this is the other thing you're going to use all of the time with s trace you since
00:20:33.580 everything in UNIX is a file knowing which files the program's interacting with or not interacting with is a great
00:20:40.870 way to diagnose issues so in this case I I wanted to enter combination 0 that's
00:20:48.940 the first number in my example so I find Bank pin 0 is 6 9 7 oh and I run else
00:20:56.080 off on it and it has a FIFO open that's the name of it now I could have known
00:21:02.380 this by looking at the screen that where Jimmy ran the bank but I don't have that screen so this gives me the information
00:21:08.230 that I needed that was lost from his terminal so again I think everyone can imagine places where you might use this
00:21:14.800 in a production server situation so I correctly echo foo into that file and
00:21:22.740 the 0 combination gets entered blah blah blah do all five starts on now let's we
00:21:32.530 lost Jimmy's terminal because he ran his bank script and you know ran it under screen or he he logged out or whatever
00:21:39.670 you know it's in the background we don't have access to it if you want to use the sort of as a kid there's this awesome
00:21:46.660 game called wizardry and I was sort of like Dungeons and Dragons for the Apple 2 and there was just one spell if you
00:21:53.230 got powerful enough it's called tilt await and tilt away just destroyed everything the equivalent is gdb it's
00:22:01.090 the tilt away dove debugging tools because this allows you to actually attach to the running process which remember Ruby is a C it's C code it's
00:22:08.320 right kind of object files it's got functions they got called you can actually attach to it and inspect
00:22:14.309 variables and you can also call C functions in the running program so if
00:22:20.800 you want to like really impress people at a party pull out gdb and attach to the process and close the
00:22:28.270 file handle okay what I did here is I want to get back Jimmy's terminal so I
00:22:34.210 find out what TTY so what teletypewriter what virtual terminal I'm attached to it speed dev pts for and I say you know
00:22:41.170 what forget Jimmy's standard out I'm Jimmy standard out you closed file handle number one and you remember we
00:22:47.860 talked about how file handles automatically fill back in well now the first open available slot in Jimmy's
00:22:54.190 process here is file number one so when I call open now I my dev pts for now has
00:23:00.880 all of Jimmy standard out and so when I issue that same command from the previous slide the output comes here now
00:23:09.670 you go through the five pins and Jimmy's money dumps out onto the screen and it's
00:23:16.210 awesome his bank count was empty you got it all let's go drinking so okay that's
00:23:22.480 great if you're not on the server where it's running so let's talk a little bit more about some concepts that affect
00:23:30.790 security a lot but also have to do is sort of the heart of how UNIX sees process management process forking I
00:23:38.410 used Fork earlier I just took for granted than ever knew what fork was but there's some key ideas about Fork that
00:23:44.730 there's very simple but you got to really understand them everything in
00:23:50.440 Unix starts with Fork the only program that ever gets run by the kernel is in it from that point forward Fork is doing
00:23:58.840 everything so the tree is a literally a tree and it all comes from an it and the
00:24:05.710 parent and child when you call fork are almost identical things so just you've
00:24:11.290 Erin you talked about the copy-on-write issues there's some simulation going on
00:24:16.810 to make it be quick but there's a complete copy of the memory there's a complete copy of the file handles a
00:24:22.240 whole bunch of stuff is exactly identical there's a few things that are different that you can read the man page about but basically they're totally
00:24:28.270 totally exact copies the only difference is the parent sees a return value a nonzero return value that's an
00:24:34.690 indication that you're the parent in the code you're like Who am I because it's not like when you have a
00:24:42.100 child you're pretty aware at the end of the process whether or not you're the parent of the child I've seen the
00:24:48.640 process and it was very obvious at the end but in this case since the processes
00:24:54.760 are identical how would you know so you need the return value of fork to tell you that and then further proving that the the
00:25:05.230 designers of eunuchs nailed the metaphor the parent does have to clean up after the child and the way that the parent
00:25:11.860 does that is they call one of the variants of the weight to system call and in doing that the parent then finds
00:25:20.410 out like well if the child exited correctly and cleans up various resources okay so that's how process
00:25:27.370 forking works since this is a ruby conference i just ruby mumble mumble i should probably talk about some actual
00:25:32.679 ruby stuff because of the way that fork works because the memory is all copied
00:25:38.080 completely as as aaron did allude to in his talk in singapore of course you want
00:25:44.530 to try to maximize the compilation before fork right because you're gonna get the the fact that all these classes
00:25:49.720 are already compiled is are automatically true in in the child process as well so i don't I got this
00:25:57.340 exactly right but the the general thought that I've read about is you want eager load paths to be maximal so that
00:26:04.750 as many classes as possible are loaded in rails before the forking occurs and
00:26:10.630 of course in any non development environment config cache classes is already going to be true but that's an important part of it right because
00:26:17.590 otherwise you you would load all the classes and then on the first request you would dump the the cache that would
00:26:24.429 be useless so an example would be something like a you know your own web server pool you can load up all of your your active record models and all that
00:26:31.510 stuff ahead of time and then the children when they fork they they're already good to go
00:26:36.580 and after a recompile in every child okay there's my requisite actual Ruby discussion
00:26:43.100 okay so fork exec is a pattern that we see all the time and basically goes like
00:26:48.810 this so you got a process it's gonna start some other process a different
00:26:54.060 process so it calls fork gets new process but it's copy of myself so that's not very useful and then you call
00:26:59.910 a second system call exec all exec does is just say you know what don't run this
00:27:05.430 code run that code and so you have the same file handles that you had before
00:27:11.360 but you have a different binary image so
00:27:16.400 what you get is all of the environment and file handles from the parent running a different program and there are lots
00:27:22.500 of variations of this you could read the man page it's kind of boring with how they all differ but this is how every
00:27:31.440 demon the sshd and you know HTTP this is how they all work well no HTTP doesn't
00:27:37.980 but the scripts that start HPV so a pattern that you see is fork exec as
00:27:45.450 enhancement so what that means is this is like a bin stub works this way well
00:27:51.840 not put the bin stubs that are written in pure Ruby but if something is a bin stub or a shim you you take a script you
00:27:58.530 add some behaviors that you want to the environment and then you exec into a different program and so what happens is
00:28:04.620 you basically have enhanced that program by giving it some behaviors that it didn't have before okay
00:28:11.120 when you call exec and bash there is no bash program anymore you try this you
00:28:18.060 can go to your console and type like I don't know exact whatever program you want to run the bash shell script the
00:28:25.560 bash start the batter session doesn't exist anymore okay so this fork exec as
00:28:31.350 enhancement is a pattern that you see all the time and I'm going to come back to that in just one second because it's
00:28:37.320 really important in the technique we're gonna use to hack into Jimmie server okay one last thing is there's this
00:28:44.580 concept of inter process communication again in the script I took it for granted that you could make a FIFO but what are these things well you've got
00:28:51.570 the idea of pipes well we saw those in my example of you know gray
00:28:56.690 and tea and all that a pipe is just a unidirectional file handle so one
00:29:02.990 process can write to the pipe and one can read from it and then you have this idea of UNIX sockets it's basically a
00:29:10.190 bi-directional pipe if you want to think of it that way parent-child communication is done with this with
00:29:18.110 UNIX sockets communication between processes when you connect to like Postgres you say psql and you hit return
00:29:24.560 if you're doing that on your localhost there's a socket file somewhere it's a UNIX domain socket and your psql program
00:29:31.640 is writing into that and the Postgres server is writing back answers and you're all you know reading from each other so this IPC as we call it inter process
00:29:40.640 communication has representations in Ruby that you can play with it's really awesome but the the important programs
00:29:48.920 on your UNIX box are using the heck out of these things in a way that we're about to see an example of last thing is
00:29:57.590 everyone should know permissions we have this sort of we have file permissions so
00:30:05.390 there's that weird list of dashes and ours and w's and stuff the first or the
00:30:11.840 very first character there tells you about the file I there's a lot of different things that can be there D for
00:30:17.900 a directory and s for set UID bit blah blah blah the next three say what can
00:30:23.210 the owner of this file do so the owner is Nick he can read and write this file
00:30:28.280 he can't execute it because it's not a program it's just file anyone in the
00:30:33.740 group staff can read the file are - - but they can't write it or execute it
00:30:38.960 the same thing for anybody in the world group now one special rule is that root isn't bound by any of this root can do
00:30:46.040 anything they want to any file at any time so there you have it there are
00:30:51.590 programs if you oh look a new man section five there's a POSIX ackles or
00:30:58.810 including the set faculty can actually say instead of just a group level or
00:31:04.100 just a world level you can say actually Anton can read this file and that's that's an extended access control
00:31:10.140 so man set faculty want to but just to
00:31:16.110 recap here we've got process working with enhancement we've got inter process
00:31:22.799 communication and we've got permissions can anyone guess what program I'm about
00:31:28.770 to talk about that uses all of these things extensively ssh agent right so
00:31:39.929 how does ssh agent work well here on by the way attribution this is from the
00:31:47.640 University of Illinois at Chicago where my son goes I did not attribute them on
00:31:53.160 this like so I couldn't figure out how to do that in rabbit but I'm sending them tens of thousands of dollars over
00:31:58.440 the next four years so we're good and I said it out loud so here on
00:32:05.340 Jimmy's laptop at home he looked he started up an SSH agent and he loaded his keys into it and his SSH program is
00:32:11.610 communicating over UNIX socket back and forth the SSH agent to provide the
00:32:17.360 authentication that he needs to get into what's labeled as remote one here it's the bastion host okay well when he logs
00:32:25.260 in sshd spawns off a shelf and not labeled on this diagram is it also
00:32:30.660 creates an SSH authorization socket on that server then when he goes to SSH -
00:32:37.740 what's label does remote - which is the vault the SSH invocation in his shell
00:32:44.480 talks to the SSH authorization socket on the bastion host which talks to SSH D
00:32:51.000 which is which is read by sshd which goes back through the TCP
00:32:56.070 connection to SSH which sends a request over the UNIX socket to SSH agent which
00:33:02.429 says I got some keys for you here you go sends them down the wire all the way back over to the bastion host where the
00:33:09.179 key is sent over the wire to the vault to get in so a lot of people I set them
00:33:15.480 up with this contest and they may lay back and said I think I got everything working but I don't have an account on
00:33:21.419 the vault yeah exactly you don't have an account on the fault but Jimmy has an account on the
00:33:26.440 vault and so here's what things look like Jimmy's at home he's got his sh a
00:33:33.310 sock he's logged in to Bastion he's got a shell running he's left it running there and meanwhile over here on
00:33:39.490 the vault there's just the SSH daemon and his bank script there's no connection
00:33:50.640 here's the exploit that gets you into the server now the way I set this up for
00:33:56.170 this this contest I actually deliberately set the permissions on
00:34:01.210 Jimmy's authorization socket to be wide open anybody who logged into the server could
00:34:08.310 read and write to jimmy's off hawk and by doing that there are SSH / by simply
00:34:13.450 just saying export and you setting the environment variable export s SH off Saw equals that file all of a sudden there
00:34:19.060 ssh client had access to jimmy's key not really jimmy's key by the way keep that
00:34:26.470 somewhere else now this is just a demo but root can always do this so that's
00:34:34.090 something to think about if you're forwarding your ssh keys that means
00:34:40.119 there's going to be an ssh off sock file on the servers that you log into and anyone with root access on that server
00:34:47.129 can get access to all of the keys you have loaded back at home
00:34:55.879 how many of you already knew that okay good I'm glad I caught you guys
00:35:01.770 something oh yeah so don't do that dangerous so yeah using this exploit
00:35:09.680 when someone else logs in here connects to Jimmy's SSH ah sock and boom they're
00:35:16.230 into the vault and at that point the rest of the exploit the very simple exploit that we already showed is very
00:35:21.390 straightforward to execute and if you use gdb you get to even see the output so that's how the hijack Keys go down
00:35:28.640 okay so in summary UNIX is awesome and it's yoda it's really important to
00:35:36.270 understand unix design and tools you if you really get to know these tools you'll find that you solve problems very
00:35:42.540 quickly especially in production environments and lastly agent forwarding
00:35:48.059 is extremely risky so evaluate your own environment and decide if it makes sense
00:35:53.670 for you okay so here's steely I've got
00:35:59.670 some winners from the contest that was announced on slack and and on Facebook I'd like to call them up for
00:36:06.589 congratulations uh first prize winner is Lincoln Lee
00:36:14.670 this was unbelievable I put the contest out there and he solved it less than 12
00:36:19.750 hours so congratulations you've earned every dime of it
00:36:44.160 yeah congratulations
00:36:49.620 and lastly Hassanein IMed congratulations okay
00:37:01.510 great that's that's my talk does anyone have any questions sorry guys
00:37:11.020 a taste-off yep any questions for Nick about UNIX security SSH
00:37:17.530 agent what not to do
00:37:26.700 no yes please Alex sir we made a deal in the break room I can
00:37:32.170 ask the question is there a recommend another one that's unique UNIX for
00:37:38.500 everyone yeah absolutely sorry there's a book by a guy named Jesse stormy err it's called in to bear
00:37:49.420 with me for a second Jesse stormy or
00:37:55.260 next working with UNIX processes this
00:38:02.450 fantastic book it's all all the UNIX work in the news now that process it's all written with Ruby examples so it's
00:38:10.250 gonna teach you a lot of the concepts of Unix but if you're a rubyist it will be read
00:38:15.320 very cleanly it's not like during the man pages can be very daunting I think for some people because if they aren't C
00:38:20.420 programmers that's fantastic book I think it's only available as an e-book it's like 35 bucks USD so it's not super
00:38:28.340 cheap but I read it a couple months ago just to sort of review some of these
00:38:34.970 concepts and I found it to be awesome also any more
00:38:44.100 excuse me did you raise your hands no did you raise your hands okay that's enough
00:38:50.480 let's go ahead use
00:38:56.920 4:22 what's that the SS has to be on 422
00:39:02.349 doesn't have to be no when you run the oscillation you can choose which report it listens on it's a convention that it listens on port 22 one thing though is
00:39:11.410 well there's the rule about port numbers in UNIX which is that only root can never bind to anything less than port 1
00:39:18.589 or 2 4 you'll want sshd generally to run as root because it needs to be able to
00:39:25.339 spawn a shell for any user who logs in it is conceivable to run sshd as a as a
00:39:32.810 single user but that sshd would only be able to spawn shells for that user but
00:39:38.810 it doesn't have to be port 22 and actually in terms of security I'm not a
00:39:43.820 security expert at all don't hire me to do your security if that's the only thing you're looking for do web development but moving stuff around to
00:39:51.829 different ports is sort of a security through obscurity trick the thing that I
00:39:59.240 showed earlier the bank vault pin combination there is a networking equivalent of that called port knocking
00:40:04.760 that people know what port knocking is anyone yeah you basically require someone who wants to connect to try to
00:40:11.270 connect to a series of ports in a well-defined sequence and once they do that then the IP tables or the IP
00:40:18.290 firewall will then allow them to connect to the the port that they actually want to connect to so it's like the right and
00:40:25.970 then you're allowed to connect to port 22 so you can move sshd to a different port if you want security in that case
00:40:32.270 or you can implement tricks like SSH port knocking good question
00:40:38.119 I oh one more just out of curiosity when you first started your presentation you
00:40:44.220 opened your presentation using SSH in terminal so like I couldn't help but
00:40:49.800 notice that you open it that way oh that's curious how oh that's that's laughs that's like a program yeah that's
00:40:56.220 rabbit it is a presentation tool written in Ruby really biased towards Ruby
00:41:04.410 presentations and so the entire presentation was just a markdown file so
00:41:09.510 if you go to github Nick Martin on github you'll actually see my whole presentation there and the whole thing is just one markdown file and and a
00:41:15.030 directory full of images I found rabbit to be its first time I use it I found it to be a little bit confusing
00:41:20.880 documentation is a little bit in Japanese I don't speak any Japanese I think that would be easier for some
00:41:26.430 other people but as always just walk through the source code and you can figure it out there was there were some
00:41:31.530 pride by bug sessions in my presentation writing all right thanks yeah actually I
00:41:40.320 was I want to be like Matt it's never gonna happen but like in one little way I thought I would use this presentation tool and I could like channel them a
00:41:45.930 little bit all right if that is all please put your hands
00:41:51.420 together thank you so much Nick
Explore all talks recorded at RubyConf MY 2017
+16