Search
Flaky Tests
Embarking on Your Journey Through the World of Unit Testing
Edit
#unit-testing
#rspec
#automated-testing
#software-testing
#test-driven-development
#ruby
#flaky-tests
Summarized using AI

Embarking on Your Journey Through the World of Unit Testing

Kim Diep • September 27, 2022 • online • Talk

The video titled "Embarking on Your Journey Through the World of Unit Testing" features a talk by Kim Diep presented at the WNB.rb meetup on September 27, 2022. The session aims to support newcomers and those looking to enhance their knowledge of unit testing, essential for ensuring software robustness and effectiveness.

Main Topics Covered

The discussion primarily focuses on unit testing and its significance within software development. Key points include:
- Importance of Testing: Diep emphasizes that testing is crucial for ensuring that software is robust and meets user’s needs. Without testing, developers may produce unreliable and unpredictable results.
- Testing Pyramid Structure: Diep explains the structure of testing in software development, which consists of:
- Unit Tests: The foundation of the testing pyramid, unit tests evaluate the functionality of individual methods or classes quickly and efficiently.
- Integration Tests: These tests verify that different components of the application work together.
- End-to-End Tests: The final layer, which tests the application’s overall behavior and user interface.
- Unit Testing Purpose: Unit tests are described as friendly tools that make developing software easier by catching issues early and effectively documenting what the code is supposed to do.

Practical Examples

  • RSpec Framework: Diep provides a demonstration of using the RSpec framework for writing unit tests in Ruby. She illustrates the Arrange-Act-Assert (AAA) structure—setting up the need, acting upon the test, and asserting the expected outcome.
  • Sample Code: Through a simple example, Diep shows how to write a test for a class called 'Bubble Tea Time' which includes a method for summing three numbers. She highlights writing clear and readable tests that serve as a form of documentation for the code.

Key Takeaways

  • Testing Strategy: The necessity of writing tests deterministically to avoid flaky tests that yield inconsistent results.
  • Control over Tests: Diep discusses the importance of isolating tests and controlling random elements, such as time in code.
  • Final Encouragement: The talk concludes with an encouragement to engage with unit testing, highlighting its benefits in understanding code functionality and supporting collaboration among developers.

By the end of the talk, participants should feel equipped to implement unit testing in their projects or advocate for its use in their teams. Kim Diep's background as a Software Engineer and her passion for tech education enrich the discussion, stressing the value of community contributions to learning and support in the tech space.

Embarking on Your Journey Through the World of Unit Testing
Kim Diep • September 27, 2022 • online • Talk

This talk was given at the WNB.rb meetup on September 27, 2022.

Are you new to unit testing? Maybe you've already written some unit tests and would like to refresh and expand your understanding? Well, you're in luck because this talk is just for you. How can we ensure that the software we build is fit-for-purpose, robust and meets users' needs? Well, that's where software testing comes in handy and even better, automated software testing. In this practical talk, I will demystify what software testing is and the purpose of unit testing. We'll have a look at the unit testing structure and a code example in the RSpec unit testing framework and I'll be sharing some unit testing top tips to help you get the most out of it.By this end of this talk, you'll be able to give software testing a go, start to use it in your learning projects or advocate for it within your teams. It's not as yucky as it sounds; automated testing is a key part of building great software and making your Software Engineering life sweeter!

Kim Diep is a Software Engineer at Butternut Box and she lives in London, England, UK. By day, she builds software with Ruby code. She also loves building things with the C# programming language, .NET ecosystem and Unity. She only just started Ruby a little over 3 months ago. Before then, she was working primarily in C#. She is passionate about creating things for tech education and loves sharing her experiences with the community to encourage others to learn and grow in their technical skills and confidence. She currently volunteers as a mentor for those who are underrepresented in tech and/or come from socially disadvantaged backgrounds. In Winter 2021, Kim created Curiously Code as a discovery platform providing technology, career and wellbeing inspiration for Curious Humans. By night and on the weekends, you can find Kim practising Kendo, working out at the gym or something random like playing video games and doing some Instax photography!

https://www.wnb-rb.dev/meetups/2022/09/27

WNB.rb Meetup September 2022

00:00:00.960 um in terms of your tests what coverage
00:00:04.020 you would get throughout the software
00:00:05.759 life cycles at the top you have the very
00:00:08.639 manual tests so technically you could go
00:00:11.700 and click a button click through an app
00:00:13.920 just to make sure it works but that's
00:00:15.719 not very efficient and also costs a lot
00:00:18.060 of money so what software Engineers do
00:00:20.340 is they incorporate testing throughout
00:00:23.400 the coding process so at the very bottom
00:00:26.519 of the pyramid we have something called
00:00:28.199 Unit tests unit tests are focused on a
00:00:33.120 um function or methods within a class
00:00:36.320 testing that it works so it's a small
00:00:39.059 list part of your code the reason it's
00:00:41.940 at the bottom of the pyramid is because
00:00:43.739 you can write many of them fairly
00:00:45.660 quickly fairly cheaply and you can keep
00:00:47.460 running them again and again in an
00:00:49.200 automated way save some time when you go
00:00:52.680 up the pyramids you can see we've got
00:00:54.480 something called integration tests you
00:00:56.640 can see integration tests as a layer
00:00:58.500 above unit test in the sense that in
00:01:00.420 tester instead of testing each thing in
00:01:03.719 isolation you're testing that things
00:01:06.240 knit together correctly that each
00:01:07.979 component can speak to another component
00:01:10.020 and then at the top you've got your
00:01:13.140 end-to-end testing so this could be like
00:01:15.299 a smoke test on the UI doing a click
00:01:17.700 through test the reason that we've got
00:01:20.100 this pyramid is because integration
00:01:21.840 tests and and tests are a bit more
00:01:24.600 expensive to run they take longer to run
00:01:26.939 and so we tend to only cover the key
00:01:30.060 flies of the application for these tests
00:01:32.400 but today we're just going a focusing
00:01:34.380 unit test because this is such a big
00:01:36.180 topic so we're not going to cover all
00:01:37.979 the pyramid
00:01:39.119 today so what is the purpose of unit
00:01:42.600 testing then
00:01:44.220 I think unit tests are your friends I
00:01:46.920 don't claim
00:01:48.119 um
00:01:49.140 yeah don't worry about this I just took
00:01:51.119 it online so don't show this about
00:01:54.659 um but they are your friends
00:01:56.640 and you can easily write and run loads
00:01:59.399 of them
00:02:00.600 and when you do it goes well if you
00:02:03.780 don't write any tests at all it can be a
00:02:06.619 nightmare because they can do unexpected
00:02:09.959 things and your app can you can't really
00:02:14.220 trust what you're building essentially
00:02:16.620 if you don't have to test there so
00:02:20.040 what is a unit test then
00:02:22.860 um so now that we know what what that
00:02:26.280 needs to do
00:02:28.140 um we can think of its structure I know
00:02:30.180 pusheen's sad and I we've got a range
00:02:33.020 act as uh so the way that unit test is
00:02:36.840 structured is you set up the things you
00:02:39.420 need to test you act on it so you run
00:02:43.019 your production code that you want to
00:02:44.640 test and assert is you're comparing
00:02:47.280 actual versus expected result
00:02:50.940 cool
00:02:52.019 and inside
00:02:53.940 or programming languages but for Ruby
00:02:56.640 we've got the R spec unit testing
00:02:59.040 framework which contains neat methods
00:03:02.400 that you can then use to write your
00:03:04.800 tests in a simple way so I'm going to
00:03:07.379 show you in a second what this means but
00:03:09.420 we've got the sky
00:03:10.680 so your system on the test or Sut is the
00:03:14.700 thing that you're trying to test within
00:03:16.920 your application so it's a class or
00:03:18.900 module
00:03:20.519 um context is a scenario or conditions
00:03:23.220 that you're running your tests in
00:03:25.200 and then your it is usually your
00:03:27.959 assertion's expected versus actual
00:03:30.300 result so before further I do let's have
00:03:34.140 a quick look at some some code then so
00:03:36.840 I've got like a
00:03:38.340 a simple
00:03:40.140 um repo set up I've got a spec file and
00:03:44.760 this was what I mentioned before is that
00:03:46.500 too small should I zoom in a bit
00:03:49.200 I can zoom in there you go much better
00:03:51.900 so this is my described block so that's
00:03:55.680 describing the system on the test that I
00:03:57.900 mentioned before so my system on the
00:04:00.420 test here is called bubble tea time
00:04:02.700 because that is the class that I'm
00:04:05.040 trying to test which is within my um
00:04:08.819 live folder and if we look inside that
00:04:13.260 I've just got an empty
00:04:15.599 um class called Bubble Tea Time
00:04:18.260 so usually when we start testing it's
00:04:20.880 just good to just run
00:04:23.040 um our spec which is the command to run
00:04:25.199 the test and at the moment I've got no
00:04:28.199 test cases in there so that's why it's
00:04:31.139 zero so that's a good place to start all
00:04:34.800 right so let's set up one
00:04:39.180 um thing that this bubble tea time class
00:04:41.460 can do so say
00:04:43.320 let's say it can sum three numbers
00:04:45.900 together all right so I can use this lap
00:04:50.100 and what lap does but
00:04:53.240 see time once it starts is it just sets
00:04:57.419 up a nice
00:04:58.979 um reusable reusable sort of um name
00:05:01.860 that I can reuse in my test
00:05:04.320 and I can do something fancy which is
00:05:08.759 describe
00:05:11.460 new and what this does is it will one
00:05:15.120 block of code
00:05:16.680 so the describe classical T times up new
00:05:19.199 create a new instance of that class and
00:05:21.900 assign it against this unusable variable
00:05:24.180 that I can use great
00:05:26.880 so I can
00:05:29.039 have a number of the Sky Block now that
00:05:31.380 I've described what I'm trying to test I
00:05:34.500 can describe say a method that that
00:05:37.940 class has so say it has a sum method
00:05:43.380 I could use this describe block to wrap
00:05:47.039 my tests in because essentially what you
00:05:49.440 want to do is within your test you want
00:05:51.240 to make it read like documentation so if
00:05:54.539 someone is new to the project and hasn't
00:05:57.000 seen that code before they can go to
00:05:59.400 your project and be able to lift off the
00:06:02.100 test code and be able to read through
00:06:04.139 what your class is intending to do so
00:06:07.440 here I'm saying that this method sum
00:06:10.020 will return the summer three numbers
00:06:12.300 and what I can do there is I can use my
00:06:16.620 um methods that I have from artifact and
00:06:19.979 I can reuse this
00:06:21.780 we'll see time and I can pour some put
00:06:26.400 in three numbers and I'm expecting that
00:06:30.000 to equal let's check my maths six oh six
00:06:34.500 there you go
00:06:36.000 so this is my
00:06:38.120 assertion that I mentioned before
00:06:40.199 testing
00:06:41.880 your actual versus expected result
00:06:46.440 um and then this is the describe block
00:06:49.440 that describes what I'm trying to test
00:06:51.360 so here it's the method and up here
00:06:54.900 for the whole file I'm testing the
00:06:57.660 bubble tea time class so if I run this
00:07:01.860 um
00:07:02.580 it's not implemented yet
00:07:04.800 so what I can do is I can go and
00:07:06.960 implement
00:07:08.600 implement the method within that class
00:07:12.720 so I can sum three numbers together
00:07:16.560 and then
00:07:18.840 we shall see the test pass
00:07:25.440 all right so if I run it again you can
00:07:27.479 see it's passing now great
00:07:29.880 so that's the general structure range
00:07:33.300 Act
00:07:34.860 and assert so here I've got acting a
00:07:37.680 certain one line
00:07:39.960 so that's the general structure of a
00:07:42.479 unit test but how do we want the unit
00:07:44.520 test to behave the first thing is when
00:07:47.280 you're trying to test something it's
00:07:48.780 good to think about the scope of what
00:07:50.460 you're testing for so what is it that
00:07:52.440 you're trying to test
00:07:54.720 number of times that I've seen tests
00:07:56.699 written in a weird way it can get quite
00:07:59.160 painful if you don't think about these
00:08:01.319 things such as making your test
00:08:02.720 deterministic so what do I mean by that
00:08:05.099 every time you run your test in the same
00:08:08.099 conditions initial starts and conditions
00:08:10.440 you should get the same result every
00:08:12.419 time so there's no sort of weird
00:08:14.660 Randomness involved because if you have
00:08:17.699 that you might get false positive
00:08:19.740 results or false negative results
00:08:22.440 test sampling is important because you
00:08:26.520 want to be able to test the right number
00:08:29.699 of values so for example if I wanted to
00:08:32.520 run like a survey across the US and they
00:08:36.959 wanted to check the number of um Russ
00:08:41.219 Johnson specific states in the US I
00:08:43.860 wouldn't just pick one data point from
00:08:46.020 one state I would pick it from and
00:08:48.120 Sample it from many many states
00:08:51.480 and you need to make sure they're well
00:08:53.160 isolated and fast as well so what
00:08:55.320 isolated means that each one of these
00:08:58.500 describe an IT blocks is can be run by
00:09:02.519 themselves without dependency
00:09:05.700 um on other tests because if you start
00:09:08.100 to make tests dependent on one another
00:09:10.459 then you will find tests failing
00:09:13.800 randomly and we don't want that to
00:09:15.600 happen
00:09:17.459 so
00:09:19.019 let's make an example then so
00:09:23.399 I'm not gonna do
00:09:25.980 task first for this one because I want
00:09:28.680 to demonstrate
00:09:30.380 how it might look if someone doesn't do
00:09:33.959 it the test first so let's say have a
00:09:36.540 time for method
00:09:38.519 that returns a certain value for a
00:09:43.980 specific time of day that I run the code
00:09:46.740 so let's say I've got
00:09:49.620 um
00:09:51.120 let's say I do time that now
00:09:55.339 uh tried to code on the Fly movie which
00:09:59.820 is not not my first language so just
00:10:03.420 bear with me
00:10:05.100 um so let's say I format it
00:10:07.620 into hours
00:10:09.300 and minutes
00:10:11.640 Okay so
00:10:14.940 oh rather smell so I'm just gonna
00:10:17.820 introduce this lunchtime constant
00:10:20.820 because it just makes it look nicer so
00:10:24.180 let's say lunch time is 12 o'clock as a
00:10:28.260 string and let's re-slap down because
00:10:30.600 it's a constant so when it's lunch time
00:10:32.700 I want it to return it's time for lunch
00:10:37.080 okay so
00:10:39.320 uh let's say
00:10:42.120 when
00:10:43.560 in a Time
00:10:45.240 uh time for dinner whatever time you eat
00:10:49.440 dinner else let's say the default case
00:10:53.100 is bubble tea time okay cool so we've
00:10:57.120 got
00:10:58.260 some code there all right so the trouble
00:11:02.700 with this is because this piece of code
00:11:05.040 is running the time that the users
00:11:09.240 running this code
00:11:11.339 there's some issues when it comes to
00:11:13.680 testing right
00:11:15.240 um I'll do an example so let's say we
00:11:18.660 describe
00:11:22.740 um time four time four methods that
00:11:25.560 we've just introduced
00:11:27.899 okay so let's say it returns
00:11:32.820 a message for lunchtime massage I don't
00:11:36.839 want that
00:11:38.180 message for lunch time and then if I try
00:11:43.079 and
00:11:43.980 I expect
00:11:46.019 that's the bubble tea time Dot
00:11:50.880 bubble tea time and then if I call time
00:11:53.880 for
00:11:55.860 to equal
00:11:58.920 time for lunch
00:12:01.100 so if I run this test
00:12:04.440 what's going to happen is I'm expecting
00:12:07.440 it to fail
00:12:10.560 oh yeah I didn't initialize a constant
00:12:12.899 so let's initialize that that was a bug
00:12:17.220 so let's say then the time is
00:12:19.980 6 p.m
00:12:22.380 or let somebody run that test
00:12:25.920 so you can see popping between the test
00:12:28.320 and the code is quite helpful all right
00:12:30.779 so you can see oh no I've got bubble tea
00:12:34.560 time instead of time for lunch this is
00:12:38.399 the scenario of the test being
00:12:40.459 non-deterministic I'm trying to run the
00:12:43.440 test but it's failing randomly just
00:12:45.839 because it happens to be
00:12:48.060 at 1723 where I am running this piece of
00:12:53.100 code so to make the test deterministic
00:12:56.940 and testable and give the same result
00:13:00.720 every time you need to be able to
00:13:02.700 control
00:13:03.959 the random elements of your code in this
00:13:06.720 case we need to be able to control time
00:13:09.660 because we know that the time could
00:13:11.459 change so for this scenario yep flaky
00:13:14.519 tests we don't want flaky tests we want
00:13:17.220 to control the test now so we need to
00:13:19.139 control time
00:13:20.700 so a good way to do that is to use
00:13:23.220 something called timecop but if you're
00:13:25.320 doing rails that's probably what the
00:13:27.300 veils um
00:13:29.399 time
00:13:31.260 freeze thing that's built in I just
00:13:33.300 forgot the name of it
00:13:35.459 so let's say
00:13:37.260 um make a new time for this test uh
00:13:40.380 let's give it the 20th of September I'm
00:13:43.260 gonna pretend that for this test I'm
00:13:45.720 running it at 12 noon
00:13:48.060 brilliant and then I can make a block
00:13:52.260 for that put this assertion so put the
00:13:56.519 code that I'm trying to run and test
00:13:58.260 inside that block and what that would do
00:14:01.380 is it would run it on the condition that
00:14:03.660 the time will always be 12 noon so if I
00:14:06.899 run the test now I thank you
00:14:09.959 we can see it passing
00:14:12.420 so you can run it again and again and it
00:14:14.940 will pass all right
00:14:17.880 so
00:14:19.019 why did I want to show that I wanted to
00:14:21.420 show that because the number of times
00:14:24.420 that I've seen in repositories where
00:14:26.760 there's flaky tests which means it's
00:14:28.500 sometimes failing sometimes passing it's
00:14:30.420 worth looking into just making sure you
00:14:32.700 know what you're testing so it's well
00:14:34.740 isolated and it can be run many many
00:14:37.260 times and still yield the same result
00:14:40.620 and what I just did then only touches
00:14:43.740 upon this topic I don't have time to
00:14:45.899 discuss test doubles in detail it'll
00:14:48.000 probably be a nice sort of future talk
00:14:50.339 but in that sense what I'm trying to do
00:14:52.860 is I'm trying to stop
00:14:55.500 so control or hard code the response of
00:14:58.920 time so instead of using the real time
00:15:02.459 I'm substituting the time object for
00:15:06.360 a different time object in my tests and
00:15:10.440 I really like this quote from Gerald my
00:15:13.500 son of us I can't pronounce the name but
00:15:16.320 it's like a stunt double in a movie
00:15:17.940 whenever you're trying to work with
00:15:20.040 dependencies inside your test it's good
00:15:22.380 to think about substituting those
00:15:25.260 dependencies for pretend objects so you
00:15:27.959 can have more control around your test
00:15:30.779 and your different scenarios
00:15:32.820 all right
00:15:34.620 so all I wanted to say is unit tests are
00:15:38.399 your friends
00:15:40.139 um please you know try and consider
00:15:42.600 using them and learning how to test
00:15:45.240 because it really helps make you
00:15:47.760 understand what you're trying to code so
00:15:50.880 what the criteria is what's your actual
00:15:53.940 and expected result and it also helps
00:15:56.940 another developer because you're
00:15:58.980 essentially writing the documentation of
00:16:01.380 your code by writing your task code so
00:16:04.380 for my example I'm new to Ruby but
00:16:07.320 because I've done testing before in C
00:16:09.060 sharp I was able to pick up repositories
00:16:12.180 just by looking at the test first and
00:16:14.940 that's been really helpful for me for my
00:16:16.920 learning
00:16:18.480 um yeah I'd like to say thank you that
00:16:21.060 was a really really quick tour of unit
00:16:23.880 testing but I hope you found something
00:16:26.300 useful there so yeah that's it you can
00:16:30.180 follow me on Twitter if you like but
00:16:32.220 I'll probably be just rambling about
00:16:34.339 random stuff let's see sharp and maybe
00:16:37.980 well learn the language all right I'm
00:16:40.380 gonna stop presenting and if you have
00:16:43.320 any questions yeah let me know you can
00:16:46.019 see
00:16:46.860 there's some comments in there like time
00:16:48.899 and date logic always great candidates
00:16:51.420 to introduce like a test random so yeah
00:16:54.120 anything to do with Randomness uh time
00:16:58.500 um and date you have to be quite careful
00:17:01.500 when you go around testing and
00:17:04.140 yeah
00:17:05.459 live coding is scary but really
00:17:08.160 appreciate all of the
00:17:10.799 um support in in the chat yeah
KD
Kim Diep
1 talk
Explore all talks recorded at WNB.rb Meetup
+21