00:00:19.039
so i'm very happy to be proven wrong
00:00:20.480
that i can see all of you from here
00:00:23.439
uh
00:00:24.480
so yeah my name is tanya i'm a developer
00:00:26.640
at southworks
00:00:27.920
based here in singapore
00:00:29.519
the singaporeans among you may be able
00:00:31.439
to tell from my accent that i'm not
00:00:32.640
really from here but singapore is my
00:00:34.079
second home
00:00:35.920
a bit of a plug
00:00:37.280
i co-organize rail skills in singapore
00:00:40.079
and we have an event coming out tomorrow
00:00:42.079
morning at 9 30
00:00:44.800
and we have a coach reaping at nine so
00:00:47.760
i'm hoping to see some of you there
00:00:49.360
tomorrow
00:00:52.320
so today i'm going to talk about
00:00:54.079
collaborating with contracts and let's
00:00:56.239
start with real life collaboration
00:00:59.120
when two parties collaborate in real
00:01:00.719
life what do they do
00:01:03.120
i think there are two possibilities
00:01:05.519
one they can just do it
00:01:08.400
and this can happen if the two parties
00:01:11.119
trust each other already
00:01:13.760
but this depends on goodwill because
00:01:15.920
the obligations of both parties are not
00:01:18.000
explicitly stated anywhere
00:01:20.400
or they can negotiate a contract and
00:01:22.640
adhere to the contract
00:01:24.479
this can be tedious at the beginning but
00:01:26.560
once a contract is there
00:01:28.479
both parties can refer to it and
00:01:31.520
when one of them don't fulfill any
00:01:33.200
obligation the other party knows exactly
00:01:35.920
what that is
00:01:38.799
so this is an example of a contract in
00:01:40.560
real life we have let's say a
00:01:42.560
supermarket who wants to get some some
00:01:45.200
ice cream from some supplier
00:01:47.360
so the contract will specify obligations
00:01:49.200
on both sides in this case for the
00:01:51.280
consumer that the consumer has to order
00:01:52.960
a week in advance and that it has to pay
00:01:55.520
at the time of order
00:01:57.840
and on the provided side it has
00:01:59.920
it specifies that the provider has to
00:02:01.520
deliver within three business days and
00:02:03.520
that the product must not expire within
00:02:04.960
two months
00:02:07.280
that's all i have about real life now
00:02:08.879
moving along to not real life
00:02:12.480
uh when i mentioned collaboration i
00:02:14.720
meant it in a context of services
00:02:17.120
can i get a show of hands who has had to
00:02:19.040
deal with services whether consuming it
00:02:21.200
or providing it
00:02:24.000
right quite a few
00:02:27.920
so just to refresh our minds a little
00:02:29.520
bit what is the service
00:02:32.160
a service is a reusable software
00:02:33.920
component encapsulating a business
00:02:35.599
function
00:02:37.360
and it can be exposed over http q or any
00:02:39.920
other transfer protocol
00:02:43.200
for every service there can be
00:02:45.440
there's a single provider and one or
00:02:47.200
many consumers
00:02:49.120
and as a developer you may be
00:02:50.480
responsible for all of this or some or
00:02:52.480
just one piece
00:02:56.160
then there's a term service contracts or
00:02:58.720
contracts when it comes to services
00:03:01.280
at the most basic level
00:03:03.280
this can contain just an expectation
00:03:05.440
about request and response
00:03:08.480
so and they are usually written from the
00:03:10.400
point of view of the provider so the
00:03:12.400
request will be the request that the
00:03:13.920
provider understands and the response is
00:03:16.319
the response that the provider sends
00:03:17.840
back
00:03:18.560
to all consumers regardless of what they
00:03:20.400
actually need
00:03:23.840
service contracts can also
00:03:25.920
have other things for example
00:03:27.200
performance characteristics
00:03:28.959
for example that their
00:03:30.720
the provider has to return a response
00:03:32.640
within 100 milliseconds under certain
00:03:34.560
kind of load
00:03:36.239
but for today we'll focus on request and
00:03:38.159
response
00:03:41.040
just to visualize this a bit better
00:03:43.120
in this person service example
00:03:45.440
the response part of the contract will
00:03:47.200
specify these three fields
00:03:49.760
id name and age regardless of what the
00:03:52.319
consumers actually need
00:03:57.040
at my workplace thoughtworks i've been
00:03:58.959
involved in a ruby project for over a
00:04:00.400
year now and one of the things i learned
00:04:03.040
is contract test i fiddle around quite a
00:04:05.599
bit with it and i hope to share some
00:04:07.200
cases where it might be useful
00:04:10.239
so with that let's go to the first one
00:04:16.799
uh coming back to the person service
00:04:18.320
example let's say that
00:04:20.320
the provider again returns three fields
00:04:22.400
id name and h
00:04:24.160
and there are two consumers consumer a
00:04:26.320
only cares actually about two fields id
00:04:28.560
and h
00:04:29.840
the fact that the provider returns
00:04:31.120
another failed name
00:04:32.800
it doesn't really care about that
00:04:34.639
and then there's another consumer
00:04:36.160
consumer b that expects also two fields
00:04:39.199
but a different one id and name
00:04:43.680
and then let's assume that there is
00:04:45.120
another consumer coming in
00:04:47.120
which expects first name and last name
00:04:50.960
well obviously now to please consumer c
00:04:53.280
uh the provider has to re has to send
00:04:55.360
over first name and last name
00:04:59.520
let's suppose that the team in charge a
00:05:01.280
provider of the provider just updates
00:05:04.400
just removes the field name and replaces
00:05:06.479
it with first name and last name
00:05:08.400
what will happen
00:05:11.840
uh consumers a and b will break because
00:05:14.880
um
00:05:15.759
this
00:05:16.880
actually consumer b o a break because it
00:05:18.960
is the only one that expects uh first
00:05:20.720
name and last name
00:05:22.240
so that's not great um of course the
00:05:25.039
provider team can
00:05:26.639
could have they could have kept name and
00:05:28.400
just introduced the two new fields on
00:05:29.840
top of the existing ones
00:05:31.919
but suppose they really do have to
00:05:33.440
remove name
00:05:34.720
will they at least know that the
00:05:35.919
consumers a and b will break before the
00:05:38.479
consumer the consumers themselves
00:05:40.320
complain
00:05:42.800
they will not if the provider doesn't
00:05:44.800
even know what the consumers a and b
00:05:46.800
expect from it
00:05:48.240
but it will if the expectations are
00:05:50.880
explicit and known to them
00:05:54.400
so that is what consumer-driven
00:05:55.919
contracts aim to help with
00:05:58.160
consumer-driven contracts make consumer
00:06:00.240
expectations explicit
00:06:02.320
unlike service contracts which are
00:06:03.759
written from the provider's side
00:06:06.000
or with the provider in mind and so many
00:06:08.319
different contracts are written from the
00:06:10.000
point of view of the consumer
00:06:12.400
so
00:06:13.280
we will have one consumer different
00:06:14.960
contract per consumer
00:06:17.039
in this case contract a will specify
00:06:18.800
that consumer a needs
00:06:20.639
id and h to be returned from the
00:06:22.240
provider
00:06:23.440
contract b will specify that consumer b
00:06:25.520
is id and name and so on
00:06:27.919
so taken individually
00:06:29.919
the consumer driven contract is not
00:06:32.000
exhaustive
00:06:33.440
but if we sum all of them up
00:06:36.240
then the brother the provider can find
00:06:37.919
out
00:06:38.720
what it needs to return to satisfy all
00:06:41.039
of the consumers
00:06:42.800
so in this case if we sum up contract a
00:06:45.120
b and c
00:06:46.160
we will find out that the provider has
00:06:47.759
to return
00:06:49.039
id h name first name and last name
00:06:54.560
why is this useful
00:06:57.520
like any tool services are only useful
00:06:59.759
when they are actually used and so the
00:07:01.440
consumer has to know how to deal with
00:07:03.599
the response that the provider returns
00:07:05.680
otherwise it's not really going to be
00:07:07.840
used
00:07:09.440
and the second point is that consumer
00:07:11.680
different contracts make consumer
00:07:13.759
expectations explicit
00:07:15.680
so when something breaks
00:07:18.639
you can find out which particular
00:07:20.080
consumer is affected
00:07:21.759
how and how to fix it
00:07:27.199
going one step further there is also
00:07:29.360
consumer-driven contract test it's quite
00:07:31.919
a mouthful
00:07:33.199
but basically they have
00:07:35.199
like any other test
00:07:36.720
they have the goodness of consumer
00:07:38.000
driven contracts and more
00:07:40.400
they are executable
00:07:42.720
so we don't have to refer to some long
00:07:44.160
document just just to uh just execute
00:07:46.720
something
00:07:47.919
and they can be run as part of the build
00:07:50.000
pipeline so we can get quick and
00:07:51.520
frequent feedback
00:07:54.400
also
00:07:55.360
we know how word documents become stale
00:07:57.840
but this is a test and so as long as we
00:08:00.400
run them regularly we are kind of forced
00:08:02.160
to update it and so it becomes a living
00:08:04.800
documentation of source of the system
00:08:10.160
here is an example of when it is useful
00:08:13.280
let's assume that there is an expensive
00:08:14.879
call or computation that the provider
00:08:17.120
needs to do to provide the field friends
00:08:20.879
but none of the consumer driven contract
00:08:22.639
tests break when we remove that field
00:08:28.240
so that means that tells us that it is
00:08:30.080
safe to remove
00:08:31.440
if not because of the consumer different
00:08:32.959
contract test
00:08:34.320
it will be harder to detect this so we
00:08:36.479
have to let's say go through some
00:08:37.599
20-page document to find out that is not
00:08:39.440
used and then remove it
00:08:44.800
the other thing about consumer driven
00:08:46.160
contract test is that it plays well with
00:08:48.160
tdd
00:08:50.080
and here's a workflow
00:08:52.240
consumer c is the one that's new so we
00:08:54.160
write consumer driven contract tests for
00:08:56.000
consumer c
00:08:58.240
um and at first they will fail
00:09:00.399
it will fail because consumer c at first
00:09:02.480
will not even make the right request to
00:09:04.640
the provider so that that's exactly what
00:09:06.959
we need to do to make it pass
00:09:09.040
this is the normal red green effector
00:09:10.640
cycle that we have with tdd
00:09:12.880
so after that we can refactor and that's
00:09:14.480
the that's what we have to do on the
00:09:16.160
consumer side
00:09:18.240
and then we move over to the provider
00:09:19.920
side and we have the same cycle red
00:09:21.920
green effector it will be red at first
00:09:24.160
because the provider when given the
00:09:26.320
response that's specified under contract
00:09:28.640
will not return what is expected
00:09:32.320
and so to make it green we have to make
00:09:34.080
the provider return what that particular
00:09:36.000
consumer expects
00:09:37.839
and then we refactor
00:09:40.480
the last part that's really important is
00:09:43.200
that we have to make sure that the other
00:09:44.720
consumer-driven contract test
00:09:46.720
for the other consumers
00:09:48.560
still passes as well because we want to
00:09:50.480
make sure that all the existing
00:09:52.000
consumers still work
00:09:55.760
so that is the first use case and to tie
00:09:58.160
this back up this is all about
00:10:00.480
making consumer expectations explicit to
00:10:03.200
the provider
00:10:06.480
so moving forward to the second one
00:10:10.640
whenever we have service provider and
00:10:12.560
consumer communicating with each other
00:10:15.120
to make unit tests more deterministic we
00:10:17.200
often introduce
00:10:18.880
we often test around the service
00:10:20.399
boundary
00:10:21.519
so on the consumer side we often have a
00:10:23.600
provider stop
00:10:25.040
for example using something like webmock
00:10:29.760
so then on the unit test for the
00:10:31.440
consumer side you will use the provider
00:10:33.360
stub and it will verify that the
00:10:35.279
consumer processes the stock response
00:10:37.440
properly
00:10:40.240
moving on to the provider's side the
00:10:42.240
provider side will typically have some
00:10:43.760
kind of request from some fake consumer
00:10:46.240
and then the test will verify that the
00:10:48.480
provider returns the right response
00:10:50.320
given their request
00:10:53.360
usually at the beginning everything will
00:10:55.200
be good
00:10:56.399
but after a while there could be a
00:10:57.760
mismatch
00:10:58.880
for example here the consumer expects
00:11:01.440
address
00:11:03.440
and the provider stop returns that
00:11:06.880
but the provider site itself is not
00:11:08.880
updated and so the unit test and also
00:11:11.839
the unit test for the provider doesn't
00:11:13.920
even know that this is happening so
00:11:15.839
everything seems okay from the
00:11:17.040
provider's point of view the unit test
00:11:18.959
for both sides will continue to pass
00:11:22.000
but the application may actually blow up
00:11:24.320
because of this mismatch
00:11:27.600
so how can contract test help
00:11:30.560
it can help by verifying both sides
00:11:33.040
against the same contract
00:11:35.200
thereby making sure that there is no
00:11:36.560
mismatch
00:11:38.959
so one way this could look is we write
00:11:42.480
the we write expectations from the
00:11:44.000
consumer side again this is putting some
00:11:46.240
emphasis on the consumer side
00:11:48.160
and from there we generate the contract
00:11:51.279
and then we verify that the provider
00:11:53.120
adheres to the same contract
00:11:57.519
to tie this back up this case is all
00:11:59.440
about making sure that the provider and
00:12:01.200
consumer do their job and not just think
00:12:03.839
they do their job
00:12:05.279
making sure that they can really
00:12:06.320
communicate with each other
00:12:10.240
so moving on to case three
00:12:13.600
um in my team we have had to keep two
00:12:16.079
systems in sync via services
00:12:19.600
and one end is handled by our team and
00:12:21.839
another is handled by a separate team
00:12:24.079
for some business functions we are in
00:12:25.920
charge of the providers and for some
00:12:27.440
others we are in charge of the consumers
00:12:31.600
we were starting from scratch so at the
00:12:33.519
beginning there was nothing
00:12:35.760
none of the services have been built and
00:12:38.240
there was no schema no interface
00:12:41.279
so we started with that we started with
00:12:43.279
deciding on the service contract
00:12:44.639
specifically the interface and the
00:12:46.240
schema for the request and the response
00:12:49.839
but we didn't want that to just be a
00:12:51.519
document that will go stale
00:12:53.600
so we wanted an executable contract or
00:12:55.680
effectively contract test
00:12:58.959
with an executable contract or contract
00:13:00.959
test the consumer team can go off on
00:13:03.200
their own and when they are ready they
00:13:04.880
can just execute the contract test and
00:13:07.040
make sure they adhere to it
00:13:09.040
and the same goes for the provider side
00:13:11.680
the provider team can go off on their
00:13:13.360
own and when they are ready they can
00:13:14.800
execute a contract test and if it passes
00:13:17.200
that means they are done
00:13:20.480
so with that both teams can develop
00:13:22.399
their site independently they don't they
00:13:23.920
don't have to wait for each other
00:13:26.399
and this is particularly useful because
00:13:28.160
we had a lot of services and so we don't
00:13:30.399
have to think hey let's do this one
00:13:31.920
first
00:13:32.720
and then let's do this other one
00:13:35.279
um
00:13:36.240
so if those tests pass on both the
00:13:38.399
consumer and the providers side
00:13:40.240
we know that both sides will be able to
00:13:42.000
communicate but of course after that we
00:13:44.480
still have to do
00:13:45.760
integration testing
00:13:49.680
so this case was all about enabling
00:13:51.680
parallel development
00:13:54.720
with that i'll go on to introduce you to
00:13:56.639
a gem that we have been using to write
00:13:58.320
our contract test it's called pact and
00:14:00.880
it is designed to write consumer driven
00:14:02.639
contract tests
00:14:06.720
the packed workflow goes like this and
00:14:08.880
i'll have to close over some of the
00:14:10.240
details
00:14:13.519
first we write tests for the consumer
00:14:15.120
site
00:14:16.480
and
00:14:17.920
we specify the
00:14:19.839
the service that we are testing that
00:14:21.279
will be that will have to be configured
00:14:22.720
beforehand
00:14:24.160
and then in the given when then style we
00:14:26.800
specify given a person with too many
00:14:29.120
friends for example
00:14:30.720
upon receiving requests for person
00:14:32.800
details so this is just some description
00:14:36.720
and then we specify the request
00:14:38.480
specification
00:14:40.560
so this is what we expect the request to
00:14:42.160
be like
00:14:43.600
the details would be things like http
00:14:45.600
method the path the query params form
00:14:48.240
params and header
00:14:50.959
this will later be used to verify that
00:14:52.800
the consumer makes the right call and
00:14:54.639
also this is the call that the provider
00:14:56.639
must understand
00:14:59.279
and then we move on to specifying the
00:15:01.680
providers the response
00:15:03.920
similarly there are
00:15:05.440
things like cp status and headers and
00:15:07.680
then there's the body
00:15:10.800
the other thing is that pect runs a mock
00:15:13.040
server when you run the test
00:15:15.199
and the mock server will return this
00:15:16.800
exact response when given the request
00:15:19.040
that matches the request that we
00:15:20.639
specified just before
00:15:25.279
and then on the next step we call the
00:15:27.279
method on our on the app
00:15:29.440
on the consumer side that makes the
00:15:31.040
actual http request
00:15:32.800
and we verify that it it processes the
00:15:35.199
response from the provider or from the
00:15:37.600
mock provider properly
00:15:41.360
so we run it
00:15:43.839
and we will see it fail at first again
00:15:46.399
because the consumer
00:15:48.240
doesn't even make the right http call
00:15:50.560
right now probably not not any http call
00:15:53.040
at all so
00:15:55.839
um at this stage
00:15:57.519
to make it past we just need to make
00:15:58.880
that http call and process it
00:16:01.759
and we don't need to care whether the
00:16:03.279
provider is doing the right thing or
00:16:04.800
whether it has even been implemented
00:16:06.800
because that will be tested separately
00:16:12.320
when we run the consumer site test
00:16:15.839
it will generate a pact or a json
00:16:18.160
representation
00:16:19.440
of the contract containing all the
00:16:21.519
service all the
00:16:23.199
requests and response specifications
00:16:26.639
the packed file will then need to be
00:16:28.079
shared over to the provider
00:16:30.720
this can be done either on your own or
00:16:33.199
using the tools that pack tests
00:16:35.120
pact actually has a suite of tools one
00:16:36.880
of them is called pack broker which also
00:16:38.959
handles things like versioning
00:16:43.839
so then
00:16:45.839
we go to the provider's side and we have
00:16:47.920
to verify that the provider follows the
00:16:49.759
pact or
00:16:50.800
the contract and we do that just by
00:16:53.600
doing a rate packed verify
00:16:56.000
this fires actual http requests to the
00:16:58.320
provider and so it exercises the full
00:17:00.639
stack of the provider from the
00:17:02.079
controller the serializer the service
00:17:04.400
the model the whole thing so it is
00:17:06.480
exactly the same way that your actual
00:17:08.240
consumer will exercise that
00:17:12.799
so a bit more unpacked
00:17:15.439
it also comes with an out of process or
00:17:18.000
standalone stub so
00:17:20.400
we saw earlier that you can you can run
00:17:22.799
your unit test against the stop
00:17:25.679
but with this auto process tab you can
00:17:27.600
also run your actual consumer let's say
00:17:29.520
a single page application against this
00:17:31.760
provider stub and you know that whatever
00:17:34.480
response you get from here will be the
00:17:36.000
exact one that your actual provider will
00:17:38.160
return
00:17:42.320
pact also has something called provider
00:17:44.400
states so we saw this earlier given
00:17:47.280
a person with too many friends
00:17:49.919
this is actually a provided state and
00:17:51.760
what it gives us is
00:17:54.240
the ability to set up and tear down data
00:17:56.559
for that particular provider state
00:17:59.200
so this data setup will be run on the
00:18:02.320
provider's side just before the request
00:18:04.799
is
00:18:05.600
processed by the provider so in this
00:18:07.840
case i'm just inserting 10 000 friends
00:18:10.799
to the person
00:18:15.280
by default peg tries to match the actual
00:18:18.000
response from the provider with the one
00:18:19.840
that is specified and packed exactly so
00:18:22.480
type and value both
00:18:24.880
but they it is also possible to just
00:18:27.520
verify on type using something like this
00:18:30.400
using actually
00:18:31.840
something like
00:18:33.360
um and also it is possible to specify a
00:18:36.000
regex pattern and and match based on
00:18:38.559
that pattern
00:18:42.799
to sum up we
00:18:44.640
we went through some use cases
00:18:47.280
first being
00:18:48.720
making consumer expectations explicit
00:18:51.520
making sure both the consumer and the
00:18:53.360
provider do their job
00:18:55.600
and enabling parallel development
00:18:58.000
and we also covered pact
00:19:00.400
with this of course the devil is in the
00:19:02.000
details for example and how to integrate
00:19:04.400
it with your existing test
00:19:06.320
and also
00:19:07.520
how much to assert on the response
00:19:10.080
because each code path on your consumer
00:19:11.840
side may have different expectations
00:19:15.600
and your my your mileage may vary as
00:19:17.600
well because
00:19:19.280
let's say if there's only one consumer
00:19:21.360
for your service then making consumer
00:19:23.760
expectations explicit is probably not
00:19:25.600
going to help that much
00:19:27.440
it also depends on how much influence
00:19:29.039
you have on the provided the team in
00:19:31.200
charge of the provider and consumer
00:19:33.120
because you need to assert some kind of
00:19:34.559
influence for them to actually do this
00:19:36.559
because this has to be done on both
00:19:37.919
sides
00:19:40.160
so these are not silver bullets but i
00:19:42.400
hope i've broke some thoughts in your
00:19:44.000
head to start considering and start
00:19:45.520
thinking about contract tests when you
00:19:46.960
see collaboration using services
00:19:50.799
and like other tests
00:19:52.559
add them in your build pipeline for the
00:19:53.919
quick and frequent feedback
00:19:56.480
that's all i have for today
00:20:05.600
thank you so much tanya do you guys have
00:20:07.120
any questions for her
00:20:09.760
yeah okay
00:20:12.799
hi there uh my name is zach
00:20:14.960
so um at the company i kind of work for
00:20:17.760
we're using kind of rails
00:20:19.440
um almost to build a big monolith to
00:20:21.679
pretty quickly explore the kind of
00:20:23.200
problem space that we're in
00:20:24.960
and very quickly once we understand it
00:20:26.960
we're kind of thinking about carbon
00:20:28.080
weight services
00:20:29.760
and maybe using more exotic languages
00:20:31.440
where we can kind of squeeze more out of
00:20:32.720
the boxes so one of the problems we have
00:20:35.280
is as soon as we start doing that we'll
00:20:36.559
need to sort of define these contracts
00:20:39.200
but we'll need to do it in kind of a
00:20:40.480
more language agnostic way that's not
00:20:42.559
necessarily bonded to ruby or rails or
00:20:44.720
anything like that
00:20:46.559
are there any sort of more general
00:20:47.760
solutions that aren't so
00:20:49.360
like pact is pretty glued to ruby but uh
00:20:51.679
i mean ideally we'd want something where
00:20:53.840
we can kind of define a json
00:20:55.120
specification that would work on
00:20:57.840
you know whatever situation we'd want so
00:21:00.000
i'm kind of curious about that
00:21:02.400
okay uh so pac actually as i mentioned
00:21:05.280
it has a suite of tools not just this
00:21:07.360
gem and not just pack broker it also has
00:21:09.919
um
00:21:11.200
an equivalent to ratepack verify but for
00:21:13.840
providers they are not written in ruby
00:21:16.000
so there are tools around pact that are
00:21:18.880
specifically for this but having said
00:21:20.799
that there is another tool which i
00:21:22.080
haven't used it's called pecto it's very
00:21:25.039
similar it's just an o at the end
00:21:28.799
and that one also um
00:21:31.360
supports non
00:21:33.039
uh non-ruby service consumers and
00:21:34.960
providers so those two things um are the
00:21:37.360
ones that you can check out
00:21:39.760
thanks a lot
00:21:41.600
all right if you do you can find tanya
00:21:43.360
during the break and let's thank her
00:21:45.200
again for coming out
00:21:46.720
thank you
00:21:59.440
you