00:00:00.199
please welcome m w shitski he's a ruby
00:00:04.560
developer and he will talk about showing
00:00:07.520
progress of background jobs with TBO
00:00:10.120
please give him a round of
00:00:16.439
applause
00:00:19.600
hello before I tell you about the topic
00:00:22.119
of my presentation I will quickly
00:00:24.160
introduce myself uh my name is m
00:00:26.840
winitzki if you like challenges you can
00:00:29.000
try to pronounce my last name but you
00:00:31.119
don't have to uh I'm a ruby developer at
00:00:35.040
visuality
00:00:36.960
uh before we start uh I like to ask you
00:00:41.000
a few questions I know that yesterday I
00:00:43.760
already made a little research what is
00:00:47.120
the state of you and what you think
00:00:50.000
about turbo hot wire uh but I have
00:00:53.680
another question how many people
00:00:56.879
completed the tutorial uh this hot r
00:01:00.320
turbo hot wire tutorial please raise
00:01:02.719
your
00:01:04.479
hand okay few of you and how many of you
00:01:08.880
want to use hot wire on
00:01:12.119
production
00:01:15.799
want lots of you want to do that that's
00:01:18.960
great uh maybe after my talk I will
00:01:21.960
motivate you to try either the Hotwire
00:01:25.840
tutorial or to start using Hotwire on
00:01:29.720
produ production we will
00:01:33.560
see and the topic of my presentation is
00:01:36.799
how to display progress of something
00:01:39.759
that is happening on the back end with
00:01:41.720
hot wire I thought that this topic is
00:01:45.119
nice because it shows a use case that
00:01:48.920
might
00:01:50.079
be that you may need at some point in
00:01:53.119
the development because sooner or later
00:01:55.880
you will have some background job and
00:01:58.640
sooner or later you will have a
00:02:00.200
background job that takes more than few
00:02:02.680
seconds and it's nice to inform users
00:02:05.600
what is the state of this job and when
00:02:08.119
it will be
00:02:09.119
finished in such case we can easily use
00:02:12.760
hot
00:02:14.720
wire but let's back let's get back to
00:02:18.560
history to 1985 and how it was done at
00:02:23.120
that
00:02:25.959
year so before if you want to achieve
00:02:29.400
that you have background job and you
00:02:31.920
want to display a progress bar for
00:02:34.239
example you need to add one more Ruby
00:02:36.959
Jem and there is a great Ruby Jam
00:02:39.239
sidekick status probably there are more
00:02:41.519
that can achieve that but this is uh to
00:02:44.879
me it's more popular and it provides a
00:02:48.239
very convenient API that you could use
00:02:51.239
but as we know from yesterday it's
00:02:53.480
against the laziness we are we are
00:02:56.480
adding one more ruby gem to our game
00:02:58.959
file and also it provides the API that
00:03:02.080
might be confusing a little if you are a
00:03:04.680
new developer or if you are new in the
00:03:06.920
project you need to go through this
00:03:09.200
documentation of this gem to read what
00:03:12.959
those methods are doing total at it
00:03:16.640
sounds like a
00:03:17.879
magic also it's getting more complex if
00:03:21.599
the job becomes more complex and if it
00:03:24.080
invokes more than one service few
00:03:27.239
Services then you need to pass this
00:03:30.560
magic methods inside those Services etc
00:03:34.200
etc so there are many challenges with
00:03:36.680
this approach it's easy at the first
00:03:39.040
glance but it's it has many
00:03:45.120
drawbacks now it works okay but we live
00:03:48.879
in a modern era of hot wire at least
00:03:51.360
those of us who are using it on
00:03:52.920
production so we can achieve the same
00:03:56.799
with hot wire and in my presentation I
00:04:00.239
won't uh tell you about the theory about
00:04:04.760
how hot wire exactly works because you
00:04:07.879
all heard about it and you know that hot
00:04:11.120
wire consists of Turbo stimulus and
00:04:14.439
Strada in my presentation you will see a
00:04:18.040
little of Turbo and a little of
00:04:21.759
stimulus but the piece of theory that we
00:04:24.600
need for the purpose of this
00:04:26.759
presentation is the fact of Broadcasting
00:04:30.080
updates happening on the back end to the
00:04:32.360
front end we could do that before
00:04:35.000
Hotwire with action cable but now we
00:04:38.639
have a really convenient API that is
00:04:41.240
doing that
00:04:43.440
easily and what rails provides to us is
00:04:47.000
a really convenient API that we can hook
00:04:51.759
to changes happening on the database
00:04:53.919
level so when something changes in the
00:04:57.400
database we can easily broadcast a
00:05:00.479
message to the front end that hey
00:05:02.560
something was
00:05:06.680
updated example
00:05:09.440
time for the purpose of this
00:05:11.479
presentation we will use a very simple
00:05:13.639
application that I created and it
00:05:15.840
consists of two models uh one of them is
00:05:19.120
a joke and another one is a joke request
00:05:21.919
so imagine you are a user that wants to
00:05:24.240
have some fun and you create a joke
00:05:27.319
request uh in there you uh you specify
00:05:31.000
how many jokes you want to fetch from
00:05:33.440
the API that provides jokes we will use
00:05:36.600
Chu noris
00:05:37.880
API because it's free and it's it has
00:05:40.720
really nice end point to provide a
00:05:42.360
random joke about Chu
00:05:44.520
Norris so the idea is that we create a
00:05:48.039
joke request and when you hit the button
00:05:51.840
submit and you specify how many jokes
00:05:54.639
you want to get there is you hit the end
00:05:57.720
point and in the end point we we spawn a
00:06:00.479
background job and the background job
00:06:03.000
downloads the jokes and we want to
00:06:05.960
display the progress of this
00:06:09.199
process also I need to mention that uh I
00:06:12.199
put some jokes that I get from this API
00:06:15.800
into my slides some of them might
00:06:19.240
be inappropriate maybe but that's not my
00:06:23.759
fault that's this
00:06:27.560
API uh so this is the plan for my
00:06:30.639
presentation actually I will show you
00:06:32.720
different ways how we can achieve that
00:06:35.599
uh maybe some Alternatives uh a little
00:06:39.280
bit of JavaScript and also I have a
00:06:42.160
bonus at the very end so let's go with
00:06:45.479
with the first
00:06:46.840
solution it's that
00:06:49.639
simple with hot
00:06:51.960
wire we can use callbacks that can
00:06:56.520
invoke methods which will broadcast the
00:06:59.199
update to the front end and for example
00:07:02.800
we have a method broadcast replace to
00:07:05.720
that says hey write to open that to the
00:07:09.479
stream that is open with the specific ID
00:07:13.160
and Target a specified HTML element on
00:07:16.840
the page and replace it with a new
00:07:19.520
element with updated
00:07:23.360
state in the invocation of this method
00:07:26.160
probably the F first argument is the
00:07:28.360
most important because it specifies the
00:07:31.440
exact stream we are writing to and
00:07:34.759
usually the biggest challenge is how to
00:07:37.120
identify those streams because when I
00:07:40.520
create a joke request and you create one
00:07:43.680
we want to broadcast the updates to the
00:07:46.720
correct
00:07:48.120
stream but in our application we are in
00:07:52.120
a good position because when you want to
00:07:54.800
fetch jokes you create a record in the
00:07:56.960
database so this jokes request record in
00:08:00.199
the database can serve the purpose of
00:08:03.120
identification of the
00:08:06.280
stream the target is the ID HTML element
00:08:10.879
on the page and then we can provide a
00:08:13.000
partial that will replace that element
00:08:15.800
and as you can guess for the progress
00:08:17.680
bar it means just render the updated
00:08:20.840
progress bar before we fetched for
00:08:23.240
example just one joke after we fetch the
00:08:26.840
the other one we just have a progress
00:08:28.960
bar that is one step
00:08:33.120
later so we did it on the active record
00:08:37.240
uh uh level let's say to make it fully
00:08:40.640
work we also need to update the front
00:08:42.680
end we modify the HTML file we need to
00:08:46.320
open the stream and you can see that the
00:08:48.720
name of the stream is exactly the same
00:08:51.279
as it was in the previous slide so we
00:08:54.519
identify that by joke request and the
00:08:57.760
name jokes progress bar and and that's
00:08:59.839
exactly the same here and we render of
00:09:02.640
course the partial with Do's progress
00:09:04.720
bar and it works it's really that simple
00:09:08.200
and it works the solution is easy it's
00:09:11.279
fast but it also has some drawbacks and
00:09:15.600
maybe one of them is that we were using
00:09:18.200
a
00:09:19.440
callback we don't like callbacks
00:09:23.200
much and it it can easily escalate into
00:09:27.920
so-called C's h especially in Legacy
00:09:30.839
applications so maybe there is
00:09:33.120
alternative approach that could be
00:09:35.760
better also what we did we bind the
00:09:40.680
logic responsible for updating something
00:09:43.440
on the front end with the active record
00:09:47.079
uh level so even if you create a record
00:09:51.160
in the database or somewhere somewhere
00:09:53.480
else this broadcast is being
00:09:56.920
invoked that's not so nice
00:10:00.600
and the most important you cannot always
00:10:03.279
use this solution if imagine that we we
00:10:07.040
are supposed to fetch jokes but we don't
00:10:09.640
save them into database in such case we
00:10:13.040
we don't have a callback that we can
00:10:14.839
hook on the active record model so let's
00:10:18.519
search for the Alternatives the second
00:10:20.880
solution which is a little
00:10:25.000
better does almost exactly the
00:10:28.320
same but we invoke the broadcast from
00:10:32.079
the level of the service or the job that
00:10:34.959
is doing those actions so we are not
00:10:39.000
doing that in the active record model we
00:10:41.720
are doing that in the service for
00:10:43.480
example and we can use exactly the same
00:10:45.920
method we need to just specify on what
00:10:48.279
class it's invoked it's turbo streams
00:10:51.000
Channel but all other arguments are
00:10:53.720
almost the same since we are independent
00:10:56.639
from the active record we also need to
00:10:59.320
update the partial a little to provide
00:11:02.120
the input it needs now it receives not
00:11:04.800
the active record um model now it
00:11:08.880
receives the number of the pro actual
00:11:11.920
progress and the end stage of the
00:11:16.279
progress and modification for the front
00:11:19.000
end we actually just modify the state of
00:11:22.800
the
00:11:23.880
partial uh the the input of the partial
00:11:26.959
the stream stays the same
00:11:30.920
and it works so we have no call backs
00:11:34.519
this solution is easy to test because we
00:11:36.800
just have one specific uh place that
00:11:40.079
makes a broadcast it's easy to
00:11:43.600
identify
00:11:45.360
and this is a huge Advantage especially
00:11:48.600
on the production when when you not when
00:11:52.079
you just not have one broadcast but 10
00:11:54.959
places or 100 places it's really
00:11:58.240
convenient to easily find the places
00:12:01.000
that that are actually doing those
00:12:03.320
broadcasts it's it will be more
00:12:06.160
convenient to debug any problems if you
00:12:08.279
have
00:12:09.399
them but it looks to be not the default
00:12:12.399
way uh I asked you about this uh hot
00:12:15.560
rails tutorial and as I remember there
00:12:18.639
is the default way of adding broadcasts
00:12:21.959
in this tutorial is uh using the active
00:12:24.600
record approach so maybe it's not the
00:12:27.440
most typical approach there and also we
00:12:31.800
still need to identify those
00:12:35.040
streams we are in a really good position
00:12:37.800
that we are still we still have this
00:12:40.399
jokes request object in the database and
00:12:43.880
we are using it for identification but
00:12:47.760
if the requirements are changed a little
00:12:50.600
and we say that hey don't touch anything
00:12:53.320
in the database please provide me an
00:12:55.480
endpoint that fetches the specific
00:12:58.560
number of jobs
00:13:00.800
then identifying uh this stream
00:13:03.920
correctly will be much more challenging
00:13:06.760
most probably we will need to create
00:13:08.720
some ID or U ID on the Fly and pass it
00:13:12.639
through all the methods to have it in
00:13:15.199
the background worker and also have it
00:13:17.680
on the front end maybe store it in the
00:13:23.800
URL so that was the second
00:13:26.720
solution we have also the third solution
00:13:29.360
solution which is actually not a
00:13:31.320
background
00:13:34.360
worker you can have a different approach
00:13:37.120
that will work for the requirement that
00:13:39.600
I just U mentioned if you don't want to
00:13:42.519
touch database at all you could use and
00:13:46.600
if you are able to get rid of the
00:13:49.360
background jobs you can use a turbo
00:13:52.240
frame tag and you can say that hey when
00:13:55.959
you are on this
00:13:57.320
page go uh add a source attribute to the
00:14:01.920
turbo frame tag and when the user enters
00:14:05.120
this page the turbo automatically hits
00:14:07.800
the end point behind this
00:14:11.160
source and the endpoint can just fetch
00:14:14.880
jokes that would also work it would be
00:14:19.040
even um if you want you can do it even
00:14:24.079
asynchronously and you can you can add
00:14:26.959
additional attribute lazy loading
00:14:30.040
which will make the turbo make this uh
00:14:34.279
heat to the end point only at the moment
00:14:36.720
when user enters this specific place at
00:14:39.759
the page so if it's on the bottom of the
00:14:42.560
P at the page user will need to scroll
00:14:45.839
down to actually see those
00:14:51.519
results in this uh last scenario there
00:14:55.519
is no background processing there is no
00:14:58.560
database
00:15:00.399
and well it's to be honest it's
00:15:03.079
completely different solution so it
00:15:05.279
depends on your requirements if you need
00:15:07.440
this solution with background jobs or
00:15:09.519
this alternative
00:15:16.120
approach okay now probably you are
00:15:21.839
wondering what if we have more than just
00:15:25.800
one job so we were talking about
00:15:29.199
background jobs and it's easy to imagine
00:15:32.959
that they escalate quickly if you want
00:15:36.079
to fetch 100 jokes maybe it will be a
00:15:39.079
nice feature to make it parallel maybe
00:15:42.040
let's say we will download one job will
00:15:45.279
download 25 jokes so for the bigger
00:15:48.279
number we will just spawn more
00:15:50.920
jobs but the problem is that those jobs
00:15:54.480
are running at the same time and they
00:15:56.839
will try to update the progress bar at
00:15:59.480
the same time so there will be
00:16:02.519
stuttering and most probably progress
00:16:05.000
bar won't finish at
00:16:09.040
100% how to solve this issue if we get
00:16:13.120
one step back and we look at the
00:16:16.319
previous
00:16:17.959
Solutions we had a very simple
00:16:21.120
design we had the state of the front end
00:16:24.199
and when something happened on the back
00:16:26.160
end the back end decided that well let's
00:16:29.959
update the front end and the method
00:16:32.639
broadcast replay to was doing this
00:16:35.480
update the back end was responsible for
00:16:38.040
that to make it work with more
00:16:40.880
background jobs we need to shift our
00:16:44.399
thinking and we need to give more power
00:16:47.240
to front end to
00:16:49.560
JavaScript so the solution in this case
00:16:53.360
is to use a mediator stimulus controller
00:16:57.040
that will take care of those updates
00:16:59.519
and the stimulus controller will decide
00:17:02.199
if you need to update the progress bar
00:17:04.079
or not and how to update it so basically
00:17:09.079
background background jobs are telling
00:17:11.760
that hey there is an update update
00:17:13.959
update but the stimulus controller will
00:17:16.520
make sure that this update of progress
00:17:18.520
bar is
00:17:24.559
smooth lots of code
00:17:30.120
I'm not sure if you want me to explain
00:17:32.880
everything what is happening if you want
00:17:34.960
to use stimulus maybe now I will ask you
00:17:37.120
a question uh who has ever used
00:17:41.720
stimulus okay so like more than half of
00:17:45.280
the room is already familiar with that
00:17:47.440
so you know what this magic does you
00:17:49.799
need to provide a data controller you
00:17:52.760
can provide some values as the input for
00:17:55.159
the controller you can have a Target
00:17:58.640
that that uh will be used for targeting
00:18:01.440
HTML
00:18:03.520
elements so the key point is that you
00:18:06.120
need to modify the current HTML
00:18:09.280
structure and use the convention
00:18:11.520
provided by stimulus to provide the
00:18:13.679
needed data by
00:18:15.880
stimulus and now I will show you some
00:18:19.679
JavaScript not sure if you saw this
00:18:22.039
sticker it's it's from the men's
00:18:25.720
bathroom but but that's how I feel when
00:18:28.840
I need to write some
00:18:32.720
JavaScript so adding stimulus means we
00:18:36.440
create a new stimulus controller and
00:18:39.240
stimulus controller is a JavaScript code
00:18:42.360
that will be invoked on a specific Pages
00:18:45.320
we know on which Pages because we
00:18:47.200
provided the data controller
00:18:50.600
attribute and we are using those
00:18:52.919
attributes that we specified in HTML
00:18:56.039
values and targets to for for a
00:18:59.360
definition for the controller but what
00:19:02.000
is the most important is the connect
00:19:05.159
function this is the function that is
00:19:07.200
invoked when the page is loaded and the
00:19:10.360
JavaScript from the controller that is
00:19:12.520
used on the contol on the on this page
00:19:15.280
is
00:19:16.440
invoked so what our connect uh function
00:19:20.760
needs to do we will use quite a smart
00:19:25.120
trick here and we will overwrite the
00:19:28.400
exist existing turbo action there is
00:19:31.760
before stream render
00:19:34.120
event which we can
00:19:36.799
overwrite we are basically telling hey
00:19:39.679
when you want to add a new joke don't do
00:19:42.320
it and there is a if condition that if
00:19:47.320
you want to add this exactly joke
00:19:50.200
element to the specific Target then
00:19:54.080
let's run some custom code and the
00:19:57.320
custom code you will see in moment is
00:19:59.679
responsible for updating the progress
00:20:01.640
bar but then we can fall back to the
00:20:04.120
default which is just rendering a new
00:20:06.400
element on the page the
00:20:09.919
joke and what happens in our custom
00:20:13.679
function we increment the state of the
00:20:17.200
progress bar we have the internal value
00:20:20.039
that represents the actual state of a
00:20:22.200
progress bar we update the HTML element
00:20:25.880
and we can update also other elements we
00:20:28.080
can have also the count of the
00:20:32.159
jokes and it should
00:20:36.440
work here I don't have pros and cons I
00:20:40.039
just have a feeling and this fault that
00:20:43.520
when you need to do something more
00:20:45.120
complex on the front end you usually
00:20:47.840
don't have a choice you need to give
00:20:50.799
some give more power to JavaScript in
00:20:53.840
our case to
00:20:55.919
stimulus but maybe
00:20:58.799
something uh that that is worth to
00:21:01.159
mention here when you see that this
00:21:04.679
moment comes it's better to do it
00:21:07.120
quickly like when you see that back end
00:21:10.279
is not enough to handle this complicated
00:21:13.559
logic it's better to well say the truth
00:21:17.080
and use this JavaScript and
00:21:19.840
stimulus I have also a bonus because I
00:21:22.640
have few more
00:21:24.720
minutes um when I Was preparing the the
00:21:28.840
this test application I thought well is
00:21:32.320
it possible also to make it work with
00:21:36.080
pagination is it possible that we have
00:21:39.279
this updating from the background but we
00:21:41.480
also render the pagination and we switch
00:21:44.039
to new pages when we need to right
00:21:46.960
because we don't want to uh render 100
00:21:50.159
jokes on one page and of course it is
00:21:53.360
possible uh the most challenging part is
00:21:56.240
to figure out where when we need to
00:21:59.600
switch to the new page and it's easy if
00:22:02.720
we have 10 jokes on one page on the 11
00:22:05.480
11th joke we need to clear the existing
00:22:09.120
page we need to add new joke and we need
00:22:12.240
to reender the pagination element if not
00:22:15.840
we just add a new
00:22:18.840
joke and in Turbo or in my solution
00:22:23.679
clearing the page means let's render the
00:22:27.039
GRE that consists of the job but it's
00:22:29.799
empty
00:22:32.080
and the most complex things thing here
00:22:35.320
most probably is R rendering the
00:22:37.679
pagination but it's also not so not so
00:22:42.039
complex because this is what we see
00:22:44.200
usually in the controller in
00:22:47.640
our daily uh end points that display
00:22:50.720
many records right so that's the same
00:22:53.279
pagination it's just updated and we can
00:22:56.279
render that if we have a partial for
00:22:58.240
that
00:23:03.039
oh let's sum up so
00:23:06.520
yesterday yesterday we heard that rails
00:23:09.159
is Magic and to me the hot wire is
00:23:14.080
really significant part of this
00:23:16.679
magic and in my presentation um to be
00:23:21.520
honest I didn't want you to just give
00:23:23.600
you a solution how to solve exactly this
00:23:26.279
problem of the progress bar with wire I
00:23:29.760
wanted to give you a taste of using a
00:23:32.279
hot wire and you can see that if you
00:23:35.360
want to use hot wire you need to have
00:23:38.279
slightly different approach how the back
00:23:41.720
back end needs to communicate to front
00:23:43.720
end or how to divide responsibilities
00:23:46.400
between back end and the front end but
00:23:49.200
it's nice so give it a
00:23:54.080
try if you want there are some links
00:23:57.400
that describe this solution and the uh
00:24:00.960
the solutions that are similar to that
00:24:04.279
also if you want to look at the
00:24:06.480
repository there is a
00:24:11.640
link
00:24:13.919
oh
00:24:17.760
okay
00:24:22.240
okay so now is thank you thank you
00:24:34.600
thank um there are any questions I
00:24:37.480
assume you're available now to have a
00:24:39.679
chat maybe you can uh come here and then
00:24:43.360
talk to M if there any questions and we
00:24:46.440
continue with the next talk at 2:30 and
00:24:50.320
I think we're going to learn about
00:24:51.919
Anonymous
00:24:54.360
databases okay thank you
00:24:59.520
we we do
00:25:00.600
the we do the Q&A just um over here so
00:25:04.880
mik will will stay here and then
00:25:10.720
um and there's a speaker gift of course