00:00:15.320
hi so um today I'm talking about web transport um a new uh protocol over the web that you might think of um and in
00:00:22.119
shorthand as websocket 2.0 um by not calling it websocket they're able to let go of the Legacy and uh try to uh uh
00:00:29.400
paint a new picture for the web um in terms of bidirectional um unidirectional uh streaming um as well as datagrams and
00:00:36.160
that sort of thing which we'll talk about um we'll also look at uh websockets and ssse which is the current
00:00:42.160
state of streaming in Ruby um it it's what we're doing today it's what we've been doing for many years um but it is
00:00:47.960
evolving and so we'll look at how it's evolving and then um finally we'll take a look at chunking um which is a
00:00:53.520
venerable um uh late 1900s um style of streaming um but it's something that um
00:01:00.320
modern projects outside of Ruby are actually utilizing today in novel ways so we'll take a look at that um briefly
00:01:06.320
as well um so web transport um this is a
00:01:12.799
spec that is semi supported at the moment um there are folk using it in Earnest and it's it's going to be um
00:01:20.400
probably getting a lot more um popularity here soon um after its drafts are released um so right now um the
00:01:27.280
worldwide Web Consortium um the WC3 um has a draft living standard um for uh
00:01:34.840
web transport so uh browsers do support it um Firefox and chrome currently and
00:01:40.439
Safari shortly um the internet uh engineering task force um also has a well-developed
00:01:47.520
um HTTP 3 spec for web transport and an upcoming http2 spec um so this is
00:01:54.479
something we'll see um throughout the talk is that there's a pairing of the browser side to get a web API
00:02:00.439
with the ietf side to get a transport protocol the transport protocols can be used without the browser um the web API
00:02:07.680
gives you the hooks in the browser um to be able to use it from Firefox Chrome Safari um the browsers we love and all
00:02:14.080
the others um so web transport unlike the protocols we use today is in a
00:02:20.599
single transport both unidirectional and bidirectional um so what does that look
00:02:25.720
like um o o over the wire um we have data represented here as 1 2 3 4
00:02:33.640
5 six um it all goes from server to client unidirectionally or the other way around um from client to server
00:02:40.680
bidirectionality is the same thing as unidirectionality but both ways at once full duplex we used to call it and then
00:02:46.640
partial reliability even where maybe some key frames um make it through um
00:02:52.239
because in web transport we have a concept of priority and so in things like video you can have frames that are
00:02:57.959
essential for rendering the next frame and then you can have filler frames that if they don't make it the user may not notice um but it's going to provide
00:03:04.360
additional information and there's lots of times where we want to say I want partial reliability some of my packets
00:03:09.959
need to stream with retries until they make it um whereas others try once or
00:03:16.080
twice maybe try once and let them go because they're no longer going to be relevant um so web transport doesn't
00:03:23.120
just support streams um web websocket and S is streaming only um web transport
00:03:29.360
also datagram um it has less overhead um
00:03:34.879
it is unreliable it's not going to it's not going to try again if the packet is dropped by a proxy or just doesn't make
00:03:40.760
it for unknown reasons it's gone um and it's unordered um if one of the packets
00:03:46.319
make it makes it before the other um the the it will be there first and it's not
00:03:52.200
going to ensure that one comes before two BEC comes before three um but today's talk is about datagrams that's
00:03:58.439
about all I'm sorry about stram so that's all I'm going to say about datagrams um so it's interesting that
00:04:05.040
this web transport is actually htb3 with htb2 fallback um because
00:04:12.439
uh it's starting actually um with the the later protocol and we'll see later
00:04:18.040
why that's a bit strange um it does have some real advantages like M toin sessions to streams um which we don't
00:04:24.759
have at all on today's protocols so this means that we can have many sessions over a single web transport connection
00:04:31.400
of which a browser can have eight um we can have many streams and they um just thought let's pick a number 64,000
00:04:37.759
sounds good um most browsers are saying how about 100 um and and uh so we have
00:04:45.080
kind of like if you're going between Ruby on both ends with a web transport you can do quite a few streams the streams will share lower layer transport
00:04:51.759
frames um they will be able to share sessions um and the the flow control and
00:04:56.919
congestion control uh will be done at the um web transport layer for htb2 and at the
00:05:02.880
quick layer for htb3 so um web transport soon but not yet um this is exciting um because it
00:05:09.919
gives us some novel capabilities some of which we haven't ever seen because they're novel um but it also allows us
00:05:15.320
to use some of the protocols that we've gotten accustomed to um firsthand and we'll see how that works later so um but
00:05:21.680
first up let's look at the present so up up to the up to current we've been talking about web transport which is
00:05:26.960
sort of forward looking if you do it right now you're certainly Cutting Edge um websockets are battle hardened um
00:05:33.919
just like web transport they have a uh WC3 um living standard the browsers it's
00:05:39.759
just whatever they're currently doing that's that's the spec um and they also have an internet engineering um task
00:05:45.880
force rfc's multiple rfc's because we've had years and years of websockets at this point um this is pre- websocket at
00:05:53.080
the end of the 1900s which we mentioned um HTP 1.1 um brings about um you know modernity um and
00:06:00.240
uh with it um uh persistent connections um it takes 12 years for us to see
00:06:06.960
websockets on the stream on the scene on the stream um with RFC uh
00:06:12.120
9455 um so these rfcs are are actually pretty good reads they're not um that um
00:06:18.080
technical um for being technical documents I guess they are technical there's technicals you can get but they're readable um and what this brings
00:06:25.840
to us in 2011 um is the upgrade method a new uh um a new uh method for headers so
00:06:34.560
uh we look at a get request um in htb 1.1 this is pretty typical um we say
00:06:40.199
upgrade um to what type websocket um and this is a connection upgrade it's basically saying we want to replace this
00:06:46.880
uh TCP socket with another one um of this type can you do that um the spec says give me a 101 response um so 101 is
00:06:55.240
I'm switching protocols like yes we'll do um to what a web socket um and so
00:07:00.759
this handshake additional handshake um takes a full hijack of the TCP um connection and basically says start a
00:07:06.039
new one um and instead of htps um start it with WSS started with a websocket one
00:07:11.080
so we may have some issues with we're now sort of going to a different place um the full hijack part is supported by
00:07:17.039
rack um rack adds this shortly um after these so it can uh stream um it can
00:07:23.240
basically make a regular um connection to rack Rack has a way to say yeah we're
00:07:28.800
not handling this um we're going to say this this is getting hijacked um just like we're
00:07:34.479
switching protocols um so it lets us support websockets um we have htb2 come um then
00:07:41.440
in 2015 um shortly thereafter and a lot of folks said why did we just get um uh
00:07:47.479
this new protocol because htb2 is also full duplex um it can go in both
00:07:53.759
directions at the same time um so our web stock is dead um and and folk could
00:08:00.360
immediately kind of begin um communicating with hh2 um back and forth um between like say a ruby program and
00:08:06.960
another Ruby program um but there's no web API um and three years later U we
00:08:12.840
see no web sockets are not dead um they are going to be supported by RFC
00:08:17.919
8441 um which bootstraps websockets on top of http2 um and actually this brings with
00:08:24.440
it not only compatibility but some real advantages um there's a new header um we
00:08:30.879
connect websocket a single header um and that can go over an http2 stream um
00:08:38.320
which because H2 supports multiplexing you don't have to start a new connection you can just say start a websocket
00:08:44.159
stream we can do lots and lots of these um and that way we have no new handshake
00:08:50.160
um no um reconnection um over a separate um socket and we get some nice um uh
00:08:57.839
things like hpack or depending on um your perspective maybe not nice um because um in HTTP 1.1 um you could just
00:09:04.920
look over the wire at the asky characters um that would read get Etc and now you look at a bunch of gibberish
00:09:10.959
binary um but hpac is um significantly quicker and the http2 protocol I'm not
00:09:16.720
going to go into detail but it's very interesting um is able to do handshakes faster um and and take on um some lower
00:09:22.720
latency especially with connections um so rack 3 streaming also
00:09:29.000
pairs with http2 gave us some interesting capabilities where that we didn't have before so let's first look
00:09:34.560
at what is r um sorry two streaming before we look at Rack 3 streaming so rack 2 you could do chunks you could
00:09:40.880
support that HTTP 1.1 standard from 99 um and you could do it by having an
00:09:46.120
enumerable so as long as it responds to each um and that means you can use an enumerator you can put sleeps in there
00:09:51.560
um you can basically you can pause you can wait for you know the next part of a CSV to be available or whatever you want to do um but you're not going to be able
00:09:58.920
to handle the stream you're just giving the the rack something that can be iterated over I mean it's going to take
00:10:05.440
those chunks and and and pass them on um but in rack three we got a new instead
00:10:11.040
of responding to each a new uh streaming style and this gives us something that response to call um you pass it one
00:10:18.440
argument which is the stream itself and then you can write to that stream you can read from that stream too so it's
00:10:23.519
bidirectional um very much unlike uh iterating over each there's no way to get anything back and in importantly you
00:10:29.959
can close the stream um which both lets you say we're done but it also lets you communicate information back to the
00:10:36.720
browser um so in 2021 we then had HTTP 3 um quick which
00:10:44.639
quick folk don't usually use it as an initialism it's just you just say quick but it is UDP um fast UDP and what that
00:10:52.240
really means is in TCP we have ordered reliable connections UDP is used for
00:10:57.519
these datagrams that we talked about in web transport and they're great for things like games where you just got to it's like if it
00:11:02.560
didn't get to you it's too late we're not retrying like the next packet is the one we care about now um they're
00:11:07.839
unordered if one's slow and you see the time stamp out of date you just throw it away you don't care um but quick takes
00:11:13.399
on top of this UDP protocol a lower level protocol in many ways than TCP and it adds in things like reliability and
00:11:22.160
order as optional things and it does it in a way that pays more heed to things
00:11:27.800
like flow control and congestion in and also um into flexibility so um why are
00:11:33.839
we doing this um in large part um head of line blocking gets significantly
00:11:39.040
better when you try to do congestion control on top of UDP as opposed to TCP
00:11:44.399
TCP head of line blocking is if something is inead of you and it gets blocked you don't go anywhere so it's
00:11:51.120
like if you're in a line um at the grocery store and someone ahead of you gets stuck there's no like they have
00:11:57.639
other L other lanes that you can go into um but if they don't then the whole line gets stuck um so htb3 lets websockets
00:12:05.440
work just like http2 only one year later they announced that we could bootstrap there because they just followed the hdp
00:12:11.920
spec like you're going to see that that that request and response will just look exactly like htb2 but you're going to
00:12:18.399
get that same thing with less head of line blocking so in Ruby we'll be able to use websockets and if the underlying
00:12:23.720
protocol is H3 it's just less likely that one of your earlier packets or a packet from something else that you're streaming like SS
00:12:29.920
will block the whole world um it's kind of like stop the world garbage collection it's not good we want to avoid it and each iteration of this
00:12:36.720
we're doing better um so it uses connect websocket but it's a little different as
00:12:42.160
a header over the wire because hpac just compressed connect websocket down a bit
00:12:48.079
um whereas qac actually did some analysis it's what they use in quick sorry the Q for quick um but they
00:12:53.800
actually did some analysis of what are the popular things we're sending over the web and it turns out connect websocket is extremely popular
00:12:59.760
so you just get to send a single bite now 15 15 means that you got yourself a
00:13:04.959
websocket um there's a lot less head of line blocking to um so what we've talked
00:13:10.000
about to this point is basically websockets and websockets over htb2 and
00:13:17.360
htb3 and then yikes we don't want to be held back by the mistakes like looking at the ietf um uh meeting notes which
00:13:24.560
are online on YouTube you can just watch the the the standards folk talking about the standards they're building
00:13:29.880
um but they said someone said like websockets pointed the ship in the wrong direction and the chair I believe of the
00:13:35.480
committee said that ship has already sunk and so they're sort of kind of
00:13:40.600
actively trying to seek websockets um but at the same time we can't use web
00:13:46.519
transport yet and we won't be able to for quite some time so while the s ship may have already sunk according to the standards committee we're going to be
00:13:52.519
using it for years to come and we should because standards precede the real world
00:13:57.560
so SSE actually um did not prede um
00:14:02.680
websockets it was the same year um it's just it was the world uh Wide Web
00:14:08.160
Consortium acting unilaterally without the need for any internet engineering uh
00:14:13.560
protocol uh fuss so they focused on the web API um what they don't have is B
00:14:20.360
directionality it's a one-way stream from server only to client you can't go from client back to server um and it
00:14:26.800
looks very much um like the uh TCP that it sits on top of um but it sends an
00:14:33.040
HTML page that completes the end of um HT end HTML element and then um events
00:14:39.279
stream after it ordered reliable events so uh let's look at a little demo
00:14:45.000
um the front end code here um is going to be HTM X um I'm just doing this so I
00:14:50.600
don't have to write any JavaScript um but you could you could instead um use
00:14:55.720
some of the many other um libraries that help us um get here like unpo um etc etc
00:15:01.920
um so I'm also going to use um little extension um for HTM X called s and
00:15:07.560
websocket um they basically let you do the the types of things that HTM X does which is when we submit a form um it's
00:15:15.519
going to be a post because the browsers thought we'd never you know need a get put and delete right um but we do and we
00:15:21.560
need to submit forms with those because we want different network and um and serers side behavior and htx just sort
00:15:28.319
of adds those polyfill um there's actually a proposal um tryptic which is just a couple hundred lines of
00:15:33.720
JavaScript to extend HTML to be able to do these um but the extensions here truly are extensions that won't ever be
00:15:39.839
you know probably accepted into HTML that provide some things on top of s and websockets and um here we're going to
00:15:47.319
see it look like pull in the script and then the only content of this page this is literally the whole content of the
00:15:54.079
demo I'm going to show you is two divs um I put a moon class on it just for some styling um and then use the
00:16:00.360
extension the extensions work similarly except um the websocket one you specify like the behavior type whereas the S one
00:16:07.240
there's different types of events um so they're they're slightly different in the same way that they are via the JavaScript API um on the back end I'll
00:16:13.959
use Roda um which um Jeremy Evans um created fantastic Library um it's a rack
00:16:19.800
Ruby web server if you haven't used it and um R A plugins are excellent um you
00:16:25.000
take Roto which is very little on top of rack itself um but it ads in a routing tree and then you can use the plugins
00:16:32.040
that Jeremy Evans created um as well as you can create your own plugins and so um in order to uh make your Roa routing
00:16:39.279
really elegant and to provide the things that you need to reuse a reusability way to actually reuse your code um it
00:16:45.160
plugins are fantastic so I created um two um one is under my GitHub um
00:16:50.319
github.com Havenwood Roda SS um and the other one is under the async um uh GitHub repo
00:16:57.959
socketry and it's uh a socketry r- websockets um these both use the
00:17:04.160
excellent async uh uh fiber libraries um because when we're going to be streaming we want to be doing it um with a whole
00:17:10.199
bunch of clients without taking a thread per um as many of the other um Ruby Solutions do today I'm outside of as snc
00:17:17.360
so let's look at our Rota code um this is um uh the the plugin for render which
00:17:22.559
ships with Roa lets us show an HTML page easily by convention um and use tilt um
00:17:28.120
as well as on the ones I wrote for S and websockets and we're going to um animate an emoji Moon spinning um so uh on the S
00:17:37.919
route this is the plugin we say if it's s meaning it's a get request and etc etc
00:17:43.280
then we're going to put the SSE headers on it and uh and and and use the rack 3 streaming interface under the hood here
00:17:50.280
um to actually write um the payload so for S we'll talk about this more but um
00:17:55.440
we say data colon because this is the world web the the WC 3 that wrote this spec so they just said let's just put
00:18:00.960
data in front of the data and like um ID in front of the ID and put it in a simple string and then we'll use new
00:18:06.120
lines to say we're done um and then in the websockets very similar um we just flush to say we're done um flush the
00:18:12.280
socket so um if we look at how these look um they're pretty similar um
00:18:19.000
they're not going to get too out of sync because if one blocks it's going to actually end up blocking the other um if
00:18:24.679
we kill the process and let it hang up um they don't stop exactly at the same time S one will uh restart on its own um
00:18:31.559
due to the spec um but you could emulate that same behavior in websockets and F typically do um so the web API for SEC
00:18:38.840
is what makes it different it's what distinguishes it and in many ways it's also what we can just replicate at the JavaScript layer but you have to write
00:18:44.480
JavaScript um so event type callbacks you can do this if you want to send Json over websockets and then figure out what
00:18:49.679
you sent and you know you come up with your own protocol for it um last event ID header is in case of failure the browser will automatically send back the
00:18:56.200
if you if you send it an ID which is optional people think that s is self-healing or something it really isn't you have to you have to care um
00:19:02.159
you have to send it an ID and then the browser when it retries will um automatically in the spec really the
00:19:07.799
real spec from from the WC3 says retry in a few seconds um which browsers have
00:19:13.760
interpreted as of course the spec three or five um and they didn't specify Jitter or exponential fallback which I
00:19:19.880
think in retrospect we wish they had um but we typically provide those like HTM X plugins provide that both for HT both
00:19:25.640
for S and for websockets um they provide Jitter so the initial like let's say your whole server goes down you come
00:19:32.080
back up you don't want three or 5 seconds later depending on your browser the whole Stampede to hit you at once
00:19:38.440
and so putting in some Jitter so each person is like I can I'm going to connect at 2.5 seconds I'm going to connect it 5.5 seconds and then they can
00:19:44.559
each exponentially fall back um so you're not going to have um these stamping herds um so these are some
00:19:51.559
simple abstractions um when we actually use them they're complicated under the hood but you can use them on the browser
00:19:57.400
and use them on um the server with only you know a couple lines um so they also
00:20:03.880
um look very much like each other so we can use a proc um to represent both of these streams there's a slightly
00:20:09.600
different body so I had to say s stream to get that data thing in there um but when you look at them in Roa the plugins
00:20:15.760
make them you serve the SS stream serve the websocket stream you want to serve web transport well we can save some
00:20:21.200
overhead by saying it's unidirectional but it essentially looks the same and I think the important thing here is for
00:20:27.120
unidirectional streaming we can write a plugin that tries to connect over web transport and if it doesn't have an H2
00:20:32.880
or H3 client falls back to websocket which will connect over 1.1 and then 2%
00:20:39.320
of websocket connections typically just fail um they they say um you know please
00:20:45.320
U switch protocols and the server dutifully says yes switching protocols and um then no traffic ever comes over
00:20:51.919
that new socket um we don't exactly know why um but probably proxies um and and
00:20:58.880
this is probably going to be an ongoing problem um and it's only going to get way worse for web transport because even
00:21:04.600
when people are are are using enginex to reverse proxy http2 typically like rubius often do um web transport needs
00:21:12.520
real http2 um it it can't just proxy over to htb 1.1 because no servers can
00:21:18.400
handle um that content negotiation it's actually incredibly complex to do on top of htb2 and preferably should just be
00:21:23.720
done on top of quick so let's look at s from the CLI though because wow s is a
00:21:29.400
web browser spec with no protocol for how you send it over the wire you can use it um from the client um most people
00:21:35.559
don't um use it directly from Ruby or any other language they use the browser um but if you wanted to implement s just
00:21:42.200
to see what the browser is doing um you can um like here we have a Rota app um
00:21:47.400
that the plugin I wrote you can take last event ID it just pulls the header off um and then we're going to stream
00:21:53.039
and then you see that raise line on line 20 um we're actually every seven requests going to simulate an eror um to
00:21:59.320
allow to do it self healing so our front end um there people just really don't
00:22:04.960
use SC as a browser protocol so it's no shame on Ruby that no one wrote an event stream parser so I just wrote one for this um and um a little SSE Ruby client
00:22:14.159
to pretend like I'm a browser um sort of do the same know browser behavior of of retrying I put some Jitter in because I
00:22:19.760
just couldn't pick three second flat um so here's some code um let's look at what this looks like so and this this
00:22:26.400
same Roa app works with a browser um so here we're streaming um some Emoji
00:22:33.400
um if it reset to the beginning it'd go back to zero pineapples but as you can
00:22:38.600
see um it carries on um from where it left off and this is basically the the
00:22:44.480
Ruby is acting as the browser you could have a browser connect the same back end at the same time and that would work um
00:22:51.000
so I don't recommend using S as a protocol um but it is interesting to look at it's a fun exercise and fairly
00:22:56.840
easy exercise to implement um the browser behavior for um so s also
00:23:02.120
supports types Beyond message um so messages if you don't give S A type it says we'll call that message and that's
00:23:09.120
in the spec um so let's look at types so here we have another Roa app um I'm
00:23:14.200
going to specify um uh a message is basically being pushed onto the Stream
00:23:19.559
So what's the message um here the message is going to be some HTML um so we have a div um the class is going to
00:23:26.240
be a grid item and we're going to s swap so we're going to we're going to use um the HTM x s extension to just swap the
00:23:34.679
div um with the same with the pixel number um of the pixel that we're wanting to swap with a random color and
00:23:41.080
then I just have a couple Ruby methods down there below to pick a random pixel um to give it a random color um so the
00:23:48.400
front end here um is including the extensions and then U we serve up the
00:23:53.520
initial screen which is just going to be um a grid item of all all black squares
00:23:58.600
so these this grid I me we're just using divs this is a screen of divs there's no content in this except for divs and
00:24:04.200
we've got two divs wide two divs high four divs per second we're gonna we're just push going to push over the wire um
00:24:09.720
with Roa so there's a couple extra headends in here um for quicker Roa connections um but here you see um those
00:24:18.080
four um divs changing color via that one ssse connection so what if we want to up
00:24:23.960
it um to 20 by 20 um at and do 40 per
00:24:29.159
second so um let's restart Falcon it it can handle this no problem um you can do
00:24:34.600
quite a few of these um if you want to go to Havenwood dodev I have it running right now I won't leave it running but
00:24:39.640
um um and this is just a lot of divs on the screen um if one of these gets
00:24:45.720
blocked it's going to block all the other updates um and if you have a bad
00:24:51.320
internet connection like in a hotel conference room um that's more likely so
00:24:56.679
um yeah you can see how if it's but it remarkably works most of the time as you bump it up this is 40,000 divs um
00:25:04.919
updating at and we'll refresh we we restarted Falcon and you see here we're starting to see it um if you want to go
00:25:11.760
to Havenwood Dev and look at your web inspector you're going to see a flurry of um events as thousands a second are
00:25:17.679
trying to go across the network um but this is not really appropriate um for um
00:25:23.960
sending video over the web um but what if we tried um let's uh go ahead and use f FFM
00:25:32.760
Peg to uh to turn a video into pngs and then let's down reses it and then let's
00:25:39.399
use image magic to compare every frame and let's do a diff um to see which if any pixels change and if a pixel doesn't
00:25:46.480
change do nothing but if a pixel does change fire off an event for that pixel to send a div with its background color
00:25:52.279
over the web to the browser um so for for the sake for for the sake of the demo um I uh I started I
00:26:00.120
did not pre-fill the actual image um I started with just a black screen but this is only my diff passing um
00:26:07.320
thousands of divs the update for its background color using SS over the web no modern
00:26:13.039
protocols um and uh it works
00:26:20.360
um um so what what does web transport actually open up for us um web codex so
00:26:25.880
this is coming down the spec pipeline right now it's not final ized but right now if you want to send video over the web it's web RTC and web RTC NX is also
00:26:33.520
coming down the pipeline which can work on top of quick and can even maybe web transport um but what web codex let us
00:26:39.080
do is say you know yeah we're not going to be sending divs across the wire for like video we smart people have worked
00:26:44.640
on this um and they've come up with a better way than I have um and and those codecs can be easily integrated into web
00:26:50.440
transport web transport can pay attention to the priority of the codec and can send um the bits across the wire
00:26:55.919
in the appropriate way whether it's partially reliable whether it's datagrams um and this is not something we can do with SS or websocket today
00:27:02.440
which is why we don't do it um but this is going to open up new things Beyond video so I think part of this talk is to
00:27:07.880
say I don't know what we're going to do with this but it's going to open Horizons and we're probably going to want it um so SSE issues um obviously
00:27:15.880
unidirectional that's huge head of line blocking suffers from the same um thing as the as websockets but it does just
00:27:21.799
work um it's not 2% or 30% um of requests that fail um it tends to just work so you see s 2012 2015 2021 it just
00:27:30.240
got these new protocols it didn't do anything because it doesn't have a transport protocol it it uses the
00:27:35.440
underlying transport um websockets um uh got uh the initial version the same year
00:27:41.159
as s um and then it took a while to get the H2 um and H the H3 version came
00:27:46.399
right along web transport going in a different way um it will never support HTTP 1.1 um it is adding things on top
00:27:53.760
of the TCP protocol that are frankly incredibly difficult to do on H2 um so H2 spec is not in browsers today you
00:28:00.720
cannot connect to web transport over htb2 that is scheduled for mid next year um and won't even be in you know Chrome
00:28:07.200
and Firefox until um maybe um around this time next year um so we're going to
00:28:12.399
see htb3 come first for web transport which is the first um it may actually come in the coming weeks um uh it it it
00:28:20.200
uh is near completion and has been implemented by the major browsers um we're going to see the http2 version um
00:28:27.000
maybe finalize around around sometime in 2025 maybe early
00:28:32.200
2026 so um this is a real challenge because this means that we're going to
00:28:38.480
have to one Implement htb3 in Ruby um or be able to get it from somewhere um but
00:28:43.640
then two we're going to need real http2 um web servers in Ruby we can't just reverse proxy like we've been doing for
00:28:49.039
ages to get at web transport so let's take a look at Ruby's future um we have
00:28:55.919
um a async um http p a great um htpu client so if you want to send from the
00:29:01.080
Ruby side to a server we have a we have good um clients um htpx um pardon my um
00:29:08.600
inversion there um and net http2 use ILO gregoric from Google's seminal work um in Ruby natively implementing hdp2 and
00:29:16.039
Ruby um very cool project um and then tyus actually uses the um uh the curl um
00:29:25.039
lib curl um under the hood um to be able to uh do htb2 um even if we don't have a
00:29:32.440
a ruby implementation um htb2 clients um do have some differences between them um
00:29:39.880
um so you may this is a spreadsheet I've done um that includes a lot more but these are some htb2 ones and the
00:29:45.080
standard Library ones um but so we might need to some more um HB cents no we don't have those um but there are some
00:29:51.760
in the works and I think it's important work and we need some diversity in the ecosystem um htb servers um not Puma um
00:29:59.279
Falcon so we have one um there may be more um I'd love to hear about them if you um are creating one or see one um
00:30:05.519
please let me know um Havenwood on GitHub um and IRC and Discord and things
00:30:11.360
um hb3 servers no um also coming but no
00:30:17.320
um so let's quickly digress um into the world of JavaScript um chunking is both
00:30:24.120
like it came out in 1999 um with uh uh with HTP 1.1 um but
00:30:30.799
people are still like using it today in novel ways so we've looked at websockets
00:30:36.360
and this websocket 2.0 web transport we've looked at s um these are the things we use in Ruby um but you can do
00:30:42.919
these without a web API um and that just looks like just keep sending more um but
00:30:50.720
when you do HTML um you can't pick where it goes it'll always be at the bottom of
00:30:56.240
the page unless maybe you do some fancy CS um uh but you're just chasing with
00:31:01.360
script tags but people do this for real um nextjs with react suspense actually just released a new version and they are
00:31:08.760
uh not doing exactly this is a pseudo code but they're basically doing the tactic of stream your like do a chunk
00:31:16.279
that renders the HTML page um but don't close the body or the HTML tag just sort
00:31:21.559
of leave it hanging with an open connection and then you can push the next chunk with a div that's hidden and
00:31:28.039
they I just put it a display none but they're they're hiding it off the screen of the browser and and then you basically juggle with the virtual Dom
00:31:34.679
and unhide it um over the div that it's replacing um this gives you incredible
00:31:39.760
flexibility to push your JavaScript over the wire um after your payload um because you can do anything you'd like
00:31:46.279
from the server um but it also means you are writing JavaScript um for the
00:31:52.159
browser on the server um so we probably with Ruby um want to do this in a
00:31:58.159
different way um so our hope is um in Amsterdam in April there are beautiful
00:32:05.519
flowers no that's not it it's it's um quick quick in the kernel um is quick is
00:32:11.080
incredibly difficult to implement and unlike things like UDP and TCP they don't say Ruby would you please Implement these they say please use them
00:32:18.159
from C or from your implementation whatever language it's in use the use the the kernel implementation of these
00:32:23.720
things quick is not in the kernel and um I Aquatics assisted me with um who does
00:32:29.639
the um async and and assisted me with the Roa s um plugin and um mentioned you
00:32:35.240
know he would like to see as I would um quick and the kernel to save all of these languages the hassle of
00:32:41.200
implementing them some I'm already have um looking at you uh rust go U python
00:32:46.440
Elixir airling um but you know we are not um necessarily terribly close but at
00:32:52.360
least we do have work in progress um to get quick for Ruby um and this is going to be something that you know we may not
00:32:58.760
have urgent need for it yet um but these Protocols are in progress and are going
00:33:04.159
to become real are going to become used and we're going to want to have them um especially the the the truly mainstream
00:33:09.639
ones like websockets 2.0 so I Aquatics um you can sponsor um online but this is
00:33:15.399
one person who's doing great work we need more people working on these protocols we need more people supporting
00:33:20.480
I aquatic's work um with documentation assistance with um bug fixes and and
00:33:25.720
with Enterprise support to add features that most of us don't use um but I Aquatics worked on things like Falcon
00:33:31.880
our one HTP A2 web server like async one of our four htb2 clients all the async
00:33:38.679
um libraries that support them um the underlying protocols um that are super
00:33:44.320
important to make it work and then things that actually land in Ruby um like the async fiber schedule the
00:33:49.440
scheduler that landed in Ruby um 3.0 like native fiber co- routines so we
00:33:54.519
have um co- routines that are native on top of our architecture and operating system um and things like iio event and
00:34:00.760
iio weight and and iio buffer so we have uh ways uh to work with our buffer and
00:34:06.639
we have um ways to get at eole and KQ um for for evented Io um from Ruby itself
00:34:12.240
at the library level and not just for these things but any of us can use this to make new Solutions so I'd love to see
00:34:18.720
um sponsorship for protocol Qui um protocol qac and protocol web transport
00:34:24.040
in particular protocol quick is the biggest one because you may not believe that protocol web web transport um is going to take off but protocol Qui is is
00:34:30.639
already 30% of the internet traffic according to Cloud flare it's three times the HTTP 1.1 traffic most traffic
00:34:36.560
is H2 but a lot of that isn't true H2 it's reverse proxy um to a 1.1 web server on the back end so I also want to
00:34:43.720
say I love Roa and its plugins um thank you to Jeremy Evans um for making it and there are a lot of other beautiful um
00:34:50.520
libraries in Ruby um to do uh to build your own web framework to compose it the way you'd like um I like the way you can
00:34:57.040
do it with plugins other Frameworks like Hanami um compose it internally in a different way um but really great to
00:35:02.839
work with um and a joy um in Ruby so um this was streaming over the web um I
00:35:09.680
have a lot more to say on this but only 30 minutes and I've gone over um but I'd love to take any questions since we're up against lunch but if anyone would
00:35:15.599
like to leave please uh feel free um uh lunch is beginning in 10
00:35:26.640
minutes for
