Supporter Talk by Ubicloud: Build a Cloud in Thirteen Years

Summarized using AI

Supporter Talk by Ubicloud: Build a Cloud in Thirteen Years

Daniel Farina • November 13, 2024 • Chicago, IL • Talk

In his talk at RubyConf 2024, Daniel Farina discusses the evolution and current practices of Ubicloud, a company he co-founded, which is building an open-source alternative to established cloud services. The talk delves into the unique engineering approaches used at Ubicloud, particularly the decision to utilize Ruby—specifically not relying on Rails—which he suggests is unconventional within the Ruby community. Farina outlines the development history and functionality of key libraries employed in Ubicloud's infrastructure, highlighting the following core concepts:

  • Ruby's Role: Farina emphasizes the practicality of using Ruby over alternatives, detailing his shift from forming a small team at Heroku to developing extensive cloud services at Ubicloud.
  • Key Libraries: The talk presents crucial libraries like SQL, OMM, and Roa, focusing on their development process and integration within Ubicloud’s architecture. He notes that these libraries maintain high quality, with Jeremy Evans, a key contributor, ensuring seamless updates and minimal issues over years of evolution.
  • Development Practices: Farina highlights the advantage of deploying every merged pull request immediately, facilitating real-time feedback and reducing complexity in regressions. He notes an established practice of keeping all bugs closed, ensuring high standards of quality in the software libraries used.
  • Performance of Ruby Libraries: Emphasizing the performance and efficacy of libraries used, he discusses their features and how they accommodate rapid development cycles without sacrificing functionality or stability. He challenges the ideas around Ruby’s capabilities, positioning it as a robust and economical language for current development needs, contrary to the notion that it is only suited for rapid prototyping.
  • Request for Community Support: Farina concludes with a call to action, encouraging the audience to test their services and provide feedback in the form of bug reports to foster further development.

In essence, Farina aims to broaden the community's view on Ruby's applications in infrastructure and cloud computing, promoting an understanding of its practical strengths and long-term viability in the industry.

Supporter Talk by Ubicloud: Build a Cloud in Thirteen Years
Daniel Farina • November 13, 2024 • Chicago, IL • Talk

In 2011, I joined a tiny team at Heroku, managing servers with a custom Ruby program. By 2023, I co-founded Ubicloud, developing the fourth generation of such a program...again, in Ruby. Unlike 2011, the choice of Ruby in 2024 raises questions, even from Ruby programmers.

The Ubicloud team's engineering approach is unusual: most of the team has used Ruby for infrastructure cloud programs, none of them depending upon Rails. I'll share our unconventional practices and assessment of Ruby's strengths, aiming to broaden perspectives on Ruby's capabilities and applications.

RubyConf 2024

00:00:15.599 okay well thank you everybody for coming
00:00:17.240 here
00:00:19.080 um so I'm Daniel I'm a founder of EB
00:00:22.279 Cloud uh and what we're trying to do is
00:00:24.480 build a open source alternative to
00:00:26.679 Amazon web services and Azure or gcp so
00:00:30.240 Cloud our focus is currently a managed
00:00:32.680 Service uh which you can use today at
00:00:35.040 icloud.com and we focus on um value
00:00:38.640 that's a cost Effectiveness and the
00:00:40.800 quality of core Services rather than a
00:00:42.680 huge number of services uh so for
00:00:45.280 example virtual machines load balancers
00:00:47.320 and databases per slide and I really
00:00:50.239 hope uh you can use it and then
00:00:52.320 especially help us improve it with your
00:00:54.120 feedback and Bug reports always my
00:00:56.840 favorite uh thing about customer is
00:01:00.280 their bug reports um it's written in
00:01:02.879 Ruby which used to not be remarkable in
00:01:05.600 2011 when I started writing programs
00:01:07.320 like this uh but it's become remarkable
00:01:10.799 these days uh UB cloud is the fourth in
00:01:13.600 a series of programs I've written that
00:01:14.960 are like this
00:01:17.799 uh I started I considered each time
00:01:20.159 whether I should change the language but
00:01:21.840 so far I've always ruled out all the
00:01:24.520 Alternatives uh for I think strictly
00:01:27.119 practical reasons uh originally I joined
00:01:30.159 Heroku Department of data as a postgress
00:01:32.240 person and it was a Sinatra application
00:01:34.320 that used
00:01:35.640 SQL
00:01:37.159 um so this is where I actually got into
00:01:39.159 Ruby I was a postest person they had
00:01:40.759 some a few postest databases and then we
00:01:43.399 built it up from our tiny team of three
00:01:45.040 into whatever you see today at Heroku
00:01:47.000 postgress uh as a result I never have
00:01:49.360 used rails which already puts me in
00:01:52.360 extremely narrow company in the Ruby
00:01:55.320 Community uh and I never really engaged
00:01:59.000 with Ruby's cultural institutions uh
00:02:01.439 such as this conference or rails conf or
00:02:04.399 even the user group in San Francisco uh
00:02:07.640 but in 2024 uh ireena who organized the
00:02:10.720 San Francisco User Group heard that this
00:02:12.840 startup called UB Cloud used Ruby to
00:02:14.760 write control plane software and she
00:02:16.920 assumed rails and uh I said oh yeah well
00:02:20.720 we don't first of all yes we are Ruby
00:02:22.400 but also not rails said okay well that's
00:02:24.120 extremely weird and then you're
00:02:24.959 extremely weirder so you should tell us
00:02:26.840 about why you're so strange and so here
00:02:28.680 I am giving uh this
00:02:30.959 talk uh I'm going to focus on two areas
00:02:35.400 uh one is I want to talk about some core
00:02:37.680 libraries or key libraries that we use
00:02:40.560 uh and actually I want to discuss how
00:02:43.720 they're developed uh which I think is
00:02:46.159 quite extraordinary and then furthermore
00:02:49.239 I think that this is the news you can
00:02:50.599 use part of the talk you can use some of
00:02:52.200 these libraries in your own work and
00:02:54.080 hopefully benefit as I have uh but then
00:02:56.879 after kind of setting your mind in how I
00:02:58.720 view these libraries and what's so
00:02:59.840 extraordinary about them I want to
00:03:01.360 discuss the strengths of Ruby as I see
00:03:03.200 them which I think is different than
00:03:04.720 most since I come from kind of both the
00:03:07.080 inside having done for so long and the
00:03:10.080 outside uh so first what did all these
00:03:12.040 things do uh SQL sometimes called well
00:03:16.760 it's very easy to make it confused with
00:03:18.599 SQL uh does database access an omm
00:03:21.480 that's probably the most well-known of
00:03:22.599 these libraries and Roa does http
00:03:25.720 routing so it took the place of Sinatra
00:03:28.239 in my stacks in the last two
00:03:30.280 projects and uh rodo does authentication
00:03:33.640 which took the place of devis which is
00:03:36.239 very kind of
00:03:37.680 railay and uh with access to all three
00:03:41.280 of these things I was able to build a
00:03:43.120 full stack application um with libraries
00:03:46.519 actually maintained in a very
00:03:48.200 similar regime so main question is what
00:03:52.200 makes a library key what makes a library
00:03:55.640 key is it has a very wide interface to
00:03:57.319 my programs uh you know like SQL ques
00:04:00.439 are everywhere in a we application or
00:04:02.840 many applications not even just web you
00:04:05.920 you have calls from everywhere they're
00:04:07.239 very invasive and how you write the
00:04:08.439 program you can't just swap them out uh
00:04:10.840 and same is true for uh the web
00:04:12.599 framework as well like how you organize
00:04:13.959 everything has a lot to do with how it
00:04:15.680 works and they're necessarily invasive
00:04:18.000 to your program a counter example saying
00:04:19.799 you want high quality but not invasive
00:04:21.280 is a database protocol driver so this is
00:04:22.919 why I focus so much on the key libraries
00:04:24.759 and they are a perfectly good reason to
00:04:25.960 use any particular
00:04:27.960 language uh but why do we care so much
00:04:30.160 or I care so much about how they're
00:04:31.440 developed you know do you just look at
00:04:33.000 the spec sheet and say oh yeah it looks
00:04:34.720 like it does what I want and it looks
00:04:35.919 pretty good to use so I'll just use it
00:04:39.360 uh this talk is actually not about how
00:04:41.600 you use the stacks it just couldn't
00:04:43.360 possibly fit in the half hour while
00:04:45.199 discussing want to discuss um but it
00:04:49.120 there are certain ways these libraries
00:04:50.479 are developed that actually inspired me
00:04:52.360 over uh the last 13 years when I first
00:04:55.080 encountered them uh that Ruby could be
00:04:58.759 very different than I think most people
00:05:01.199 uh think of it
00:05:02.440 as uh normally there wouldn't be much to
00:05:04.880 say about like oh you know Gem's
00:05:05.880 released once in a while there's a new
00:05:07.080 version blah blah blah well this is the
00:05:09.800 exception where it's extremely
00:05:13.199 odd uh so it's all about this guy who
00:05:15.680 will be here tomorrow and speaking on
00:05:18.360 Friday you should see his talk all three
00:05:20.759 of these uh key libraries are maintained
00:05:22.840 by Jeremy Evans and they work together
00:05:26.360 cohesively and don't drift apart for
00:05:28.000 that reason and SQL was his first
00:05:30.520 well-known work which we used at
00:05:32.960 Heroku we started using in 2011 by 2012
00:05:35.560 we were so perplexed at how he
00:05:37.120 maintained it that we invited him to
00:05:38.639 give a talk just about what his process
00:05:42.360 was uh particular we
00:05:45.039 were uh confused how he had well that's
00:05:49.000 a spoiler okay um so there's a in if you
00:05:53.720 go to this slide deck um I'll post a
00:05:56.120 link later uh you can go and um click on
00:06:00.319 these links like the presentations he
00:06:01.759 has Archives of his presentations and
00:06:04.039 you can find that one take a look and um
00:06:07.599 he's definitely one of the few software
00:06:09.080 Engineers like I've ever seen that kind
00:06:11.880 of redefined what is possible to do not
00:06:14.599 only with Ruby but just in general but I
00:06:16.800 think May's practices have something to
00:06:18.240 do with how Ruby
00:06:19.759 Works uh so I've been trying to
00:06:22.039 duplicate his efficiency uh for the last
00:06:24.479 decade uh because I haven't quite
00:06:27.199 replicated it yet but um getting closer
00:06:30.639 it only it's only taking me 13 years um
00:06:33.800 so because of this I want to dwell on
00:06:35.360 some of the outward signs of how this
00:06:37.680 works okay so first bizarre Jeremy Evans
00:06:40.680 fact there at least a few if you kind of
00:06:43.280 just blink in his talks or interviews
00:06:45.560 you'll miss something because he's kind
00:06:47.319 of humble but sometimes he gets cornered
00:06:50.039 and will like let a fact leak that if
00:06:52.520 you think about is actually just
00:06:54.720 insane for example this remark it gives
00:06:57.639 me a sense of accomplishment to be able
00:06:59.599 to fix bugs in Ruby that have been known
00:07:01.919 but unfixed for many years that's a
00:07:04.000 situation it doesn't happen by open
00:07:05.400 source projects and you might say like I
00:07:07.400 can't literally be true he must mean
00:07:09.639 something kind of
00:07:11.240 metaphorical but no it's actually
00:07:13.759 literally true where every single issue
00:07:16.879 is always closed on all of his projects
00:07:19.000 not because he told you go to hell and
00:07:20.840 go away he actually just fixes
00:07:22.759 everything also
00:07:24.919 1,00 issues over let's see 2007 14 years
00:07:29.080 is not that many issues and so whenever
00:07:31.319 I say this people say well maybe
00:07:32.319 nobody's using it I'm said No it's it's
00:07:34.440 the it's the weird one it's the case
00:07:36.240 where there's actually not many bugs and
00:07:37.840 he closes them all
00:07:40.720 um SQL is not a micro RM it has like
00:07:44.800 every feature it always seems to have
00:07:46.280 new postgress features while postgress
00:07:48.159 is in beta so in beta It'll like oh Json
00:07:50.879 b or upserts it'll have new data set
00:07:54.599 methods to do to let you wrap in the
00:07:57.319 DSL um now I mean zero issues only 1200
00:08:01.159 issues over 14 years and zero open
00:08:03.120 issues at any given time uh that's a
00:08:06.199 level of quality I like to see in a key
00:08:09.120 Library another thing that's extremely
00:08:11.199 bizarre is there's monthly releases of
00:08:13.560 the gem these are the most invasive
00:08:15.360 dependencies in my program there's like
00:08:16.800 tons and tons and tons of call sites
00:08:18.800 everywhere uh how can you release it
00:08:20.840 every month and not break everybody all
00:08:22.240 the time uh but actually I do and then
00:08:26.720 it doesn't break I've maybe had three
00:08:28.560 regressions of my Min sorts in the last
00:08:30.879 13 years and most of them are backloaded
00:08:32.479 I haven't had one in a long time I just
00:08:34.640 upgrade every
00:08:36.279 month uh I look at the dip sometimes the
00:08:39.399 way he does month-to month and they
00:08:41.120 usually aren't that big I think oh yeah
00:08:42.680 you know like in principle I could have
00:08:44.000 done it but could have Shoulda would a
00:08:48.000 didn't and somehow his relentlessness
00:08:50.560 just adds up and makes it work I've just
00:08:52.680 never seen everything comparable to this
00:08:54.480 in any kind of uh release I don't do you
00:08:56.959 know of a single gem other than one that
00:08:58.600 he writes that does stuff like this I've
00:09:00.640 never seen it have you even seen like
00:09:02.240 any python library or node library or go
00:09:04.519 I mean I've never seen anything like it
00:09:07.760 um you know it's just the breath and the
00:09:10.600 stability for such a huge interface in
00:09:12.160 your program is just unheard
00:09:13.959 of I do have
00:09:16.560 a a kind of a rhyming
00:09:20.000 um uh practice that we use for services
00:09:22.680 for UB cloud and all its predecessors is
00:09:26.440 I think this is more common we people
00:09:27.800 than most but we deploy every single
00:09:30.279 pull request right after it's merged so
00:09:33.040 whenever you see in our history like hey
00:09:35.399 we merged a thing into UV Cloud's get
00:09:37.399 have repo it's in production like with
00:09:39.279 the next few seconds um and the reason
00:09:42.399 we do this is uh one it works it works
00:09:44.760 fine uh we don't need to you know we
00:09:46.800 don't feel a need to bake things or or
00:09:48.519 whatnot um it allows the author to
00:09:50.640 supervise the outcomes without
00:09:52.279 interfering changes and it helps if you
00:09:54.480 have a good test Suite to be able to do
00:09:55.800 this uh but it works great so that
00:09:57.640 whenever there's abnormalities the
00:09:59.399 author knows it was them and they you
00:10:01.120 know figure out to do a roll back um so
00:10:04.079 we actually do in some ways more
00:10:05.360 releases than even he does uh but I
00:10:07.720 think this is somewhat more normal
00:10:09.600 probably a lot of you are used to
00:10:10.760 merging PL requests and deploying
00:10:12.360 website I think I'm not sure if everyone
00:10:13.920 batches their
00:10:16.120 changes okay so next weird thing uh his
00:10:21.279 projects are all based on a huge number
00:10:23.640 of plugins uh because I kind of swapped
00:10:28.399 in Roa for uh Sinatra people think oh
00:10:32.120 are these micro Frameworks and I would
00:10:33.519 say they're they're definitely not rails
00:10:36.320 but they're not micro there's just a
00:10:38.480 huge number of first-party plugins uh
00:10:41.000 this is like a really small screenshot
00:10:42.680 of all plugins I think in SQL on the
00:10:44.959 left so there's Pages like that for all
00:10:47.480 three um the plugins are much of many of
00:10:51.720 them are first party also written by him
00:10:53.839 and you can almost think of him as lazy
00:10:55.240 code loading he's very particular about
00:10:57.360 not loading code you don't need and so
00:10:59.600 actually everything is um loads very
00:11:01.639 fast which is great for tests in
00:11:03.399 particular uh so you can even think of
00:11:05.120 more of like configuration or lazy code
00:11:07.440 than a plugin I'm very impressed with
00:11:09.880 their compact size and scope if you read
00:11:11.440 implementations it takes a little while
00:11:12.760 to understand the very short code but it
00:11:15.600 is there's not much to audit once you
00:11:16.800 get it uh for example and they do things
00:11:19.160 that are both trivial to fundamental
00:11:21.120 streaming responses in HTTP that's a
00:11:23.680 plugin you don't have to load it but if
00:11:25.680 you need streaming responses you load it
00:11:27.920 processing inbound email
00:11:29.959 is a
00:11:31.079 plugin I mean we don't do that yet but
00:11:33.440 if we did I guess there's a plugin for
00:11:34.839 that it's kind of like there's an app
00:11:36.720 for that uh I just never seen so much
00:11:39.200 cohesion delivered optionally and it
00:11:42.360 keeps apps that don't use all the
00:11:43.920 features loading very
00:11:46.279 quickly uh that said although the plugin
00:11:48.399 is very broad I kind of interested being
00:11:50.519 practical have some areas where they
00:11:52.639 have no opinion that you kind of have to
00:11:54.240 do your own thing one
00:11:56.519 is no opinions on JavaScript
00:12:00.760 uh now one question is do you really you
00:12:03.519 need JavaScript you know our Cloud
00:12:05.320 console is an Enterprise app what do
00:12:07.560 Cloud consoles do they show you tables
00:12:09.560 it's like yo bro I heard you like tables
00:12:11.000 they put some tables in your tables so
00:12:12.320 you can browse where you browse it's
00:12:13.440 kind of what cloud you know it's just
00:12:14.959 Table after table tables of VM tables of
00:12:17.160 load balances what do they really to do
00:12:18.560 there so for us actually static Pages
00:12:21.279 you know or dynamic you know just
00:12:22.720 Dynamic server side pages are fine and
00:12:25.600 they're easy to test and you can
00:12:26.920 generate screenshots easily and you can
00:12:28.519 do accessibility and you know all sorts
00:12:30.199 of stuff that gets really expensive with
00:12:31.560 single page apps uh so that's one answer
00:12:35.360 is maybe don't use the JavaScript but I
00:12:37.639 know that in this world you sometimes
00:12:39.079 really do need some fancy interaction uh
00:12:42.079 for JavaScript and in that case you
00:12:43.560 basically have to decide how you're
00:12:44.800 going to do it and how you going to test
00:12:46.639 it uh in principle I've heard there are
00:12:48.959 robust ways to test JavaScript but
00:12:50.639 whenever I ask someone can you show me a
00:12:52.079 project that does it um is there the SQL
00:12:57.079 of node or I just can't no one seems to
00:13:00.519 come up with anything so there in lies
00:13:02.680 the rub is there an arsp is there an
00:13:05.040 rspec of node all kind of but can you do
00:13:07.320 the partial locks well maybe you know
00:13:08.880 it's kind of a little bit
00:13:10.920 hairy another area I think would be very
00:13:12.959 interesting is it doesn't really have
00:13:14.560 any PS on mobile applications I see is a
00:13:16.600 pretty nice thing that rails has
00:13:18.880 opinions about is how to make you know
00:13:20.199 turbo native um so that's kind of cool
00:13:22.600 and also not covered by the plugins so
00:13:24.279 to give you a sense if you're building
00:13:25.399 something where pure HTML apps uh I have
00:13:28.760 no about using this stack if you need a
00:13:30.680 lot of JavaScript you kind of have to be
00:13:32.560 ready to have
00:13:33.639 opinions so there's the
00:13:36.560 preo uh but many of you so this is kind
00:13:39.040 of an obscure stack and so many of you
00:13:41.440 have prior commitments and so I thought
00:13:43.240 I'd talk about some useful plugins uh
00:13:46.360 that might give you some of the benefits
00:13:48.880 uh there's rad off rails and SQL rails
00:13:52.959 uh rodo rails I think is the most
00:13:55.360 compelling uh because rodo is a very
00:13:57.639 good authentication framework
00:13:59.959 uh as you can see here it has a pretty
00:14:02.680 big list of features uh TFA recovery
00:14:05.040 emails uh magic links all optional and
00:14:08.440 plugins by the way uh audit logging you
00:14:11.560 know all this kind of stuff also a very
00:14:13.959 strict security stands where it
00:14:15.560 separates the password hashes so you you
00:14:17.320 can't it's hard to implicate you in a
00:14:19.199 credential stuffing
00:14:20.959 attack uh there's a very this plugin is
00:14:23.639 wellmaintained uh rodo rails and you
00:14:25.680 could think of it instead of maybe using
00:14:26.920 devis that's a very practical news you
00:14:29.720 can use the other um plugin is um SQL
00:14:35.240 rails which replaces the orm with SQL
00:14:38.399 this is useful I think you have an
00:14:39.519 organization that has both orms or if
00:14:42.120 you really need some of the power you
00:14:43.880 have a lot of you know would benefit
00:14:45.720 from powerful database queries because
00:14:47.199 SQL is kind of known for having a very
00:14:48.759 powerful data set um model uh it also is
00:14:53.720 faster uh like the way micro like micro
00:14:56.440 compositions for allocation so forth and
00:14:58.000 that actually really adds up in the
00:14:59.399 tests if you have a lot of tests the
00:15:02.040 speed of allocation so forth really
00:15:03.839 matters just doing new not even doing
00:15:06.079 insert it does add up to speed up your
00:15:08.360 test cycle um we actually used it uh in
00:15:11.519 the second iteration of the system and
00:15:13.399 it we had to fiddle with it sometimes
00:15:14.800 but work surprisingly well so those are
00:15:16.240 two pieces of news you can
00:15:18.079 use here are some fun uh quotes about
00:15:23.000 both of the kind of libraries uh one is
00:15:26.279 I just pulled from the internet thank
00:15:27.519 you uh futures David I always go for
00:15:30.759 sequels just a better design overall
00:15:32.480 100% bug free seriously when you last
00:15:34.279 see an issue on the GitHub page that was
00:15:35.759 more than a couple of days
00:15:37.360 old the other second comment was
00:15:39.519 provided on uh rodof by one of the
00:15:42.839 engineers that works at UB Cloud but is
00:15:44.360 working for rodof For the First Time
00:15:46.199 some months
00:15:47.240 ago and he says as he submits the pr to
00:15:51.040 add to fa we use the Roff library for
00:15:53.519 authentication which offers built-in
00:15:54.959 first P plugins for popular two pay
00:15:56.680 methods upon enabling these plugins they
00:15:59.480 function seamlessly without additional
00:16:01.240 configuration it was like a miracle it
00:16:04.319 just kind of worked and it still
00:16:05.519 continued
00:16:07.680 working just because there's these
00:16:09.639 really broad um plugins doesn't mean
00:16:11.959 that his fa are slow actually they're
00:16:14.199 incredibly fast uh by stands of Ruby
00:16:16.880 programs so if you look here the little
00:16:19.079 lines at the bottom are Sinatra and
00:16:21.800 rails and up at the upper left you know
00:16:24.000 that big yellow line is Roa and he's
00:16:26.480 going to be talking a little about
00:16:27.480 performance in his talk on Friday Friday
00:16:29.160 so maybe should go check it out it's
00:16:30.360 maybe the only piece of software over 10
00:16:32.160 years it gets
00:16:33.800 faster um you can see at the lower right
00:16:36.959 a commit he made to UB Cloud where he
00:16:38.399 says avoid four allocations for every
00:16:40.360 request so that stuff adds up as it
00:16:42.880 turns out and he applies this kind of
00:16:45.440 merciless uh optimization to his own
00:16:47.759 libraries which really matter again
00:16:49.360 especially in your
00:16:52.639 tests he has 100% Branch coverage on all
00:16:55.519 of his
00:16:56.800 libraries uh this is a page his open
00:16:59.600 source page and these are all the branch
00:17:02.160 coverages they're all
00:17:03.640 100% uh when I first heard this in 2012
00:17:07.160 I was um pretty shocked that seemed like
00:17:10.839 you know the conventional wisdom was oh
00:17:11.959 after 80% doesn't really matter no no no
00:17:14.679 there's the difference between 95 and
00:17:16.199 100 and 100 is much better than 95 uh
00:17:19.240 I've been doing this since 20 I've
00:17:21.079 adopted it to UB Cloud so most of it is
00:17:23.160 at 100% regime
00:17:26.039 um and I've done it also at cus and at
00:17:29.559 crunchy and i' after you know I guess
00:17:32.440 almost 10 years I've de some insight
00:17:33.919 into why it's a good idea so here are
00:17:36.760 some quick tips uh one is you should do
00:17:40.120 it there also news you can use uh it's
00:17:43.600 not really what about people think it is
00:17:45.280 it's it's true that with more test you
00:17:47.120 tend to decrease in bug count but really
00:17:49.679 it helps you administer the introduction
00:17:52.880 of gaps into your test Suite much more
00:17:55.679 effectively every time you see red it's
00:17:58.080 like okay I just did that because it was
00:17:59.679 green before and now there lines that
00:18:01.080 are red obviously it's me furthermore
00:18:04.000 there's always a test that got to the
00:18:05.440 adjacent line so it's like okay I'll
00:18:06.720 find the test that gets to that branch
00:18:09.120 and then I will put in a new test it's
00:18:10.799 always easy to add the next
00:18:12.520 test I think these are kind of the key
00:18:14.520 things that make it work so well uh so
00:18:17.360 this is why I have advised like choose
00:18:20.159 one file and make it 100% and enforce it
00:18:23.280 don't worry about having area file 95
00:18:25.080 try to get one file to 100 that way the
00:18:28.520 the quickness of Gap analysis is there
00:18:31.120 for at least a part of your code base so
00:18:32.960 much better to have 100 on one than like
00:18:35.640 maybe 80 everywhere it's kind of uh what
00:18:38.320 I'm saying here it's also although it's
00:18:41.480 kind of you could say developed or
00:18:44.240 practiced by solitary programmer Jeremy
00:18:46.039 is mostly Sol solitary on his libraries
00:18:48.520 it works very well in teams and the
00:18:50.720 reason it works very well in teams is
00:18:52.919 because um when people are building
00:18:55.320 their projects you know there's kind of
00:18:57.240 the promotion packet they got to build
00:18:59.000 or whatever and if someone didn't leave
00:19:00.400 you a testing methodology you're like oh
00:19:02.120 do I really want to rip up all that code
00:19:03.240 to figure out how to test it and then
00:19:04.760 they don't then the cracks get bigger
00:19:06.080 and bigger this kind of allows okay
00:19:07.720 there's always a test and to add a test
00:19:09.400 it's about even cost book to your
00:19:11.120 project so I think this is also works
00:19:12.679 very well in larger
00:19:14.840 teams uh now you might be saying okay
00:19:17.720 well but you're infrastructure Cloud so
00:19:18.919 why are you talking about all this web
00:19:20.000 stuff is that really necessary to you
00:19:22.400 know is this really the core of what you
00:19:23.799 do and I would say it sort of is
00:19:26.720 definitely bookkeeping is a big part
00:19:28.720 part of doing infrastructure clouds just
00:19:30.679 like hey list servers between these two
00:19:32.240 create ad dates run this shell command
00:19:34.480 tell me what it says coate it and then
00:19:36.400 you know Bill a report um so like very
00:19:39.679 strong orm is key that's why we kind of
00:19:41.000 started with that uh at
00:19:42.840 Heroku um relational databases and
00:19:45.720 Powerful orms really help you with
00:19:47.039 bookkeeping errors and those cause a lot
00:19:48.320 of outages so that kind of makes that
00:19:49.840 less likely and L security program
00:19:52.120 problems are caused by excessive
00:19:53.559 complexity in the web application uh or
00:19:56.039 the API and you need implementation
00:19:57.880 polish and power to avoid incidental
00:20:00.559 complexity to bloat your you know line
00:20:03.080 counts and other things that make hard
00:20:04.320 to
00:20:05.240 audit and so my longtime collaborator
00:20:07.840 will who I mentioned the first I mean on
00:20:09.600 the slide uh described our life's work
00:20:12.280 in the previous three efforts as a fancy
00:20:14.840 SSH client and now the fourth is a fancy
00:20:17.960 SSH client with a web page previously
00:20:21.080 the previous ones talked to an API to
00:20:23.159 another system so also special thanks
00:20:25.840 toit States for working so well for all
00:20:27.640 these years we kick them a small
00:20:29.320 sponsorship because apparently no one
00:20:30.600 else does and I felt bad because I've
00:20:32.640 been using it for 10 years so good
00:20:35.000 Library
00:20:37.360 NS so one of the main assets of uh rails
00:20:40.159 or Jango is a prescription of how
00:20:41.559 components fit together you don't really
00:20:43.159 get that from each libr individually but
00:20:44.520 there's this handy if not super uh
00:20:47.240 slickly named project called Roa SQL
00:20:49.120 stack where Jeremy is like okay here's
00:20:51.080 how I suggest you glue things together
00:20:53.600 and that's what we based ucloud on and
00:20:55.679 then we added Rod off per the manual and
00:20:58.280 you know you should consider using this
00:20:59.600 and looking at that's a link by the way
00:21:01.919 okay so I want to move on now you kind
00:21:04.640 of have a sense of how I what I'm
00:21:05.679 impressed by and what I feel about kind
00:21:07.919 of these libraries I want to talk about
00:21:09.440 reinterpreting what ruby is good at
00:21:14.159 um because I think I see it very
00:21:16.080 differently than most people because
00:21:17.360 frankly I've been just sitting in a hole
00:21:19.039 writing these kinds of control planes
00:21:20.520 for 13 years and never going to
00:21:22.080 conferences or running in rails apps so
00:21:23.679 I I don't really see things the same way
00:21:26.279 here's kind of a common list of
00:21:27.559 interpretations I pulled from the net
00:21:29.480 about what ruby is kind of all about oh
00:21:31.919 it's inspired artistic beautiful it's
00:21:33.760 productive just developer happiness but
00:21:36.000 also kind of unprincipled or a little
00:21:38.840 wild or hard to maintain or only good
00:21:41.400 for
00:21:42.799 prototypes
00:21:45.120 uh personally well I've come to not see
00:21:48.600 things this way uh here is how I see it
00:21:52.200 it's very stable uh it's rigorous
00:21:55.840 prudent restrained and economical so you
00:21:58.799 know I come from the the database world
00:22:00.600 so I'm very much a
00:22:02.200 killjoy uh you know when you get right
00:22:04.440 down to it postgress people I mean
00:22:06.000 they're fun to hang around with but
00:22:07.000 mostly because of the Gallows humor
00:22:11.320 um it's for these reasons I use Ruby not
00:22:14.559 for you know kind of the
00:22:16.559 more romantic reasons perhaps I'm I'm
00:22:19.559 very
00:22:20.279 boring um and what else can you say
00:22:23.480 about a software package that releases
00:22:26.159 every month with 100% Branch coverage is
00:22:27.880 that not stable and rigorous I
00:22:31.200 mean you know that's kind of how I feel
00:22:34.360 about
00:22:35.400 Ruby another thing is if you well this
00:22:38.960 is if you look at our line counts U
00:22:42.440 Cloud supports all this stuff but it's
00:22:44.520 only uh 20,000 lines or so of
00:22:47.039 application code and then another 20,000
00:22:49.039 Lin of test code uh so we're getting a
00:22:51.480 lot of economy out of what we do with
00:22:53.880 100% Branch coverage as well that's
00:22:56.120 includes not every feature we want but
00:22:58.039 VM VMS DNS load balancers hosted
00:23:00.559 postgress minio blob stores a rich
00:23:03.120 authorization system web console and
00:23:05.159 then block device Key Management you
00:23:06.840 know so like all that's in there in 20
00:23:08.159 in 20 klck um so you know I kind of use
00:23:11.200 Ruby because I just it's so much more
00:23:13.039 economical to build than if I were to
00:23:14.559 especially I mean static language would
00:23:16.600 be very difficult and then I would say
00:23:18.400 that probably the other Dynamic
00:23:19.520 languages would be uh you know I
00:23:22.279 probably lose at least 30% I
00:23:25.440 think uh another thing is the language
00:23:27.440 has gotten more stable over all these
00:23:29.120 years uh in 2013 it turned out to be a
00:23:31.480 watershed year when they had a keyword
00:23:33.320 arguments and the language kind of
00:23:34.279 started looking like it looks today and
00:23:36.080 they stopped changing stuff this was
00:23:38.440 very important because I was pretty
00:23:40.120 scarred by the um 2011 experience when
00:23:43.120 there was Ruby 18 and 19 was very
00:23:44.880 painful and 2o was like lesser painful
00:23:46.840 but who knows if it would come back um
00:23:49.039 that's back when my sequel Ruled the
00:23:50.480 Earth at that time um Ruby and postos
00:23:53.400 were not yet married in the United
00:23:54.600 States rails was a MySQL program and
00:23:57.120 then at Heroku people people always ask
00:23:58.960 why are you supporting postgress that's
00:24:01.240 weird um so language stability also
00:24:04.240 helps I just upgrade these days it's
00:24:06.880 fine uh another times people ask oh you
00:24:10.120 know are you only doing all these tests
00:24:11.279 because you have stack typing I find
00:24:13.159 stack typing only actually gives you
00:24:14.799 very superficial
00:24:17.159 reliability guarantees if you can even
00:24:19.159 call it that uh they help catch typos
00:24:22.279 but what you really need is that huge
00:24:23.520 mass well not huge because it's actually
00:24:25.440 you need that you need that the huge M
00:24:27.559 Huish Mass of tests so that every time
00:24:30.039 you get the interesting kind of fault in
00:24:32.000 production the expensive way you can
00:24:33.679 capture it into the knowledge base of
00:24:35.000 your test Suite with 100% Branch
00:24:36.559 coverage you leak very little
00:24:38.120 information anytime you cut you get an
00:24:39.720 interesting error message interesting
00:24:41.520 result from like oh a dis a dis fails
00:24:43.960 and this particular path and get this
00:24:45.240 kind of message from Lipsy saying IO
00:24:46.799 eror you can mock it in without you know
00:24:49.760 tearing your hair out um so I find that
00:24:52.159 the tests are much better at
00:24:53.039 assimilating information than than
00:24:55.880 types so I when I tell people like well
00:24:58.000 it's actually more rigorous than types
00:24:59.320 because you can get pretty much
00:25:01.480 everything uh is St typing interesting I
00:25:03.679 think it is mostly for typo detection
00:25:05.360 and navigation so I do think the type
00:25:07.559 you know interest in typing in Ruby is
00:25:09.240 probably well placed rather than maybe
00:25:10.520 changing the language too much um but
00:25:14.080 you know we can kind of slow roll it uh
00:25:15.880 I'm very impressed with say Ruby LSP it
00:25:17.840 seems like it's come a long way um and
00:25:20.080 also when I use sorbet at the previous
00:25:22.039 uh project it was very good at finding
00:25:23.840 typos but it wasn't really great at
00:25:25.200 finding or accumulating information
00:25:27.440 about bugs
00:25:29.760 another thing that we use pretty
00:25:30.760 extensively is the repple uh here you
00:25:33.559 can see me commenting on a page with an
00:25:36.559 exact stack trace of what the problem
00:25:38.919 was so it's contemporarily noted the
00:25:41.240 methodology is very clear uh it's easy
00:25:45.039 to share I find that this is one reason
00:25:47.000 why I would probably never write this
00:25:48.000 kind of program in a non-real language
00:25:49.760 as to say almost certainly every static
00:25:51.600 language um so pretty much any static
00:25:55.080 language is pretty much out and really
00:25:56.880 you're talking about which Dynamic
00:25:57.960 language use because it's just the
00:25:59.320 note-taking abilities of using the rep
00:26:02.200 The Interpreter is I just don't see how
00:26:04.480 you could possibly write an efficient
00:26:06.000 operation without
00:26:08.039 it uh here's uh a fun example of this
00:26:10.919 there's this ad hoc report which you
00:26:12.279 can't even see because you can actually
00:26:14.480 I just pasted this into uh the terminal
00:26:17.840 to do this crazy report on it was
00:26:20.080 combining the GitHub API some secure
00:26:22.000 shell commands and um and
00:26:26.799 some and some database models in order
00:26:29.240 to report some stuff and that's very
00:26:30.480 useful to just hack that out solve your
00:26:32.679 problem understand understand the issue
00:26:34.760 move
00:26:36.559 on but the most important uh
00:26:38.919 recontextualization or re interpretation
00:26:40.640 I say for last and I feel like whenever
00:26:42.919 I you know even when I was invited to
00:26:44.200 give these talks kind of the
00:26:45.039 undercurrent of anxiety about
00:26:47.399 popularity uh so the thing is for
00:26:50.799 someone who's never been using rails
00:26:53.520 I've never been popular you know like my
00:26:56.679 you know my dependencies have always
00:26:58.000 been
00:26:59.039 uh extremely obscure and many ways you
00:27:01.159 know postgress was the odd one out back
00:27:03.440 then where said why are you in rails my
00:27:05.600 sequel world trying to roll this boulder
00:27:07.200 up hill at Heroku to make it postgress
00:27:09.480 we said because it's better uh you know
00:27:12.120 that's why
00:27:14.279 um and so barely Ruby and you know
00:27:16.640 postgress have swapped places where
00:27:17.840 postgress is now kind of in and like you
00:27:20.240 know pretty common currency and Ruby is
00:27:22.000 like oh yeah I've heard of it and yeah
00:27:24.000 know I guess it's good for some stuff
00:27:25.240 it's kind of they kind of swap places
00:27:27.000 but anything I got more strength by
00:27:28.559 getting of Sinatra and going to you know
00:27:31.080 what is the lesson here uh it's only
00:27:33.000 important to be adequately popular I
00:27:34.760 think uh really only a precious few eent
00:27:37.000 crack people are involved in good
00:27:38.520 libraries and I think more Engineers
00:27:40.640 should maybe have some Faith their
00:27:42.320 ability to assess what is good and what
00:27:44.320 is bad especially by popping open the
00:27:47.039 release process and source code of the
00:27:49.159 libraries they use and you'll find some
00:27:51.039 interesting stuff in there like for
00:27:52.320 example the Jeremy Evans
00:27:54.960 methodology uh so last thing is I could
00:27:58.279 really use your help and by using your
00:27:59.760 help I mean you could try out UV cloud
00:28:02.519 and Report bugs that's what I mean by
00:28:04.000 helping yes also you could pay a little
00:28:05.200 bit of money but really I want your bug
00:28:08.399 reports um thank you I have two minutes
00:28:12.600 left do you have any questions comments
00:28:18.679 stories
00:28:20.279 so active records and playing a long
00:28:23.120 game of catchup with I mean not only SQL
00:28:25.919 because I don't think they necessarily I
00:28:27.159 mean sometimes they probably look at it
00:28:28.320 but SQL kind of has is part of the
00:28:30.399 category of design data set chaining and
00:28:33.000 they've been you know it was kind of a
00:28:34.200 massive effort but you know they added
00:28:35.320 RL and so forth to kind of make it more
00:28:37.640 powerful but I think there's even still
00:28:39.159 some stuff you can't express the Gap I
00:28:41.159 think maybe has been narrowing somewhat
00:28:43.240 although I think a lot of people who are
00:28:44.399 efficient Auto say well you know SQL is
00:28:46.120 faster and more expressive and more
00:28:47.279 straightforward by the way really good
00:28:48.720 to use llms to help it write the DSL
00:28:51.799 good good tip um even I know it I I
00:28:54.960 still use it to avoid it's coing the
00:28:57.200 manual I I think the test performance is
00:28:59.240 probably why I would do it you if you're
00:29:01.480 really into going the 100% which I think
00:29:03.640 you know that is where really could add
00:29:05.279 up okay 60 seconds all
00:29:09.080 right yes please help us find bugs rough
00:29:11.840 edges and we'll understand you a little
00:29:13.399 better when you tell us what's wrong and
00:29:14.960 um also you might get a kick out of be
00:29:16.240 able to understand the patch that fixes
00:29:17.720 your problem because unlike other
00:29:19.960 language communities you can't just
00:29:21.440 crack it open be like ah yeah that's
00:29:22.679 what they
00:29:24.039 did oh wow that was dumb why' they do it
00:29:26.360 that way originally
00:29:31.039 all right
Explore all talks recorded at RubyConf 2024
+64