00:00:00.880
um so hello R uh as she said he's an Natoli I'm Christian we work at zendesk
00:00:07.439
UH zenes is a pretty big company that was founded and still to today uses rails uh and we're going to talk about
00:00:14.120
the history of the last 15 years of it um I think it's actually very nice that
00:00:19.279
we are talking here in Verona because I see a certain relationship of the city or of the most famous story about the
00:00:24.760
city with the real Community you know again this talk is going to be about the history of rails and history of RA
00:00:31.800
community and you will be thinking what are you talking about how is this related okay spoiler alert so if someone
00:00:38.800
doesn't know what Romeo and Juliet is about you you can leave the the room now I mean you're 400 years old 4 400 years
00:00:45.960
late but well let's say Romeo and Julet is the story of these two families the montos and the capuletos that they have
00:00:52.480
fighted for generations and then two teenagers of each family fall in love and this kind of triggers a huge drama
00:01:15.640
no it's maybe the biggest drama in history so how is this crazy drama
00:01:21.560
related to a ruby Community I mean think again about that how is drama related to
00:01:27.680
the ru community I mean come on I did a recopilation of
00:01:32.840
Dramas apparently there was a website called ruid drama.com that that put all
00:01:37.920
of them and you see only you can stop the Ruby drama millions of unicorns die each year due to un necessary drama in
00:01:43.640
the Ruby Community I mean don't get me wrong I I love drama huge entertainment
00:01:48.960
value like for a while I tried to be a Scala developer that was terrible times and I went to conference in Scala and
00:01:55.960
you see like the typical speaker in those conferences looks like this you know it seems like
00:02:02.119
AI generated while our our typical speakers in a Ru conference look like
00:02:17.920
but I mean others we some of us know how to address to impress but he is a bit
00:02:23.360
like this okay so introducing the talk uh this there is a reason why I wanted to
00:02:29.120
talk about the passage of time it's a bit of a special moment in my life uh I am originally from Barcelona but I mov
00:02:35.400
abroad for 10 years first to Denmark then to the US and I I moved back
00:02:40.599
recently I also made my 10th anniversary working for zenas and finally this kind
00:02:46.080
of a ner thing but I watching this TV show called feren does anybody know what
00:02:51.159
is this okay so so basically friend is this typical anime in Award of elves
00:02:56.760
Warriors wizards but the main idea that they use there is the passage of time
00:03:01.800
because friend is an immortal being and and they play a lot with the idea this person has seen it all before moreover
00:03:08.280
in my opinion uh in Fr they use magic as something that can be read as technology
00:03:13.840
or at least you can see the progress of magic that follows similar patterns to the progress of Technology this actually
00:03:20.080
not something super new my favorite fantasy novel uh a wizard of SE by UL
00:03:25.760
lein that's that's that's incredible yeah I always read it in the same way so in in the magic in this world what was
00:03:33.040
the what the following way if you know the true name of things you have po over
00:03:38.360
that thing I always think that it's about it's like object in the programming you create a perfect
00:03:43.480
abstraction that is impossible then you you you gain power over over over those objects we all work it with terrible
00:03:49.920
abstractions and we know the difference no I mean again all these ideas are not mine arer has a very famous quote about
00:03:56.599
the relationship between uh Magic and technology and it's L advanced technology is indistinguishable from
00:04:02.920
Magic but well going back to time and and Fen You need to think that in this
00:04:08.000
world uh it's very structure like the show is very structur around flashbacks so you see here when magic was something
00:04:14.680
very very rare then you see her one her years before where there was a spell that was considered incredible and then
00:04:20.759
you see here in the present that that same spell is now the basic the the base of everything it's something similar to
00:04:26.440
H that happens in technology that something amazing 10 years ago ago now it's completely normal more thing for to
00:04:33.520
and me it was natural for us to identify with his character like frin is an elf
00:04:38.759
that has Liv over a thousand years old uh she's the that makes her the possessor of forbidden forgotten magical
00:04:46.039
knowledge and I would say that we are equally mythical me we we are Engineers
00:04:51.400
that have worked in the same company without changing for 10 years that makes us the possessor of
00:04:57.360
Forgotten forbidden magical knowledge and we are here to share it uh we will
00:05:04.360
start the discussion talking about the monolith both how the monolith evolve at zendesk because U zes was originally a
00:05:12.199
pure rails monolith created in 2007 but also about the concept of monolith I've seen a lot of things changes like I
00:05:19.000
started working with Ruby late 2000s early 2010s um I was was way younger
00:05:25.080
then but you know the 2010s were weird times you know we like all kind of
00:05:30.440
strange stuff uh and and then in the late 2010s the idea of monolith became
00:05:36.039
not cool anymore I remember going to an interview and when I said yes I think monolith makes sense in the context of
00:05:41.759
startup they look at me like go away you know uh now kind of the concept of Mor
00:05:47.360
is trendy again or at least not as bad as it used to be for me this is not a
00:05:52.560
surprise I personally believe that a lot of uh debates that we have in in Tech uh they they behave like pendulums
00:05:59.039
something that was was completely mainstream 10 years ago then becomes terrible and then goes back to a more
00:06:05.080
balanced position uh understand here I don't mean that you can say whatever and nothing
00:06:10.919
nothing matters there are debates that uh that uh suggest a very dangerous
00:06:17.000
pendulum for example uh six seven eight years ago I think uh relational
00:06:22.120
databases were there were at this trend like no SQL which makes sense in certain context but there was a lot of people
00:06:27.280
saying that we shouldn't use relational databases at all and SQL was was kind of dead or in the way out well that was
00:06:33.880
that wasn't the case eventually and if you think that this was a uh spicy take
00:06:40.039
I have one way more spicy I actually think that the boom of microservices in the last 10 years is a zero interest
00:06:47.160
rate phenomena and by this I mean tech companies had so much money that uh they
00:06:52.440
went in a lot of cases for pre preemptive optimization I am not the only one that
00:06:57.800
thinks this actually uh one or two months after I did this presentation the first time dhh put this this tweet which
00:07:04.639
I kind of agree maybe I wouldn't WR it that way but it's the HH um but I mean I
00:07:11.160
can I I've seen this because uh I remember a friend of mine that was talking me about his new startup he just
00:07:17.080
joined and he was describing to
00:07:25.720
me he was describing to me this incredibly complex architect Ure I couldn't understand but I was thinking
00:07:31.840
well I'm happy for you because this seems like very popular said no no we have 10 customers it was like okay man I
00:07:37.280
don't know moreover uh now that unfortunately like uh you know there is less money in
00:07:43.520
the market when companies need to reduce amount of Engineers having to support uh
00:07:49.120
2,000 Services is going to be very very very very hard like this from a newsletter I found where some company
00:07:54.879
complained about this say this I do believe that microservices may sense in
00:08:00.240
in a lot of us cases I have a very different opinion from let's say El musk that called microservices blw I mean I
00:08:08.400
find this hilarious but I I wouldn't be go that far uh my opinion is more similar to Jason Warner the C of GitHub
00:08:15.319
he believe that there is like a spectrum a lot of times when you are a startup or you create an application that needs to
00:08:21.720
try to find market field let's call it uh you need to start with something that you can evolve very fast and then slowly
00:08:28.639
break it down into to different Services actually with this we go back to zendesk because internally at zes we
00:08:35.080
had a lot of debates about that but somehow what we end up doing is kind of like that we we broke some parts of the
00:08:42.479
of the monolith into services but I still think that if you are someone on boarding into Z in4 you will feel that
00:08:50.080
it's similar to a monolith it's pretty monolith uh said this we had different
00:08:55.760
Visions during all these 15 years originally our our first Morton had this idea of I'm going to
00:09:02.160
create a monolith in rails but then I'm going to extract part of the logic so it's perfectly modular and then in the
00:09:08.640
future I will be we will be able to spin new rails applications that have a lot in
00:09:13.800
common if you check the the gem file of the monolith desk we have a lot of gems like that and actually not we don't
00:09:20.800
create applications like this anymore but we have a few applications that do that they share code through
00:09:27.120
gems then we move to what I call the service ERA this came a lot because we
00:09:32.959
acquir companies and we wanted them to share data across the the ecosystem for
00:09:38.399
example we acquire weire a chat company we wanted to share accounts it didn't make sense for us that this the chat
00:09:45.079
product calls the support product to to ask about accounts so we created an account service and finally these days
00:09:51.920
we are doing more of an event driven architecture so if something happens in the monolith it generates a gafka
00:09:57.399
message that's consumed by other services and this triggers multiple actions but going back to the monolith
00:10:04.160
our way to to grow it now is experimenting more with mization uh we are playing a bit around with pwork that
00:10:10.720
is a gem created by Shopify that helps in in this regard and I would be but I wanted to
00:10:16.920
comment a little bit on the front end what happened in the last 15 years in the in the front end side of creating
00:10:22.600
applications well again uh I joined the the the community in 2010 which was
00:10:28.440
strange times and at that time is it's when it started
00:10:34.240
one of the most violent conflicts in human history which I call the great JavaScript Wars I
00:10:40.720
mean I mean if you if you remember if you you were active in those times and
00:10:45.880
you entering Hacker News Chan is that every week there was a new a new JavaScript framework this still happens
00:10:52.160
today but the difference is that then all of them were somehow getting traction it was very hard to follow like
00:10:58.399
at Z weu used a lot this three uh backbone Ember and react uh and I'm sure
00:11:04.200
that somewhere we use also angular and other stuff but but this one very very used I mean there is something about
00:11:09.639
JavaScript Engineers that they seem to love creating new Frameworks we have something called CJs KJS you don't know it for sure
00:11:17.120
because the real is that someone called cart nobody at Zer remembers his last name anymore created his own
00:11:23.079
framework and is undocumented of course and we are building on top of CJs it's
00:11:29.000
something fascinating um but still it seems that in the last years we kind of agreed to
00:11:34.760
go on the r way yeah there is there are other Frameworks but seems like react is kind of the king of the hill at the moment and I like react I I use react at
00:11:42.240
work I use specifically react native a lot for my own applications but sometimes I feel that the this victory
00:11:47.920
of react is just because people got too tired you know it was just just hard to follow to follow with the JavaScript uh
00:11:56.000
ecosystem the next part I would like to talk about deployment uh we don't have much time and this is like a very long
00:12:01.480
long story but uh let's say that when uh Z was started we went using private
00:12:07.399
server servers like engine yard Rackspace and with each of them we kind
00:12:12.720
of went over the limit that what they were expecting uh it it actually at that
00:12:18.040
point it just made sense to go uh for own metal I think at that point we were
00:12:23.560
kind of a little bit pioneering pioneering this like we didn't have the right tools So eventually we moved to AWS and we've been in in AWS uh since
00:12:31.800
then if you want to know more about this uh we have a series of of articles about that with a with an interview with the
00:12:38.240
first C of zenas that can illuminate that better than me the next part yes I know that this is
00:12:44.240
a ruby conference but I wanted to to talk about how do you adopt new technologies in a big
00:12:49.880
company again we are a ruby shop more or less but rub is very important for us
00:12:56.800
but then of course you you start thinking how do you adapt new technologies into the stack well when you're a startup it's very easy a lot of
00:13:03.440
times you can be even one engineer that decides to hey let's use K yes or let's use something and you talk with your
00:13:10.399
manager and then that's it but the problem is that when you're a company that has around a thousand a th000 Engineers that that doesn't scale that
00:13:17.639
becomes foolish um it's the the cognitive load of having to support hundreds of languages is just too much
00:13:24.720
so we created something called the tech menu so the idea of the tech menu is that hey you want to add I don't know
00:13:31.839
Elixir to to the to to the systems that we use at sendes to the pring languages great do a proposal we have an
00:13:38.600
architecture team that decides which in which path we should go and if it's accepted that's it you can create applications on
00:13:44.760
Elixir uh it's funny that I use l here as an example because the I think French have a word for that like C CBR like
00:13:51.560
famous case uh was about which language should we use for creating Services uh I
00:13:57.120
was in team Elixir that's why it came to my mind and I kind of lost so we we don't build
00:14:02.399
services in using alexir at zesk we use two languages jav and Scala one thing
00:14:08.920
that was very fascinating is that both were accepted because two very significant parts of the company opposed
00:14:14.600
a bit they they both defended them um and for a while it was completely accepted and equally defended to use
00:14:21.360
Scala and Java uh I will tell you that as 2024 uh we kind of prefer to uh create
00:14:28.759
service in Java and interestingly the reason is that it was very hard for us to hire people in Scala uh to train
00:14:35.360
people in Scala it just became very hard to maintain our Scala systems so we just prefer Java these
00:14:41.199
days but actually the the main way that we adopt new technologies that desk is be Acquisitions you can say whatever but
00:14:48.000
if the com you can say whatever about python for example but if you if the COO
00:14:53.079
once a company and this company is using python python python enters the stack
00:14:58.199
for as are important around half of our products came be Acquisitions and I will just ex
00:15:04.920
exemplify this with the our first acquisition that was a company in Singapore called zopim zopim is a chat
00:15:11.399
company and the product now what this is called zenes chat as I was putting example they were using Python and
00:15:18.320
python is a zes language but from a software engineering perspective I think the most interesting part what I call
00:15:24.720
domain Fusion we are a customer support company we have the idea of ticket of user of ticket is the the main Center a
00:15:32.079
chat company has a concept of chat I guess conversation whatever it's completely different
00:15:37.880
domains and then suddenly all the chats in this company need to become ticket somehow because they are now not a chat
00:15:44.240
a neutral chat company they are part of a a customer support product so like in St Trek think they're called as zcs you
00:15:51.160
know that you need to absorb and make two one out of two so again what is a ticket what is a chat now we have a
00:15:57.959
protocol things and this voice that phone calls are also part of this customer support uh flow I mean yeah
00:16:05.120
this is a product problem because it's very important from a pro perspective that your customers feel like they have
00:16:10.440
a unified experience but it obviously has theep consequences on the technical side now
00:16:16.199
some something that was modeled for years as a conversation needs to be part of a ticket eventually what I felt is
00:16:22.800
that what we needed is uh shared features uh common features into very different objects I would like in
00:16:29.759
interfaces and the final solution is that everything eventually becomes a ticket for us so any object created but
00:16:36.680
by an application calls the monolith uh originally via an API call now through
00:16:42.759
Kafka messages and generates the basic slate for everything that is the ticket here okay and after this kind of PR
00:16:50.040
conversation an here will tell you reality is about performance test test test hear
00:16:56.319
me so I had a very boring 30 slides and then I talk to badar and now my slides
00:17:01.959
are not boring anymore I hope all
00:17:07.439
right this is the photo I have taken probably U I took probably maybe two years
00:17:13.760
ago has anybody uh been to Dublin about Dublin weather okay quite a few people so we
00:17:21.160
believe that we don't have any summer and if you want to have summer you you come to Italy right well this is what I was thinking
00:17:28.319
and then I I checked Wikipedia and and this is what I found how many Sun hours we have per
00:17:34.320
year in Verona versus Dublin I was shocked I was shocked so D supposed to have zero hours
00:17:43.400
no it's not it's not zero and why am i showing this slide is the
00:17:51.760
system uh we think about a large
00:17:56.960
system it's very similar to where better we think it's something one but
00:18:02.039
it's something else it's completely something else and now attention B has
00:18:07.520
to smile and everybody has to
00:18:13.120
say I love it okay so this is the photo I took uh 7 years ago in
00:18:21.240
Dublin with three three days of snow uh no like no traffic No Buses no flights
00:18:27.960
nothing so the system we work with at zendesk is
00:18:34.520
similar it's unpredictable very unpredictable this the photo from San
00:18:39.559
Francisco where we have a headquarter it's supposed to be sunny but it's not so again it's a a
00:18:46.679
challenging weather and sometimes the system makes its own decision for you so you just you
00:18:53.400
you don't ask and and system behaves one way but you can't explain
00:19:00.000
and next slide I stole from Christian I think that was my only funny slide and I made it even more
00:19:06.760
funnier all happy systems are alike each unhappy system is unhappy in a own
00:19:13.640
way yeah I have to take a 50 seconds pause but I don't have too much time for
00:19:20.120
that uh yeah it's like it's very it's yeah okay I'll keep it because I don't
00:19:26.919
have time as then transition from AWS uh from data center to the cloud in 2016 I
00:19:34.679
believe 17 yeah that's when we started the process I think we ended in 2018 I think 2018 when we finished and if you
00:19:41.720
ask what was the uh the biggest challenge with zesk
00:19:46.840
uh it was the database so database performance was the reason why did it take
00:19:52.760
longer and why we had many problems than we expected and does anybody you know I'm
00:19:59.120
the law I'm the law so if your system is stateless you have no data it scales
00:20:06.120
forever it scales like well but does anybody have no data in your system no
00:20:11.760
no hands one hand no hands so as soon as you have small data
00:20:17.360
set performance degrades but as as soon as you have large data set like we have at zesk you hit the moment of contention
00:20:25.320
and now I should have badar slide I don't know why I don't have it
00:20:30.559
here I think I have it a couple slides later yes
00:20:36.280
uh Happy large systems have one thing in common and this is the data this is
00:20:42.679
Happy databases happy performance databases uh
00:20:48.600
there's no large system which is safe reliable past with a unperformed
00:20:56.280
database and performance database have one thing in common
00:21:02.240
and you have to pick two out of three okay you either have consistent run time
00:21:08.080
and complex queries but small data set you have complex queries large data
00:21:13.720
set but forget about
00:21:19.120
performance that's it okay thank you and this is exactly what happened to
00:21:25.919
zes this is our past data set was small query run time was
00:21:33.240
relatively consistent and we could do anything we wanted anything absolutely anything the future
00:21:41.799
actually present is little bit different uh query around time took the back seat
00:21:48.520
because data set took over the future is different the future is no complex
00:21:56.960
queries whatsoever uh large data set will dominate and performance of the
00:22:02.440
database and API and points will be consistent back to Christian yeah and
00:22:09.720
after this that actually is what you should keep in mind because what I say is mainly mainly stories um um we'll
00:22:16.200
talk a little bit about testing and reliability like how we use test zendesk to keep our hundreds of thousand of customers uh
00:22:24.440
happy but before that I would like to comment a little bit on a on a typical debate in the Ruby community
00:22:30.840
that is tests versus types so in the 2010 like the proposal of Ruby was that you don't need types you have test and
00:22:37.480
you need test anyway so you don't need types uh with the time in the late 20110 or maybe more more more recently I think
00:22:44.880
in the community there been a bit debate about that I think Ruby 3 supports type checking uh natively we also have svet
00:22:51.480
that is the G that adds type checking uh so there's a debate here um I will tell you at zesk in this debate we are EX L
00:22:58.799
on the side of tests and for test test for us are incredibly important because
00:23:04.279
we have a lot of code around 1.6 million lines of code and we have a absur amount of test
00:23:11.320
how many test do you think we have like unit test I'm not counting uh i counting uh feature test or integration ones say
00:23:20.880
numbers no no how many test un need test
00:23:27.159
um yeah yeah okay so well close 55,000
00:23:58.840
uh this is his real Avatar in the slack let me merch or I will cry like literally a few months ago we were on
00:24:04.520
boarding a new Junior engineer and I was waiting for him to complete the feature say hey Christian I'm sorry I couldn't
00:24:10.039
match there is this guy that is telling me that my test are wrong and I cannot match this and I was like oh you met
00:24:16.960
Ryan um some of this you may know Ryan because he's also the Creator and maintainer of mini test and actually
00:24:24.240
this brings me to the second comment that most Junior engineers make at the desk why mini test and by why mini test
00:24:30.919
they mean why not a respect which is I think the most popular test framework in in the Ruby Community well there's a
00:24:37.919
good reason we do have a lot of test and to make it even worse we have we run uh
00:24:44.000
all the all the test Suite twice every time there is a push later will explain why so this means that every time that
00:24:50.120
someone does get push to GitHub we're running 11,572
00:24:56.240
deaths uh so the advantage of me test against our spec is that smaller less magic and it's way faster I told with
00:25:03.279
Ryan and he told me that other big companies using using rails like uh Shopify also prefer mini test for this
00:25:09.520
reason in fact when we try to modify mini test we try to make it less less
00:25:14.840
magic less faster we care about that still despite that we have pain points in our testing
00:25:21.200
pipeline they are flaky test a flaky test for us can be terrible for productivity because again in imagine uh
00:25:28.960
this is like a a real Snapshot from from our PRS there is like
00:25:34.000
120 I know how it's called checks being run each check is around 2,000 test so
00:25:39.880
if you one test is flaky you need to rerun it this means that you need to rerun 2,000 test that they can take
00:25:45.240
around 20 minutes that's pretty painful moreover now we're in a much better world because I don't know if you
00:25:51.679
remember but until three years ago you couldn't rerun one check in GitHub actions you need to run all of them so
00:25:58.679
this means if you have one flaky test this means run over 100,000 test that was very painful and was a very happy
00:26:05.480
day when when you have released that um but say this I would say that the true secret for uh for uh reliability for us
00:26:12.440
is ownership everything needs to have an owner why well we have a lot of
00:26:17.960
code and we have a lot of committers I checked the the monolith and there's been 2,459 different accounts of with
00:26:26.320
that uh push code into into my in domain and okay you may be thinking well this application has been around for 17 years
00:26:32.919
of course most probably lately there's way less people well the last year 570
00:26:39.000
different people so a lot of people pushing code and if you don't have pretty clear
00:26:45.039
processes you're going to end up in a fight it's going to be very hard to know who did what who uh had responsibility
00:26:51.960
of fixing certain things so we have a rule that you all all files are owned by
00:26:58.240
at least one team uh like this it makes it way easier if something breaks to find who who
00:27:03.760
needs to fix it or who needs to work on that is super important but still of course I couldn't have a talk about uh
00:27:11.480
history of rails without explaining what happens when you upgrade rails because again zenes is kind of ancient uh what
00:27:19.120
started in rails one actually there is a there is an a story about that uh like uh the original founder of of of zenes
00:27:26.679
Mikel uh used to work with dhh before before dhh created rail so they know each
00:27:32.559
other uh we used tests for upgrading rails how do we do that well remember I
00:27:39.039
told you that we run all the test twice all so what we do is like of course we run all the test with the current
00:27:44.760
version of rails and then we run them again with the verion that we move want to move next the idea for that is that
00:27:52.320
if it's red that's optional that's fine but once a test becomes green we do not
00:27:57.679
accept that this test goes back to red again like this we because imagine like
00:28:02.919
you are trying to upgrade the version you're fixing problems but someone introduces something new that isn't
00:28:08.440
compatible you will never end like this we we can we can uh guarantee that uh we
00:28:13.679
will have dations I think they're called um still we have had historically pain
00:28:19.360
points on the on when raing rails and I think this something can be pretty interesting for you because again I've
00:28:25.399
been around since 2010 at the time the there was something that was incredibly popular in this community that was not
00:28:32.279
popular at all in other programming communities and it's one reason why I started doing Ruby and the next slide
00:28:40.399
hurts my heart because I love the book but this is the the reason meta programming I literally fell in love
00:28:47.240
like I literally have the book signed by by Paulo Pera like with know like he's my idol but oh my God metam is such a
00:28:54.600
pain when you go try to upgrade we had until rails 5 we had our or Code full
00:28:59.919
full full of monkey Poes some ghost methods like we had gems that we were
00:29:05.080
using the main branch but then we had modified versions of the gem for us it was terrible eventually it doesn't scale
00:29:10.360
it became just too the cognitive load is just too hard still we move for a lot of
00:29:16.320
years we were keeping this app until it got to a point of upating to rails 5 basically when we try toate ra five all
00:29:22.679
test broke like it was nothing worked so at that point we decided to go hard mode
00:29:28.640
and remove all the metaprogramming or as much meta programing as possible from the from our
00:29:35.240
app moving from rails four to five took two years and a half of and with people
00:29:40.880
dedicated to it uh this is like the last year of of
00:29:46.519
progression amount of Errors failures how how much it went and you'll be thinking what were the specific issues
00:29:52.279
that were crazier where well we had our own Branch basically the most important things were ISS ues that were uh
00:29:59.240
distributed across the whole the whole code base for example stronger parameters we were using a a fork of
00:30:06.000
stronger parameters and when uh when we arrive at five we decide this doesn't
00:30:11.399
make sense we don't want to be maintaining a stronger parameters forever our own version let's go back to rails the rails way this was hard still
00:30:20.080
the most famous issue that uh we had and it's something that the community got a bit angry at the time was dirty
00:30:27.080
attributes that is what actually a thing with ra 52 I don't know if you remember this but the idea is that so you have an
00:30:33.919
object in ra 51 you change the the an attribute and you save it you do user do
00:30:41.279
object do change it will return true in ra 52 was decided that if the change has
00:30:46.320
been committed change the method change should return false we're using a lot of
00:30:51.720
logic based on that and actually this was not even like oh there's an exception now it's just that it just
00:30:57.519
returns different value uh so this actually I was talking with Ryan about this and he is very
00:31:03.639
strong believer that this should this should have a minor change and should have been in real six but it it was
00:31:09.279
really really problematic for us I mean this thing alone added six months to the upgrade and there is a reason for this
00:31:14.600
this seems crazy but imagine that you have an application that is being used by 500 Engineers every year that means
00:31:20.919
around at least 100 teams you need to communicate with 100 teams tell please could you check that those hooks that
00:31:27.360
are broken fix them everybody have their own road maps there's communication issues you discover that some endpoints
00:31:33.320
don't have an owner um it's really hard to change the good news is that since then
00:31:39.279
it has been way easier and now more or less we keep up with the latest upgrades still maybe the lesson I want you to
00:31:45.279
take is what I call the first mover disadvantage so if you have there is something that you want to do and nobody
00:31:51.519
else has done it before there's going to be a price that you're going to have to pay one example uh sharding we've been
00:31:58.279
using sharding for a long time since probably rails to and we have our own gem our own stuff um recently well
00:32:05.960
recently no maybe five five years ago um that's probably before the pandemic uh
00:32:11.240
rail six came with sharding like uh out of the box we're still using our own
00:32:16.880
version of sharding and we want to move to to to main uh to the main branch of
00:32:23.000
rails of sharding so my advice after all this would be to join the party essentially
00:32:30.080
don't develop something by yourself closeing your little room or your little you know private repo if you find a
00:32:36.480
problem and you think there is a solution like mainstream create an issue talk with people maybe you will find
00:32:42.679
that someone is already working that solution and you can join or maybe something new and you can become a r
00:32:48.120
contributor what I don't recommend is just maintain your private solution for years because eventually there's going
00:32:54.000
to be another solution that will be public and it will be maintained by a lot of people and eventually that solution is going to
00:32:59.760
be better than yours and you're going to have to adopt it okay we are we are getting to the end
00:33:07.880
um this presentation is a bit of a sequel of the previous one that we did uh last year we were talking about uh
00:33:14.159
performance uh scaling an application but from a from a performance
00:33:19.960
perspective uh the things that iong with people I realized that not being to scale an engineer organization not being
00:33:26.279
to scale processes is way more problematic I mean in the end performance problems are kind of
00:33:32.600
easy to the de so if application is too slow it doesn't respond well to have a huge problem but uh the slowness of an
00:33:39.000
application is kind of obvious up to a certain point with processes organizations is completely different
00:33:45.559
point like even what how a a wood organization looks is kind of an object
00:33:51.200
of debate it's not very clear to me so then I start to think okay how how how
00:33:56.360
do I imagine this ideal organization how do I imagine this this ideal process at this point I start to think about the
00:34:02.399
past and you know all the things that i' I've seen at zesk or in different companies and seeing well maybe I can
00:34:09.960
take things out of what I've learned and this you know when you you've been around in a in a in a company or in a
00:34:16.599
place you feel a like Gandalf like oh I have this secret knowledge that I can share with people know I have power no
00:34:23.000
but then you start to realize and I saw this a lot that pass can make you also like s man you know so someone corrupted
00:34:28.679
like because the pass can become this little thing that talks to you say this is not the way to go I mean in 2005 you
00:34:34.760
develop an application with completely different par completely different paradigms it's such complicated and
00:34:40.760
there is an anti pattern I discovered this the other day called the Frozen caveman about this it's like Oh My My
00:34:47.560
paradig my patterns have worked for me forever so they will work like this and
00:34:52.599
it doesn't happen particularly it doesn't happen in Tech so what's the right attitude here
00:34:58.440
well I think that you need to learn the past but take some take some distance iine like a Bart you know like someone
00:35:03.800
that comes here and explains your story and you just need to pay attention maybe think oh the story was nice but I didn't
00:35:09.720
learn anything out of it or maybe say well I can apply that so you should know the past and take your decisions it's a
00:35:16.760
bit like like in Dune I don't know this is not very clear in the in the in the movie but it's more clear in the in the
00:35:23.640
in the in the novels so in the novels he's always able to see the future
00:35:28.800
but then only if you've seen the movie you remember that scene he drinks some kind of water all that and then in that moment He's Able also to see the past
00:35:35.800
and just when he's able to see the past he's able to see way way way farther into the
00:35:41.000
future um I think that if you are able to do that you will be able to to H the
00:35:46.079
Ruby Community much more and maybe you can continue doing uh presentations like this in in 10 years thank you
00:36:00.520
this was incredible and I I I recently joined a company that has some of those
00:36:07.520
problems it's uh I can't relate um do we have any questions one
00:36:14.640
over there and one over there and another one over there so shall we go
00:36:20.319
left to right um I can share my that's okay
00:36:31.040
hello um I'm curious to know uh how long does it take to run the entire test
00:36:38.440
suit and just to confirm are there 60
00:36:43.599
instances running in parallel of the and and also what do you use to parallelize
00:36:49.720
that that's interesting I I will have to check it runs less than 20 minutes uh
00:36:54.800
and because we're running all in parallel of course if not it would be it be impossible what do do you remember what we use for running the
00:37:04.160
test yeah I can check if you give me the contact I can tell
00:37:12.800
you check later after
00:37:18.200
yeah uh sorry I'll come back uh where was the over there and you right come
00:37:26.119
okay thank you uh you mentioned using local
00:37:33.240
gems to break some of your app complexity I guess one question that is did you have a dependency structure or
00:37:39.720
all those gems that we saw in that screenshot required by the main app and second about uh packwork sounds like
00:37:45.839
you're using some of that as well curious about your take on the different approaches yeah so uh actually good
00:37:52.480
point I mean uh um historically like the the idea of using private gems you know
00:37:57.560
like you take that uh eventually didn't work uh because it is what you say we didn't have like clear dependency maps
00:38:05.160
and right now some of those gems are so strongly entangled with the monolith that you cannot use them anymore like
00:38:11.960
that's uh yeah uh we have some exceptions and this because we created pretty early a pretty successful
00:38:18.200
application and in this case it's very clear that you cannot create PRS that entangle too much the the gem with the
00:38:26.280
with the month uh on pwor we actually just payment at this point I I very strongly believe in it because because I
00:38:33.079
believe in Mor but you need to be well organized Mor like sometimes people says oh Mor are like that and it's not like
00:38:39.359
that it's just a spaghetti monit you know uh that the fact that you have Mor doesn't mean that you can you know have
00:38:45.000
a complete mess up uh map of
00:38:51.359
dependencies and one more question here yes probably it's a silly question you
00:38:56.480
said that you test twice for every PR right I suppose the twice is one just as you file it to to start the review and
00:39:04.359
one is a presubmit check just before it get the merg or is something else no it's something else so it runs in
00:39:10.640
parallel they both runs but it's not exactly the same PR in the way that one run with rails the current version of
00:39:17.079
rails and the other with ra rails 7.1 for example so and the idea is that um
00:39:23.599
if uh once a test is green we don't allow it to go back to red because a lot of times our rails team says well I just
00:39:30.520
fix this but now there is a new hole so they wouldn't end it wouldn't do this but when you don't change release if you
00:39:36.000
just have a PR with no release change what release with you're using the same
00:39:41.640
ra version you're just changing something else without changing the rails version no no like like it's so
00:39:46.680
one run is with Rail 7even and the with rail 71 but the thing is that to make ra 71
00:39:52.760
completely green it takes time what we don't accept is test that are fine are are green in real 71 to break them
00:39:59.800
anymore so at any point in time you always have a current version and the next yes version okay sorry I missed
00:40:06.040
that I thought it was like a one exception event not at all events okay got it sorry MH
00:40:11.720
yeah last question I have a lot of questions but
00:40:19.079
let me ask one uh more about the how you organize the code because you're starting with rails 1.0 right so so a
00:40:26.560
couple of one question it's in tests let's say for the test what folders do you have how do you organize
00:40:33.440
them and how do you connect them with the code because when you want to change the code you want to change the test
00:40:39.079
right yeah in the monolith in in the r of the monolith we only have unit test and those corresp it's like test and
00:40:45.200
they they correspond to the file uh both for feature test integration test we actually have two
00:40:51.319
completely different projects one for the API and the other for the browser and they run completely we use Jenkins
00:40:56.760
and they are run completely you know a completely different way okay
00:41:04.520
yeah sure so still about the organizing code so are you using how do you
00:41:10.319
organize again about ra monolyth right so how do you organize your code let's a
00:41:16.040
bit about code design let say are you using name spaces so I'm not talking of P pwor I know how it works but the one
00:41:21.960
that is not there right are you namespacing things um are you yes
00:41:27.319
putting everything in up models or I don't know what how how is Oran we use quite a lot of name spacing like part
00:41:33.119
except even models are name space at sometimes but uh we use a lot of name spacing for anything related like for
00:41:39.240
example ticket again I was saying that everything is a ticket ticket itself like the model it's probably 2,000 lines
00:41:45.960
of code but we have a lot of modules that are included into it and of course they are they are Nam space under ticket
00:41:52.400
and where is this ticket in which folder do you have uh it's a app models ticket
00:41:57.440
uh so you have app models ticket RV and then you will have a folder called app models ticket yeah yeah that so yeah
00:42:03.480
ticket is a is a yeah I get it thank you thank you very very much enjoy the
00:42:57.359
oh