00:00:14.799
Okay, so I'm not a bird. This is not my first talk at Wroc Love RB. Although I've been here before, there are only two or three people in the audience who know who I am. For the rest of you, perhaps it's better that you don't.
00:00:20.070
I was here in 2015, and I gave a talk about how, as a Ruby back-end developer, I reignited my love for front-end development when I discovered ClosureScript and React. It turns out that front-end development isn't crazy; it's just that the tooling was. If everything is just 'view equals function of data', then everything becomes nice and pleasant. I actually enjoyed front-end development.
00:00:33.199
Two years later, I returned and shared my experiences with you all. I told you that I had been to the woods, and I came back to discover that you can do even back-end development with Closure and immutable data structures. You can build entire systems with this approach, and it's not just limited to front-end development. There's great tooling available with Closure Spec and other such resources, providing functionality similar to that of type checkers but without some of the hassles.
00:01:01.710
Then, two more years passed, and I gave a talk at a conference, discussing how not only can you do front-end development with this technology, but you can also replace your database with the concept of immutable data. I explored the idea of how databases might work if we only had immutable data and a solid data flow concept. It turns out that they actually look really nice.
00:01:19.440
My talk at that conference was great because it was recorded, and if you're interested in learning how to replace your relational database, NoSQL database, or Redux state machine with something more scalable that adapts to changes in your domain over time, I highly encourage you to check out that video. But I’m not going to repeat it here; instead, we have a new slot and a new topic for this talk.
00:01:52.020
I want to close the loop. I've discussed front-end development, back-end development, and how I replaced my databases. The only missing pieces are APIs. Now we have a fuller picture, and I can finally go home and write Lisp all day long. So, today, this talk will focus on how to build APIs using the concepts I’ve been sharing for the past five years.
00:02:36.410
There’s a recurring theme here. What is that theme? Well, aside from my presence every two years, there was a photo tweeted from my last visit with a caption that read, 'When you use Closure and haven’t told anyone for five minutes.' I was disappointed, and let me explain why.
00:02:51.100
They completely missed the point of my talks. If they had tweeted something else, I would have been guilty as charged, because my facial expressions change every time I hear someone describe their problem. Often, it turns out they could solve their issues using declarative data structures and simple functions. Anyone who's talked to me after a party knows this to be true.
00:03:29.520
So the question remains: how can we take this approach and build APIs with it? I’ve tried to distill this down into about a four-step process because if it takes more than four steps, nobody's going to do it. Here’s how to build an API using the concepts of immutability, EDN, context-free structures, and flat data.
00:03:54.480
Let’s consider YouTube as an example. What does YouTube look like? For someone who has been living under a rock for the last twenty years, it’s a single-page app where a lot of interactions occur. Now, if you’re a front-end developer tasked with building this app, you’d also need to handle back-end requests to render all this information. We're not going to delve into the intricate details because they are irrelevant for the moment. The key here is to consider the structure of the data being fetched.
00:04:41.160
When we query for video data from the YouTube API, we receive detailed information about the video, including title, licenses, views, share links, and all related metadata. We also need to consider the channel associated with the video, which can be a user or a physical channel. In the YouTube API, users are essentially channels, and there isn't a separate concept for users.
00:05:03.910
This means that when rendering a view, you'll need data on the channel title, avatar, number of subscribers, and much more. Additionally, for popular videos, you have the comments section, which includes information about who wrote the comment, the body of the comment, when it was posted, and the author's title and avatar.
00:05:24.070
On top of that, there's a sidebar with recommendations for the next video, which requires information like the video title, thumbnail, and potential reasons for why it was suggested—maybe the channel is popular or similar users have watched the video. All data necessary to render just a single video view must be fetched, including all these details.
00:06:00.760
If you were to use a REST API, fetching all this data would look something like this: fetch the video, fetch the channel-related info, fetch the comments for each one, get the author's info per comment, and repeat this for additional videos. Clearly, this sequential approach is not efficient; it would take ages—potentially crashing back-end systems due to the volume of data needed.
00:06:48.570
Instead, YouTube has created a more efficient way of doing this. They use something called 'parts' in their API, where different pieces of information can be retrieved through different parts. Users can fetch only the information they need by obtaining the correct part, and some parts come with different quota costs depending on their importance to the server.
00:07:20.130
So, if you wanted to get the channel title, which part would you request? The answer is the 'snippet.' The YouTube API has defined various types and their corresponding characteristics, allowing developers to differentiate what data is returned in a single query efficiently. This marks a significant improvement compared to the limitations of REST.
00:08:01.330
Now, how does GraphQL fit into this discussion? GraphQL and other solutions like it, such as those employed by Facebook and Netflix, aim to solve similar issues. The fundamental concept is that you can define types—for example, a video type with attributes like ID, title, description, etc.—and then define relationships between these types.
00:08:42.710
With GraphQL, you can write a query that specifies exactly the attributes you're interested in for a given video ID, using a map notation for related entities like channels and comments. This means you can effectively and intuitively pull the data you need—it's a far more streamlined approach than REST and reduces boilerplate code significantly.
00:09:56.020
GraphQL also empowers the client, allowing them to request only the specific information needed without breaking older clients, as they won't request new features not related to their current needs. This keeps older clients functional even as new features are added over time. The reduction of boilerplate code is a huge advantage since much front-end code is dedicated to just fetching and transforming arbitrary API responses.
00:10:47.940
However, GraphQL isn't perfect. Its main limitation is that it requires defining specific types, and this can lead to type explosions when attributes need to be repeated across different types—especially evident when building large applications. For instance, if you had a comment component that only cared about the author's title and body, you'd need to create three different types because the attributes are always nested within a specific type scope.
00:11:42.740
In contrast, one way to approach this is through 'decomposing'—or taking apart elements that are unintentionally coupled. By examining the components of data and breaking them down, you can achieve clearer structures without redundancy. This is especially useful for keeping elements that you might want to use in diverse contexts separate, thus avoiding a rigid structure.
00:12:34.750
EQL emerges as a concept that allows for clear separation of concerns because it recognizes that attributes often exist independently from their original types. By treating data as independent entities, we can build APIs that provide a more flexible querying experience, effectively reducing unnecessary type coupling.
00:13:25.710
An example of this would be the implementation of pathom within ClosureScript. Using pathom, you can seamlessly pull data based on identifiers that pass the requirements of being context-free, allowing for a much cleaner separation of responsibilities. For example, we can use 'select keys' to pull only the attributes we care about from a hash map, ensuring flexibility when navigating our data.
00:14:27.412
In any API, it’s crucial that we know how to traverse and derive new information without needing to build complex queries. JQ serves as a good model for this since it can pull out specific attributes using a string-indexed notation, making it easy to access nested data structures.
00:15:14.560
Thus, the challenge that remains is how to use these principles effectively while still returning results in an intuitive format. That is where EQL fits in, enabling us to write clear and understandable queries that can derive new kinds of information based on relationships among entities.
00:15:52.950
Within the Closure ecosystem, developers can leverage libraries built on EQL to integrate various data sources efficiently, including third-party APIs like YouTube. The spirit of immutable data is to have predictable systems where side effects are minimized, helping to create applications that are easier to reason about and maintain.
00:17:44.225
One interesting application of this concept is in the integration of different APIs. For example, if you can pull a YouTube video ID from a SpaceX API response, you can enable cross-referencing across services. This flexibility allows developers to create robust applications that can compile and manipulate data from various sources.
00:18:29.520
Imagine crafting a front-end dashboard for a website dedicated to SpaceX where, alongside launch details, you also fetch information on the associated YouTube video: its title, channel, and view count. This type of seamless integration is possible through the use of context-free identifiers, making it easy to compose rich UIs without overcomplicating our data-fetching logic.
00:19:27.840
You can structure these queries as you wish, allowing you to create meaningful dashboards or widgets without needing stringent rules on joins or structuring in your schema. This flexibility is beneficial for developers who need to tailor the API to their own needs, effectively assisting users in retrieving the specific information they desire.
00:20:20.670
Over the course of this talk, I've illustrated several principles of building effective, context-free, and immutable APIs with EQL and related approaches. While it takes careful consideration and practice to implement these ideas, the benefits will be evident in the applications you create.
00:21:16.600
In conclusion, I'm glad to have shared these ideas with you today, and I hope you've found the concepts of EQL and pathom helpful. I encourage you to explore these in your own projects to see how they can enhance your API's functionality and your overall development process.