GoRuCo 2009
Resource-Oriented Architecture With Waves
00:00:18.640
Before I get into the meat of the talk, I'd first like to thank Francis and all the great GoRuCo folks. This conference has been a lot of fun so far, so please give them a big round of applause. I'd also like to thank you all for coming out and supporting the Ruby community. Conferences like this are a big part of what makes the community special. I came out here from LA, but I grew up in Detroit, so I have no problem saying I love New York City. It's been too long since I've been here, so it's a great opportunity to meet a lot of new folks. A little bit about myself: the program states that I'm from AT&T Interactive, but as of about three days ago, that's no longer true. I moved to a startup in Santa Monica called SharesPost, where we're trying to make private equity trading possible. If you've ever thought about buying some Facebook or Twitter stock, that's what we're working on.
00:01:10.160
I'm also the lead developer for Waves. In my previous role at AT&T, I was doing a lot more managing, so my GitHub account shows a lull in commits over the last few months. I hope to become more Hands-On again, but I still am allegedly the lead developer for Waves. My contact information is there, and part of the fun for me in doing these talks is meeting all of you. I genuinely hope that if there's anything here that you find interesting or if you have ideas that might influence the direction of this project, you'll reach out and get in touch. A couple of days ago, at Google I/O, a project called Google Wave was announced. So now I'm faced with either changing the name or answering questions about what it has to do with Google, which is nothing—at least not directly related.
00:02:09.640
There was also a certain incident that forced me to remove the surfer babe that was at the end of each one of my presentations, but I won't elaborate on that. So first, I want to talk a little bit about Waves. I think I've made the mistake in the past of not talking enough about what Waves really can do today. It is under development, but it's already in use on production systems. It's a fairly sophisticated framework at this point. I tend to like to discuss what we are planning to do in the future rather than what it already does, but I don't want to sell the framework short. So, I'll quickly talk about a couple of things Waves can do to give you an idea of its capabilities.
00:03:01.040
There seems to be a couple of themes in terms of Ruby development. Initially, it was all about MVC and Rails; that was the big thing. Before Rails, there was a lot going on, but the big turning point for Ruby's growth was the emergence of a viable MVC framework. However, we started getting into a phase where people began saying they didn’t always need a full-fledged MVC framework. Many users turned to Sinatra for simpler solutions. I want to make the case that you can achieve a level of compactness similar to Sinatra just by using Waves. What you see here is an example of the power of one of Waves' core features, which illustrates just one aspect of it.
00:04:08.280
Waves has a nice domain-specific language (DSL) for accessing and mapping HTTP requests, which you can use in various contexts. It’s different from conventional MVC-style routing in Rails. I can match many things, like the accept header or query parameters, and it’s quite a natural thing to implement. In this case, we can say there must be a query parameter in the request called 'name'. There are several primitives available, and we are using the Compact Foundation which allows us to build a very compact, Sinatra-like application. If new requirements arise, we can adapt to handle more sophisticated needs. This can involve transitioning to a more traditional web app that utilizes what's called the MVC Foundation. These foundations are pluggable into Waves, giving it different character depending on your requirements.
00:05:00.759
The nice thing about that flexibility is that you can start simple, and as your application evolves, you can incorporate whatever foundation makes sense or additional layers. A foundation is made up of layers; you can even create your own foundations. One of the Rubinius developers has been working on a DSL for building more RESTful web apps, and he was able to add that in as a foundation. This is an experimental foundation that has been in development. Another example revolves around services, which I proposed but we haven’t yet implemented. As we go forward, we’ll see an attempt to represent a resource server with a DSL that expresses the abstractions around REST computing and resource-oriented concepts.
00:06:02.759
Waves is quite feature-rich. It includes inheritable configurations that Rails still does not have. I've thought about extracting that and some of the other features into a gem. There are decoupled implementations for workers and dispatchers, which allow for considerable customization. One particularly interesting aspect is the work with JRuby. We’ve had positive experiences with it. From a scalability standpoint, it is straightforward to deploy because you don’t need to figure out how many instances you need. The JVM, which has likely been optimized for various architectures, can be leveraged to scale up to the machine's limits simply by running one JVM. We’ve been running production systems that have never gone down, even under the load of hundreds of thousands of requests.
00:07:06.200
One thing I often hear after discussing some of the more abstract ideas with Waves is people saying they find them fascinating, but they don't see how it impacts their work since everything they're doing fits within the MVC paradigm. This is probably true for many, and I'm not trying to complicate anyone's life. However, it’s essential to recognize that the most successful web client, the browser, may not hold that position indefinitely. Already, we see people experimenting within browsers using techniques like AJAX and Comet. So we're moving towards environments where we need to consciously manage aspects like the accept header in requests. With the emergence of publicly available APIs—over a thousand last time I checked—our applications will increasingly need to integrate multiple APIs. Mobile development also represents a significant shift, as we see hybrid clients that want to combine native functionalities with server capabilities that permit updates without needing to release a new app.
00:08:43.680
My favorite example involves an open-source project called Boxy. With IPTV already on platforms like Apple TV, it's only a matter of time before we see Ruby implementations of such projects. Imagine the potential for interactive TV; you might be watching a news program, and as topics arise, you could click to get more information about them. We are witnessing contexts like Wolfram Alpha launching, demonstrating the ability to semantically parse web content, which allows for robust analysis. We are moving toward a very different kind of web. Should it not have impacted you yet, it’s something to keep in mind: the MVC view of the world and browser-centric perspective won’t endure indefinitely. One profound notion to grasp is that HTTP itself doesn’t conform to MVC principles.
00:10:43.960
What I want to emphasize is that while discussing not just Ruby but the web's potential, we've only scratched the surface of what it can do. Waves aims to assist as our collective requirements evolve beyond MVC by helping us figure out how to craft frameworks and applications that take full advantage of web capabilities. Earlier, I mentioned the foundations and layers; Waves is much more flexible than traditional frameworks. It allows you to mix in desired features and build from there, starting simply and expanding as needed. A pivotal term here is REST, which remains controversial. Roy Fielding, who coined the term, often finds himself engulfed in debates about its meaning. If we wish to move beyond discussions on REST, an appropriate term to use is 'resource-oriented architecture'. If you step outside Roy's strict definition of REST, you can adopt 'resource-oriented' as a broader concept.
00:12:39.840
Resource-oriented architecture is not merely a set of constraints; it allows for real architectural discussions, identifying HTTP-based conventions and standards. This is unlike REST, which doesn't address HTTP specifically. What I will discuss from here out is HTTP-based resource-oriented architecture. The impressive aspect of the web is its robust, proven infrastructure—we don't need to reinvent protocols. A recent discussion with a colleague showcased the drawbacks of implementing a custom protocol; you essentially sacrifice the benefits of the existing tools like load balancers and firewalls that support scaling. Additionally, managing HTTP-based services has its advantages as it's already constructed to monitor implementations proficiently. HTTP itself is a complex protocol, but many misunderstandings arise from the various verbs and their proper applications.
00:14:51.760
I believe that a distributed hash table is an extremely powerful abstraction. It underlies many applications, including caching and messaging. HTTP essentially serves as a distributed hashtable, which clarifies the verbs—GET, PUT, DELETE, and the use of HEAD to check for values. POST serves as a fallback, utilized when something doesn't fit neatly within the constructs of a defined resource. The distinction is clear; the other verbs follow robust, defined semantics, while POST allows for variation. Resources consist of key-value pairs, represented as objects. The representation becomes crucial, especially in a distributed context where clients and servers may operate across different architectures. Our context assumes that the client and server might use different platforms, necessitating a neutral approach that reevaluates and clarifies object abstraction in distributed systems.
00:17:06.679
Moving from object-oriented programming terms to a broader focus, we need to split the notion of an object into two parts: the entity and its representations. This distinction aligns with the concepts we've encountered in computing, and it is essential to be aware of the usability of resources, which drive our interactions across applications. The Resource Description Framework (RDF) is worth exploring, as it unifies resource description with schema concepts into an extensible framework. Tools like Freebase exemplify the power of RDF, allowing for a generic point of entry into a set of resources. The importance lies in creating interfaces that don't require programmers to delve into documentation to understand how to call services. The principles of least surprise should govern design to yield efficiency across the board.
00:18:54.760
Waves aims to make services machine-readable, discoverable, and dynamically composable. This principle extends to RSS and Atom feeds, where end-users need not know the underlying structure of a service. Instead, they can consume content without concern for the internals of the implementation. This exemplifies loose coupling—where outside changes to URLs do not impede workflows. The basis of our goal is to render our clients generic; if we can share an RDF for our server, depending on the given services, we can maximize our efficiency. By avoiding the pitfalls of tightly coupling clients and servers, we enhance discoverability and dynamic composition. Understanding these principles is vital for us moving forward.
00:21:05.960
Let’s delve deeper into specific examples of how HTTP concepts are enhanced through Waves. One example is edge caching. HTTP provides intelligent caching mechanisms, allowing clients to cache data, eliminating network traversals altogether. A request may never even happen if the client knows the cached data is likely still valid. This kind of optimization is crucial to scalability. In Waves, we can set up a simple server that handles these caching mechanics effectively. Caching, along with making networks smarter, can simplify complex procedures, such as security protocols, by shifting responsibilities to firewall-type nodes. This way, once a method is correctly implemented, it can be utilized across multiple servers seamlessly.
00:23:57.560
An often overlooked best practice is ensuring that resource semantics are meaningful. For instance, rather than hardcoding URLs that directly interact with followers in an application like Twitter, we ought to embed this information as data, allowing for flexibility in resource representation. Therefore, a client can receive links embedded within returned data, maintaining functionality even if URL specifications change. Another common mistake is involving format specifications in URLs. Instead, clients should specify their desired representations through appropriate headers when requesting resources. Such design principles enhance the dynamic interaction between clients and servers, providing a more robust application structure. It encourages developers to focus on linking data fluidly without predetermining formatted outcomes.
00:26:09.679
The overarching question is what Waves is truly trying to do. In our ongoing discussions, we’re working on various DSLs aimed at optimizing resourceful web applications and services. There’s debate on prioritizing web applications or service development at the moment, and this is being discussed actively on our Google Groups. We would love to have more participants in these discussions to help refine our DSL offerings, which we refer to as foundations. This concept signifies a new paradigm that grants us the leverage of a distributed object-based architecture that is protocol and platform neutral. We are merely at the tip of the iceberg in discovering the true potential of what can be accomplished.
00:28:11.160
Please check out Waves on GitHub, where you can find the latest stable and edge releases. The recent Edge version is the foundation for what's expected in the upcoming version. Your engagement, experimentation with foundations, and various contributions would be greatly appreciated. The contact information is available, and I’d be happy to take any questions that you might have.
00:30:22.960
So, you mentioned the necessity of not specifying formats in requesting URLs. Your example of requesting a specific shoe in XML raises an interesting discussion—what happens when one wants different language representations? It should be handled using the Accept-Language header. However, there's an underlying caution regarding how some network layers don’t implement these conventions effectively, highlighting the pitfalls we face as developers. The header-related complexities can often result in non-conforming behaviors from browsers or systems. Many challenges will arise in overcoming inconsistencies as we move forward in the web development landscape.
00:32:07.760
Navigating these issues brings us closer to understanding the substantial advantages of RESTful architecture. While it may not seem to affect individual projects significantly, the broader implications of adopting these methodologies lie in scalability, flexibility, and improved interoperability. Striving toward achieving synergies among diverse services will ultimately prove beneficial, not only to individual developers but also to the evolution of service-based applications and the cloud computing paradigm. Transitioning into agile development paradigms could allow for the assembly of functional apps that tap into the remarkable potential offered by well-architected distributed systems to a degree we have yet to fully explore.
00:34:03.960
In exploring the relationship between diverse technologies and practices, we continually make strides to create an environment conducive to innovation. Engaging product management and development teams in collaborative brainstorming processes enables us to break down silos that impede communication. Eventually, we’ll arrive at an optimized, efficient ecosystem where open-source projects advance at greater velocities and serve the broader community's needs.
