Reimagining 2D graphics and game development with Ruby

by Tom Black

In this presentation titled "Reimagining 2D graphics and game development with Ruby," Tom Black discusses the potential of using Ruby in the realm of graphics and game development, highlighting a more intuitive and accessible approach to programming. The video focuses on several key themes and insights derived from Black's journey in coding and game development:

The main takeaway from the presentation is the encouragement to leverage Ruby as a tool for simplifying and enhancing the graphics programming experience and to build a vibrant community focused on education and creative coding.
Tom concludes by inviting listeners to engage with the Ruby community to explore these advancements further and reminds them that his notes and code samples will be shared online.

00:00:10.630 I'm Tom Black, a developer, designer, and product strategist. You can find me online in most places at Black TM on GitHub and Twitter, and I use Ruby for both work and client projects.

00:00:25.610 Before we get started, I'm going to upload notes and code samples to the conference website, so you don’t have to jot down everything. Today, I’m going to share some of my experiences and thoughts about reimagining 2D graphics and game development.

00:00:36.620 For me, this journey began here in New York City, specifically in the Flatiron District. I was new to the city, spending a lot of time meeting people and taking everything in. Around 2012, I became part of the "learn to code" movement.

00:00:49.479 Despite all the hype surrounding it, I found learning to code to be quite challenging. I feel excited to share that experience and realize there are many like-minded people. I attended events and meet-ups to connect with others, many of which included coding boot camps and discussions.

00:01:10.010 While I was learning, I noticed that many of the paradigms used for teaching coding and development were often externalized. When you're immersed in the world of computers, it can feel overwhelming, and not much seems to have changed.

00:01:17.930 In numerous ways, things have undoubtedly progressed – we now enjoy millions of colors, richer audio speakers, and diverse input devices. However, while the keyboard may remain a constant from yesteryear, new technologies offer a multitude of possibilities.

00:01:28.209 Our learning styles vary; some are visual, some auditory, and some require hands-on interaction. As such, I was deeply interested in how we could take this multisensory experience and make it accessible.

00:01:44.590 In 2012, I created this demo to explore the vision I imagined—what if you could engage with Ruby through a terminal, harnessing its capabilities while creating interactive learning experiences? I envisioned a setup where you could easily prototype mini-games and plug in various controllers.

00:02:06.130 This would facilitate real engagement, allowing learners to dive into coding through meaningful interactions. Built with real Ruby code and libraries, this prototype was established to validate the idea.

00:02:19.810 Every good prototype is just a façade set to validate an idea. So, I tested it with people who were eager to learn. This kickstarted my interest in digging deeper to see if I could build something more substantial.

00:02:40.360 We conducted research to discover what already existed in the realm of graphics and game development, tracing concepts back to around 2000. A rich history emerged, presenting diverse ideas that utilized graphics in different ways.

00:03:01.840 My instinct was to identify what must-have features I would require if I wanted to innovate in this area. I'd like to create an environment that is cross-platform and operates both natively and on the web while remaining easy to understand. This system would need to nurture a deeper understanding of what's happening internally as well.

00:03:22.450 Ultimately, I wanted a platform that wouldn’t just cater to education but could also be engaging for creatives. The quicker you can see your ideas in action, the more motivation you have to continue.

00:03:46.910 I felt a strong sense of inspiration from taking existing ideas and mixing them together. There are benefits to building upon the fundamental concepts while exploring new technologies and methodologies. Sure, learning something new can feel daunting at first.

00:04:03.360 Yet, I thought starting with a clear vision in mind would help demystify things. The idea of developing a Minimum Viable Product (MVP) was at the forefront of my mind in terms of experience that would resonate.

00:04:24.900 Envisioning a user-friendly experience, I began breaking down what a project might entail—starting with a window, the most fundamental interface. I found that most operating systems provide low-level libraries to create windows.

00:04:46.610 For instance, on macOS, you have NS classes and their documentation stating that an NS window is simply a window displayed on the screen. With some trial and error, I learned how to build a window using Cocoa classes.

00:05:03.240 From a technical perspective, constructing a window may sound simple, but it opens the door to more functionalities. For example, one has to consider the graphical context and how to acquire input from various devices.

00:05:13.030 You start imagining the pathways required for that complete experience. It’s engaging, but overwhelming when you remember the additional layers and inconsistencies.

00:05:25.890 The reality is that developers have been tackling these issues for quite some time, with many efforts dating back 20 years when they began thinking about universal game delivery systems across different operating systems.

00:05:38.270 The advent of media libraries helped smooth out the inconsistencies across platforms, providing a nice API for developers to work with. I found three major ones—SDL, SFML, and Allegro—while SDL appeared to be the most balanced for my project.

00:06:01.380 It’s been utilized by both AAA titles and indie games alike, which adds to its credibility. With SDL, we could build out our window efficiently without mulling over the various inconsistencies of the operating systems.

00:06:13.730 This brings us back to the path we want to take. You may wonder why I’m discussing system programming within the context of a Ruby conference. The truth is, C remains highly relevant.

00:06:32.920 Its standards continuously evolve, giving rise to advanced tools and infrastructure. While it may not be used for web applications, it excels in system programming, which fits quite nicely with Ruby.

00:06:46.700 Ruby thrives on C, especially since its primary implementation, MRI, is also written in C. This connection allows Ruby to engage with the native world beautifully and, uniquely, we're not just limited to interpreting it.

00:07:02.940 As we return to the Square application, we recognize that we are interacting with an operating system that facilitates essential access to hardware.So we now can work in a manner that adapts across all these environments.

00:07:16.660 The good news is that we don't need to communicate directly with the hardware; we can instead leverage the abundant cross-platform APIs in existence, such as OpenGL, which has capabilities for both 2D and 3D rendering.

00:07:32.570 The beauty of OpenGL lies in its programmatic simplicity. Although I’m oversimplifying it, the programmer's perspective involves writing C programs with OpenGL calls.

00:07:48.210 It's crucial to understand rendering pipelines in contemporary graphics programming. This programming often starts by transforming points into triangles, the simplest 2D shape.

00:08:09.240 Every vertex is defined by its coordinates, which we can manipulate within the coordinate system as needed. In addition, we require a projection to portray how we perceive the world.

00:08:23.210 In our digital environment, we can use perspective matrices, wherein the vanishing point influences how we render depth, but we also want to consider orthographic projections.

00:08:39.790 Understanding these concepts aids in crafting a unique coordinate space that allows for efficient rendering without delving too deeply into complications.

00:08:54.780 The next step involves rasterization, where we interpolate vertex values and fill in colors. This intricate process ultimately renders that visual output to the frame buffer.

00:09:10.760 This approach successfully translates all the mathematical processes into visual representation, allowing the graphics to engage users effectively.

00:09:24.240 Now that we understand this approach via OpenGL, we begin creating with a more modern take using pipelines and shaders to achieve flexible rendering.

00:09:39.220 We also engage with the architecture that allows us to draw anywhere on the screen. Utilizing this, we can create applications that include informative visual experiences.

00:09:52.660 Returning to our tech stack, we see the interplay between the operating system, SDL, OpenGL, and our application code through event loops and graphics integrations.

00:10:06.420 While this may seem scattered, our end goal is to package those concepts neatly. This organization leads to clearer implementations.

00:10:20.120 So we explored the native graphics programming with development in mind. We wanted to establish clear paths for creating cross-platform applications.

00:10:37.720 Bringing Ruby into the mix involves employing tools such as MRI or using MRuby for smaller applications. MRuby is a lightweight implementation of Ruby, designed for situations like this.

00:10:50.480 With it, we can compile Ruby down into machine code. By employing the MRuby compiler, we can generate easily embeddable code, which opens more accessibility possibilities.

00:11:06.580 To proceed to the Ruby layer, we would need to write a native extension that allows us to interact seamlessly with Ruby and our lower layers.

00:11:18.520 In many cases, differences between the MRI native API and MRuby may complicate things, but we can establish mapped definitions, making integration smoother.

00:11:37.480 By defining modules and classes in our Ruby extension, we finally connect the Ruby layer with our application, allowing developers to harness Ruby's convenience.

00:11:48.600 At this point, we have seamlessly integrated Ruby into our stack, achieving the goal we sought out from the beginning: a cross-platform application experience.

00:12:02.045 Transitioning to the web adds an interesting layer, as we can leverage WebGL to facilitate graphics right through the browser. We won’t need SDL but will extract the essential features available from its environment.

00:12:17.780 Creating JavaScript from our Ruby applications via compilation simplifies the process further. Although the JavaScript output may not be beautiful, it will effectively render across platforms.

00:12:37.560 You can enjoy the additional benefits of local JavaScript extension compatibility through simple libraries instead of mirroring those in native code.

00:12:56.240 This gives us versatility across platforms without compromising Ruby’s expressiveness, and it makes the process much easier. We’re integrating a level of abstraction that streamlines our development.

00:13:13.700 Back to our programming approach – the key goal was to build a user-friendly DSL (domain-specific language) for graphical programming in Ruby.

00:13:33.720 This involves creating intuitive syntax, allowing developers to express actions in a manner superior to traditional text. By adopting more expressive computing semantics, we aim to create immersive experiences.

00:13:52.340 Drawing inspiration from historical examples like HyperTalk and existing languages becomes vital. These platforms emphasized an approachable language that connected with the end-user.

00:14:06.280 This understanding allows us to overhaul how we think about our applications and how we can make those interactions more natural with simple English-like commands.

00:14:22.020 For instance, in Ruby, instead of commands that seem fragmented, we can present cleaner alternatives that maintain clarity for developers.

00:14:39.980 As a demonstration, creating a simple 640 by 480 window in Ruby could look like this, welcoming new users into this engaging domain seamlessly.

00:14:57.780 This approach demonstrates how easily we can bridge the gap between Ruby and 2D graphics through a streamlined interface. I envision it maintaining simplicity while maximizing user participation.

00:15:15.180 With that visual clarity, graphics can be tested effectively, capturing essential actions from controllers seamlessly. By connecting existing community efforts, we can refine the standard application usability.

00:15:34.960 The community-driven developer ecosystem offers invaluable resources, assuring that even unconventional controllers universally align with game development practices.

00:16:00.440 Mapping inputs for varying controllers to a common interface promotes inclusivity and versatility. With so many possibilities available, the Ruby community stands to gain immensely.

00:16:17.020 The existing tools and libraries built upon Ruby already grant developers the power to over-come challenges and innovate solutions smoothly. It leads to the creation of dynamic applications that remain accessible.

00:16:38.020 When we compile Ruby to a single binary, we establish a platform that bridges the gap effectively. Packaging everything together allows for simpler distributions, whether for the web or mobile applications.

00:16:55.260 Potentially publishing your applications to places like the App Store becomes accessible, as Ruby has the opportunity to serve as the transitional language that strides between these platforms efficiently.

00:17:14.230 Thus far, we’ve discussed various approaches to rethink what’s possible with game development and 2D graphics using Ruby. We’ve started to scratch the surface of this creative intersection and it's starting to evolve.

00:17:38.760 Ruby is poised to facilitate educational initiatives, engage its community, and encourage explorations of graphics and game development. It truly embodies an accessible language!

00:17:57.240 I'd like to showcase the advancements we can achieve through Ruby graphically. As an example, I would love to demonstrate a simple n-body simulation of celestial physics.

00:18:13.309 What we are visualizing is how gravitational forces interact among bodies in space, denoting their relative velocities with differing colors.

00:18:30.220 I envision an exciting array of possibilities using Ruby and I'm passionate about the growth of the community. The Ruby fringe track embodies a small, but growing, group eager to innovate in graphics and games.

00:18:53.810 If you're interested in cultivating a vibrant community and exchanging ideas, I'd love for you to join us in this endeavor! The future shines brightly for Ruby enthusiasts and developers.

00:19:12.400 Thank you so much for your time, and just a reminder: the notes from today will be made available.