Native Extensions Served 3 Ways

Talks

Tejas Deinkar

1 talk

Native Extensions Served 3 Ways

by Tejas Deinkar

In the talk 'Native Extensions Served Three Ways' by Tejas Deinkar at Garden City Ruby 2014, the focus is on building native extensions in Ruby and how to integrate them effectively within applications. The presentation primarily covers three methods: C extensions, Foreign Function Interface (FFI), and SWIG (Simplified Wrapper and Interface Generator).

Key Points Discussed:

Introduction to Native Extensions: Tejas highlights the importance of building native extensions to interact with new libraries, enhance performance, and leverage existing C/C++ libraries within Ruby applications.
C Extensions: The talk emphasizes the straightforward nature of creating Ruby C extensions using the 'ruby.h' header. It discusses the significance of memory management and the use of macros for handling memory correctly, given Ruby's garbage collection system.
FFI: FFI allows Ruby to interact with C libraries easily, without the complexity of C extensions. Tejas explains how FFI offers automatic conversion between Ruby types and C primitives, which facilitates the integration of native libraries across various Ruby implementations.
SWIG: SWIG is introduced as a tool that automates the process of generating wrapper code for C and C++ libraries. By annotating header files, developers can produce bindings for multiple languages, including Ruby and Python. Tejas mentions the initial overhead of setup but stresses the long-term advantages it provides for cross-language integrations.
Performance Considerations: The speaker addresses the need to manage the Global Interpreter Lock (GIL) when writing native extensions for Ruby, especially during long-running tasks, to ensure thread safety.
Testing Native Extensions: Tejas discusses the varying methods to test native extensions based on their functionality, with a recommendation for mocking external dependencies for database connections while using integration tests for simpler functionalities.

In conclusion, Tejas Deinkar emphasizes that native extensions can be built effectively with C extensions, FFI, and SWIG. For those not maintaining a library, FFI is often the simplest choice. For developers involved in multiple language support, SWIG presents a compelling option. This talk equips developers with the foundational knowledge and practical tools needed to integrate native extensions within their Ruby applications.

00:00:24.410 Hi everyone! As Swan mentioned, I'm part of the team that organized this event, and it's really awesome to see such a huge crowd on a Friday morning in Bangor. Today, I'll be speaking about native extensions in Ruby. My talk is titled 'Native Extensions Served Three Ways.'

00:00:40.350 A little about myself: I’m Tejas Deinkar, a partner at Newlands Software, which is an employee-owned collective. It's a really fun place to work. If you want to reach out, you can find me on Twitter as @tea_drinker and on GitHub under the username @gja, where you can see most of my open-source contributions.

00:00:51.300 About my talk: this is actually a pretty technical presentation, so expect to see a lot of code. I hope to leave five minutes for questions at the end. If we don’t have time during the talk, feel free to catch me in the hallway, and I’ll try to answer any questions you have.

00:01:09.300 I will primarily be covering C extensions, FFI (Foreign Function Interface), and SWIG (Simplified Wrapper and Interface Generator). First, let's discuss why you might want to build a native extension for Ruby. There are several reasons for this. One is to integrate with new libraries—perhaps a new database comes out, such as Libdrizzle, which was created to work with MySQL and the Drizzle database.

00:01:40.410 You might want to port this library over to Ruby to enhance your application. Another reason is to improve the performance of critical code. While there are many ways to optimize performance, such as using JRuby or different caches, there may be situations where you have an algorithm that you want to implement in native code, or where there’s already a great library in C++ that could be beneficial for your project.

00:02:28.140 Sometimes, you might want to write code that works across different languages. After all, many developers enjoy programming in C and regard it as a badge of honor. It's exciting to feel elite by working in such a powerful language.

00:02:42.420 Before diving deeper into native extensions, let’s shift our focus for a moment and talk about Python. How many people here are Python enthusiasts? Surprisingly, there aren’t many. I recently discovered the best way to write Python code, and I’m going to share this with you now—though it's a bit of a joke. If we have a Python interpreter open, we could import Ruby, and then use Ruby's eval to run some code, showcasing how one could intertwine functionalities between these two languages.

00:03:52.490 Let's consider a simple use case where I define a recursive method in Ruby to calculate the factorial of a number. By using Ruby's eval, I can execute that method from within Python, which highlights the potential for integration between the two languages. All it takes is a few lines of code, and I can demonstrate the actual implementation.

00:05:42.690 Now, let’s transition back to native extensions in Ruby. Ruby C extensions are straightforward to build. To start, you need to include 'ruby.h', which is a header file that provides the necessary definitions for constructs exposed by the Ruby library. Most of the time, including 'ruby.h' is sufficient for your extensions. You can define methods in your C code that will be called from Ruby, such as creating a method that interprets Ruby strings and converts them into C strings with proper handling of different data types.

00:08:31.140 It's crucial to handle memory allocation correctly in C extensions because Ruby has its own garbage collection (GC) system. There are two macros available to help with this. The first macro associates a C pointer with a Ruby object, allowing you to manage the memory effectively. The second macro retrieves the pointer when needed. When the Ruby object is garbage collected, so too is the associated memory, simplifying memory management for developers.

00:10:02.490 Now, let’s discuss FFI, which allows interaction with C libraries in Ruby without the need for C extensions. FFI is remarkably easy to use; it supports all Ruby implementations, including JRuby, Rubinius, and MacRuby. FFI offers a simple way to create bindings to native libraries by automatically converting between Ruby types and C primitives, making it accessible for integration with various applications.

00:12:48.410 For instance, using FFI, you can attach to C libraries and expose functions with little overhead. This method not only allows you to maintain your Ruby style of coding but also simplifies the deployment process, as you don’t have to worry as much about Makefiles or traditional build processes. However, there is a common misconception that using FFI means you don't have to handle garbage collection; that's not true. You still need to be mindful of memory management to avoid crashes.

00:15:14.950 SWIG is another tool worth mentioning; it stands for Simplified Wrapper and Interface Generator. SWIG automates the generation of wrapper code for C and C++ libraries, allowing you to annotate your header files and produce bindings for several languages, including Ruby and Python. This approach is particularly beneficial if you're maintaining a native library and need it to integrate seamlessly across various programming languages.

00:18:31.150 While SWIG simplifies cross-language integrations, it may involve initial overhead in setup, but the long-term benefits outweigh these efforts if you frequently need to connect a library to different languages. Various options like dynamic loading and Fiddle can also be used, but FFI and SWIG are generally recommended for their ease of use and broad capabilities.

00:21:25.040 In summary, native extensions can be enjoyable and relatively simple to build with three main tools: C extensions, FFI, and SWIG. If you don't maintain the library yourself, FFI is often the best choice due to its straightforward nature. However, if you do maintain a library and need to provide access to multiple languages, SWIG may be the better option.

00:22:48.320 Thank you! I believe I have a bit of time left for questions. If anyone has any queries or needs clarification on what I've discussed, please feel free to ask.

00:23:59.560 One question people frequently ask is about handling the Global Interpreter Lock (GIL) while writing native code. When you write native extensions, you must manage the GIL appropriately, especially if you’re performing long-running tasks. This is crucial to ensure thread safety and to avoid potential issues that can arise from concurrent Ruby calls.

00:26:09.500 Another important aspect is how to test native extensions. The testing setup largely depends on your library's functionality. For database connections, you might want to mock external dependencies, while for simpler functionalities, integration tests can be beneficial. Overall, the approach to testing varies, and it’s essential to consider the specific requirements based on what your native extension does.

00:28:00.000 Thank you again for your attention. If there are no more questions, this concludes my talk!

Garden City Ruby 2014