Going beyond Ruby: Writing a simple C extension
Ruby is a high-level language with elegant syntax. But sometimes, performance-critical tasks can be slow in pure Ruby. Writing a C extension lets us move performance-critical code into native C for speed while also tapping into existing C libraries. In this article, we'll walk through building a C extension for Ruby from scratch using quicksort as our example.
Why use a C extension?
One big reason for using a C extension is to leverage existing libraries. Ruby's ecosystem sometimes has gems written on top of C libraries. For example, Nokogiri is a Ruby gem built upon existing C libraries (such as libxml2). By writing a C extension, we can directly interact with those C libraries from Ruby without reimplementing them. This means we can mix Ruby's productivity with battle-tested C code.
Another reason is that Ruby is built on C. Learning how to write a C extension can help you understand Ruby's internal workings. For instance, we can look at the source code to understand how Ruby sorts an array under the hood.
Ruby's easy-to-use features also come at the cost of speed. Those abstractions make Ruby a delight to work with, but also can make an algorithm implemented in Ruby slower than the same algorithm in C — which doesn't have those abstractions. By moving performance-critical code into a C extension, we can take advantage of the compiler that can optimize the code and still use the result seamlessly in Ruby. This is especially useful when working with large arrays or complex calculations.
To show the performance difference, consider a simple Ruby quicksort algorithm to sort an array in place:
def swap(array, i, j)
array[i], array[j] = [array[j], array[i]]
end
def partition(array, low, high)
p, i, j = [array[low], low, high]
while (i < j)
i += 1 while (array[i] <= p && i <= high - 1)
j -= 1 while (array[j] > p && j >= low + 1)
swap(array, i, j) if i < j
end
swap(array, low, j)
return j
end
def qsort_ruby!(array, low = 0, high = array.length - 1)
if low < high
pi = partition(array, low, high)
qsort_ruby!(array, low, pi - 1)
qsort_ruby!(array, pi + 1, high)
array
end
endThe corresponding C version was written up as well (this is shown later) and then compared with a simple benchmark:
require_relative "./ext/qsort/qsort"
require_relative "./lib/qsort_ruby"
require "benchmark/ips"
array = (1..10000).to_a.shuffle
Benchmark.ips do |x|
x.config(warmup: 2, time: 30)
x.report("C extension") { qsort_c!(array.dup) }
x.report("Ruby") { qsort_ruby!(array.dup) }
x.compare!
endWarming up --------------------------------------
C extension 92.000 i/100ms
Ruby 11.000 i/100ms
Calculating -------------------------------------
C extension 930.099 (± 1.7%) i/s (1.08 ms/i) - 27.968k
Ruby 114.704 (± 2.6%) i/s (8.72 ms/i) - 3.443k
Comparison:
C extension: 930.1 i/s
Ruby: 114.7 i/s - 8.11x slower
The results show that it's over eight times faster in this simple example!
Compiling and linking the extension
Ruby code doesn't need to be pre-compiled before running through the interpreter. A C extension, however, needs to first be compiled before we can link it into the Ruby code. We need to follow a few steps to compile the code and link it into our Ruby process.
- Create an
ext/directory to house the C files. - Create a
ext/extconf.rbfile with these contents:
require 'mkmf'
create_makefile 'qsort'This uses Ruby's mkmf library to generate a Makefile that can compile the C code.
- While still in
ext/, create a C file (for example,qsort.c). This is where we'll implement the C functions that will be called from Ruby. The basic contents of this file will look like:
#include "ruby/ruby.h"
void Init_qsort(void) {
}-
From the
ext/directory, runruby extconf.rb. When we run this script, mkmf will probe the environment and produce a Makefile to compile our C code. In effect, thecreate_makefilemethod tells Ruby how to name the output file and where to look for headers. -
In the
ext/, runmake. This uses the generated Makefile to compile our C code into a shared library.
After this, we should see something like qsort.so (or qsort.bundle on macOS) in that directory.
It's also useful to make a rake task to help with the compilation step:
task :compile do
puts "Compiling extension"
`cd ext && make clean`
`cd ext && ruby extconf.rb`
`cd ext && make`
puts "Done"
end- Finally, load the compiled extension in the Ruby code. For example:
require_relative 'ext/qsort'Once required, the classes and methods we defined in C will be available in Ruby. We can now call the C-backed methods just like normal Ruby methods. These steps will produce and load our C extension.
Writing the C code
With the compilation set up, let's delve into the C code itself. Ruby's C API lets us define modules, classes, and methods from C and work with Ruby objects via the VALUE type. Below are the main pieces: creating classes/modules, defining methods, handling arguments, and using the VALUE type. There is also a lot of good information on this available in this section of the Ruby Reference.
Defining methods in C
Every C extension has an initialization function that Ruby calls when the extension is loaded. The function must be named Init_<extension_name> . For our example, because our extension is named qsort , we'll create the void Init_qsort(void) function. We want to define a global method called qsort_c! . To do this, we can use the rb_define_method function and give it the object we are defining it on (rb_cObject), the name of the method (qsort_c!), a reference to a function (rb_qsort), and, finally, how many arguments this function will take (which is one).
#include "ruby/ruby.h"
static VALUE rb_qsort(VALUE self, VALUE ary) {
return ary;
}
void Init_qsort(void) {
rb_define_method(rb_cObject, "qsort_c!", rb_qsort, 1);
}If we wanted a class method instead, we'd use rb_define_singleton_method on the class object (or rb_define_module_function if we wanted it as a module function). If we want our method to take a variable number of arguments, we'd pass -1 as the number of arguments.
Creating Ruby classes and modules
It's not necessary for this example, but creating classes and modules is often important as well. We can use the rb_define_class method to define a class, or rb_define_class_under to define a class that inherits from another.
// This will define a top level Foo class. The same as:
// class Foo; end
VALUE fooClass = rb_define_class("Foo", rb_cObject);
// Assuming there is a constant Bar that exists.
VALUE barClass = rb_const_get(rb_cObject, rb_intern("Bar"))
// This will define a class "Baz" under "Bar" that inherits from "Foo". This is the same as:
// class Bar::Baz < Foo; end
rb_define_class_under(barClass, "Baz", fooClass);Handling arguments and the VALUE type
A keen eye might have noticed the VALUE type used when writing C. In Ruby, everything is an object, and those objects in C are of type VALUE, which is essentially a pointer to an internal Ruby representation. When our C method is called, it always receives VALUE self as the first parameter, and then additional VALUE parameters for each argument.
Before using these arguments, it's important to check types. For example, our quicksort expects an array, and we are going to modify it. So we can write:
static VALUE rb_qsort(VALUE self, VALUE ary) {
Check_Type(ary, T_ARRAY);
rb_ary_modify(ary);
return ary;
}The macro Check_Type(obj, T_ARRAY) verifies that ary is an array and raises a Ruby TypeError if not. This ensures safety before we treat the VALUE as an array.
It's also possible that the argument needs to be multiple different types. In this case, using a switch statement to apply different behavior to different types is helpful:
switch (TYPE(ary)) {
case T_ARRAY:
/* process Array */
break;
case T_STRING:
/* process String */
break;
default:
/* raise exception */
rb_raise(rb_eTypeError, "not valid value");
break;
}Once we've confirmed the argument is an array, we can work with it. Because this will potentially modify the array, we'll need to call rb_ary_modify as well. There are many of these little nuances when working with C that aren't needed when working with Ruby.
Ruby's C API provides helpers. For example, RARRAY_LEN(ary) gives its length, and rb_ary_entry(ary, i) gives the element at index i. These macros are the C-level equivalent of ary.length and ary[i].
Putting it all together: Quicksort
With these pieces, we can quickly implement quicksort in C using the original Ruby algorithm above.
We can write a swap method using the rb_ary_entry and rb_ary_store:
void swap(VALUE ary, long i, long j) {
VALUE temp = rb_ary_entry(ary, i);
rb_ary_store(ary, i, rb_ary_entry(ary, j));
rb_ary_store(ary, j, temp);
}We can also write a simple partition method using the first element as the pivot:
long partition(VALUE ary, long low, long high) {
// Initialize pivot to be the first element
VALUE p = rb_ary_entry(ary, low);
long i = low;
long j = high;
while (i < j) {
// Find the first element greater than
// the pivot (from starting)
while (rb_ary_entry(ary, i) <= p && i <= high - 1) {
i++;
}
// Find the first element smaller than
// the pivot (from last)
while (rb_ary_entry(ary, j) > p && j >= low + 1) {
j--;
}
if (i < j) {
swap(ary, i, j);
}
}
swap(ary, low, j);
return j;
}We can set up a recursive method to glue everything together:
void quick_sort(VALUE ary, long low, long high) {
if (low < high) {
// call partition function to find Partition Index
long pi = partition(ary, low, high);
// Recursively call quickSort() for left and right
// half based on Partition Index
quick_sort(ary, low, pi - 1);
quick_sort(ary, pi + 1, high);
}
}And finally, we can tie it into the rb_qsort method we set up earlier:
static VALUE rb_qsort(VALUE self, VALUE ary) {
Check_Type(ary, T_ARRAY);
rb_ary_modify(ary);
long len = RARRAY_LEN(ary);
quick_sort(ary, 0, len - 1);
return ary;
}We can now compile and run our quicksort algorithm written in C from our Ruby code!
Closing thoughts
Writing a C extension for Ruby can seem daunting at first because it's well outside of the norm for Ruby. But the steps to get started are straightforward. It will take some getting used to, but the result is performance-critical parts run at C speed.
We're hiring engineers who love Ruby — take a look at our open roles.
