Dynamic libraries with Rust

Sometimes you need to run some functions that are provided by a 3rd party, or the code must be configurable at runtime. This is where dynamic libraries come in handy. In Linux those libraries end with the extension .so while in macOS they have the extension .dylib. Windows uses the extension .dll for dynamic libraries.

Dynamic libraries in Rust

According to the Rust documentation there are two types of dynamic libaries:

  1. Dynamic Rust library (dylib)
  2. Dynamic system library (cdylib)

For this example, we will implement and use a dynamic Rust library, so we can use all the good Rust stuff.

Project structure

The idea is to have a trait that is defined in a common static library, used in a binary and implemented in a plugin (the dynamic library). For this, we need three projects, that we will group together in a Cargo workspace.

First we create a directory for our workspace and create the Cargo project:

$> mkdir rust-plugin
$> cd rust-plugin

In this project we then create the Cargo.toml:

[workspace]
resolver = "2"

When we create new cargo projects in a workspace directory, cargo will automatically add the new project as a member to the workspace. So we create the common library first, that contains all the traits that we want to use in our binary, and that will get implemented in our plugin:

$> cargo new library --lib

Now we can add the binary project, that will use the traits, and the plugin project, that will implement the traits:

$> cargo new binary 
$> cargo new plugin --lib

After all of that, our directory should look like the following:

$> tree .
.
├── binary
│   ├── Cargo.toml
│   └── src
│       └── main.rs
├── Cargo.toml
├── library
│   ├── Cargo.toml
│   └── src
│       └── lib.rs
└── plugin
    ├── Cargo.toml
    └── src
        └── lib.rs

Defining the common trait

As a simple example, we will create a trait, that takes a &str parameter and returns a String:

// in library/src/lib.rs

pub trait Greeter {
    fn say_hello(&self, name: &str) -> String;
}

Implementing the trait in the plugin

To implement the Greeter trait, we need to add the library as a dependency in our Cargo.toml. And since this plugin should be a dynamic Rust library, we need to add a [lib] section:

# plugin/Cargo.toml

[dependencies]
library = { path="../library"}

[lib]
name = "plugin"
crate-type = ["dylib"]

With the crate-type set to dylib, we tell Rust to compile it as a dynamic Rust library. This will create a library file in the target directory:

  • libplugin.so for Linux
  • libplugin.dylib for macOS
  • plugin.dll for Windows

The implementaion in the plugin/src/lib.rs is very straight forward:

use library::Greeter;

pub struct DynamicGreeter;

impl Greeter for DynamicGreeter {
    fn say_hello(&self, name: &str) -> String {
        format!("A dynamic hello, {name}")
    }
}

Implementing the new function in the plugin

Since we need a way to create a Greeter instance from our binary project, we need to implement a function, that we call new. This function will return a Box<dyn Greeter> and is implemented in plugin/src/lib.rs.

// plugin/src/lib.rs
impl DynamicGreeter {
    #[no_mangle]
    fn new() -> Box<dyn Greeter> {
        Box::new(Self)
    }
}

Notice the attribute #[no_mangle], it tells the compiler to not mangle the name, so code that tries to load the function can be sure, the name is as it is. For details, see the section in the Rust documentation.

Using the trait in our binary

To use the trait in our binary, we have to create a dependency to the library project in our binary/Cargo.toml:

# binary/Cargo.toml

[dependencies]
library = { path="../library"}

In the binary we want to load the dynamic library from the plugin project. When we run cargo build, Cargo will create the file in the directory target/debug/libplugin.so (when compiling in Linux)

Loading the dynamic library

To be able to load the dynamic library, we use the crate libloading, so we add the following to the Cargo.toml in our binary project, by calling cargo add libloading:

[dependencies]
libloading = "0.8.5"

First, we need to load the library at runtime:

    let lib_path = "target/debug/libplugin.so"; // for macOS and Windows the name must be changed
    let lib = unsafe {
        match Library::new(lib_path) {
            Ok(lib) => lib,
            Err(e) => panic!("Could not load library: {e}"),
        }
    };

Loading a library is an unsafe operation, so it has to be inside an unsafe block. If you want to learn more about unsafe Rust, you can read the chapter about it the Rust book.

When everything works, the match expression returns the library handle. If any error occurs, we simply panic in this example. In production code we would of course properly handle such situations!

To instantiate the Greeter implementation in the dynamic library, we load the function called new, which returns a Box<dyn Greeter>:

    let constructor: Symbol<fn() -> Box<dyn Greeter>> = unsafe {
        match lib.get(b"new") {
            Ok(constructor) => constructor,
            Err(e) => panic!("Could not load function: {e}"),
        }
    };

Calling the functions from the plugin

This code will return a function, that we can call as a regular function:

    let greeter = constructor();

Now that we have our instance of a Greeter from the plugin, we can call it:

    let greeting = greeter.say_hello("dear developer");

The complete picture

Now we can put all things together and implement the main function in our binary:

use libloading::{Library, Symbol};
use library::Greeter;

fn main() {
    let lib_path = "target/debug/libplugin.so";
    let lib = unsafe {
        match Library::new(lib_path) {
            Ok(lib) => lib,
            Err(e) => panic!("Could not load library: {e}"),
        }
    };
    let constructor: Symbol<fn() -> Box<dyn Greeter>> = unsafe {
        match lib.get(b"new") {
            Ok(constructor) => constructor,
            Err(e) => panic!("Could not load function: {e}"),
        }
    };
    let greeter = constructor();
    println!("{}", greeter.say_hello("World!"));
}

Possible problems and solutions

Segmentation fault when calling say_hello

For a cleaner code, you might want to extract the loading of the library part into a separate function like this:

// Do *NOT* do this, because it will cause a segmentation fault when calling say_hello()
fn create_greeter(lib_path: &str) -> Result<Box<dyn Greeter>, Box<dyn Error>> {
    let lib = unsafe { Library::new(lib_path)? };
    let creator: Symbol<GreeterCreator> = unsafe { lib.get(b"new")? };

    let greeter = unsafe { creator() };
    Ok(greeter)
}

But when you call the returned Greeter like this:

fn main() {
    let lib_path = "target/debug/libplugin.so";
    match create_greeter(&lib_path) {
        Ok(greeter) => {
            let greeting = greeter.say_hello("segfault!");
            println!("{greeting}");
        }
        Err(e) => eprintln!("{e}"),
    };
}

You will get a segmentation fault at runtime. This is because the variable lib in the function create_greeter will be dropped after the function ends. This is because of the Rust ownership. And as soon as the lib gets dropped, the library is also closed and freed. Calling any function from the library will then fail.

To solve this issue, we have to make sure, that the library has the same lifetime like all the functions used.

This can easily done, by bundeling the Greeter instance and the library handle in one struct:

pub struct GreeterBundle {
    greeter: Box<dyn Greeter>,
    // Make sure _lib is sorted after greeter, so that it gets dropped after it
    _lib: Library,
}

This way, the library will be available as long as the GreeterBundle is valid. Now we can implement a loading function like this:

// binary/src/main.rs

pub type GreeterCreator = unsafe fn() -> Box<dyn Greeter>;

impl GreeterBundle {
    pub fn new(path: &str, name: &str) -> Result<Self, Box<dyn std::error::Error>> {
        // name is the "base" name of the library without any platform specific parts
        #[cfg(target_os = "linux")]
        let os_lib_name = format!("lib{name}.so");
        #[cfg(target_os = "macos")]
        let os_lib_name = format!("lib{name}.dylib");
        #[cfg(target_os = "windows")]
        let os_lib_name = format!("{name}.dll");

        let lib_path = format!("{path}/{os_lib_name}");
        let _lib = unsafe { Library::new(lib_path)? };
        let creator: Symbol<GreeterCreator> = unsafe { _lib.get(b"new")? };

        let greeter = unsafe { creator() };

        Ok(Self { _lib, greeter })
    }
}

In the new method here, we also make it platform agnostic, when it comes to loading the library.

For easier usage, we let the GreeterBundle also implement the Greeter trait, so that it forwards the call to the Greeter member:

// binary/src/main.rs

impl Greeter for GreeterBundle {
    fn say_hello(&self, name: &str) -> String {
        self.greeter.say_hello(name)
    }
}

Now we can safely call the Greeter method in our main function:

// binary/src/main.rs

pub fn main() {
    let lib_path = "target/debug";
    let lib_name = "plugin";
    match GreeterBundle::new(lib_path, lib_name) {
        Ok(bundle) => {
            let greeting = bundle.say_hello("bundle");
            println!("{greeting}");
        }
        Err(e) => eprintln!("{e}"),
    }
}

Segmentation fault when dropping the GreeterBundle

In the example above, the fields in the struct GreeterBundle are correctly sorted. But if the field _lib is the first element, it can happen, that the library is dropped before the field greeter is dropped. This would cause a segmentation fault.

rust