{"id":208,"date":"2015-10-17T17:01:30","date_gmt":"2015-10-17T21:01:30","guid":{"rendered":"http:\/\/www.joshmatthews.net\/blog\/?p=208"},"modified":"2015-10-17T17:12:23","modified_gmt":"2015-10-17T21:12:23","slug":"creating-a-c-api-for-a-rust-library","status":"publish","type":"post","link":"https:\/\/www.joshmatthews.net\/blog\/2015\/10\/creating-a-c-api-for-a-rust-library\/","title":{"rendered":"Creating a C API for a Rust library"},"content":{"rendered":"<p><a href=\"https:\/\/twitter.com\/ImYoric\">Yoric<\/a> has been doing great work porting Firefox&#8217;s client backend to Rust for use in Servo (see <a href=\"https:\/\/github.com\/Yoric\/telemetry.rs\">telemetry.rs<\/a>), so I decided to create a C API to allow using it from other contexts. You can observe the general direction of my problem-solving by looking at the <a href=\"https:\/\/github.com\/Yoric\/telemetry.rs\/pull\/39\/commits\">resulting commits<\/a> in the PR, but I drew a lot of inspiration from <a href=\"https:\/\/github.com\/servo\/html5ever\/\">html5ever&#8217;s<\/a> C API.<\/p>\n<p>There are three main problems that require solving for any new C API to a Rust library:<\/p>\n<ul>\n<li>writing low-level bindings to call whatever methods are necessary on high-level Rust types<\/li>\n<li>writing equivalent C header files for the low-level bindings<\/li>\n<li>adding automated tests to ensure that all of the pieces work<\/li>\n<\/ul>\n<h3>Low-level bindings:<\/h3>\n<p>Having never used telemetry.rs before, I wrote my bindings by looking at the <a href=\"https:\/\/github.com\/Yoric\/servo\/commit\/70f90d02b2e39f5759953b5620b8067176b58161\">reference commit<\/a> for integrating the library into Servo, as well as <a href=\"https:\/\/github.com\/Yoric\/telemetry.rs\/blob\/master\/examples\/main.rs\">examples\/main.rs<\/a>. This worked out well for me for this afternoon hack, since I didn&#8217;t need to spend any time reading through the internals of the library to figure out what was important to expose. This did bite me later when I realized that the implementation for Servo is <i>significantly<\/i> more complicated than the example code, which caused me to redesign several aspects of the API to require explicitly passing around global state (as can be seen in <a href=\"https:\/\/github.com\/jdm\/telemetry.rs\/commit\/d6826a6f1e8c91ab71d3321ca2c99b7866c025ff\">this commit<\/a>).<\/p>\n<p>My workflow here was to categorize the library usage that I saw in Servo, which yielded three main areas that I needed to expose through C: initialization\/shutdown, definition and recording of histogram types, and serialization. In each case I sketched out some functions that made sense (eg. <code>count_histogram.record(1)<\/code> became <code>unsafe extern \"C\" fn telemetry_count_record(count: *mut count_t, value: libc::uint)<\/code>) and wrote placeholder structures to verify that everything compiled.<\/p>\n<p>Next I implemented type constructors and destructors, and decided not to expose any Rust structure members to C. This allowed me to use types like <code>Vec&lt;T&gt;<\/code> in my implementation of the global telemetry state, which both improved the resulting API ergonomically (many fewer <code>_free<\/code> functions are required) and allowed me to write more concise and correct code. This decision also allowed me to define a destructor on a type exposed to C; this would usually be forbidden due to changing the low-level representation of the type in ways visible to C if the structure members were exposed. Generally these API methods took the form of the following:<\/p>\n<pre>#[no_mangle]\r\npub unsafe extern \"C\" fn telemetry_new_something(telemetry: *mut telemetry_t, ...) -> *mut something_t {\r\n    let something = Box::new(something::new(...));\r\n    Box::into_raw(something)\r\n}\r\n\r\n#[no_mangle]\r\npub unsafe extern \"C\" fn telemetry_free_something(something: *mut something_t) {\r\n    let something = Box::from_raw(something);\r\n    drop(something);\r\n}<\/pre>\n<p>The use of Box in this code places the enclosed value on the heap, rather than the stack, which allows us to return to the caller without the value being deallocated. However, because C deals in raw pointers rather than Rust&#8217;s Box type, we are forced to convert (ie. reinterpret_cast) the box into a pointer that it can understand. This also means that the memory pointed to will not be deallocated until the Rust code explicitly asks for it, which is accomplished by converting the pointer back into an owned Box upon request.<\/p>\n<p>Once I had meaningful types, I filled out the API methods that were used for operating on them. The types used in the public API are very thin wrappers around the full-featured Rust types, so the step was mostly boilerplate like this:<\/p>\n<pre>#[no_mangle]\r\npub unsafe extern \"C\" fn telemetry_record_flag(flag: *mut flag_t) {\r\n    (*flag).inner.record();\r\n}<\/pre>\n<p>The code for serialization was the most interesting part, since it required some choices. The Rust API requires passing a <code>Sender&lt;JSON&gt;<\/code> and allows the developer to retrieve the serialized results at any point in the future through the JSON API. In the interests of minimizing the amount of work required on a Saturday afternoon, I chose to expose a synchronous API that waits on the result from the receiver and returns the pretty-printed result in a string, rather than attempting to model any kind of Sender\/Receiver or JSON APIs. Even this ended up causing some trouble, since telemetry.rs supports stable, beta and nightly versions of Rust right now. Rust 1.5 contains some nice <code>ffi::CString<\/code> APIs for passing around C string representations of Rust strings, but these are named differently in Rust 1.4 and don&#8217;t exist in Rust 1.3. To solve this problem, I ended up defining an opaque type named <code>serialization_t<\/code> which wraps a <code>CString<\/code> value, along with a <code>telemetry_borrow_string<\/code> API function to extract a C string from it. The resulting API works across all versions of Rust, even if it feels a bit clunky.<\/p>\n<h3>C header files<\/h3>\n<p>The next step was writing a header file that matched the public types and functions exposed by my low-level bindings (like an inverse <a href=\"https:\/\/github.com\/michaelwu\/rust-bindgen\/\">bindgen<\/a>). This was a straightforward application of writing out function prototypes that match, since all of the types I expose are opaque structs (ie. <code>struct telemetry_t;<\/code>).<\/p>\n<p>The most interesting part of this step was writing a C program that linked against my Rust library and included my new header file. I ported a simple Rust test from one I added earlier to the low-level bindings, then wrote a straightforward Makefile to get the linking right:<\/p>\n<pre>\r\nCC ?= gcc\r\nCFLAGS += -I..\/..\/capi\/include\/\r\nLDFLAGS += -L..\/..\/capi\/target\/debug\/ -ltelemetry_capi\r\nOBJ := telemetry.o\r\n\r\n%.o: %.c\r\n\t$(CC) -c -o $@ $< $(CFLAGS)\r\n\r\ntelemetry: $(OBJ)\r\n\t$(CC) -o $@ $^ $(LDFLAGS)\r\n<\/pre>\n<p>This worked! Running the resulting binary yielded the same output as the test that used the C API from Rust, which seemed like a successful result.<\/p>\n<h3>Automated testing<\/h3>\n<p>Following html5ever's example, my prior work defined a separate crate for the C API (<code>libtelemetry_capi<\/code>), which meant that the default <code>cargo test<\/code> would not exercise it. Until this point I had been running Cargo from the <code>capi<\/code> subdirectory, and running the makefile from the <code>examples\/capi<\/code> subdirectory. Continuing to steal from html5ever's prior art, I created a script that would run the full suite of Cargo tests for the non-C API parts, then run <code>cargo test --manifest-path capi\/Cargo.toml<\/code>, followed by <code>make -C examples\/capi<\/code>, and made Travis use that as its script to run all tests.<\/p>\n<p>These changes led me to discover a problem with my Makefile - any changes to the Rust code for the C API would not cause the example to be rebuilt, so I didn't actually have effective local continuous integration (the changes still would have been picked up on Travis). Accordingly, I added a DEPS variable to the makefile that looked like this:<br \/>\n<code>DEPS := ..\/..\/capi\/include\/telemetry.h ..\/..\/capi\/target\/debug\/libtelemetry_capi.a Makefile<\/code><br \/>\nwhich causes the example to be rebuilt any time any of the C header, the underlying static library, or the Makefile itself are changed. The result is that whenever I'm modifying <code>telemetry.rs<\/code>, I can now make changes, run <code>.\/travis-build.sh<\/code> and feel confident that I haven't inadvertently broken the C API.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Yoric has been doing great work porting Firefox&#8217;s client backend to Rust for use in Servo (see telemetry.rs), so I decided to create a C API to allow using it from other contexts. You can observe the general direction of &hellip; <a href=\"https:\/\/www.joshmatthews.net\/blog\/2015\/10\/creating-a-c-api-for-a-rust-library\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[24,26],"tags":[],"class_list":["post-208","post","type-post","status-publish","format-standard","hentry","category-code","category-rust"],"_links":{"self":[{"href":"https:\/\/www.joshmatthews.net\/blog\/wp-json\/wp\/v2\/posts\/208","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.joshmatthews.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.joshmatthews.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.joshmatthews.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.joshmatthews.net\/blog\/wp-json\/wp\/v2\/comments?post=208"}],"version-history":[{"count":11,"href":"https:\/\/www.joshmatthews.net\/blog\/wp-json\/wp\/v2\/posts\/208\/revisions"}],"predecessor-version":[{"id":219,"href":"https:\/\/www.joshmatthews.net\/blog\/wp-json\/wp\/v2\/posts\/208\/revisions\/219"}],"wp:attachment":[{"href":"https:\/\/www.joshmatthews.net\/blog\/wp-json\/wp\/v2\/media?parent=208"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.joshmatthews.net\/blog\/wp-json\/wp\/v2\/categories?post=208"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.joshmatthews.net\/blog\/wp-json\/wp\/v2\/tags?post=208"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}