Dimitri’s place about

defer and errdefer in Rust #

I have been doing a bunch of experiments in the past months/year around Zig, out of curiosity and because I wanted to assess whether the language could actually replace Rust in some places — turns out no (1, 2). Most of the feature of Zig can actually be expressed in Rust, but not vice-versa. For instance, there is no way to implement move semantics / ownership and borrowing in a safe way in Zig. Because the language doesn’t discriminate between safe and unsafe code, the audit surface is the whole codebase whenever it comes down to memory safety issue, which is already different in Rust, where you want to minimize the unsafe surface as much as possible — this is already a huge misconception that C developers make while considering Rust vs. Zig: you don’t write Rust as you write C, because most of the code you write in Rust is actually safe code, so you rarely need to actually use pointers (you use refs and safe abstractions).

Anyway, among the features that Zig needs (and that we don’t really need in a pure Rust environment) are defer and errdefer. It’s actually pretty simple to understand what they do:

defer <stuff> will execute <stuff> when control flow leaves the current scope, whether normally (end of block), or abruptly (early-return; try). <stuff> can be a single statement, or a block.
errdefer <stuff> will execute <stuff> when control flow leaves on an error.

`defer` and `errdefer` in Zig#

You usually use defer to deallocate resources after you are done with. For instance:

const ptr = try allocator.alloc(u8, len);
defer allocator.free(ptr);

At the end of the block or if an error occurs afterwards, ptr will be deallocated. This already creates a bunch of issues you have to be concerned with:

ptr cannot escape the current block/function, because at the end of the block, allocater.free(ptr) will be called. Zig has no way to broadcast this contract, so it’s on the programmer’s to ensure they don’t use the pointer after being freed.
Zig doesn’t have ownership, so if you copy the pointer, ensure copies do not outlive the current block, or you get into dangling pointer issues (which are just UB right now; there’s no way to check for it, even at runtime zigbin.io — the best to hope for is a segfault).

Sometimes, you might want to allocate some fields to return an owned version of them, such as:

const field1 = try allocator.create(Field1);
const field2 = try allocator.create(Field2);
// …

return .{ .field1 = field1, .field2 = field2 };

You do not want to defer-free them, because that’s on the caller to do so properly (imagine a constructor function, for instance). However, you might have spotted the problem here: what happens if the allocation of field2 fails? We leak field1. So you would need to do something like this:

const field1 = try allocator.create(Field1);
const field2 = allocator.create(Field2) catch |err| {
  allocator.destroy(field1);
  return err; // pass-through
};
// …

This is pretty annoying, so Zig has a solution: errdefer. It executes its right-side statement only in case of errors. So you can write the same code like this:

const field1 = try allocator.create(Field1);
errdefer allocator.destroy(field1);

const field2 = try allocator.create(Field2);

Those two keywords exist in Zig to make resource management easier. But what about Rust?

What about Rust?#

Rust doesn’t need defer nor errdefer because it has the concept of dropping automatically — Drop, also known as destructors. How does it work? Contrary to the value-based approach of Zig, dropping in Rust is type-based: a type that implements Drop automatically gets its values dropped by drop glue code at the appropriate place in the code. You can picture this as inserting the code defer and errdefer would put for you, but automatically. That prevents forgetting calling destructors:

let name = String::from("Chuck Norris");

At the end of the block name is declared in, name will be cleaned up by calling its Drop::drop() implementation, and eventually deallocated as part of its drop logic.

There are interesting consequences of this design:

Dropping values implies code running, which means that some code runs when a value is dropped automatically, which is not apparent — you don’t see a call to Drop::drop(), as the compiler inserts that call at the right places for you.
Users cannot forget to run destructors/cleanup logic, which is safer in terms of memory management, and combined with ownership, greatly enhance resource management. However, it is possible to leak values to defer call them Drop glue code by using std::mem::leak(). That has some usecases, but should be avoided in 99,9999% for obvious reasons.

An interesting aspect of Drop is that if you have several ways of implementing the cleanup logic for a given value, you can create as many types as logic of dropping you can think of, and move the value in the appropriate type wrapper to select the drop logic.

That last point led me to a funny idea: we can actually implement defer and errdefer in Rust! Why you would want to is open to discussions, but among the main reasons:

Running some logic at the end of the code. For instance, incrementing an atomic counter.
Implementing a C-based API without having to introduce Drop-types. This is probably the main incentive for defer, as you might need to perform the cleanup logic in a single function. Introducing a type just for that function might look a bit overkill, so defer could be nice there.

`defer` in Rust#

Implementing defer in Rust is pretty straight-forward. Remember what Zig does: it actually inserts code that runs at the right place for you — Zig people keep stating that everything is explicit in Zig, but it’s not completely true, since defer and errdefer statements will run at various places that the compiler will insert at; it’s not apparent and thus is very similar to Drop in that regard; you just need to manually ask the compiler to do so.

Let’s look at the Zig example again:

const ptr = try allocator.alloc(u8, len);
defer allocator.free(ptr);

The defer will call allocator.free(ptr) when we leaves the current block, which is akin as having a value dropped at the end of the current scope in Rust. We can do that by introducing a simple type wrapper (3):

struct Defer<F>
where
    F: FnMut(),
{
    drop: F,
}

impl<F> Drop for Defer<F>
where
    F: FnMut(),
{
    fn drop(&mut self) {
        (self.drop)()
    }
}

We can then introduce a simple way to construct such a value in the scope of the call site:

macro_rules! defer {
    ($($item:tt)*) => {
        let _defer = Defer {
            drop: || { $($item)* }
        };
    }
}

Why the let _defer? Because Rust, by default, is smart enough to realize that we won’t be using the result, and will likely immediately deallocate it without waiting for the end of the scope. By using an ignore-binding like that, we force Rust to drop the value at the end of the scope.

Using this is easy:

pub fn main() {
    let mut counter = std::sync::atomic::AtomicU32::new(0);

    defer! {
        println!("should be called last: {counter:?}");
        counter.store(1, std::sync::atomic::Ordering::Relaxed);
    }

    {
        defer! {
            println!("should be called first: {counter:?}");
            counter.store(2, std::sync::atomic::Ordering::Relaxed);
        }
    }

    println!("should be called second: {counter:?}");
}

The first defer! will be called last — at the end of main, and the second one will be called when the enclosing scope exits, which will then make it the first one to execute. The full output will be:

should be called first: 0
should be called second: 2
should be called last: 2

Now, how would we take care of implementing errdefer in Rust, though?

`errdefer` in Rust#

errdefer is trickier, because it requires the compiler to insert code when an error occurs. But Rust doesn’t have the concept of error baked-in. Result<A, E> and Option<A> can be used for error handling. Actually, anything implementing the std::ops::Try trait.

Besides checking whether the thread panicked during a Drop implementation, we cannot really detect and run code when an error occurred. We have to manually introduce the concept. The idea is actually a combination of two principles:

A value which type implements Drop will call code when it’s dropped (end of scope, like for defer).
We can prevent code from running when exiting on success paths, since we do have control on those.

The second idea is key. Because we eventually need to return something, we will have to wrap our result in an Ok-like enum variant. The whole idea goes like so:

Build a value by passing a closure that will run in case of errors. We type this value with errdefer: ErrDefer<F>, which is similar to Defer<F>, but this time we want it to be explicitly named and used by the user.
The function should return an ErrResult<A, E>, which is a type wrapper over a Result<A, E>, but can only be constructed from an ErrDefer<F>.
Whenever we want to return a success value, we just call something like errdefer.ok(value), which wraps value in a ErrResult<A, E>. This call is responsible for ensuring the error closure will not run when the ErrDefer<F> is dropped.
Whenever we want to return an error value, we call something like errdefer.err(err_value). This will simply wrap err_value in an ErrResult<A, E>, and will let the ErrDefer<F> be dropped, calling the closure.
We will optionally need a way to try function calls returning ErrResult<A, E>. More on that below.

Let’s start with ErrDefer<F>:

struct ErrDefer<F>
where
    F: FnMut(),
{
    err: Option<F>,
}

impl<F> Drop for ErrDefer<F>
where
    F: FnMut(),
{
    fn drop(&mut self) {
        if let Some(mut f) = self.err.take() {
            f();
        }
    }
}

Here, we wrap the closure in an Option<F> so that we can specifically decide whether we want to call it in Drop if it’s Some(f), or just drop the function without calling it with None. That decision is taken later when returning values with ErrResult<A, E>, which we can have a look at right now:

struct ErrResult<A, E> {
    result: Result<A, E>,
}

impl<A, E> ErrResult<A, E> {
    fn into_result(self) -> Result<A, E> {
        self.result
    }
}

Nothing fancy, just a type wrapper over a Result<A, E>. Now let’s see how we can build it:

impl<F> ErrDefer<F>
where
    F: FnMut(),
{
    fn new(f: F) -> Self {
        Self { err: Some(f) }
    }

    fn ok<A, E>(mut self, success: A) -> ErrResult<A, E> {
        self.err = None;
        ErrResult { result: Ok(success) }
    }

    fn err<A, E>(self, e: E) -> ErrResult<A, E> {
        ErrResult { result: Err(e) }
    }
}

If you look at ErrDefer::ok(), you can see that before wrapping the success: A into an ErrResult<A, E>, we set the Option<F> closure in ErrDefer<F> to None. That will cause its Drop implementation not to call the closure. However, if you look at ErrDefer::err(), can see that the closure is not dropped, and as such, when the ErrDefer<F> will be dropped, it will call its carried error closure.

You will also see the ErrDefer::new() method, which will not be practical to use, as we will have to wrap our code in a closure. We will use the same idea as with Defer<F> here and use a macro:

macro_rules! errdefer {
    ($($block:tt)*) => {
        ErrDefer::new(|| { $($block)* })
    }
}

We can then write code like this:

fn success() -> ErrResult<i32, &'static str> {
    let result = errdefer! {
        println!("will not be called");
    };

    result.ok(10)
}

fn fail() -> ErrResult<i32, &'static str> {
    let result = errdefer! {
        println!("an error occurred");
    };

    result.err("nope")
}

This is great, but not very useful. Indeed, something that would be more useful would be to able to compose functions that might fail like we do with the try keyword in Zig; or with the ? operator in Rust. The key here is that ErrResult<A, E> type. As with the previous problem where the compiler was not aware of error paths, the compiler here cannot help us understand that a function call returning ErrResult<A, E> can actually make the current function fail, so we need to help it:

macro_rules! errtry {
    ($errdefer:ident, $res:expr) => {
        match $res.result {
            Ok(success) => success,
            Err(err) => return $errdefer.err(err),
        }
    };
}

This macro is not production-ready and just there to demonstrate the idea, but it still holds: it inspects the content of the ErrResult<A, E> (actually, anything that has a err() method as first argument, and result: Result<_, _> as second’s field argument, but this is not important), and convert it to either A, or wrap the E error into the current ErrDefer<A, E>.

You can use this like so:

fn transitive() -> ErrResult<i32, &'static str> {
    let result = errdefer! {
        println!("something failed deeper the stack…");
    };

    let failed = errtry!(result, fail());

    result.ok(failed * 10)
}

Combining and calling all of our previous functions:

fn main() {
    let a = success().into_result();
    println!("a = {a:?}");

    let b = fail().into_result();
    println!("b = {b:?}");

    let c = transitive().into_result();
    println!("c = {c:?}");
}

We can this output:

a = Ok(10)
//an error occurred
b = Err("nope")
//an error occurred
//something failed deeper the stack…
c = Err("nope")

Something interesting to notice here is that errtry! works only for a function, which is also the case in Rust with the ? operator. This is similar to Zig, but there is a slight difference with my current implementation: Zig will evaluate the exit scope value, meaning that if we wanted to have the same result in Rust, we would need to exit all scopes (blocks) manually with a errdefer.ok(value) to have the same effect. Another idea would be to have the function stored as F directly, in a bool — defaulted to false — set to true when errdefer.err(_) is called, allowing inspecting that value in Drop::drop() to call if needed.

Full code.

Now, let’s have a rationale.

Okay and… should I use that?#

The honest and short answer is: no, probably not. However, it can be useful in some circumstances, as mentioned earlier, especially when writing code to interface with C. Sometimes, you might need to have that kind of cleanup that requires defer and errdefer, and having those macros can massively help you.

On the other side, you could also just go for a type wrapper that implements Drop for you, and you have the same logic that can be run for both situations automatically (defer / errdefer) without thinking too much about it. This whole blog entry was an experiment I wanted to make to show that type-based Drop — proper destructors — are more general and reliable than defer and errdefer — you can’t forget to call them.

With hindsight, I find it a bit a pity to have the Zig compiler infrastructure be able to inject code at scope exit / on error, but require the user to do it. I know Zig people would tell you that everything should be explicit, which I think is both a lie and not a strong argument. I think a better argument would be to state that without move semantics, which is pretty hard to implement correctly (to my knowledge, only Rust does it correctly; more on that in the next paragraph), Zig cannot have destructors, otherwise, how do you make a difference between a function that returns an ArrayList(T) — and thus must not deallocate it! — and a function that allocates one, performs some stuff with it, and wants to get rid of it upon exit? This kind of questioning is why I think people are wrong about defer and errdefer: they are not a feature per-se, but a required mean because Zig doesn’t have ownership and move semantics. That strengthened my opinion about the fact that holistic language archictecture moves problems from user-land into compiler / language. The lack of a static construct forces you to introduce keywords and solutions that wouldn’t be required if you had the construct.

And why do I think only Rust implements move semantics correctly? The only other language that I know of which has move semantics is C++, and the way this is implemented in C++ is completely flawed by constructors. Because bindings cannot be invalidated in C++; when a value is constructed, it can have its content moved-out from another value. You then need to just copy those fields (bit-wise copy), but manually invalidate the internals of the other value. That means that the other value’s destructor will still be called, and it will have to inspect its state to realize its content has been moved. I hate this design. It’s overly complicated, and super error-prone.

As always, I hope you enjoyed this small experiment and that it gave you ideas.

Keep the vibes!

↑ Go up ↑

defer and errdefer in Rust #

defer and errdefer in Zig#

What about Rust?#

defer in Rust#

errdefer in Rust#

Okay and… should I use that?#

`defer` and `errdefer` in Zig#

`defer` in Rust#

`errdefer` in Rust#