rust - How does for<> syntax differ from a regular lifetime bound?

Question

Consider the following code:

trait Trait<T> {}

fn foo<'a>(_b: Box<dyn Trait<&'a usize>>) {}
fn bar(_b: Box<dyn for<'a> Trait<&'a usize>>) {}

Both functions foo and bar seem to accept a Box<Trait<&'a usize>>, although foo does it more concisely than bar. What is the difference between them?

Additionally, in what situations would I need for<> syntax like that above? I know the Rust standard library uses it internally (often related to closures), but why might my code need it?

Answer 1

for<> syntax is called higher-ranked trait bound (HRTB), and it was indeed introduced mostly because of closures.

In short, the difference between foo and bar is that in foo() the lifetime for the internal usize reference is provided by the caller of the function, while in bar() the same lifetime is provided by the function itself. And this distinction is very important for the implementation of foo/bar.

However, in this particular case, when Trait has no methods which use the type parameter, this distinction is pointless, so let's imagine that Trait looks like this:

trait Trait<T> {
    fn do_something(&self, value: T);
}

Remember, lifetime parameters are very similar to generic type parameters. When you use a generic function, you always specify all of its type parameters, providing concrete types, and the compiler monomorphizes the function. Same thing goes with lifetime parameters: when you call a function which have a lifetime parameter, you specify the lifetime, albeit implicitly:

// imaginary explicit syntax
// also assume that there is TraitImpl::new::<T>() -> TraitImpl<T>,
// and TraitImpl<T>: Trait<T>

'a: {
    foo::<'a>(Box::new(TraitImpl::new::<&'a usize>()));
}

And now there is a restriction on what foo() can do with this value, that is, with which arguments it may call do_something(). For example, this won't compile:

fn foo<'a>(b: Box<Trait<&'a usize>>) {
    let x: usize = 10;
    b.do_something(&x);
}

This won't compile because local variables have lifetimes which are strictly smaller than lifetimes specified by the lifetime parameters (I think it is clear why it is so), therefore you can't call b.do_something(&x) because it requires its argument to have lifetime 'a, which is strictly greater than that of x.

However, you can do this with bar:

fn bar(b: Box<for<'a> Trait<&'a usize>>) {
    let x: usize = 10;
    b.do_something(&x);
}

This works because now bar can select the needed lifetime instead of the caller of bar.

This does matter when you use closures which accept references. For example, suppose you want to write a filter() method on Option<T>:

impl<T> Option<T> {
    fn filter<F>(self, f: F) -> Option<T> where F: FnOnce(&T) -> bool {
        match self {
            Some(value) => if f(&value) { Some(value) } else { None }
            None => None
        }
    }
}

The closure here must accept a reference to T because otherwise it would be impossible to return the value contained in the option (this is the same reasoning as with filter() on iterators).

But what lifetime should &T in FnOnce(&T) -> bool have? Remember, we don't specify lifetimes in function signatures only because there is lifetime elision in place; actually the compiler inserts a lifetime parameter for each reference inside a function signature. There should be some lifetime associated with &T in FnOnce(&T) -> bool. So, the most "obvious" way to expand the signature above would be this:

fn filter<'a, F>(self, f: F) -> Option<T> where F: FnOnce(&'a T) -> bool

However, this is not going to work. As in the example with Trait above, lifetime 'a is strictly longer than the lifetime of any local variable in this function, including value inside the match statement. Therefore, it is not possible to apply f to &value because of lifetime mismatch. The above function written with such signature won't compile.

On the other hand, if we expand the signature of filter() like this (and this is actually how lifetime elision for closures works in Rust now):

fn filter<F>(self, f: F) -> Option<T> where F: for<'a> FnOnce(&'a T) -> bool

then calling f with &value as an argument is perfectly valid: we can choose the lifetime now, so using the lifetime of a local variable is absolutely fine. And that's why HRTBs are important: you won't be able to express a lot of useful patterns without them.

You can also read another explanation of HRTBs in Nomicon.