Pin and self-referential futures

Rust · Expert · 20 min

Pin shows up whenever you implement Future, store futures or write async combinators, and its rules confuse almost everyone at first.

An async block that keeps a reference to one of its own locals across an .await compiles into a self-referential struct: one field points into another field of the same struct. If that struct were moved in memory after the first poll, the internal pointer would dangle.

`Pin<P>` wraps a pointer P (such as &mut T or Box<T>) and guarantees that the pointee will not move again until it is dropped. That is why Future::poll takes self: Pin<&mut Self>: once polled, a future stays put.

Most types do not care about being moved, so they implement the auto trait `Unpin`, and for them Pin is a no-op: Pin::new(&mut x) works and get_mut gives back &mut T. Compiler-generated futures are !Unpin, as is any type with a PhantomPinned field. For those, getting a &mut T out of a Pin is unsafe, because you could mem::swap it away.

Ways to pin a !Unpin value:

  • `Box::pin(value)` pins on the heap; the result Pin<Box<T>> is itself Unpin and easy to store, e.g. Pin<Box<dyn Future<Output = T> + Send>>.
  • `std::pin::pin!(value)` pins on the stack for local use, giving Pin<&mut T>.

When you write your own future that contains another future, you need pin projection: turning Pin<&mut Outer> into Pin<&mut Inner> for the structurally pinned field and a plain &mut for the rest. Doing it by hand requires unsafe code and careful rules (no moving pinned fields, a compatible Drop), so use the pin-project or pin-project-lite crates. Or sidestep it, as in the example, by storing the inner future as Pin<Box<F>>, which keeps the outer type Unpin at the cost of one allocation.

Pin only constrains the pointee's address. It does not make a type immutable, and it is not needed for ordinary code: you meet it when implementing Future or Stream by hand.

Example

use std::future::{ready, Future};
use std::pin::{pin, Pin};
use std::task::{Context, Poll, Waker};

// Pin<Box<F>> is Unpin, so Counted<F> is Unpin and needs no unsafe pin projection.
struct Counted<F> { inner: Pin<Box<F>>, polls: u32 }
impl<F: Future> Future for Counted<F> {
    type Output = (F::Output, u32);
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let this = self.get_mut(); // allowed only because Self: Unpin
        this.polls += 1;
        this.inner.as_mut().poll(cx).map(|v| (v, this.polls))
    }
}

fn main() {
    // self-referential: `r` points into the future's own state across an .await
    let fut = async { let s = String::from("pinned"); let r = &s; ready(()).await; r.len() };
    // Pin::new(&mut fut) would not compile: async blocks are !Unpin
    let mut task = pin!(Counted { inner: Box::pin(fut), polls: 0 }); // pinned on the stack
    let mut cx = Context::from_waker(Waker::noop());
    if let Poll::Ready((len, polls)) = task.as_mut().poll(&mut cx) {
        println!("len {len} after {polls} poll"); // len 6 after 1 poll
    }
}

Remember

  • Futures may be self-referential, so once polled they must not move
  • Pin<P> guarantees the pointee stays put; for Unpin types it changes nothing
  • Pin with Box::pin or pin!; use pin-project (or a boxed inner future) for projection

Watch out for

  • Thinking Pin makes a value immutable; it only forbids moving !Unpin values out from behind the pointer.

Check yourself

  1. Why does `Future::poll` take `self: Pin<&mut Self>` instead of `&mut self`?
  2. For a type that implements `Unpin`, what does `Pin<&mut T>` restrict?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: doc.rust-lang.org/std/pin/index.html

Builds on

Unlocks

Nothing builds on this yet.