Memory model and atomic ordering

Lock-free code and fast synchronisation depend on memory ordering; getting it wrong produces bugs that appear only on some CPUs and some days.

Compilers and CPUs reorder memory operations freely as long as a single thread can't tell. Other threads can. The C++ memory model defines what one thread is guaranteed to see of another's writes, through the happens-before relation. A program without data races behaves as if operations were interleaved in an order consistent with happens-before; a program with a data race has undefined behaviour.

Atomic operations take a std::memory_order:

seq_cst (the default): all seq_cst operations appear in one global order that every thread agrees on. The easiest to reason about, and a bit more expensive on weakly ordered CPUs (ARM, POWER) and for stores on x86.
release (on stores) and acquire (on loads): if an acquire load reads the value written by a release store, everything the writer did before the store happens-before everything the reader does after the load. This is the publish/subscribe pattern behind mutexes, queues and one-time initialization.
relaxed: the operation itself is atomic, but there are no ordering guarantees for other memory. Fine for statistics counters and for compare-and-swap loops where only the atomic value matters.
acq_rel for read-modify-write operations that both acquire and release. (consume exists but compilers treat it as acquire, and it's being removed.)

x86 is strongly ordered, so a missing acquire/release often only shows up on ARM, which is a good reason to rely on the model rather than on testing.

compare_exchange_weak(expected, desired) is the building block of lock-free algorithms. On failure it loads the current value into expected, so it belongs in a loop; the weak form can fail spuriously and is cheaper on LL/SC architectures. C++20 also adds atomic::wait/notify_one for efficient blocking and std::atomic<std::shared_ptr<T>>.

Start with mutexes or seq_cst; weaken orderings only with a proof and a benchmark.

Example

#include <atomic>
#include <cassert>
#include <string>
#include <thread>

std::string payload;                     // ordinary, non-atomic data
std::atomic<bool> ready{false};

void producer() {
    payload = "hello";                                 // (1) plain write
    ready.store(true, std::memory_order_release);      // (2) publishes everything before it
}

void consumer() {
    while (!ready.load(std::memory_order_acquire))     // (3) synchronises with (2)
        std::this_thread::yield();
    assert(payload == "hello");                        // (4) safe: (1) happens-before (4)
}

std::atomic<int> max_seen{0};
void update_max(int v) {                               // lock-free CAS loop
    int cur = max_seen.load(std::memory_order_relaxed);
    while (cur < v && !max_seen.compare_exchange_weak(cur, v, std::memory_order_relaxed)) {
        // on failure `cur` now holds the latest value; the loop re-checks it
    }
}

int main() {
    std::jthread c{consumer}, p{producer};
    std::jthread a{update_max, 7}, b{update_max, 3};
}

Remember

A release store read by an acquire load makes all prior writes visible
relaxed = atomicity only; fine for counters, wrong for publishing data
Default seq_cst is easiest; weaken only with proof and measurements

Watch out for

Publishing data with a relaxed flag: it may work on x86 and fail on ARM.

Check yourself

In the producer/consumer above, both orders are changed to memory_order_relaxed. What is the consequence?
Why is compare_exchange_weak normally called in a loop?

Official docs: en.cppreference.com/w/cpp/atomic/memory_order

Example

Remember

Watch out for

Check yourself

Builds on

Unlocks