Recursive versus non-recursive make

Make · Expert · 12 min

Large Makefile-based projects live or die by this design choice.

A common layout has a top-level Makefile that runs $(MAKE) -C lib and $(MAKE) -C app, each directory with its own Makefile. It's simple, but it has well-known problems, described in Peter Miller's paper "Recursive Make Considered Harmful":

  • Each sub-make sees only part of the dependency graph. If app depends on a header generated in lib, the top level must order the sub-makes by hand, and getting it wrong yields stale or broken builds.
  • Parallelism suffers: sub-makes are serialized at directory boundaries, and each one re-reads and re-stats files.
  • Even a no-op build runs every sub-make.

The non-recursive alternative: one make process, one complete graph. Each directory contributes a fragment (lib/module.mk) that adds to shared variables and defines rules with paths relative to the root, and the top-level Makefile includes them all. make then knows every edge, parallelises across directories and does nothing when nothing changed.

If you do recurse (for vendored projects with their own Makefiles, for instance), always call $(MAKE), not make: it passes flags such as -j and -n and connects to the jobserver. Variables reach sub-makes only if exported (export CFLAGS) or passed on the command line.

At large scale, generators like CMake (producing Ninja files) or Meson usually replace hand-written Makefiles altogether.

Example

# top-level Makefile: one make, one complete graph
CFLAGS  := -Wall -O2 -Iinclude
OBJ     :=
PROGS   :=
# the fragments define targets first, so name the default goal
.DEFAULT_GOAL := all

include lib/module.mk
include app/module.mk

all: $(PROGS)

%.o: %.c
	$(CC) $(CFLAGS) -c $< -o $@

# ---- lib/module.mk ----
# LIB_OBJ := lib/strings.o lib/hash.o
# OBJ += $(LIB_OBJ)
# lib/libcore.a: $(LIB_OBJ)
# 	$(AR) rcs $@ $^

# ---- app/module.mk ----
# PROGS += app/app
# app/app: app/main.o lib/libcore.a
# 	$(CC) -o $@ $^

Remember

  • Recursive make splits the dependency graph into pieces that can't see each other
  • Non-recursive: include per-directory fragments into one Makefile
  • If you must recurse, call $(MAKE) so flags and the jobserver propagate

Watch out for

  • Calling plain make in a recipe, which ignores -j limits and -n dry runs.

Check yourself

  1. What is the main correctness problem with recursive make?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: www.gnu.org/software/make/manual/make.html#Recursion

Builds on

Unlocks

Nothing builds on this yet.