Refs, packfiles and garbage collection

Git · Expert · 14 min

Understanding storage explains repository size, slow operations and what gc and maintenance actually do.

Refs are names for object IDs. Traditionally each is a file: .git/refs/heads/main contains a commit hash, .git/refs/tags/v1.0 a tag or commit hash, .git/refs/remotes/origin/main the remote-tracking ref. .git/HEAD is usually a symbolic ref: ref: refs/heads/main. With thousands of refs, Git moves them into one .git/packed-refs file; a loose ref file, if present, overrides the packed one. Newer Git also offers the reftable backend (git init --ref-format=reftable), a binary format that scales to millions of refs and updates them atomically.

Objects start out loose: one zlib-compressed file each, at .git/objects/ab/cdef... (the first two hex characters form the directory). That is simple but wasteful for many similar versions.

Packfiles fix that. git gc, git repack and network transfers bundle objects into .git/objects/pack/pack-*.pack with a matching .idx index. Inside a pack, objects can be stored as deltas against similar objects, often an older or newer version of the same file. Clones and fetches send packs, which is why they are compact.

Garbage collection (git gc, also triggered automatically by gc.auto) packs loose objects, packs refs, expires old reflog entries and deletes unreachable objects older than the prune window (two weeks by default). git maintenance schedules incremental versions of these tasks in the background.

Diagnostics:

  • git count-objects -vH: loose vs packed size.
  • git verify-pack -v .git/objects/pack/*.idx | sort -k3 -n | tail: largest objects.
  • git for-each-ref: list refs regardless of storage format.

Example

cat .git/HEAD                     # ref: refs/heads/main
cat .git/refs/heads/main          # 9c1e2ab4f... (if not packed)
git for-each-ref --format='%(refname) %(objectname:short)' refs/heads
git count-objects -vH
# count: 312  size: 1.4 MiB  in-pack: 48210  size-pack: 61.0 MiB
git gc                            # pack objects + refs, prune unreachable
git count-objects -vH             # count: 0 ...
# Biggest objects in the pack
git verify-pack -v .git/objects/pack/pack-*.idx | sort -k3 -n | tail -3

Remember

  • Refs are names to IDs: loose files, packed-refs, or reftable
  • HEAD is normally a symbolic ref to the current branch
  • Packfiles hold delta-compressed objects; loose objects are per-file zlib
  • gc packs, expires reflogs and prunes unreachable objects

Watch out for

  • Editing ref files by hand — use git update-ref so packed-refs, reftable and the reflog stay consistent.

Check yourself

  1. Where are deltas between file versions stored?
  2. What does a symbolic ref like HEAD usually contain?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: git-scm.com/book/en/v2/Git-Internals-Packfiles