Loaders, caching and auto-reload

Jinja · Advanced · 10 min

Loader and cache setup decide where templates come from and how fast rendering is in production.

The loader maps a template name to source. Built-in options:

  • FileSystemLoader(["templates", "shared"]): searches directories in order
  • PackageLoader("myapp", "templates"): loads from an installed Python package, ideal for libraries
  • DictLoader({...}): in-memory, great for tests
  • FunctionLoader(fn): any callable returning source, for example from a database
  • ChoiceLoader([a, b]): tries loaders in order, so an app's templates can override a library's
  • PrefixLoader({"admin": loader}): namespaces, "admin/index.html"

Template names always use forward slashes and cannot escape the loader's root with ...

Caching. get_template compiles a template once and keeps it in an LRU cache (cache_size=400 by default). With auto_reload=True (the default), Jinja checks the source's modification time on each get_template call and recompiles if it changed, which is convenient in development. In production, auto_reload=False skips those filesystem checks.

Bytecode cache. Compiling is the expensive step. A FileSystemBytecodeCache (or memcached-based cache) stores compiled bytecode on disk so new processes, for example many short-lived workers, skip recompiling.

env.list_templates() lists everything a loader can find, useful for pre-compiling or checking that all templates parse in CI.

Keep one Environment per application. Creating a new Environment per request throws away the cache and recompiles every template each time, a common performance bug.

Example

import os, tempfile
from jinja2 import ChoiceLoader, DictLoader, Environment, FileSystemLoader, PrefixLoader

with tempfile.TemporaryDirectory() as d:
    with open(os.path.join(d, "hello.txt"), "w") as f:
        f.write("v1 from disk")
    env = Environment(loader=ChoiceLoader([
        FileSystemLoader(d),                                    # app templates win
        DictLoader({"hello.txt": "default", "footer.txt": "(c) lib"}),
        PrefixLoader({"admin": DictLoader({"index.txt": "admin home"})}),
    ]))
    print(env.get_template("hello.txt").render())        # v1 from disk
    print(env.get_template("footer.txt").render())       # (c) lib
    print(env.get_template("admin/index.txt").render())  # admin home
    print(env.get_template("hello.txt") is env.get_template("hello.txt"))  # True (cached)

    with open(os.path.join(d, "hello.txt"), "w") as f:
        f.write("v2 edited")
    os.utime(os.path.join(d, "hello.txt"), (0, 10**10))   # make the change visible
    print(env.get_template("hello.txt").render())        # v2 edited (auto_reload)

Remember

  • FileSystemLoader for apps, PackageLoader for libraries, ChoiceLoader for overrides
  • Compiled templates are cached; auto_reload re-checks sources
  • Use one shared Environment; a bytecode cache speeds up cold starts

Watch out for

  • Creating a new Environment for every render, recompiling templates each time.

Check yourself

  1. Why should a web app create its Jinja Environment once at startup?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: jinja.palletsprojects.com/en/stable/api/#loaders