TopolearnWriting a config or data file in place leaves a half-written file behind if the program crashes mid-write.
open(path, "w") truncates the file immediately. If the program crashes, the disk fills up, or the power fails before the write finishes, readers find an empty or partial file. The fix is the write-then-rename pattern:
1. Write the new contents to a temporary file in the same directory (rename only works atomically within one file system). 2. Flush Python's buffer with f.flush(), then os.fsync(f.fileno()) so the operating system pushes the data to disk. 3. os.replace(tmp, path) swaps the new file into place.
On POSIX, rename is atomic: any reader sees either the complete old file or the complete new one, never a mix. On Windows, os.replace is the right call too, though the guarantee is weaker.
For full durability after a crash, also fsync the directory on Linux so the rename itself is recorded: open it with os.open(dirname, os.O_RDONLY), call os.fsync on that descriptor, and close it.
tempfile.NamedTemporaryFile(dir=..., delete=False) or tempfile.mkstemp(dir=...) create a uniquely named temp file safely. If anything fails before the replace, delete the temp file so it doesn't accumulate.
Note that the new file gets fresh permissions and ownership; copy them with shutil.copymode if they matter.
Example
import json
import os
import tempfile
def write_atomic(path, text):
folder = os.path.dirname(os.path.abspath(path))
fd, tmp = tempfile.mkstemp(dir=folder, prefix=".tmp-")
try:
with os.fdopen(fd, "w") as f:
f.write(text)
f.flush()
os.fsync(f.fileno())
os.replace(tmp, path) # atomic swap on POSIX
except BaseException:
os.remove(tmp)
raise
write_atomic("settings.json", json.dumps({"theme": "dark"}))
write_atomic("settings.json", json.dumps({"theme": "light"}))
print(open("settings.json").read()) # {"theme": "light"}
print(os.listdir(".")) # ['settings.json']Remember
- Write to a temp file in the same directory, fsync, then
os.replace flushempties Python's buffer;os.fsyncasks the OS to persist to disk- Clean up the temp file on failure
Watch out for
- Creating the temp file in /tmp; renaming across file systems is not atomic and may fail with EXDEV.