TopolearnLocking files, exclusive creation, pipes and inheritance between processes are all expressed in terms of descriptors.
Underneath every Python file object is a file descriptor: a small integer indexing the process's table of open files. f.fileno() returns it; 0, 1 and 2 are stdin, stdout and stderr.
os.open(path, flags, mode=0o777) is the raw system call. Flags are OR-ed together:
os.O_RDONLY,os.O_WRONLY,os.O_RDWRos.O_CREATcreate if missing,os.O_EXCLfail if it exists (an atomic "create only if new", ideal for lock files),os.O_TRUNC,os.O_APPEND
os.read(fd, n) and os.write(fd, data) move bytes with no buffering; os.write may write fewer bytes than given and returns the count. os.close(fd) releases it. Wrap a descriptor in a normal file object with os.fdopen(fd, "w") or open(fd, closefd=True).
Other descriptor tools:
os.pipe()returns(read_fd, write_fd)of a unidirectional channel.os.dup2(fd, 1)redirects stdout at the OS level, which also affects C extensions and child processes.- Since Python 3.4 new descriptors are non-inheritable by default (
O_CLOEXEC);os.set_inheritablechanges that. - The
opener=argument of the built-inopenlets you supply custom flags while still getting a buffered file.
Leaking descriptors eventually triggers OSError: [Errno 24] Too many open files.
Example
import os
fd = os.open("app.lock", os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o644)
os.write(fd, str(os.getpid()).encode())
os.close(fd)
try: # a second exclusive create fails atomically
os.open("app.lock", os.O_WRONLY | os.O_CREAT | os.O_EXCL)
except FileExistsError:
print("already locked") # already locked
r, w = os.pipe()
os.write(w, b"ping")
os.close(w)
print(os.read(r, 100)) # b'ping'
os.close(r)
def append_only(path, flags):
return os.open(path, flags | os.O_APPEND, 0o600)
with open("audit.log", "a", opener=append_only) as f:
print(f.fileno() > 2) # TrueRemember
- File objects wrap integer descriptors;
os.open/read/write/closework on them directly O_CREAT | O_EXCLgives an atomic create-if-absent, the basis of lock files- Descriptors are non-inheritable by default; close every one you open
Watch out for
- Ignoring the return value of
os.write; it can write fewer bytes than requested on pipes and sockets.