[gaborbernat]

SoftReadWriteLock keeps its whole state as files under <path>.rw: generation records in gen and one holder record per participant in holders. 🐛 Two of its storage operations trusted a name they had checked earlier. A tree swapped in after the first acquire redirected the lock's creates, renames and unlinks into the directory a symlink or junction named. The holder heartbeat wrote through, or blocked on, any file renamed over its record. A swapped tree also ended the heartbeat thread without a report. The fixes live in the storage and protocol layers. Stacked on #839. This resolves GHSA-wfhq-q4j2-c37m and GHSA-fj95-fr6c-h845.

Records reached by name through a swapped tree

OsFiles.prepare checked with lstat that <path>.rw, gen and holders are real directories. It did so once per lock object since #799 added the _prepared shortcut, and every later create, link, replace, unlink and listdir resolved the full path again. O_NOFOLLOW applies to the last path component alone (open(2)). Between two acquisitions of a long-lived lock there was no timing to win: when another process renamed the checked tree aside and put one in its place whose gen was a symlink into a directory of the lock's own account, the next write_lock() returned without error after creating epoch and two generation records there and renaming a record over a file named HEAD. With holders swapped instead, the sweep unlinked files with 32-hex-digit names in the target. The lock refused the planted tree in a fresh process, so the check existed but no live instance consulted it again. The generation log came with #738.

sequenceDiagram
    participant W as Worker (one lock object)
    participant O as Other process
    participant T as Target directory
    rect rgb(207, 228, 255)
        W->>W: first write_lock(): prepare() checks the tree, remembers it
    end
    rect rgb(255, 214, 214)
        O->>O: renames <path>.rw aside, plants a tree with gen -> T
        W->>T: second write_lock(): creates epoch and generations in T
        W->>T: replace() renames a record over T/HEAD
    end
    rect rgb(216, 243, 220)
        Note over W: now each operation opens <path>.rw and gen without following symlinks and works relative to them
    end

Loading

Where the platform supports dir_fd (openat(2)), every record operation now opens <path>.rw and then gen or holders with O_NOFOLLOW and acts relative to that descriptor. A swapped-in symlink fails with a TreeReplacedError, a RuntimeError subclass, and prepare creates and inspects the directories the same way. Each operation also compares the device and inode of the <path>.rw, gen and holders it opened with the ones prepare recorded, and repeats the ownership check from #839 when they differ. A live lock object therefore meets a tree another account created after a recovery the way a fresh one does. No descriptor outlives the operation, so the documented recovery of renaming <path>.rw aside still moves live instances to a fresh log. Each operation costs two extra directory opens and one fstat per opened directory, which may show on network filesystems.

Windows has no dir_fd, and os.lstat reports a junction (a Windows directory that points at another directory) as a directory with the reparse flag set. There the directory check now refuses a reparse point, and each operation holds the directories open without following one and without sharing deletion, so no process can rename or replace them until the operation finishes. Other platforms without dir_fd keep the path-based calls.

Holder refresh wrote through any file at the record's name

A holder refreshes its record on every poll of a waiting acquire and on every heartbeat by reopening holders/<token> by name and writing at offset zero. Since #738 that open was a plain blocking O_WRONLY | O_NOFOLLOW with no fstat of what it opened, while read() next to it used O_NONBLOCK and a regular-file check. O_NOFOLLOW refuses a symlink but not a hard link (a second name for the same file) or another file renamed over the name. Where the kernel permits linking a file one does not own, a hard link renamed over the record received the holder record in its first 135 bytes while the holder saw an ordinary Timeout.

A FIFO (a named pipe) renamed there needs no such setting: opening a FIFO for writing blocks until a reader appears, so write_lock(timeout=2) did not return. A holder's heartbeat thread stayed asleep the same way, on_compromise did not run, and after stale_threshold a second process took the lock while the first received no report.

sequenceDiagram
    participant H as Holder (heartbeat)
    participant O as Other process
    rect rgb(255, 214, 214)
        H->>H: create holders/<token>, close it
        O->>O: renames a hard link or FIFO over holders/<token>
        H->>H: overwrite(): open by name, write at offset 0
        Note over H: hard link, another file overwritten. FIFO, open blocks past any timeout
    end
    rect rgb(216, 243, 220)
        H->>H: now open with O_NONBLOCK, fstat, compare device and inode, require one link
        Note over H: anything else counts as the record being gone: contender polls on, holder reports evicted
    end

Loading

create now returns the record's device and inode (the kernel's identity for a file, independent of its name), and the participant keeps them. The refresh opens with O_NONBLOCK and writes only when the descriptor is that regular file with a single link. The protocol does not link holder records, so a second name also marks a file that inherited a removed record's inode number. A FIFO with no reader fails the non-blocking open with ENXIO, which counts as the record being gone: a contender keeps polling, and a holder reports lost, which surfaces as the evicted compromise it would get for a deleted record. The Files protocol signature changes to match; it is internal to _soft_rw. On Windows the same identity check refuses a hard-link swap, and named pipes cannot live in a regular directory, so there is no FIFO case there.

A swapped tree ended the heartbeat without a report

The refresh opens <path>.rw through a descriptor and raises RuntimeError for a symlink or junction found there. Participant.heartbeat() handled OSError only, so the exception ended the heartbeat thread, on_compromise never ran, and the holder kept reporting its generation. The renamed-aside tree is the recovery the how-to describes. If a symlink or junction sat at <path>.rw for one heartbeat and was removed later, a new process started a fresh log and acquired the write lock while the first holder still believed it held it. Before the descriptor route, the by-name open followed the link, found no record and reported evicted.

sequenceDiagram
    participant H as First holder (heartbeat)
    participant O as Other process
    participant S as Second holder
    rect rgb(255, 214, 214)
        O->>O: renames <path>.rw aside, puts a symlink at <path>.rw
        H->>H: refresh raises RuntimeError, thread ends
        Note over H: compromise stays None, no report
        O->>O: removes the symlink
        S->>S: acquire_write() starts a new log at generation 1
        Note over H,S: two holders, the first never told
    end
    rect rgb(216, 243, 220)
        Note over H: now the refresh raises TreeReplacedError, heartbeat() returns lost
        H->>H: report evicted through on_compromise
    end

Loading

The storage layer raises TreeReplacedError, a RuntimeError subclass, when a protocol directory is a symlink, a junction, a file or belongs to another account. heartbeat() returns lost for it, as for a removed record, and treats it like an OSError when reading the log or sweeping. leave() returns without raising, since a record that cannot be reached has nothing to commit out of or remove. The refresh loop also reports any other exception from heartbeat() as refresh-failed, so the thread does not end with the hold still published. LeaseCompromise.error accepts any Exception for that reason.

Behavior changes

A symlink or junction swapped in for <path>.rw, gen or holders now fails the operation with RuntimeError instead of following it. On Windows a rename of <path>.rw during an operation fails with PermissionError and succeeds once the operation ends. A holder whose record another file replaced, or whose <path>.rw a symlink or junction replaced, reports evicted at its next heartbeat rather than writing into that file. A lock object that used a tree earlier now refuses a replacement tree owned by another account under a sticky parent, as a new object does. LeaseCompromise.error is typed Exception | None.