I actually looked at fstat, but the "check for deletions" piece, given I'm polling at 1kHZ, was the reason I decided not to use it. Older hardware actually made this a big issue but it's fast enough now I decided it wasn't a problem.
I'll ignore the malicious ones bc [out of scope declaration]. Object paranoia is an artifact of build trama and I respect that lmao.
I've just looked into the device number and system clock issues. I think what i'll end up doing is actually a combo of ncruces's above comment and your feedback: a 1kHZ data_version and a 10HZ stat() with version check. This gets around syscall load, avoid clock issues, avoids the WAL truncation issues that others have mentioned, and is both lighter weight and less bugabooable than my previous design.
One clarification: by "check for deletions" I didn't mean that you need to read back through the filesystem; you can check for deletions for free using fstat(2)'s result. The number of hard links to a file descriptor's underlying description returned by fstat includes the "existential" hard link of the file itself, and drops to zero when the file's deleted and the open handle is an orphan:
import os
import time
from threading import Thread, Event
f = '/tmp/foo.test'
ev = Event()
Thread(target=lambda: ev.wait() and os.unlink(f), daemon=True).start()
with open(f, 'w+') as fh:
print("before delete:", os.fstat(fh.fileno()).st_nlink)
ev.set()
time.sleep(1)
print("after delete:", os.fstat(fh.fileno()).st_nlink)
I actually looked at fstat, but the "check for deletions" piece, given I'm polling at 1kHZ, was the reason I decided not to use it. Older hardware actually made this a big issue but it's fast enough now I decided it wasn't a problem.
I'll ignore the malicious ones bc [out of scope declaration]. Object paranoia is an artifact of build trama and I respect that lmao.
I've just looked into the device number and system clock issues. I think what i'll end up doing is actually a combo of ncruces's above comment and your feedback: a 1kHZ data_version and a 10HZ stat() with version check. This gets around syscall load, avoid clock issues, avoids the WAL truncation issues that others have mentioned, and is both lighter weight and less bugabooable than my previous design.
Thanks again.