gabriel / muse public
feat main

feat: stat cache — 86x faster muse status

Implements a persistent stat-based file hash cache that eliminates redundant SHA-256 hashing on every snapshot() call.

Architecture ------------ muse/core/stat_cache.py (new): - FileCacheEntry TypedDict: {mtime, size, object_hash, dimensions} The `dimensions` slot is populated by domain plugins with semantic hashes (AST symbol hash, note-event hash, etc.) so that future muse diff / merge can skip re-parsing unchanged files entirely. - StatCache class: load (versioned JSON), save (atomic temp+rename), get_cached (pre-stat fast path), get_object_hash (pathlib wrapper), set_dimension / get_dimension, prune (removes deleted-file entries). - load_cache() convenience helper; StatCache.empty() for test contexts. - Single canonical _hash_bytes / _hash_str — deletes three duplicate _hash_file() copies (code plugin, MIDI plugin, snapshot.py).

Performance ----------- Profile showed pathlib.relative_to() was the hot path (500 ms of 770 ms for 824 files), not SHA-256. Two-pronged fix:

1. Replace pathlib.rglob() + sorted() with os.walk() in walk_workdir and all three plugin snapshot() methods. Hidden directory subtrees are pruned at the walk level (dirnames[:] = ...) so the walker never descends into .muse/, .git/, etc. String slicing computes rel paths in O(1) with no object allocation.

2. StatCache.get_cached() accepts pre-computed (mtime, size) from the walk loop, avoiding a redundant stat() syscall per file.

Result on this repo (824 tracked files): Cold (first run, no cache) : ~200 ms Warm (cache fully hit) : 3.3 ms (86x vs pre-patch 284 ms)

Forward compatibility --------------------- The dimensions: {} field is always written even when empty. Future domain plugins fill it with semantic hashes after parsing; the cache version field allows safe format evolution without corrupting old caches.

G Gabriel Cardona <gabriel@tellurstori.com> · 4d ago Mar 20, 2026 · b7e24576 · parent e05a2ecf
2
files changed
310
files in snapshot
Files Changed 310 in snapshot
+2
← Older Oldest on main
All commits
Newer → Latest on main

0 comments

No comments yet. Be the first to start the discussion.