Sprint: Algorithm-Prefixed Storage Paths
Every content-addressed ID in Muse carries its algorithm as a prefix in the
value (sha256:<hex>). The same principle must hold in the filesystem path:
the algorithm must appear as a path segment, never as a prefix inside a
filename. This makes storage algorithm-agnostic, future-proof, and
Windows-safe (colons are not valid in filenames on Windows).
The object store already does this correctly:
.muse/objects/sha256/<2-char>/<62-char> — every other storage area must
follow the same pattern.
For nested resources (tags, releases) the path carries the algorithm of each ID independently:
.muse/tags/<repo-algo>/<repo-id>/<tag-algo>/<tag-id>.msgpack
This handles heterogeneous hashing across time — a repo created with sha256 can have tags created with blake3, and the path reflects both without ambiguity.
There is no backwards compatibility layer. All paths move to the new convention in a single atomic refactor.
repo_id: UUID → Genesis Fingerprint
repo_id is currently stored as a UUID in .muse/repo.json. UUIDs must not
be used for any system-level entity. Every system entity must be identified by
a content fingerprint computed at genesis: a one-time hash of the
initialisation inputs (timestamp + domain + path), never re-hashed when
content changes.
The code at init.py:481 already calls content_hash({"created_at": ..., "domain": ..., "path": ...}) — the correct shape. The stored value must also
carry the algorithm prefix (sha256:<hex>), and all path segments derived
from it must use the bare hex under an algo directory.
Agent Quick-Reference
Each row below is one atomic unit of work. An agent should complete one row per session, run only the tests listed for that row, and commit on the task branch for that row.
| # | Area | Source file(s) | New path shape | Utility method | Tests to run |
|---|---|---|---|---|---|
| 1 | Commits | store.py |
.muse/commits/sha256/<hex>.msgpack |
commit_path(root, commit_id) |
test_cmd_commit_*.py, test_store_commits.py |
| 2 | Snapshots | store.py |
.muse/snapshots/sha256/<hex>.msgpack |
snapshot_path(root, snapshot_id) |
test_cmd_snapshot*.py, test_store_snapshots.py |
| 3 | Tags | store.py |
.muse/tags/sha256/<repo-hex>/sha256/<tag-hex>.msgpack |
tag_path(root, repo_id, tag_id) |
test_cmd_tag*.py, test_store_tags.py |
| 4 | Releases | store.py |
.muse/releases/sha256/<repo-hex>/sha256/<release-hex>.msgpack |
release_path(root, repo_id, release_id) |
test_release_supercharge.py, test_store_releases.py |
| 5 | Harmony patterns | harmony.py |
.muse/harmony/patterns/sha256/<hex>/ |
pattern_dir(root, pattern_id) |
test_harmony_*.py |
| 6 | Harmony resolutions | harmony.py |
.muse/harmony/patterns/sha256/<p-hex>/resolutions/sha256/<r-hex>.json |
resolution_path(root, pattern_id, resolution_id) |
test_harmony_*.py |
| 7 | repo_id genesis | store.py, init.py, repo.json |
stored as sha256:<hex>; path segments use bare hex under sha256/ |
repo_genesis_id(inputs) |
test_cmd_init*.py, test_store_repo.py |
Detailed Grid
1 — Commits
| Field | Value |
|---|---|
| Current path | .muse/commits/<64-hex>.msgpack |
| Target path | .muse/commits/sha256/<64-hex>.msgpack |
| Current construction | _commits_dir(root) / f"{commit_id.removeprefix('sha256:')}.msgpack" |
| Target construction | commit_path(root, commit_id) |
| Utility method | commit_path(repo_root: Path, commit_id: str) -> Path |
| Utility location | muse/core/store.py — replaces current commit_path() at line 1381 |
| Utility contract | Accepts sha256:<hex> or bare <hex>. Strips prefix, extracts algo, returns _commits_dir(root) / algo / f"{hex}.msgpack". Raises ValueError for malformed IDs. |
| Algo extraction | algo, _, hex_id = commit_id.partition(':') if : in id, else algo = "sha256" |
| Directory helper | _commits_dir(root) at line 1359 — no change needed |
| All callers | write_commit, read_commit, commit_exists in store.py; any bundle/pack path referencing commits directly |
| Glob pattern update | _commits_dir(root).glob("*.msgpack") → _commits_dir(root).glob("*/*.msgpack") |
| Docstring requirement | Module-level docstring in store.py layout section; function docstring on commit_path explaining algo-in-path convention and why no colons in filenames |
| New test file | tests/test_store_commits.py |
| Test cases | test_commit_path_sha256, test_commit_path_bare_hex, test_commit_path_rejects_invalid, test_commit_path_algo_in_dir, test_write_read_commit_roundtrip, test_commit_path_windows_safe (no colon in any segment) |
2 — Snapshots
| Field | Value |
|---|---|
| Current path | .muse/snapshots/<64-hex>.msgpack |
| Target path | .muse/snapshots/sha256/<64-hex>.msgpack |
| Current construction | _snapshots_dir(root) / f"{snapshot_id.removeprefix('sha256:')}.msgpack" |
| Target construction | snapshot_path(root, snapshot_id) |
| Utility method | snapshot_path(repo_root: Path, snapshot_id: str) -> Path |
| Utility location | muse/core/store.py — replaces current snapshot_path() at line 1386 |
| Utility contract | Same contract as commit_path: accepts prefixed or bare, extracts algo, returns _snapshots_dir(root) / algo / f"{hex}.msgpack". |
| All callers | write_snapshot, read_snapshot, snapshot_exists, apply_manifest in store.py |
| Glob pattern update | _snapshots_dir(root).glob("*.msgpack") → _snapshots_dir(root).glob("*/*.msgpack") |
| Docstring requirement | Function docstring explaining algo-in-path; cross-reference to commit_path |
| New test file | tests/test_store_snapshots.py |
| Test cases | test_snapshot_path_sha256, test_snapshot_path_bare_hex, test_snapshot_path_rejects_invalid, test_snapshot_path_algo_in_dir, test_write_read_snapshot_roundtrip, test_snapshot_path_windows_safe |
3 — Tags
| Field | Value |
|---|---|
| Current path | .muse/tags/<repo-id>/<tag-id>.msgpack (repo-id is UUID; tag-id has sha256: prefix embedded in filename — invalid on Windows) |
| Target path | .muse/tags/sha256/<repo-hex>/sha256/<tag-hex>.msgpack |
| Current construction | _tags_dir(root, repo_id) / f"{tag.tag_id}.msgpack" (line 2156) |
| Target construction | tag_path(root, repo_id, tag_id) |
| Utility method | tag_path(repo_root: Path, repo_id: str, tag_id: str) -> Path |
| Utility location | muse/core/store.py — replaces inline construction at lines 2152, 2179 |
| Utility contract | Both repo_id and tag_id accepted with or without sha256: prefix. Extracts algo from each independently. Returns _tags_dir(root) / repo_algo / repo_hex / tag_algo / f"{tag_hex}.msgpack". Each algo segment comes from its respective ID. Raises ValueError for malformed IDs. |
| Two-level algo | repo_algo may differ from tag_algo — path encodes both. Example: tags/sha256/<repo-hex>/blake3/<tag-hex>.msgpack is valid. |
| All callers | write_tag, delete_tag, get_all_tags, get_tag_by_name in store.py |
| Directory helper update | _tags_dir(root, repo_id) becomes _tags_dir(root) (no repo_id arg); scoping moves into tag_path |
| Glob pattern update | tags_dir.glob("*.msgpack") → tags_dir.glob("*/*/*/*.msgpack") to match <repo-algo>/<repo-hex>/<tag-algo>/<tag-hex>.msgpack |
| Docstring requirement | Explain two-level algo, Windows-safety, and why colons are forbidden in filesystem paths |
| repo_id dependency | After sprint row 7, repo_id will be sha256:<hex> instead of UUID — tag_path must handle both during the row-7 transition, then only the new format |
| New test file | tests/test_store_tags.py |
| Test cases | test_tag_path_both_sha256, test_tag_path_mixed_algos, test_tag_path_rejects_invalid_repo_id, test_tag_path_rejects_invalid_tag_id, test_tag_path_no_colon_in_any_segment, test_write_read_tag_roundtrip, test_delete_tag, test_list_tags |
4 — Releases
| Field | Value |
|---|---|
| Current path | .muse/releases/<repo-id>/<release-hex>.msgpack (repo-id is UUID) |
| Target path | .muse/releases/sha256/<repo-hex>/sha256/<release-hex>.msgpack |
| Current construction | _releases_dir(root, repo_id) / f"{release.release_id}.msgpack" (line 2223) |
| Target construction | release_path(root, repo_id, release_id) |
| Utility method | release_path(repo_root: Path, repo_id: str, release_id: str) -> Path |
| Utility location | muse/core/store.py — replaces inline construction at lines 2223, 2232, 2293 |
| Utility contract | Same two-level algo contract as tag_path. Each ID's algo is extracted independently. |
| All callers | write_release, read_release, list_releases, delete_release in store.py |
| Directory helper update | _releases_dir(root, repo_id) → _releases_dir(root) — scoping moves into release_path |
| Glob pattern update | releases_dir.glob("*.msgpack") → releases_dir.glob("*/*/*/*.msgpack") |
| Docstring requirement | Same as tags; cross-reference tag_path for the two-level algo pattern |
| New test file | tests/test_store_releases.py |
| Test cases | test_release_path_both_sha256, test_release_path_mixed_algos, test_release_path_no_colon_in_any_segment, test_write_read_release_roundtrip, test_delete_release, test_list_releases |
5 — Harmony Patterns
| Field | Value |
|---|---|
| Current path | .muse/harmony/patterns/<64-hex>/ |
| Target path | .muse/harmony/patterns/sha256/<64-hex>/ |
| Current construction | patterns_dir(root) / _id_hex(pattern_id) where _id_hex strips sha256: prefix (line 599) |
| Target construction | pattern_dir(root, pattern_id) |
| Utility method | pattern_dir(repo_root: Path, pattern_id: str) -> Path |
| Utility location | muse/core/harmony.py — replaces _pattern_entry_dir at line 599 |
| Utility contract | Accepts sha256:<hex> or bare hex. Extracts algo, returns patterns_dir(root) / algo / hex_id. Creates parent dirs on write. Raises ValueError for unknown algo or malformed ID. |
_id_hex helper |
Retire _id_hex(); replace all call sites with explicit split_id(id) → (algo, hex) |
| All callers | Every function in harmony.py that calls _pattern_entry_dir or constructs pattern paths |
| Glob pattern update | patterns_dir(root).glob("*/") → patterns_dir(root).glob("*/*/") |
| Docstring requirement | Explain that the pattern directory is NOT content-addressed (directory name = pattern_id hex, not hash of directory contents); the algo prefix is for future hash agility only |
| New test file | tests/test_store_harmony_paths.py |
| Test cases | test_pattern_dir_sha256, test_pattern_dir_bare_hex, test_pattern_dir_algo_in_path, test_pattern_dir_no_colon |
6 — Harmony Resolutions
| Field | Value |
|---|---|
| Current path | .muse/harmony/patterns/<p-hex>/resolutions/<r-hex>.json |
| Target path | .muse/harmony/patterns/sha256/<p-hex>/resolutions/sha256/<r-hex>.json |
| Current construction | Derived from _pattern_entry_dir(root, pid) / "resolutions" / f"{_id_hex(rid)}.json" |
| Target construction | resolution_path(root, pattern_id, resolution_id) |
| Utility method | resolution_path(repo_root: Path, pattern_id: str, resolution_id: str) -> Path |
| Utility location | muse/core/harmony.py — new function, replaces inline construction |
| Utility contract | Builds on pattern_dir. Extracts resolution algo from resolution_id. Returns pattern_dir(root, pattern_id) / "resolutions" / r_algo / f"{r_hex}.json". |
| All callers | All resolution read/write/list/delete functions in harmony.py |
| Glob pattern update | resolutions_dir.glob("*.json") → resolutions_dir.glob("*/*.json") |
| Docstring requirement | Cross-reference pattern_dir; note that resolution IDs and pattern IDs may use different algorithms |
| New test file | Extend tests/test_store_harmony_paths.py |
| Test cases | test_resolution_path_sha256, test_resolution_path_mixed_pattern_resolution_algos, test_resolution_path_no_colon, test_write_read_resolution_roundtrip |
7 — repo_id: UUID → Genesis Fingerprint
| Field | Value |
|---|---|
| Current value | UUID string, e.g. "51db136e-ae54-42c2-91c2-230f8171a488" in .muse/repo.json |
| Target value | "sha256:<64-hex>" stored in repo.json; bare <64-hex> used in filesystem paths under a sha256/ segment |
| Current creation | init.py:481 — content_hash({"created_at": ..., "domain": ..., "path": ...}) — already correct shape, but the stored value on disk is a UUID for this repo (legacy initialization) |
| Target creation | Same content_hash(...) call; store the full sha256:<hex> string in repo.json |
| Utility method | repo_genesis_id(created_at: str, domain: str, path: str) -> str |
| Utility location | muse/core/store.py — new function wrapping content_hash, documents the genesis contract |
| Utility contract | Computes content_hash({"created_at": created_at, "domain": domain, "path": path}). Returns sha256:<hex>. This ID is immutable after genesis — it does not change when content changes. |
| Validation update | validation.py — validate_repo_id must accept sha256:<hex> (prefixed) and reject bare UUIDs. Update the validator and all call sites. |
| Scope | repo.json schema, init.py, store.py, validation.py, and all 376 files that reference repo_id — this is a large cross-cutting refactor. Treat as its own multi-session sprint. |
| Path impact | Tags and releases scoped under repo_id change from tags/<uuid>/ to tags/sha256/<repo-hex>/ — rows 3 and 4 depend on this being settled first, but can use a split_id() helper that handles both formats during transition |
| No UUID fallback | No legacy UUID support. The repo is initialized fresh with the new format. Existing repos must be re-initialized or their repo.json updated manually. |
| Docstring requirement | New repo_genesis_id function docstring must explain: (1) genesis = one-time computation, (2) not re-hashed on content change, (3) content-indexed not content-addressed in the traditional sense, (4) algo in prefix enables future migration |
| New test file | tests/test_store_repo.py |
| Test cases | test_repo_genesis_id_format, test_repo_genesis_id_is_deterministic, test_repo_genesis_id_is_prefixed, test_repo_id_in_path_uses_bare_hex, test_repo_id_validation_rejects_uuid, test_repo_id_validation_accepts_sha256_prefixed |
Shared Utility: split_id()
Every row above needs to extract (algo, hex) from a potentially-prefixed ID.
This must live in one place.
| Field | Value |
|---|---|
| Function | split_id(object_id: str) -> tuple[str, str] |
| Location | muse/core/store.py (or a new muse/core/ids.py if the module gets crowded) |
| Contract | If : present: algo, _, hex_id = object_id.partition(':'). If absent: algo = "sha256", hex_id = object_id. Validates hex_id is 64 lowercase hex chars. Raises ValueError otherwise. |
| Usage | Called by commit_path, snapshot_path, tag_path, release_path, pattern_dir, resolution_path, repo_genesis_id |
| Docstring | Must document: (1) why algo is extracted not assumed, (2) the bare-hex fallback defaults to sha256 for legacy callers only, (3) the function is the single source of truth for ID parsing |
| Test file | tests/test_store_ids.py |
| Test cases | test_split_id_prefixed, test_split_id_bare_hex_defaults_sha256, test_split_id_rejects_empty, test_split_id_rejects_non_hex, test_split_id_rejects_wrong_length, test_split_id_rejects_colon_in_wrong_position |
Object Store (reference — already correct)
.muse/objects/sha256/<2-char>/<62-char> — no changes needed.
object_store.py:object_path() is the canonical model every other utility
method should follow. Read it before implementing any new path utility.
Execution Order
Rows must be executed in this order due to dependencies:
split_id() ← prerequisite for everything
↓
commits (1) ← no dependencies
snapshots (2) ← no dependencies
harmony (5, 6) ← no dependencies
↓
repo_id (7) ← must land before tags/releases can finalize
↓
tags (3) ← depends on repo_id format being settled
releases (4) ← depends on repo_id format being settled
split_id and rows 1, 2, 5, 6 can be parallelized. Row 7 is the blocker for
rows 3 and 4. Do not start rows 3 or 4 until row 7 is merged to dev.