# Sprint: Algorithm-Prefixed Storage Paths Every content-addressed ID in Muse carries its algorithm as a prefix in the value (`sha256:`). The same principle must hold in the **filesystem path**: the algorithm must appear as a path segment, never as a prefix inside a filename. This makes storage algorithm-agnostic, future-proof, and Windows-safe (colons are not valid in filenames on Windows). The object store already does this correctly: `.muse/objects/sha256/<2-char>/<62-char>` — every other storage area must follow the same pattern. For nested resources (tags, releases) the path carries the algorithm of each ID independently: ``` .muse/tags////.msgpack ``` This handles heterogeneous hashing across time — a repo created with sha256 can have tags created with blake3, and the path reflects both without ambiguity. There is **no backwards compatibility layer**. All paths move to the new convention in a single atomic refactor. --- ## repo_id: UUID → Genesis Fingerprint `repo_id` is currently stored as a UUID in `.muse/repo.json`. UUIDs must not be used for any system-level entity. Every system entity must be identified by a **content fingerprint computed at genesis**: a one-time hash of the initialisation inputs (timestamp + domain + path), never re-hashed when content changes. The code at `init.py:481` already calls `content_hash({"created_at": ..., "domain": ..., "path": ...})` — the correct shape. The stored value must also carry the algorithm prefix (`sha256:`), and all path segments derived from it must use the bare hex under an algo directory. --- ## Agent Quick-Reference Each row below is one atomic unit of work. An agent should complete one row per session, run only the tests listed for that row, and commit on the task branch for that row. | # | Area | Source file(s) | New path shape | Utility method | Tests to run | |---|------|---------------|----------------|----------------|--------------| | 1 | Commits | `store.py` | `.muse/commits/sha256/.msgpack` | `commit_path(root, commit_id)` | `test_cmd_commit_*.py`, `test_store_commits.py` | | 2 | Snapshots | `store.py` | `.muse/snapshots/sha256/.msgpack` | `snapshot_path(root, snapshot_id)` | `test_cmd_snapshot*.py`, `test_store_snapshots.py` | | 3 | Tags | `store.py` | `.muse/tags/sha256//sha256/.msgpack` | `tag_path(root, repo_id, tag_id)` | `test_cmd_tag*.py`, `test_store_tags.py` | | 4 | Releases | `store.py` | `.muse/releases/sha256//sha256/.msgpack` | `release_path(root, repo_id, release_id)` | `test_release_supercharge.py`, `test_store_releases.py` | | 5 | Harmony patterns | `harmony.py` | `.muse/harmony/patterns/sha256//` | `pattern_dir(root, pattern_id)` | `test_harmony_*.py` | | 6 | Harmony resolutions | `harmony.py` | `.muse/harmony/patterns/sha256//resolutions/sha256/.json` | `resolution_path(root, pattern_id, resolution_id)` | `test_harmony_*.py` | | 7 | repo_id genesis | `store.py`, `init.py`, `repo.json` | stored as `sha256:`; path segments use bare hex under `sha256/` | `repo_genesis_id(inputs)` | `test_cmd_init*.py`, `test_store_repo.py` | --- ## Detailed Grid ### 1 — Commits | Field | Value | |-------|-------| | **Current path** | `.muse/commits/<64-hex>.msgpack` | | **Target path** | `.muse/commits/sha256/<64-hex>.msgpack` | | **Current construction** | `_commits_dir(root) / f"{commit_id.removeprefix('sha256:')}.msgpack"` | | **Target construction** | `commit_path(root, commit_id)` | | **Utility method** | `commit_path(repo_root: Path, commit_id: str) -> Path` | | **Utility location** | `muse/core/store.py` — replaces current `commit_path()` at line 1381 | | **Utility contract** | Accepts `sha256:` or bare ``. Strips prefix, extracts algo, returns `_commits_dir(root) / algo / f"{hex}.msgpack"`. Raises `ValueError` for malformed IDs. | | **Algo extraction** | `algo, _, hex_id = commit_id.partition(':')` if `:` in id, else `algo = "sha256"` | | **Directory helper** | `_commits_dir(root)` at line 1359 — no change needed | | **All callers** | `write_commit`, `read_commit`, `commit_exists` in `store.py`; any bundle/pack path referencing commits directly | | **Glob pattern update** | `_commits_dir(root).glob("*.msgpack")` → `_commits_dir(root).glob("*/*.msgpack")` | | **Docstring requirement** | Module-level docstring in `store.py` layout section; function docstring on `commit_path` explaining algo-in-path convention and why no colons in filenames | | **New test file** | `tests/test_store_commits.py` | | **Test cases** | `test_commit_path_sha256`, `test_commit_path_bare_hex`, `test_commit_path_rejects_invalid`, `test_commit_path_algo_in_dir`, `test_write_read_commit_roundtrip`, `test_commit_path_windows_safe` (no colon in any segment) | --- ### 2 — Snapshots | Field | Value | |-------|-------| | **Current path** | `.muse/snapshots/<64-hex>.msgpack` | | **Target path** | `.muse/snapshots/sha256/<64-hex>.msgpack` | | **Current construction** | `_snapshots_dir(root) / f"{snapshot_id.removeprefix('sha256:')}.msgpack"` | | **Target construction** | `snapshot_path(root, snapshot_id)` | | **Utility method** | `snapshot_path(repo_root: Path, snapshot_id: str) -> Path` | | **Utility location** | `muse/core/store.py` — replaces current `snapshot_path()` at line 1386 | | **Utility contract** | Same contract as `commit_path`: accepts prefixed or bare, extracts algo, returns `_snapshots_dir(root) / algo / f"{hex}.msgpack"`. | | **All callers** | `write_snapshot`, `read_snapshot`, `snapshot_exists`, `apply_manifest` in `store.py` | | **Glob pattern update** | `_snapshots_dir(root).glob("*.msgpack")` → `_snapshots_dir(root).glob("*/*.msgpack")` | | **Docstring requirement** | Function docstring explaining algo-in-path; cross-reference to `commit_path` | | **New test file** | `tests/test_store_snapshots.py` | | **Test cases** | `test_snapshot_path_sha256`, `test_snapshot_path_bare_hex`, `test_snapshot_path_rejects_invalid`, `test_snapshot_path_algo_in_dir`, `test_write_read_snapshot_roundtrip`, `test_snapshot_path_windows_safe` | --- ### 3 — Tags | Field | Value | |-------|-------| | **Current path** | `.muse/tags//.msgpack` (repo-id is UUID; tag-id has `sha256:` prefix embedded in filename — invalid on Windows) | | **Target path** | `.muse/tags/sha256//sha256/.msgpack` | | **Current construction** | `_tags_dir(root, repo_id) / f"{tag.tag_id}.msgpack"` (line 2156) | | **Target construction** | `tag_path(root, repo_id, tag_id)` | | **Utility method** | `tag_path(repo_root: Path, repo_id: str, tag_id: str) -> Path` | | **Utility location** | `muse/core/store.py` — replaces inline construction at lines 2152, 2179 | | **Utility contract** | Both `repo_id` and `tag_id` accepted with or without `sha256:` prefix. Extracts algo from each independently. Returns `_tags_dir(root) / repo_algo / repo_hex / tag_algo / f"{tag_hex}.msgpack"`. Each algo segment comes from its respective ID. Raises `ValueError` for malformed IDs. | | **Two-level algo** | `repo_algo` may differ from `tag_algo` — path encodes both. Example: `tags/sha256//blake3/.msgpack` is valid. | | **All callers** | `write_tag`, `delete_tag`, `get_all_tags`, `get_tag_by_name` in `store.py` | | **Directory helper update** | `_tags_dir(root, repo_id)` becomes `_tags_dir(root)` (no repo_id arg); scoping moves into `tag_path` | | **Glob pattern update** | `tags_dir.glob("*.msgpack")` → `tags_dir.glob("*/*/*/*.msgpack")` to match `///.msgpack` | | **Docstring requirement** | Explain two-level algo, Windows-safety, and why colons are forbidden in filesystem paths | | **repo_id dependency** | After sprint row 7, `repo_id` will be `sha256:` instead of UUID — `tag_path` must handle both during the row-7 transition, then only the new format | | **New test file** | `tests/test_store_tags.py` | | **Test cases** | `test_tag_path_both_sha256`, `test_tag_path_mixed_algos`, `test_tag_path_rejects_invalid_repo_id`, `test_tag_path_rejects_invalid_tag_id`, `test_tag_path_no_colon_in_any_segment`, `test_write_read_tag_roundtrip`, `test_delete_tag`, `test_list_tags` | --- ### 4 — Releases | Field | Value | |-------|-------| | **Current path** | `.muse/releases//.msgpack` (repo-id is UUID) | | **Target path** | `.muse/releases/sha256//sha256/.msgpack` | | **Current construction** | `_releases_dir(root, repo_id) / f"{release.release_id}.msgpack"` (line 2223) | | **Target construction** | `release_path(root, repo_id, release_id)` | | **Utility method** | `release_path(repo_root: Path, repo_id: str, release_id: str) -> Path` | | **Utility location** | `muse/core/store.py` — replaces inline construction at lines 2223, 2232, 2293 | | **Utility contract** | Same two-level algo contract as `tag_path`. Each ID's algo is extracted independently. | | **All callers** | `write_release`, `read_release`, `list_releases`, `delete_release` in `store.py` | | **Directory helper update** | `_releases_dir(root, repo_id)` → `_releases_dir(root)` — scoping moves into `release_path` | | **Glob pattern update** | `releases_dir.glob("*.msgpack")` → `releases_dir.glob("*/*/*/*.msgpack")` | | **Docstring requirement** | Same as tags; cross-reference `tag_path` for the two-level algo pattern | | **New test file** | `tests/test_store_releases.py` | | **Test cases** | `test_release_path_both_sha256`, `test_release_path_mixed_algos`, `test_release_path_no_colon_in_any_segment`, `test_write_read_release_roundtrip`, `test_delete_release`, `test_list_releases` | --- ### 5 — Harmony Patterns | Field | Value | |-------|-------| | **Current path** | `.muse/harmony/patterns/<64-hex>/` | | **Target path** | `.muse/harmony/patterns/sha256/<64-hex>/` | | **Current construction** | `patterns_dir(root) / _id_hex(pattern_id)` where `_id_hex` strips `sha256:` prefix (line 599) | | **Target construction** | `pattern_dir(root, pattern_id)` | | **Utility method** | `pattern_dir(repo_root: Path, pattern_id: str) -> Path` | | **Utility location** | `muse/core/harmony.py` — replaces `_pattern_entry_dir` at line 599 | | **Utility contract** | Accepts `sha256:` or bare hex. Extracts algo, returns `patterns_dir(root) / algo / hex_id`. Creates parent dirs on write. Raises `ValueError` for unknown algo or malformed ID. | | **`_id_hex` helper** | Retire `_id_hex()`; replace all call sites with explicit `split_id(id)` → `(algo, hex)` | | **All callers** | Every function in `harmony.py` that calls `_pattern_entry_dir` or constructs pattern paths | | **Glob pattern update** | `patterns_dir(root).glob("*/")` → `patterns_dir(root).glob("*/*/")` | | **Docstring requirement** | Explain that the pattern directory is NOT content-addressed (directory name = pattern_id hex, not hash of directory contents); the algo prefix is for future hash agility only | | **New test file** | `tests/test_store_harmony_paths.py` | | **Test cases** | `test_pattern_dir_sha256`, `test_pattern_dir_bare_hex`, `test_pattern_dir_algo_in_path`, `test_pattern_dir_no_colon` | --- ### 6 — Harmony Resolutions | Field | Value | |-------|-------| | **Current path** | `.muse/harmony/patterns//resolutions/.json` | | **Target path** | `.muse/harmony/patterns/sha256//resolutions/sha256/.json` | | **Current construction** | Derived from `_pattern_entry_dir(root, pid) / "resolutions" / f"{_id_hex(rid)}.json"` | | **Target construction** | `resolution_path(root, pattern_id, resolution_id)` | | **Utility method** | `resolution_path(repo_root: Path, pattern_id: str, resolution_id: str) -> Path` | | **Utility location** | `muse/core/harmony.py` — new function, replaces inline construction | | **Utility contract** | Builds on `pattern_dir`. Extracts resolution algo from resolution_id. Returns `pattern_dir(root, pattern_id) / "resolutions" / r_algo / f"{r_hex}.json"`. | | **All callers** | All resolution read/write/list/delete functions in `harmony.py` | | **Glob pattern update** | `resolutions_dir.glob("*.json")` → `resolutions_dir.glob("*/*.json")` | | **Docstring requirement** | Cross-reference `pattern_dir`; note that resolution IDs and pattern IDs may use different algorithms | | **New test file** | Extend `tests/test_store_harmony_paths.py` | | **Test cases** | `test_resolution_path_sha256`, `test_resolution_path_mixed_pattern_resolution_algos`, `test_resolution_path_no_colon`, `test_write_read_resolution_roundtrip` | --- ### 7 — repo_id: UUID → Genesis Fingerprint | Field | Value | |-------|-------| | **Current value** | UUID string, e.g. `"51db136e-ae54-42c2-91c2-230f8171a488"` in `.muse/repo.json` | | **Target value** | `"sha256:<64-hex>"` stored in `repo.json`; bare `<64-hex>` used in filesystem paths under a `sha256/` segment | | **Current creation** | `init.py:481` — `content_hash({"created_at": ..., "domain": ..., "path": ...})` — already correct shape, but the stored value on disk is a UUID for this repo (legacy initialization) | | **Target creation** | Same `content_hash(...)` call; store the full `sha256:` string in `repo.json` | | **Utility method** | `repo_genesis_id(created_at: str, domain: str, path: str) -> str` | | **Utility location** | `muse/core/store.py` — new function wrapping `content_hash`, documents the genesis contract | | **Utility contract** | Computes `content_hash({"created_at": created_at, "domain": domain, "path": path})`. Returns `sha256:`. This ID is immutable after genesis — it does not change when content changes. | | **Validation update** | `validation.py` — `validate_repo_id` must accept `sha256:` (prefixed) and reject bare UUIDs. Update the validator and all call sites. | | **Scope** | `repo.json` schema, `init.py`, `store.py`, `validation.py`, and all 376 files that reference `repo_id` — this is a large cross-cutting refactor. Treat as its own multi-session sprint. | | **Path impact** | Tags and releases scoped under `repo_id` change from `tags//` to `tags/sha256//` — rows 3 and 4 depend on this being settled first, but can use a `split_id()` helper that handles both formats during transition | | **No UUID fallback** | No legacy UUID support. The repo is initialized fresh with the new format. Existing repos must be re-initialized or their `repo.json` updated manually. | | **Docstring requirement** | New `repo_genesis_id` function docstring must explain: (1) genesis = one-time computation, (2) not re-hashed on content change, (3) content-indexed not content-addressed in the traditional sense, (4) algo in prefix enables future migration | | **New test file** | `tests/test_store_repo.py` | | **Test cases** | `test_repo_genesis_id_format`, `test_repo_genesis_id_is_deterministic`, `test_repo_genesis_id_is_prefixed`, `test_repo_id_in_path_uses_bare_hex`, `test_repo_id_validation_rejects_uuid`, `test_repo_id_validation_accepts_sha256_prefixed` | --- ## Shared Utility: `split_id()` Every row above needs to extract `(algo, hex)` from a potentially-prefixed ID. This must live in one place. | Field | Value | |-------|-------| | **Function** | `split_id(object_id: str) -> tuple[str, str]` | | **Location** | `muse/core/store.py` (or a new `muse/core/ids.py` if the module gets crowded) | | **Contract** | If `:` present: `algo, _, hex_id = object_id.partition(':')`. If absent: `algo = "sha256"`, `hex_id = object_id`. Validates hex_id is 64 lowercase hex chars. Raises `ValueError` otherwise. | | **Usage** | Called by `commit_path`, `snapshot_path`, `tag_path`, `release_path`, `pattern_dir`, `resolution_path`, `repo_genesis_id` | | **Docstring** | Must document: (1) why algo is extracted not assumed, (2) the bare-hex fallback defaults to sha256 for legacy callers only, (3) the function is the single source of truth for ID parsing | | **Test file** | `tests/test_store_ids.py` | | **Test cases** | `test_split_id_prefixed`, `test_split_id_bare_hex_defaults_sha256`, `test_split_id_rejects_empty`, `test_split_id_rejects_non_hex`, `test_split_id_rejects_wrong_length`, `test_split_id_rejects_colon_in_wrong_position` | --- ## Object Store (reference — already correct) `.muse/objects/sha256/<2-char>/<62-char>` — no changes needed. `object_store.py:object_path()` is the canonical model every other utility method should follow. Read it before implementing any new path utility. --- ## Execution Order Rows must be executed in this order due to dependencies: ``` split_id() ← prerequisite for everything ↓ commits (1) ← no dependencies snapshots (2) ← no dependencies harmony (5, 6) ← no dependencies ↓ repo_id (7) ← must land before tags/releases can finalize ↓ tags (3) ← depends on repo_id format being settled releases (4) ← depends on repo_id format being settled ``` `split_id` and rows 1, 2, 5, 6 can be parallelized. Row 7 is the blocker for rows 3 and 4. Do not start rows 3 or 4 until row 7 is merged to dev.