docs_muse_foundations.html
html
sha256:f99af7b1a7f36c4d537d1c630d4b71fc39222b1255f82e930929e2fc89015e11
fix: relax browse_repo perf budget to 500ms — 200ms was too…
Sonnet 4.6
101 days ago
| 1 | {% extends "musehub/base.html" %} |
| 2 | |
| 3 | {% block container_extra_class %} page-container{% endblock %} |
| 4 | {% block title %}Foundations — Muse Developer Docs{% endblock %} |
| 5 | {% block page_json %}{"page":"docs-foundations"}{% endblock %} |
| 6 | |
| 7 | {% block content %} |
| 8 | <div class="devdocs"> |
| 9 | <div class="devdocs-layout"> |
| 10 | |
| 11 | {# ── Sidebar nav ─────────────────────────────────────────────────────────── #} |
| 12 | <aside class="devdocs-sidebar"> |
| 13 | <nav class="devdocs-nav" aria-label="Docs navigation"> |
| 14 | <div class="devdocs-nav-group"> |
| 15 | <div class="devdocs-nav-group-label">Sections</div> |
| 16 | {% for slug, num, title, desc in phases %} |
| 17 | <a class="devdocs-nav-link devdocs-nav-link--phase{% if slug == current %} devdocs-nav-link--active{% endif %}" |
| 18 | href="/muse/{{ slug }}">{{ num }} {{ title }}</a> |
| 19 | {% endfor %} |
| 20 | </div> |
| 21 | |
| 22 | <div class="devdocs-nav-group"> |
| 23 | <div class="devdocs-nav-group-label">On this page</div> |
| 24 | <a class="devdocs-nav-link devdocs-nav-link--sub" href="#object-store">Object store</a> |
| 25 | <a class="devdocs-nav-link devdocs-nav-link--sub" href="#commit-record">CommitRecord</a> |
| 26 | <a class="devdocs-nav-link devdocs-nav-link--sub" href="#snapshot">SnapshotManifest</a> |
| 27 | <a class="devdocs-nav-link devdocs-nav-link--sub" href="#branch-dag">Branch DAG</a> |
| 28 | <a class="devdocs-nav-link devdocs-nav-link--sub" href="#round-trip">Round-trip walkthrough</a> |
| 29 | <a class="devdocs-nav-link devdocs-nav-link--sub" href="#museignore">.museignore</a> |
| 30 | <a class="devdocs-nav-link devdocs-nav-link--sub" href="#serialization">Serialization</a> |
| 31 | <a class="devdocs-nav-link devdocs-nav-link--sub" href="#on-disk">On-disk layout</a> |
| 32 | <a class="devdocs-nav-link devdocs-nav-link--sub" href="#on-disk-server">Server store</a> |
| 33 | <a class="devdocs-nav-link devdocs-nav-link--sub" href="#cli">CLI reference</a> |
| 34 | <a class="devdocs-nav-link devdocs-nav-link--sub" href="#push-pull">Push / pull</a> |
| 35 | </div> |
| 36 | </nav> |
| 37 | </aside> |
| 38 | |
| 39 | {# ── Content ─────────────────────────────────────────────────────────────── #} |
| 40 | <article class="devdocs-content"> |
| 41 | |
| 42 | <div class="devdocs-breadcrumb"> |
| 43 | <a href="/muse">Developer Docs</a> |
| 44 | <span>›</span> |
| 45 | <span>Foundations</span> |
| 46 | </div> |
| 47 | |
| 48 | <div class="devdocs-phase-header"> |
| 49 | <span class="devdocs-phase-num">PHASE 01</span> |
| 50 | <h1 class="devdocs-phase-title">Foundations</h1> |
| 51 | <p class="devdocs-phase-desc"> |
| 52 | Everything in Muse is content-addressed. Every object — blob, snapshot, |
| 53 | commit — has a deterministic ID derived from its content. There is no |
| 54 | mutable global state to coordinate: if two writers produce identical |
| 55 | content, they produce identical IDs, and only one copy is stored. |
| 56 | This section explains the three core record types, how the commit DAG |
| 57 | is built, how objects are serialized and stored on disk, and how push/pull |
| 58 | moves state between repositories. |
| 59 | </p> |
| 60 | </div> |
| 61 | |
| 62 | {# ── Object store ─────────────────────────────────────────────────────── #} |
| 63 | <section class="devdocs-section" id="object-store"> |
| 64 | <h2 class="devdocs-section-title"><a href="#object-store">Object store</a></h2> |
| 65 | |
| 66 | <p> |
| 67 | Every byte stored by Muse lives in the <strong>object store</strong> — a |
| 68 | content-addressed key-value store keyed by |
| 69 | <code>sha256:<64-hex></code>. The algorithm prefix is part of the ID |
| 70 | (not decoration), making the format self-describing and algorithm-agnostic. |
| 71 | When a new hash function is needed, old and new IDs coexist without collision. |
| 72 | </p> |
| 73 | |
| 74 | <p> |
| 75 | IDs are produced by <code>content_hash()</code> for structured data and |
| 76 | <code>blob_id()</code> for raw bytes: |
| 77 | </p> |
| 78 | |
| 79 | <div class="devdocs-code-block"> |
| 80 | <div class="devdocs-code-header"> |
| 81 | <span class="devdocs-code-lang">python</span> |
| 82 | <span class="devdocs-code-label">muse.core.types</span> |
| 83 | </div> |
| 84 | <pre><code><span class="tok-kw">def</span> <span class="tok-fn">content_hash</span>(obj: JsonValue) -> str: |
| 85 | <span class="tok-str">"""Canonical SHA-256 ID for any JSON-serializable value. |
| 86 | |
| 87 | Canonical form: json.dumps with sort_keys=True, separators=(",",":"), |
| 88 | ensure_ascii=True, UTF-8 encoded — then SHA-256. |
| 89 | Returns "sha256:<64-hex>" (always 71 chars). |
| 90 | """</span> |
| 91 | |
| 92 | <span class="tok-kw">def</span> <span class="tok-fn">blob_id</span>(data: bytes) -> str: |
| 93 | <span class="tok-str">"""SHA-256 of raw bytes. Same "sha256:<64-hex>" format."""</span></code></pre> |
| 94 | </div> |
| 95 | |
| 96 | <p> |
| 97 | The canonical JSON form — sorted keys, no whitespace — means two writers |
| 98 | constructing the same logical object always produce the same ID, regardless |
| 99 | of insertion order or formatting. This is what makes content-addressed |
| 100 | merges safe: identical content is identical, provably. |
| 101 | </p> |
| 102 | |
| 103 | <h3 class="devdocs-subsection-title"><a href="#object-types">Object types</a></h3> |
| 104 | |
| 105 | <p>Three distinct record types live in the object store:</p> |
| 106 | |
| 107 | <table class="devdocs-table"> |
| 108 | <thead><tr><th>Type</th><th>ID source</th><th>Storage path</th><th>Notes</th></tr></thead> |
| 109 | <tbody> |
| 110 | <tr> |
| 111 | <td>Blob</td> |
| 112 | <td>SHA-256 of raw bytes</td> |
| 113 | <td><code>objects/<algo>/<2-hex>/<62-hex></code></td> |
| 114 | <td>File contents; <code>blob <size>\0<bytes></code> on disk</td> |
| 115 | </tr> |
| 116 | <tr> |
| 117 | <td>Snapshot</td> |
| 118 | <td><code>content_hash(snapshot_dict)</code></td> |
| 119 | <td><code>objects/<algo>/<2-hex>/<62-hex></code></td> |
| 120 | <td>Path → blob-ID manifest; <code>snapshot <size>\0<json></code> on disk</td> |
| 121 | </tr> |
| 122 | <tr> |
| 123 | <td>Commit</td> |
| 124 | <td><code>content_hash(commit_dict)</code></td> |
| 125 | <td><code>objects/<algo>/<2-hex>/<62-hex></code></td> |
| 126 | <td>Snapshot ID + provenance + signature; <code>commit <size>\0<json></code> on disk</td> |
| 127 | </tr> |
| 128 | </tbody> |
| 129 | </table> |
| 130 | |
| 131 | <h3 class="devdocs-subsection-title"><a href="#object-graph">Object graph</a></h3> |
| 132 | <p>The three types form a strict DAG. Arrows point from referencing object to referenced object — blobs are always leaves:</p> |
| 133 | <div class="devdocs-code-block"> |
| 134 | <pre><span class="tok-cmt"> refs/heads/dev ──► CommitRecord (sha256:9e21b8...) |
| 135 | │ parent_commit_id |
| 136 | ▼ |
| 137 | CommitRecord (sha256:3f8a1c...) |
| 138 | │ snapshot_id |
| 139 | ▼ |
| 140 | SnapshotRecord (sha256:04ee5e...) |
| 141 | │ manifest |
| 142 | ┌────┼────────────┐ |
| 143 | ▼ ▼ ▼ |
| 144 | Blob Blob ... Blob |
| 145 | (sha256:a1b2c3...) (sha256:d4e5f6...)</span></pre> |
| 146 | </div> |
| 147 | |
| 148 | <div class="devdocs-callout"> |
| 149 | {{ icon("info", 16, "devdocs-callout-icon") }} |
| 150 | <div>Objects are written atomically: <code>mkstemp → write → fsync → os.replace</code>. |
| 151 | A partial write can never produce a readable but corrupt object. |
| 152 | Objects are immutable once stored — a collision on content_hash would require |
| 153 | a SHA-256 preimage attack.</div> |
| 154 | </div> |
| 155 | </section> |
| 156 | |
| 157 | {# ── CommitRecord ─────────────────────────────────────────────────────── #} |
| 158 | <section class="devdocs-section" id="commit-record"> |
| 159 | <h2 class="devdocs-section-title"><a href="#commit-record">CommitRecord</a></h2> |
| 160 | |
| 161 | <p> |
| 162 | A commit is the top-level record in the Muse DAG. It points to exactly one |
| 163 | snapshot and zero, one, or two parent commits (zero for the genesis commit, |
| 164 | two for a merge). |
| 165 | </p> |
| 166 | |
| 167 | <div class="devdocs-code-block"> |
| 168 | <div class="devdocs-code-header"> |
| 169 | <span class="devdocs-code-lang">python</span> |
| 170 | <span class="devdocs-code-label">muse.core.commits — CommitRecord</span> |
| 171 | </div> |
| 172 | <pre><code><span class="tok-kw">@dataclass</span> |
| 173 | <span class="tok-kw">class</span> <span class="tok-type">CommitRecord</span>: |
| 174 | <span class="tok-cmt"># Core identity</span> |
| 175 | commit_id: str <span class="tok-cmt"># sha256:<64-hex></span> |
| 176 | branch: str |
| 177 | snapshot_id: str <span class="tok-cmt"># sha256:<64-hex></span> |
| 178 | message: str |
| 179 | committed_at: datetime.datetime |
| 180 | |
| 181 | <span class="tok-cmt"># Graph edges</span> |
| 182 | parent_commit_id: str | None <span class="tok-cmt"># None for genesis commits</span> |
| 183 | parent2_commit_id: str | None <span class="tok-cmt"># Set for merge commits (two-parent)</span> |
| 184 | |
| 185 | <span class="tok-cmt"># Authorship</span> |
| 186 | author: str <span class="tok-cmt"># handle</span> |
| 187 | metadata: dict |
| 188 | |
| 189 | <span class="tok-cmt"># Structured delta (populated by domain plugin's diff())</span> |
| 190 | structured_delta: StructuredDelta | None |
| 191 | |
| 192 | <span class="tok-cmt"># Semantic versioning</span> |
| 193 | sem_ver_bump: Literal["none", "patch", "minor", "major"] |
| 194 | breaking_changes: list[str] |
| 195 | |
| 196 | <span class="tok-cmt"># Agent provenance — all empty strings for human commits</span> |
| 197 | agent_id: str <span class="tok-cmt"># e.g. "claude-code"</span> |
| 198 | model_id: str <span class="tok-cmt"># e.g. "claude-sonnet-4-6"</span> |
| 199 | toolchain_id: str |
| 200 | prompt_hash: str <span class="tok-cmt"># sha256 of system prompt</span> |
| 201 | |
| 202 | <span class="tok-cmt"># Ed25519 signature</span> |
| 203 | signature: str <span class="tok-cmt"># "ed25519:<base64url>"</span> |
| 204 | signer_public_key: str <span class="tok-cmt"># "ed25519:<base64url>"</span> |
| 205 | signer_key_id: str <span class="tok-cmt"># fingerprint of signing key</span> |
| 206 | |
| 207 | <span class="tok-cmt"># Labels, review metadata</span> |
| 208 | reviewed_by: list[str] |
| 209 | test_runs: int |
| 210 | labels: list[str] |
| 211 | status: str |
| 212 | notes: list[str] |
| 213 | score: float | None</code></pre> |
| 214 | </div> |
| 215 | |
| 216 | <h3 class="devdocs-subsection-title"><a href="#agent-commits">Agent commits</a></h3> |
| 217 | |
| 218 | <p>When an agent commits, it populates <code>agent_id</code>, <code>model_id</code>, |
| 219 | and signs with its derived Ed25519 key. The absence of these fields is itself |
| 220 | a signal that a human committed directly — the format encodes the distinction |
| 221 | structurally rather than by convention.</p> |
| 222 | |
| 223 | <div class="devdocs-code-block"> |
| 224 | <div class="devdocs-code-header"> |
| 225 | <span class="devdocs-code-lang">bash</span> |
| 226 | </div> |
| 227 | <pre><code><span class="tok-cmt"># Agent commit — full provenance chain</span> |
| 228 | muse commit -m <span class="tok-str">"feat: add rate limiting"</span> \ |
| 229 | --agent-id claude-code \ |
| 230 | --model-id claude-sonnet-4-6 \ |
| 231 | --sign |
| 232 | |
| 233 | <span class="tok-cmt"># Human commit — no provenance flags</span> |
| 234 | muse commit -m <span class="tok-str">"chore: update config"</span></code></pre> |
| 235 | </div> |
| 236 | |
| 237 | <div class="devdocs-code-block"> |
| 238 | <div class="devdocs-code-header"> |
| 239 | <span class="devdocs-code-lang">json</span> |
| 240 | <span class="devdocs-code-label">muse read --json</span> |
| 241 | </div> |
| 242 | <pre><code>{ |
| 243 | <span class="tok-key">"commit_id"</span>: <span class="tok-str">"sha256:6ab243df7bdb..."</span>, |
| 244 | <span class="tok-key">"branch"</span>: <span class="tok-str">"dev"</span>, |
| 245 | <span class="tok-key">"snapshot_id"</span>: <span class="tok-str">"sha256:04ee5ecd07ec..."</span>, |
| 246 | <span class="tok-key">"message"</span>: <span class="tok-str">"feat: add rate limiting"</span>, |
| 247 | <span class="tok-key">"committed_at"</span>: <span class="tok-str">"2026-04-21T23:00:00Z"</span>, |
| 248 | <span class="tok-key">"parent_commit_id"</span>: <span class="tok-str">"sha256:4c9e7959..."</span>, |
| 249 | <span class="tok-key">"parent2_commit_id"</span>: null, |
| 250 | <span class="tok-key">"author"</span>: <span class="tok-str">"gabriel"</span>, |
| 251 | <span class="tok-key">"agent_id"</span>: <span class="tok-str">"claude-code"</span>, |
| 252 | <span class="tok-key">"model_id"</span>: <span class="tok-str">"claude-sonnet-4-6"</span>, |
| 253 | <span class="tok-key">"signature"</span>: <span class="tok-str">"ed25519:AAAA..."</span>, |
| 254 | <span class="tok-key">"signer_public_key"</span>: <span class="tok-str">"ed25519:BBBB..."</span>, |
| 255 | <span class="tok-key">"sem_ver_bump"</span>: <span class="tok-str">"minor"</span>, |
| 256 | <span class="tok-key">"labels"</span>: [<span class="tok-str">"reviewed"</span>], |
| 257 | <span class="tok-key">"score"</span>: <span class="tok-num">0.94</span> |
| 258 | }</code></pre> |
| 259 | </div> |
| 260 | </section> |
| 261 | |
| 262 | {# ── SnapshotManifest ─────────────────────────────────────────────────── #} |
| 263 | <section class="devdocs-section" id="snapshot"> |
| 264 | <h2 class="devdocs-section-title"><a href="#snapshot">SnapshotManifest</a></h2> |
| 265 | |
| 266 | <p> |
| 267 | A snapshot is an immutable mapping of every tracked path to its blob ID at |
| 268 | that point in time. It is the complete working tree — not a delta, not a patch. |
| 269 | Diffing two snapshots is an O(|paths|) set operation with no chain of deltas |
| 270 | to traverse. |
| 271 | </p> |
| 272 | |
| 273 | <div class="devdocs-code-block"> |
| 274 | <div class="devdocs-code-header"> |
| 275 | <span class="devdocs-code-lang">python</span> |
| 276 | <span class="devdocs-code-label">muse.core.snapshots — SnapshotRecord</span> |
| 277 | </div> |
| 278 | <pre><code><span class="tok-kw">@dataclass</span> |
| 279 | <span class="tok-kw">class</span> <span class="tok-type">SnapshotRecord</span>: |
| 280 | snapshot_id: str <span class="tok-cmt"># sha256:<64-hex></span> |
| 281 | manifest: dict[str, str] <span class="tok-cmt"># path → "sha256:<64-hex>"</span> |
| 282 | directories: list[str] <span class="tok-cmt"># explicit empty directories</span> |
| 283 | created_at: datetime.datetime |
| 284 | note: str |
| 285 | schema_version: int</code></pre> |
| 286 | </div> |
| 287 | |
| 288 | <div class="devdocs-code-block"> |
| 289 | <div class="devdocs-code-header"> |
| 290 | <span class="devdocs-code-lang">json</span> |
| 291 | <span class="devdocs-code-label">muse read --json --manifest (excerpt)</span> |
| 292 | </div> |
| 293 | <pre><code>{ |
| 294 | <span class="tok-key">"schema_version"</span>: <span class="tok-num">1</span>, |
| 295 | <span class="tok-key">"snapshot_id"</span>: <span class="tok-str">"sha256:04ee5ecd..."</span>, |
| 296 | <span class="tok-key">"manifest"</span>: { |
| 297 | <span class="tok-key">"musehub/main.py"</span>: <span class="tok-str">"sha256:a1b2c3..."</span>, |
| 298 | <span class="tok-key">"musehub/graph/dag.py"</span>: <span class="tok-str">"sha256:d4e5f6..."</span>, |
| 299 | <span class="tok-key">"musehub/graph/push_validator.py"</span>: <span class="tok-str">"sha256:7890ab..."</span> |
| 300 | }, |
| 301 | <span class="tok-key">"directories"</span>: [], |
| 302 | <span class="tok-key">"created_at"</span>: <span class="tok-str">"2026-04-21T23:00:00Z"</span>, |
| 303 | <span class="tok-key">"note"</span>: <span class="tok-str">""</span> |
| 304 | }</code></pre> |
| 305 | </div> |
| 306 | |
| 307 | <p> |
| 308 | Because a snapshot is just a flat map, any two snapshots can be diffed |
| 309 | instantly: paths present in both with the same blob ID are unchanged; paths |
| 310 | with different IDs are modified; paths only in one are added or removed. |
| 311 | Domain plugins receive this pair as their <code>base</code> and <code>target</code> |
| 312 | in <code>diff()</code>, then interpret the semantic meaning. |
| 313 | </p> |
| 314 | |
| 315 | <div class="devdocs-callout"> |
| 316 | {{ icon("info", 16, "devdocs-callout-icon") }} |
| 317 | <div> |
| 318 | <code>muse read --json</code> returns commit metadata and file-level changes |
| 319 | (<code>files_added</code>, <code>files_modified</code>, <code>files_removed</code>) |
| 320 | but not the full manifest. Add <code>--manifest</code> to get the complete |
| 321 | path → object_id map for every tracked file. |
| 322 | </div> |
| 323 | </div> |
| 324 | </section> |
| 325 | |
| 326 | {# ── Branch DAG ───────────────────────────────────────────────────────── #} |
| 327 | <section class="devdocs-section" id="branch-dag"> |
| 328 | <h2 class="devdocs-section-title"><a href="#branch-dag">Branch DAG</a></h2> |
| 329 | |
| 330 | <p> |
| 331 | Branches are mutable named pointers to commit IDs, stored in |
| 332 | <code>.muse/refs/heads/<branch></code>. The commit graph is an immutable DAG; |
| 333 | the branch pointer advances atomically when you commit or merge. |
| 334 | </p> |
| 335 | |
| 336 | <p> |
| 337 | Merge commits have two parents: <code>parent_commit_id</code> (the branch |
| 338 | being merged into) and <code>parent2_commit_id</code> (the branch being merged |
| 339 | from). Three-way merge is computed from the common ancestor found by walking |
| 340 | both chains backward to their lowest common ancestor. |
| 341 | </p> |
| 342 | |
| 343 | <div class="devdocs-code-block"> |
| 344 | <div class="devdocs-code-header"> |
| 345 | <span class="devdocs-code-lang">bash</span> |
| 346 | <span class="devdocs-code-label">branch lifecycle</span> |
| 347 | </div> |
| 348 | <pre><code><span class="tok-cmt"># Create and switch in one command, with intent metadata</span> |
| 349 | muse checkout -b task/rate-limiting \ |
| 350 | --intent <span class="tok-str">"implement token bucket rate limiter"</span> \ |
| 351 | --resumable |
| 352 | |
| 353 | <span class="tok-cmt"># Work, stage, commit</span> |
| 354 | muse code add src/rate_limiter.py |
| 355 | muse commit -m <span class="tok-str">"feat: token bucket rate limiter"</span> \ |
| 356 | --agent-id claude-code --model-id claude-sonnet-4-6 --sign |
| 357 | |
| 358 | <span class="tok-cmt"># Merge back</span> |
| 359 | muse checkout dev |
| 360 | muse merge task/rate-limiting <span class="tok-cmt"># three-way, harmony auto-resolves known conflicts</span> |
| 361 | muse branch -d task/rate-limiting |
| 362 | muse push local dev</code></pre> |
| 363 | </div> |
| 364 | |
| 365 | <h3 class="devdocs-subsection-title"><a href="#branch-flow">Branch flow</a></h3> |
| 366 | <table class="devdocs-table"> |
| 367 | <thead><tr><th>Branch</th><th>Role</th><th>Rule</th></tr></thead> |
| 368 | <tbody> |
| 369 | <tr><td>main</td><td>Production</td><td>Tagged releases only; never direct-pushed</td></tr> |
| 370 | <tr><td>dev</td><td>Integration</td><td>Latest deliverable state</td></tr> |
| 371 | <tr><td>feat/*</td><td>Feature</td><td>Short-lived; one atomic task; hours not days</td></tr> |
| 372 | <tr><td>task/*</td><td>Agent task</td><td>Same as feat/*; carries --intent and --resumable</td></tr> |
| 373 | <tr><td>bugfix/*</td><td>Bug fix</td><td>From dev; merges into dev</td></tr> |
| 374 | <tr><td>hotfix/*</td><td>Hot fix</td><td>From main; merges into main AND dev</td></tr> |
| 375 | </tbody> |
| 376 | </table> |
| 377 | |
| 378 | <h3 class="devdocs-subsection-title"><a href="#resumable-branches">Resumable branches</a></h3> |
| 379 | <p> |
| 380 | Branches carry <code>--intent</code> (a free-text description of the task) and |
| 381 | <code>--resumable</code> (a boolean signal that another agent may safely pick this |
| 382 | up mid-flight). Both are stored in branch metadata, readable via |
| 383 | <code>muse branch --json</code>, and surfaced in the coordination bus so |
| 384 | orchestrators can assign in-progress work to idle agents. |
| 385 | </p> |
| 386 | |
| 387 | <div class="devdocs-code-block"> |
| 388 | <div class="devdocs-code-header"> |
| 389 | <span class="devdocs-code-lang">json</span> |
| 390 | <span class="devdocs-code-label">muse branch --json (excerpt)</span> |
| 391 | </div> |
| 392 | <pre><code>[ |
| 393 | { |
| 394 | <span class="tok-key">"name"</span>: <span class="tok-str">"task/rate-limiting"</span>, |
| 395 | <span class="tok-key">"current"</span>: <span class="tok-num">false</span>, |
| 396 | <span class="tok-key">"intent"</span>: <span class="tok-str">"implement token bucket rate limiter"</span>, |
| 397 | <span class="tok-key">"resumable"</span>: <span class="tok-num">true</span>, |
| 398 | <span class="tok-key">"created_by"</span>: <span class="tok-str">"claude-code"</span>, |
| 399 | <span class="tok-key">"commit_id"</span>: <span class="tok-str">"sha256:6ab243..."</span> |
| 400 | } |
| 401 | ]</code></pre> |
| 402 | </div> |
| 403 | </section> |
| 404 | |
| 405 | {# ── Round-trip walkthrough ───────────────────────────────────────────── #} |
| 406 | <section class="devdocs-section" id="round-trip"> |
| 407 | <h2 class="devdocs-section-title"><a href="#round-trip">Round-trip walkthrough</a></h2> |
| 408 | |
| 409 | <p> |
| 410 | This walkthrough traces a single file change from working tree all the way |
| 411 | to a remote and back, showing the <code>sha256:</code> IDs at each step so |
| 412 | you can see how objects, snapshots, commits, and branch refs compose. |
| 413 | </p> |
| 414 | |
| 415 | <h3 class="devdocs-subsection-title"><a href="#round-trip-stage">Step 1 — stage</a></h3> |
| 416 | <p> |
| 417 | <code>muse code add</code> hashes each file into the object store and records |
| 418 | the path → blob-ID mapping in the staging area. |
| 419 | </p> |
| 420 | |
| 421 | <div class="devdocs-code-block"> |
| 422 | <div class="devdocs-code-header"> |
| 423 | <span class="devdocs-code-lang">bash</span> |
| 424 | </div> |
| 425 | <pre><code>muse code add src/rate_limiter.py</code></pre> |
| 426 | </div> |
| 427 | <div class="devdocs-code-block devdocs-code-block--output"> |
| 428 | <pre><span class="tok-str">staged</span> src/rate_limiter.py <span class="tok-cmt">sha256:f3a7b92c4de1…</span></pre> |
| 429 | </div> |
| 430 | |
| 431 | <p> |
| 432 | The blob <code>sha256:f3a7b92c4de1…</code> is written to |
| 433 | <code>.muse/objects/sha256/f3/a7b92c4de1…</code> immediately — it |
| 434 | exists in the object store whether or not you ever commit. |
| 435 | </p> |
| 436 | |
| 437 | <h3 class="devdocs-subsection-title"><a href="#round-trip-commit">Step 2 — commit</a></h3> |
| 438 | <p> |
| 439 | <code>muse commit</code> builds a SnapshotRecord from the staging area, hashes |
| 440 | it to get a snapshot ID, then builds a CommitRecord referencing that snapshot |
| 441 | and the current branch tip as parent. |
| 442 | </p> |
| 443 | |
| 444 | <div class="devdocs-code-block"> |
| 445 | <div class="devdocs-code-header"> |
| 446 | <span class="devdocs-code-lang">bash</span> |
| 447 | </div> |
| 448 | <pre><code>muse commit -m <span class="tok-str">"feat: token bucket rate limiter"</span> \ |
| 449 | --agent-id claude-code --model-id claude-sonnet-4-6 --sign</code></pre> |
| 450 | </div> |
| 451 | <div class="devdocs-code-block devdocs-code-block--output"> |
| 452 | <pre><span class="tok-str">committed</span> sha256:9e21b8a4f273… |
| 453 | snapshot sha256:c8d5e1f09ab3… |
| 454 | parent sha256:4c9e7959beef… |
| 455 | branch task/rate-limiting |
| 456 | signed ed25519:AAAA…</pre> |
| 457 | </div> |
| 458 | |
| 459 | <p> |
| 460 | Three new objects hit disk: |
| 461 | </p> |
| 462 | <table class="devdocs-table"> |
| 463 | <thead><tr><th>ID</th><th>Type</th><th>Contents</th></tr></thead> |
| 464 | <tbody> |
| 465 | <tr> |
| 466 | <td><code>sha256:f3a7b92c…</code></td> |
| 467 | <td>Blob</td> |
| 468 | <td>Raw bytes of <code>src/rate_limiter.py</code></td> |
| 469 | </tr> |
| 470 | <tr> |
| 471 | <td><code>sha256:c8d5e1f0…</code></td> |
| 472 | <td>Snapshot</td> |
| 473 | <td>Full path → blob-ID map for the entire working tree</td> |
| 474 | </tr> |
| 475 | <tr> |
| 476 | <td><code>sha256:9e21b8a4…</code></td> |
| 477 | <td>Commit</td> |
| 478 | <td>Message, author, agent provenance, snapshot_id, parent_commit_id, signature</td> |
| 479 | </tr> |
| 480 | </tbody> |
| 481 | </table> |
| 482 | <p> |
| 483 | The branch ref <code>.muse/refs/heads/task/rate-limiting</code> is updated |
| 484 | atomically to <code>sha256:9e21b8a4f273…</code>. |
| 485 | </p> |
| 486 | |
| 487 | <h3 class="devdocs-subsection-title"><a href="#round-trip-push">Step 3 — push</a></h3> |
| 488 | <p> |
| 489 | <code>muse push</code> computes the set of objects the remote does not have, |
| 490 | packs them into an MPack, and POSTs to the hub. |
| 491 | </p> |
| 492 | |
| 493 | <div class="devdocs-code-block"> |
| 494 | <div class="devdocs-code-header"> |
| 495 | <span class="devdocs-code-lang">bash</span> |
| 496 | </div> |
| 497 | <pre><code>muse push local task/rate-limiting</code></pre> |
| 498 | </div> |
| 499 | <div class="devdocs-code-block devdocs-code-block--output"> |
| 500 | <pre>Pushing task/rate-limiting → local |
| 501 | <span class="tok-num">3</span> objects (<span class="tok-str">1</span> blob, <span class="tok-str">1</span> snapshot, <span class="tok-str">1</span> commit) |
| 502 | <span class="tok-str">✓</span> sha256:f3a7b92c… <span class="tok-cmt">blob 4.2 kB</span> |
| 503 | <span class="tok-str">✓</span> sha256:c8d5e1f0… <span class="tok-cmt">snapshot 1.1 kB</span> |
| 504 | <span class="tok-str">✓</span> sha256:9e21b8a4… <span class="tok-cmt">commit 0.8 kB</span> |
| 505 | branch task/rate-limiting → sha256:9e21b8a4… |
| 506 | <span class="tok-str">✔</span> pushed in 142 ms</pre> |
| 507 | </div> |
| 508 | |
| 509 | <p> |
| 510 | The hub stores all three objects, verifies the Ed25519 signature, and advances |
| 511 | the branch pointer in a single transaction. |
| 512 | </p> |
| 513 | |
| 514 | <h3 class="devdocs-subsection-title"><a href="#round-trip-pull">Step 4 — pull</a></h3> |
| 515 | <p> |
| 516 | A second agent (or a second machine) pulls the branch. Muse fetches only the |
| 517 | objects it doesn't already have — content-addressability makes deduplication |
| 518 | trivial. |
| 519 | </p> |
| 520 | |
| 521 | <div class="devdocs-code-block"> |
| 522 | <div class="devdocs-code-header"> |
| 523 | <span class="devdocs-code-lang">bash</span> |
| 524 | </div> |
| 525 | <pre><code>muse pull local task/rate-limiting</code></pre> |
| 526 | </div> |
| 527 | <div class="devdocs-code-block devdocs-code-block--output"> |
| 528 | <pre>Fetching task/rate-limiting from local |
| 529 | <span class="tok-num">3</span> new objects |
| 530 | <span class="tok-str">✓</span> sha256:f3a7b92c… <span class="tok-cmt">blob</span> |
| 531 | <span class="tok-str">✓</span> sha256:c8d5e1f0… <span class="tok-cmt">snapshot</span> |
| 532 | <span class="tok-str">✓</span> sha256:9e21b8a4… <span class="tok-cmt">commit</span> |
| 533 | branch task/rate-limiting → sha256:9e21b8a4… |
| 534 | <span class="tok-str">✔</span> fast-forward, working tree updated</pre> |
| 535 | </div> |
| 536 | |
| 537 | <p> |
| 538 | Because every ID is derived from content, the pull output IDs are byte-for-byte |
| 539 | identical to the push output. There is no translation, no rebase, no rewriting — |
| 540 | the same objects are present on both sides. |
| 541 | </p> |
| 542 | |
| 543 | <h3 class="devdocs-subsection-title"><a href="#round-trip-log">Inspecting the result</a></h3> |
| 544 | |
| 545 | <div class="devdocs-code-block"> |
| 546 | <div class="devdocs-code-header"> |
| 547 | <span class="devdocs-code-lang">bash</span> |
| 548 | </div> |
| 549 | <pre><code>muse log --json</code></pre> |
| 550 | </div> |
| 551 | <div class="devdocs-code-block devdocs-code-block--output"> |
| 552 | <pre>{ |
| 553 | <span class="tok-key">"truncated"</span>: <span class="tok-num">false</span>, |
| 554 | <span class="tok-key">"commits"</span>: [ |
| 555 | { |
| 556 | <span class="tok-key">"commit_id"</span>: <span class="tok-str">"sha256:9e21b8a4f273…"</span>, |
| 557 | <span class="tok-key">"message"</span>: <span class="tok-str">"feat: token bucket rate limiter"</span>, |
| 558 | <span class="tok-key">"committed_at"</span>: <span class="tok-str">"2026-04-30T18:22:04Z"</span>, |
| 559 | <span class="tok-key">"author"</span>: <span class="tok-str">"gabriel"</span>, |
| 560 | <span class="tok-key">"agent_id"</span>: <span class="tok-str">"claude-code"</span>, |
| 561 | <span class="tok-key">"model_id"</span>: <span class="tok-str">"claude-sonnet-4-6"</span>, |
| 562 | <span class="tok-key">"parent_commit_id"</span>:<span class="tok-str">"sha256:4c9e7959…"</span>, |
| 563 | <span class="tok-key">"snapshot_id"</span>: <span class="tok-str">"sha256:c8d5e1f0…"</span> |
| 564 | }, |
| 565 | <span class="tok-cmt">…earlier commits…</span> |
| 566 | ] |
| 567 | }</pre> |
| 568 | </div> |
| 569 | |
| 570 | <div class="devdocs-code-block"> |
| 571 | <div class="devdocs-code-header"> |
| 572 | <span class="devdocs-code-lang">bash</span> |
| 573 | </div> |
| 574 | <pre><code>muse read --json --manifest</code></pre> |
| 575 | </div> |
| 576 | <div class="devdocs-code-block devdocs-code-block--output"> |
| 577 | <pre>{ |
| 578 | <span class="tok-key">"commit_id"</span>: <span class="tok-str">"sha256:9e21b8a4f273…"</span>, |
| 579 | <span class="tok-key">"snapshot_id"</span>: <span class="tok-str">"sha256:c8d5e1f09ab3…"</span>, |
| 580 | <span class="tok-key">"files_added"</span>: [<span class="tok-str">"src/rate_limiter.py"</span>], |
| 581 | <span class="tok-key">"files_modified"</span>:[], |
| 582 | <span class="tok-key">"files_removed"</span>: [], |
| 583 | <span class="tok-key">"manifest"</span>: { |
| 584 | <span class="tok-key">"src/rate_limiter.py"</span>: <span class="tok-str">"sha256:f3a7b92c4de1…"</span>, |
| 585 | <span class="tok-key">"src/main.py"</span>: <span class="tok-str">"sha256:a1b2c3d4e5f6…"</span>, |
| 586 | <span class="tok-key">"src/config.py"</span>: <span class="tok-str">"sha256:789abc012def…"</span> |
| 587 | } |
| 588 | }</pre> |
| 589 | </div> |
| 590 | |
| 591 | <div class="devdocs-code-block"> |
| 592 | <div class="devdocs-code-header"> |
| 593 | <span class="devdocs-code-lang">bash</span> |
| 594 | </div> |
| 595 | <pre><code>muse diff --staged</code></pre> |
| 596 | </div> |
| 597 | <div class="devdocs-code-block devdocs-code-block--output"> |
| 598 | <pre><span class="tok-str">+++ src/rate_limiter.py</span> <span class="tok-cmt">(new file, sha256:f3a7b92c…)</span> |
| 599 | <span class="tok-str">+</span> class TokenBucket: |
| 600 | <span class="tok-str">+</span> def __init__(self, rate: float, burst: int) -> None: |
| 601 | <span class="tok-str">+</span> self.rate = rate |
| 602 | <span class="tok-str">+</span> self.burst = burst |
| 603 | <span class="tok-str">+</span> self._tokens = burst |
| 604 | <span class="tok-str">+</span> self._last = time.monotonic() |
| 605 | <span class="tok-str">+</span> |
| 606 | <span class="tok-str">+</span> def consume(self, n: int = 1) -> bool: |
| 607 | <span class="tok-str">+</span> …</pre> |
| 608 | </div> |
| 609 | </section> |
| 610 | |
| 611 | {# ── .museignore ──────────────────────────────────────────────────────────── #} |
| 612 | <section class="devdocs-section" id="museignore"> |
| 613 | <h2 class="devdocs-section-title"><a href="#museignore">.museignore</a></h2> |
| 614 | |
| 615 | <p> |
| 616 | <code>.museignore</code> is a <strong>TOML</strong> file that tells Muse which |
| 617 | files to exclude from tracking. It has three section types: |
| 618 | </p> |
| 619 | |
| 620 | <table class="devdocs-table"> |
| 621 | <thead><tr><th>Section</th><th>Applied when</th></tr></thead> |
| 622 | <tbody> |
| 623 | <tr><td><code>[global]</code></td><td>All domains, always</td></tr> |
| 624 | <tr><td><code>[domain.<name>]</code></td><td>Only when the active domain is <code><name></code></td></tr> |
| 625 | <tr><td><code>[force_track]</code></td><td>Whitelist — exact paths that bypass all ignore rules</td></tr> |
| 626 | </tbody> |
| 627 | </table> |
| 628 | |
| 629 | <div class="devdocs-code-block"> |
| 630 | <div class="devdocs-code-header"> |
| 631 | <span class="devdocs-code-lang">toml</span> |
| 632 | <span class="devdocs-code-label">.museignore</span> |
| 633 | </div> |
| 634 | <pre><code>[global] |
| 635 | patterns = [ |
| 636 | ".DS_Store", |
| 637 | "Thumbs.db", |
| 638 | "*.tmp", |
| 639 | "*.swp", |
| 640 | ] |
| 641 | |
| 642 | [domain.code] |
| 643 | patterns = [ |
| 644 | "__pycache__/", |
| 645 | "*.pyc", |
| 646 | ".venv/", |
| 647 | "dist/", |
| 648 | "build/", |
| 649 | "*.egg-info/", |
| 650 | "node_modules/", |
| 651 | ] |
| 652 | |
| 653 | # [force_track] |
| 654 | # Exact repo-relative paths to track even if they match a secrets pattern. |
| 655 | # paths = [ |
| 656 | # "deploy/local-tls/localhost.key", |
| 657 | # ]</code></pre> |
| 658 | </div> |
| 659 | |
| 660 | <h3 class="devdocs-subsection-title"><a href="#museignore-patterns">Pattern syntax</a></h3> |
| 661 | |
| 662 | <table class="devdocs-table"> |
| 663 | <thead><tr><th>Pattern</th><th>Matches</th></tr></thead> |
| 664 | <tbody> |
| 665 | <tr><td><code>*.pyc</code></td><td>Any <code>.pyc</code> file at any depth</td></tr> |
| 666 | <tr><td><code>__pycache__/</code></td><td>Any directory named <code>__pycache__</code> (trailing <code>/</code> = directory)</td></tr> |
| 667 | <tr><td><code>/dist/</code></td><td>Only <code>dist/</code> at the repo root (leading <code>/</code> = anchored)</td></tr> |
| 668 | <tr><td><code>!important.tmp</code></td><td>Un-ignore a previously matched path (leading <code>!</code> = negate)</td></tr> |
| 669 | <tr><td><code>src/*.min.js</code></td><td>Minified JS files directly inside <code>src/</code> (<code>*</code> excludes <code>/</code>)</td></tr> |
| 670 | <tr><td><code>tests/fixtures/**</code></td><td>All contents of <code>tests/fixtures/</code> recursively (<code>**</code> includes <code>/</code>)</td></tr> |
| 671 | </tbody> |
| 672 | </table> |
| 673 | |
| 674 | <p> |
| 675 | Patterns are evaluated in order — global first, then domain-specific. The |
| 676 | <strong>last matching rule wins</strong>, mirroring gitignore semantics. |
| 677 | </p> |
| 678 | |
| 679 | <h3 class="devdocs-subsection-title"><a href="#museignore-force-track">[force_track] — override the secrets blocklist</a></h3> |
| 680 | |
| 681 | <p> |
| 682 | Muse automatically blocks certain file types from tracking (e.g. <code>*.key</code>, |
| 683 | <code>*.pem</code>, <code>.env</code>). The <code>[force_track]</code> section |
| 684 | lists <strong>exact repo-relative paths</strong> (no globs) that must be tracked |
| 685 | regardless. Use it for dev infrastructure that would otherwise be blocked. |
| 686 | </p> |
| 687 | |
| 688 | <div class="devdocs-code-block"> |
| 689 | <div class="devdocs-code-header"> |
| 690 | <span class="devdocs-code-lang">toml</span> |
| 691 | <span class="devdocs-code-label">.museignore — force_track</span> |
| 692 | </div> |
| 693 | <pre><code>[force_track] |
| 694 | paths = [ |
| 695 | "deploy/local-tls/localhost.key", |
| 696 | "deploy/local-tls/localhost.crt", |
| 697 | ]</code></pre> |
| 698 | </div> |
| 699 | |
| 700 | <div class="devdocs-callout"> |
| 701 | {{ icon("info", 16, "devdocs-callout-icon") }} |
| 702 | <div> |
| 703 | <code>muse check-ignore <path> --json</code> tells you whether a given path |
| 704 | is ignored and which rule matched. Use it when <code>muse status</code> shows |
| 705 | a file as untracked and you want to understand why. |
| 706 | </div> |
| 707 | </div> |
| 708 | </section> |
| 709 | |
| 710 | {# ── Serialization ─────────────────────────────────────────────────────── #} |
| 711 | <section class="devdocs-section" id="serialization"> |
| 712 | <h2 class="devdocs-section-title"><a href="#serialization">Serialization</a></h2> |
| 713 | |
| 714 | <p> |
| 715 | Commits and snapshots are serialized with <strong>msgpack</strong>, not JSON. |
| 716 | On real-world code repositories, msgpack is 3–6× faster to encode and decode, |
| 717 | and produces smaller files. The coordination bus uses JSON (infrequent, small |
| 718 | payloads) and the MCP wire uses JSON for tool calls, but the core object store |
| 719 | is msgpack throughout. |
| 720 | </p> |
| 721 | |
| 722 | <table class="devdocs-table"> |
| 723 | <thead><tr><th>Data</th><th>Format</th><th>Reason</th></tr></thead> |
| 724 | <tbody> |
| 725 | <tr><td>Commits / snapshots</td><td>msgpack</td><td>3–6× faster; binary-safe for blob content</td></tr> |
| 726 | <tr><td>Objects (blobs)</td><td>raw bytes</td><td>No encoding overhead</td></tr> |
| 727 | <tr><td>Coordination records</td><td>JSON</td><td>Infrequent; human-readable debugging</td></tr> |
| 728 | <tr><td>Harmony patterns</td><td>JSON</td><td>Infrequent; inspectable</td></tr> |
| 729 | <tr><td>Wire push (HTTP)</td><td>msgpack</td><td><code>application/x-msgpack</code></td></tr> |
| 730 | <tr><td>MCP tool calls</td><td>JSON</td><td>MCP protocol requirement</td></tr> |
| 731 | </tbody> |
| 732 | </table> |
| 733 | |
| 734 | <h3 class="devdocs-subsection-title"><a href="#msgpack-example">msgpack CommitRecord — decoded</a></h3> |
| 735 | <p> |
| 736 | Commits are stored as msgpack binary files. The JSON below is the decoded |
| 737 | equivalent — every field in the msgpack maps 1:1 to the CommitRecord dataclass. |
| 738 | The <code>sha256:</code> prefix is stored as a plain string; it is never |
| 739 | stripped. |
| 740 | </p> |
| 741 | |
| 742 | <div class="devdocs-code-block"> |
| 743 | <div class="devdocs-code-header"> |
| 744 | <span class="devdocs-code-lang">bash</span> |
| 745 | </div> |
| 746 | <pre><code>python3 -c <span class="tok-str">"import msgpack,json,sys; d=msgpack.unpackb(open(sys.argv[1],'rb').read(),raw=False); print(json.dumps(d,indent=2))"</span> \ |
| 747 | .muse/objects/sha256/9e/21b8a4f273…</code></pre> |
| 748 | </div> |
| 749 | <div class="devdocs-code-block devdocs-code-block--output"> |
| 750 | <pre>{ |
| 751 | <span class="tok-key">"commit_id"</span>: <span class="tok-str">"sha256:9e21b8a4f273…"</span>, |
| 752 | <span class="tok-key">"repo_id"</span>: <span class="tok-str">"sha256:0000genesis…"</span>, |
| 753 | <span class="tok-key">"branch"</span>: <span class="tok-str">"task/rate-limiting"</span>, |
| 754 | <span class="tok-key">"snapshot_id"</span>: <span class="tok-str">"sha256:c8d5e1f09ab3…"</span>, |
| 755 | <span class="tok-key">"message"</span>: <span class="tok-str">"feat: token bucket rate limiter"</span>, |
| 756 | <span class="tok-key">"committed_at"</span>: <span class="tok-str">"2026-04-30T18:22:04Z"</span>, |
| 757 | <span class="tok-key">"parent_commit_id"</span>: <span class="tok-str">"sha256:4c9e7959…"</span>, |
| 758 | <span class="tok-key">"parent2_commit_id"</span>: null, |
| 759 | <span class="tok-key">"author"</span>: <span class="tok-str">"gabriel"</span>, |
| 760 | <span class="tok-key">"agent_id"</span>: <span class="tok-str">"claude-code"</span>, |
| 761 | <span class="tok-key">"model_id"</span>: <span class="tok-str">"claude-sonnet-4-6"</span>, |
| 762 | <span class="tok-key">"toolchain_id"</span>: <span class="tok-str">""</span>, |
| 763 | <span class="tok-key">"prompt_hash"</span>: <span class="tok-str">""</span>, |
| 764 | <span class="tok-key">"signature"</span>: <span class="tok-str">"ed25519:AAAA…"</span>, |
| 765 | <span class="tok-key">"signer_public_key"</span>: <span class="tok-str">"ed25519:BBBB…"</span>, |
| 766 | <span class="tok-key">"signer_key_id"</span>: <span class="tok-str">"sha256:CCCC…"</span>, |
| 767 | <span class="tok-key">"sem_ver_bump"</span>: <span class="tok-str">"minor"</span>, |
| 768 | <span class="tok-key">"breaking_changes"</span>: [], |
| 769 | <span class="tok-key">"structured_delta"</span>: null, |
| 770 | <span class="tok-key">"reviewed_by"</span>: [], |
| 771 | <span class="tok-key">"test_runs"</span>: <span class="tok-num">0</span>, |
| 772 | <span class="tok-key">"labels"</span>: [], |
| 773 | <span class="tok-key">"status"</span>: <span class="tok-str">""</span>, |
| 774 | <span class="tok-key">"notes"</span>: [], |
| 775 | <span class="tok-key">"score"</span>: null, |
| 776 | <span class="tok-key">"format_version"</span>: <span class="tok-num">8</span> |
| 777 | }</pre> |
| 778 | </div> |
| 779 | |
| 780 | <h3 class="devdocs-subsection-title"><a href="#limits">Size limits</a></h3> |
| 781 | <table class="devdocs-table"> |
| 782 | <thead><tr><th>Limit</th><th>Value</th><th>Notes</th></tr></thead> |
| 783 | <tbody> |
| 784 | <tr><td>Max commits per push</td><td>10,000</td><td>Rejected at wire layer</td></tr> |
| 785 | <tr><td>Max objects per push</td><td>1,000</td><td>Larger batches use presigned URLs</td></tr> |
| 786 | <tr><td>Max object size (inline)</td><td>38 MB</td><td>Above this: presigned upload</td></tr> |
| 787 | <tr><td>Max msgpack file</td><td>64 MiB</td><td>Per commit or snapshot file</td></tr> |
| 788 | <tr><td>Max blob in mpack</td><td>256 MiB</td><td>MPack format limit</td></tr> |
| 789 | <tr><td>Max string (msgpack)</td><td>1 MiB</td><td>Any single string value</td></tr> |
| 790 | <tr><td>Max collection entries</td><td>1M</td><td>Array or map</td></tr> |
| 791 | </tbody> |
| 792 | </table> |
| 793 | </section> |
| 794 | |
| 795 | {# ── On-disk layout ────────────────────────────────────────────────────── #} |
| 796 | <section class="devdocs-section" id="on-disk"> |
| 797 | <h2 class="devdocs-section-title"><a href="#on-disk">On-disk layout</a></h2> |
| 798 | |
| 799 | <p> |
| 800 | Every Muse repo is a directory containing a <code>.muse/</code> subdirectory. |
| 801 | There is no index file, no packed-refs, no reflog by default — just the |
| 802 | flat object store and a handful of ref files. |
| 803 | </p> |
| 804 | |
| 805 | <div class="devdocs-code-block"> |
| 806 | <div class="devdocs-code-header"> |
| 807 | <span class="devdocs-code-lang">text</span> |
| 808 | <span class="devdocs-code-label">.muse/ directory tree</span> |
| 809 | </div> |
| 810 | <pre><code>.muse/ |
| 811 | ├── repo.json <span class="tok-cmt"># repo_id, domain, owner, created_at</span> |
| 812 | ├── HEAD <span class="tok-cmt"># "refs/heads/dev" (symbolic ref)</span> |
| 813 | ├── refs/ |
| 814 | │ └── heads/ |
| 815 | │ ├── main <span class="tok-cmt"># "sha256:<64-hex>\n"</span> |
| 816 | │ └── dev <span class="tok-cmt"># "sha256:<64-hex>\n"</span> |
| 817 | ├── objects/ |
| 818 | │ └── sha256/ |
| 819 | │ └── ab/ <span class="tok-cmt"># first 2 hex chars (sharding)</span> |
| 820 | │ └── <62-hex> <span class="tok-cmt"># commits, snapshots, and blobs — unified store</span> |
| 821 | ├── coordination/ <span class="tok-cmt"># multi-agent symbol reservations</span> |
| 822 | │ ├── reservations/ |
| 823 | │ ├── intents/ |
| 824 | │ ├── releases/ |
| 825 | │ └── heartbeats/ |
| 826 | ├── harmony/ <span class="tok-cmt"># conflict resolution memory</span> |
| 827 | │ ├── patterns/ |
| 828 | │ ├── policies/ |
| 829 | │ └── audit/ |
| 830 | └── agent.md <span class="tok-cmt"># repo-specific agent rules</span></code></pre> |
| 831 | </div> |
| 832 | |
| 833 | <p> |
| 834 | The two-level sharding on objects (<code>sha256/ab/<62-hex></code>) keeps |
| 835 | directory sizes bounded: at one million objects, each shard directory |
| 836 | holds ~3,900 files on average — well within filesystem limits on all |
| 837 | major platforms. |
| 838 | </p> |
| 839 | |
| 840 | <h3 class="devdocs-subsection-title"><a href="#on-disk-server">MuseHub server store</a></h3> |
| 841 | |
| 842 | <p> |
| 843 | The MuseHub server uses the <strong>same on-disk format</strong> per repo under |
| 844 | <code>/data/repos/</code>. Object IDs, shard directories, and ref files are |
| 845 | byte-for-byte compatible with the client store — a blob stored by a push and |
| 846 | a blob stored locally are indistinguishable at the byte level. |
| 847 | The database holds only metadata caches and the collaboration layer; |
| 848 | authoritative repo state is always on disk. |
| 849 | </p> |
| 850 | |
| 851 | <div class="devdocs-code-block"> |
| 852 | <div class="devdocs-code-header"> |
| 853 | <span class="devdocs-code-lang">text</span> |
| 854 | <span class="devdocs-code-label">Server-side per-repo tree</span> |
| 855 | </div> |
| 856 | <pre><code>/data/repos/<owner>/<slug>/ |
| 857 | ├── objects/ |
| 858 | │ └── sha256/ <span class="tok-cmt"># algorithm namespace (mldsa65/ slots in here)</span> |
| 859 | │ └── ab/ <span class="tok-cmt"># 2-char hex shard</span> |
| 860 | │ └── <62-hex> <span class="tok-cmt"># raw blob — same layout as .muse/objects/</span> |
| 861 | └── refs/ |
| 862 | └── heads/ |
| 863 | ├── main <span class="tok-cmt"># "sha256:<64-hex>" — same format as client</span> |
| 864 | └── dev</code></pre> |
| 865 | </div> |
| 866 | |
| 867 | <table class="devdocs-table"> |
| 868 | <thead><tr><th>DB table</th><th>Role</th></tr></thead> |
| 869 | <tbody> |
| 870 | <tr><td><code>musehub_commits</code></td><td>Cache — fast graph queries, search, API listing</td></tr> |
| 871 | <tr><td><code>musehub_snapshots</code></td><td>Cache — fast manifest lookups</td></tr> |
| 872 | <tr><td><code>musehub_branches</code></td><td>Cache — fast branch listing; disk ref is authoritative</td></tr> |
| 873 | <tr><td><code>musehub_repos</code></td><td>Canonical — repo metadata, visibility, owner</td></tr> |
| 874 | <tr><td><code>musehub_identities</code> / <code>musehub_auth_keys</code></td><td>Canonical — identity and auth</td></tr> |
| 875 | <tr><td><code>musehub_issues</code> / <code>musehub_proposals</code></td><td>Canonical — collaboration layer</td></tr> |
| 876 | <tr><td><code>musehub_objects</code></td><td>Canonical — <code>storage_uri</code> + <code>size_bytes</code> index for fetch path resolution</td></tr> |
| 877 | </tbody> |
| 878 | </table> |
| 879 | |
| 880 | <div class="devdocs-callout"> |
| 881 | {{ icon("info", 16, "devdocs-callout-icon") }} |
| 882 | <div> |
| 883 | Push negotiation checks object existence directly on disk — never in the DB. |
| 884 | This means force-resign, migration, or partial push failures cannot corrupt |
| 885 | the have/want walk. If the DB cache drifts from disk, |
| 886 | <code>GET /repos/{owner}/{repo}/branches/{name}/repair</code> heals it. |
| 887 | </div> |
| 888 | </div> |
| 889 | </section> |
| 890 | |
| 891 | {# ── CLI reference ────────────────────────────────────────────────────── #} |
| 892 | <section class="devdocs-section" id="cli"> |
| 893 | <h2 class="devdocs-section-title"><a href="#cli">CLI reference</a></h2> |
| 894 | |
| 895 | <p> |
| 896 | Every command accepts <code>--json</code>. The <code>--json</code> output is the |
| 897 | stable machine contract; the default terminal output is for humans and is not |
| 898 | versioned. Use <code>muse -C ~/path/to/repo <cmd></code> when your working |
| 899 | directory differs from the target repo. |
| 900 | </p> |
| 901 | |
| 902 | <h3 class="devdocs-subsection-title"><a href="#cli-basics">Core workflow</a></h3> |
| 903 | |
| 904 | <table class="devdocs-table"> |
| 905 | <thead><tr><th>Task</th><th>Command</th></tr></thead> |
| 906 | <tbody> |
| 907 | <tr><td>Initialise repo</td><td><code>muse init [--domain code|midi|identity]</code></td></tr> |
| 908 | <tr><td>Working-tree status</td><td><code>muse status --json</code></td></tr> |
| 909 | <tr><td>Stage files</td><td><code>muse code add <path></code> / <code>muse code add .</code></td></tr> |
| 910 | <tr><td>Unstage</td><td><code>muse code reset <path></code></td></tr> |
| 911 | <tr><td>Delete + stage deletion</td><td><code>muse rm <path></code></td></tr> |
| 912 | <tr><td>Commit</td><td><code>muse commit -m "msg" [--agent-id X --model-id Y --sign]</code></td></tr> |
| 913 | <tr><td>History</td><td><code>muse log --json</code></td></tr> |
| 914 | <tr><td>Inspect commit</td><td><code>muse read --json [--manifest]</code></td></tr> |
| 915 | <tr><td>Diff working tree</td><td><code>muse diff</code> / <code>muse diff --staged</code></td></tr> |
| 916 | <tr><td>Diff two refs</td><td><code>muse diff HEAD~3 HEAD --json</code></td></tr> |
| 917 | <tr><td>List branches</td><td><code>muse branch --json</code></td></tr> |
| 918 | <tr><td>Switch / create branch</td><td><code>muse checkout [-b] <branch> [--intent "..." --resumable]</code></td></tr> |
| 919 | <tr><td>Three-way merge</td><td><code>muse merge <branch></code></td></tr> |
| 920 | <tr><td>Dry-run merge</td><td><code>muse merge --dry-run <branch> --json</code></td></tr> |
| 921 | <tr><td>Shelf (stash)</td><td><code>muse shelf save [-m "msg"]</code> / <code>muse shelf pop</code></td></tr> |
| 922 | <tr><td>Tag</td><td><code>muse tag add "label" [<ref>]</code></td></tr> |
| 923 | <tr><td>Release</td><td><code>muse release add <semver></code></td></tr> |
| 924 | </tbody> |
| 925 | </table> |
| 926 | |
| 927 | <div class="devdocs-callout devdocs-callout--warn"> |
| 928 | {{ icon("alert", 16, "devdocs-callout-icon") }} |
| 929 | <div> |
| 930 | <code>muse code add .</code> stages new files, modifications, <em>and</em> |
| 931 | deletions of already-tracked files — equivalent to <code>git add -u && git add .</code> |
| 932 | combined. To remove a file from tracking without deleting it from disk, |
| 933 | use <code>muse rm --cached <path></code>. |
| 934 | </div> |
| 935 | </div> |
| 936 | |
| 937 | <h3 class="devdocs-subsection-title"><a href="#cli-json-shape">muse status --json shape</a></h3> |
| 938 | |
| 939 | <p> |
| 940 | The status JSON schema is always identical regardless of domain or staging state. |
| 941 | All keys are always present — no <code>dict.get</code> guards needed. |
| 942 | </p> |
| 943 | |
| 944 | <div class="devdocs-code-block"> |
| 945 | <div class="devdocs-code-header"> |
| 946 | <span class="devdocs-code-lang">json</span> |
| 947 | </div> |
| 948 | <pre><code>{ |
| 949 | <span class="tok-key">"branch"</span>: <span class="tok-str">"dev"</span>, |
| 950 | <span class="tok-key">"head_commit"</span>: <span class="tok-str">"sha256:abc..."</span>, |
| 951 | <span class="tok-key">"upstream"</span>: null, <span class="tok-cmt">// tracking remote name, or null</span> |
| 952 | <span class="tok-key">"ahead"</span>: null, <span class="tok-cmt">// commits ahead of remote; null when no upstream</span> |
| 953 | <span class="tok-key">"behind"</span>: null, <span class="tok-cmt">// commits behind remote; null when no upstream</span> |
| 954 | <span class="tok-key">"clean"</span>: <span class="tok-num">true</span>, <span class="tok-cmt">// true only when no staged, unstaged, or untracked files</span> |
| 955 | <span class="tok-key">"dirty"</span>: <span class="tok-num">false</span>, <span class="tok-cmt">// always NOT clean</span> |
| 956 | <span class="tok-key">"total_changes"</span>: <span class="tok-num">0</span>, <span class="tok-cmt">// tracked-file changes (added+modified+deleted+renamed)</span> |
| 957 | <span class="tok-key">"untracked_count"</span>: <span class="tok-num">0</span>, <span class="tok-cmt">// len(untracked); nonzero when dirty but total_changes==0</span> |
| 958 | <span class="tok-key">"added"</span>: [], <span class="tok-cmt">// flat union of staged + unstaged</span> |
| 959 | <span class="tok-key">"modified"</span>: [], |
| 960 | <span class="tok-key">"deleted"</span>: [], |
| 961 | <span class="tok-key">"renamed"</span>: {}, <span class="tok-cmt">// old_path → new_path map</span> |
| 962 | <span class="tok-key">"staged"</span>: { |
| 963 | <span class="tok-key">"added"</span>: [], <span class="tok-key">"modified"</span>: [], <span class="tok-key">"deleted"</span>: [] |
| 964 | }, |
| 965 | <span class="tok-key">"unstaged"</span>: { |
| 966 | <span class="tok-key">"added"</span>: [], <span class="tok-key">"modified"</span>: [], <span class="tok-key">"deleted"</span>: [], <span class="tok-key">"renamed"</span>: {} |
| 967 | }, |
| 968 | <span class="tok-key">"untracked"</span>: [], <span class="tok-cmt">// on-disk but not tracked; presence makes clean=false</span> |
| 969 | <span class="tok-key">"conflict_paths"</span>: [], |
| 970 | <span class="tok-key">"merge_in_progress"</span>: <span class="tok-num">false</span>, |
| 971 | <span class="tok-key">"merge_from"</span>: null, <span class="tok-cmt">// branch being merged; null when no merge</span> |
| 972 | <span class="tok-key">"conflict_count"</span>: <span class="tok-num">0</span>, |
| 973 | <span class="tok-key">"checkout_interrupted"</span>: <span class="tok-num">false</span>, |
| 974 | <span class="tok-key">"checkout_target"</span>: null |
| 975 | }</code></pre> |
| 976 | </div> |
| 977 | </section> |
| 978 | |
| 979 | {# ── Push / pull ──────────────────────────────────────────────────────── #} |
| 980 | <section class="devdocs-section" id="push-pull"> |
| 981 | <h2 class="devdocs-section-title"><a href="#push-pull">Push / pull</a></h2> |
| 982 | |
| 983 | <p> |
| 984 | Push sends a <code>WireMPack</code> — a compact envelope containing every |
| 985 | commit, snapshot, and object the remote doesn't already have — over |
| 986 | <code>application/x-msgpack</code> to <code>POST /{owner}/{slug}/push</code>. |
| 987 | The hub validates the mpack, stores objects atomically, and advances the |
| 988 | branch pointer in a single transaction. Pull is the reverse: the client |
| 989 | fetches an mpack from the hub and integrates it locally. |
| 990 | </p> |
| 991 | |
| 992 | <div class="devdocs-code-block"> |
| 993 | <div class="devdocs-code-header"> |
| 994 | <span class="devdocs-code-lang">python</span> |
| 995 | <span class="devdocs-code-label">WireMPack shape (musehub.models.wire)</span> |
| 996 | </div> |
| 997 | <pre><code><span class="tok-kw">class</span> <span class="tok-type">WireMPack</span>(BaseModel): |
| 998 | commits: list[<span class="tok-type">WireCommit</span>] <span class="tok-cmt"># CommitRecord dicts</span> |
| 999 | snapshots: list[<span class="tok-type">WireSnapshot</span>] <span class="tok-cmt"># SnapshotRecord dicts</span> |
| 1000 | objects: list[<span class="tok-type">WireObject</span>] <span class="tok-cmt"># raw blob bytes</span> |
| 1001 | branch_heads: dict[str, str] <span class="tok-cmt"># branch → commit_id</span> |
| 1002 | |
| 1003 | <span class="tok-kw">class</span> <span class="tok-type">WireObject</span>(BaseModel): |
| 1004 | object_id: str |
| 1005 | content: bytes <span class="tok-cmt"># raw; no base64</span> |
| 1006 | path: str = <span class="tok-str">""</span> |
| 1007 | encoding: str = <span class="tok-str">"raw"</span> <span class="tok-cmt"># "raw" | "zlib" | "delta+zlib"</span> |
| 1008 | base_id: str | None <span class="tok-cmt"># set for delta-encoded objects</span></code></pre> |
| 1009 | </div> |
| 1010 | |
| 1011 | <h3 class="devdocs-subsection-title"><a href="#push-flow">Push flow</a></h3> |
| 1012 | |
| 1013 | <p>The hub validates a push in this order before persisting anything:</p> |
| 1014 | |
| 1015 | <div class="devdocs-code-block"> |
| 1016 | <div class="devdocs-code-header"> |
| 1017 | <span class="devdocs-code-lang">text</span> |
| 1018 | </div> |
| 1019 | <pre><code>1. Verify MSign Authorization header (Ed25519, ±30s replay window) |
| 1020 | 2. Resolve repo — owner + slug → repo_id + repo_root (/data/repos/<owner>/<slug>/) |
| 1021 | 3. Confirm pusher has write access |
| 1022 | 4. Negotiate have/want — hub checks object existence on disk, not in DB |
| 1023 | 5. Validate mpack schema + ID format (<span class="tok-acc">sha256:</span><hex>, ≥32 chars) |
| 1024 | 6. Enforce push limits (max 10k commits, 1k objects, 38 MB/object) |
| 1025 | 7. Persist objects → /data/repos/<owner>/<slug>/objects/sha256/<2-hex>/<62-hex> (atomic) |
| 1026 | 8. Persist snapshots → musehub_snapshots (DB cache) |
| 1027 | 9. Persist commits → musehub_commits (DB cache) |
| 1028 | 10. Advance branch pointer → refs/heads/<branch> on disk (atomic rename), then musehub_branches (cache) |
| 1029 | 11. Update repo.pushed_at timestamp |
| 1030 | 12. Upsert reachability index (musehub_object_refs)</code></pre> |
| 1031 | </div> |
| 1032 | |
| 1033 | <div class="devdocs-code-block"> |
| 1034 | <div class="devdocs-code-header"> |
| 1035 | <span class="devdocs-code-lang">bash</span> |
| 1036 | </div> |
| 1037 | <pre><code><span class="tok-cmt"># Push dev branch to the local hub</span> |
| 1038 | muse push local dev |
| 1039 | |
| 1040 | <span class="tok-cmt"># Push to staging</span> |
| 1041 | muse push staging dev |
| 1042 | |
| 1043 | <span class="tok-cmt"># Pull from remote</span> |
| 1044 | muse pull local dev |
| 1045 | |
| 1046 | <span class="tok-cmt"># Check configured remotes</span> |
| 1047 | muse remote --json</code></pre> |
| 1048 | </div> |
| 1049 | |
| 1050 | <div class="devdocs-callout"> |
| 1051 | {{ icon("info", 16, "devdocs-callout-icon") }} |
| 1052 | <div> |
| 1053 | Push returns 404 ("Repository not found on remote") if the repo hasn't |
| 1054 | been created on the hub yet. Create it first via |
| 1055 | <code>muse hub repo create --name <name> --json</code>, then retry. |
| 1056 | </div> |
| 1057 | </div> |
| 1058 | </section> |
| 1059 | |
| 1060 | {# ── Phase nav ────────────────────────────────────────────────────────── #} |
| 1061 | <nav class="devdocs-phase-nav" aria-label="Phase navigation"> |
| 1062 | <a class="devdocs-phase-nav-btn devdocs-phase-nav-btn--prev" href="/muse/getting-started"> |
| 1063 | {{ icon("arrow-left", 14) }} |
| 1064 | Phase 00: Getting Started |
| 1065 | </a> |
| 1066 | <a class="devdocs-phase-nav-btn devdocs-phase-nav-btn--next" href="/muse/identity"> |
| 1067 | Phase 02: Cryptographic Identity |
| 1068 | {{ icon("arrow-right", 14) }} |
| 1069 | </a> |
| 1070 | </nav> |
| 1071 | |
| 1072 | </article> |
| 1073 | </div> |
| 1074 | </div> |
| 1075 | |
| 1076 | {% endblock %} |
File History
2 commits
sha256:f99af7b1a7f36c4d537d1c630d4b71fc39222b1255f82e930929e2fc89015e11
fix: relax browse_repo perf budget to 500ms — 200ms was too…
Sonnet 4.6
101 days ago
sha256:763eb2cb8675073b84c19345b27586d2ed939a9aee97c5479b69f502f1a70eff
fix(tests): update test suite to match current implementation
Sonnet 4.6
patch
123 days ago