remotes.md
markdown
| 1 | # Remotes — Muse Remote Sync Reference |
| 2 | |
| 3 | Muse supports synchronizing repositories with a remote host (e.g. MuseHub) |
| 4 | using a small set of commands modelled on Git's porcelain/plumbing separation. |
| 5 | The CLI is a **pure client** — no server runs inside `muse`. The server lives on |
| 6 | MuseHub. |
| 7 | |
| 8 | --- |
| 9 | |
| 10 | ## Table of Contents |
| 11 | |
| 12 | 1. [Overview](#overview) |
| 13 | 2. [Transport Architecture](#transport-architecture) |
| 14 | 3. [MuseHub API Contract](#musehub-api-contract) |
| 15 | 4. [PackBundle Wire Format](#packbundle-wire-format) |
| 16 | 5. [Authentication](#authentication) |
| 17 | 6. [Commands](#commands) |
| 18 | - [muse remote](#muse-remote) |
| 19 | - [muse clone](#muse-clone) |
| 20 | - [muse fetch](#muse-fetch) |
| 21 | - [muse pull](#muse-pull) |
| 22 | - [muse push](#muse-push) |
| 23 | - [muse plumbing ls-remote](#muse-plumbing-ls-remote) |
| 24 | 7. [Tracking Branches](#tracking-branches) |
| 25 | 8. [Token Lifecycle](#token-lifecycle) |
| 26 | |
| 27 | --- |
| 28 | |
| 29 | ## Overview |
| 30 | |
| 31 | The remote sync workflow mirrors Git's model: |
| 32 | |
| 33 | ``` |
| 34 | muse clone <url> # one-time: download a full copy of a remote repo |
| 35 | muse fetch [remote] # download new commits without merging |
| 36 | muse pull [remote] # fetch + three-way merge into current branch |
| 37 | muse push [remote] # upload local commits to the remote |
| 38 | muse remote [-v] # list remotes (implied); manage named remotes |
| 39 | muse remote add <n> <url> |
| 40 | muse plumbing ls-remote [remote] # list remote branches (Tier 1 plumbing) |
| 41 | ``` |
| 42 | |
| 43 | Remotes are named connections to a MuseHub repository URL. The default remote |
| 44 | name is `origin` (set automatically by `muse clone`). |
| 45 | |
| 46 | --- |
| 47 | |
| 48 | ## Transport Architecture |
| 49 | |
| 50 | ``` |
| 51 | Muse CLI process MuseHub server |
| 52 | ───────────────── ────────────── |
| 53 | HttpTransport REST API |
| 54 | └─ urllib.request (stdlib) ────HTTPS──► GET /refs |
| 55 | POST /fetch |
| 56 | POST /push |
| 57 | ``` |
| 58 | |
| 59 | The `MuseTransport` Protocol in `muse/core/transport.py` is the seam between |
| 60 | CLI command code and the HTTP implementation. It is entirely synchronous — the |
| 61 | Muse CLI has no async code. The `HttpTransport` class uses Python's stdlib |
| 62 | `urllib.request` (HTTP/1.1 + TLS), requiring zero new dependencies. |
| 63 | |
| 64 | **Why HTTP/1.1?** |
| 65 | Each `muse` CLI invocation is a short-lived OS process making 2–3 requests. |
| 66 | HTTP/2 multiplexing benefits arise within a single long-lived connection. |
| 67 | Agent scale comes from MuseHub handling millions of concurrent HTTP/1.1 |
| 68 | connections via its load balancer — not from any individual CLI process. |
| 69 | When MuseHub is ready to upgrade, the `MuseTransport` Protocol seam means |
| 70 | only `HttpTransport` changes; all CLI command code stays the same. |
| 71 | |
| 72 | --- |
| 73 | |
| 74 | ## MuseHub API Contract |
| 75 | |
| 76 | MuseHub must implement these three endpoints under each repository URL: |
| 77 | |
| 78 | ### `GET {repo_url}/refs` |
| 79 | |
| 80 | Returns the current state of the repository. |
| 81 | |
| 82 | **Response (JSON):** |
| 83 | |
| 84 | ```json |
| 85 | { |
| 86 | "repo_id": "<uuid>", |
| 87 | "domain": "midi", |
| 88 | "default_branch": "main", |
| 89 | "branch_heads": { |
| 90 | "main": "<commit_id>", |
| 91 | "dev": "<commit_id>" |
| 92 | } |
| 93 | } |
| 94 | ``` |
| 95 | |
| 96 | ### `POST {repo_url}/fetch` |
| 97 | |
| 98 | Downloads commits, snapshots, and objects the client does not have. |
| 99 | |
| 100 | **Request body (JSON):** |
| 101 | |
| 102 | ```json |
| 103 | { |
| 104 | "want": ["<commit_id>", ...], |
| 105 | "have": ["<commit_id>", ...] |
| 106 | } |
| 107 | ``` |
| 108 | |
| 109 | `want` is the list of remote commit IDs the client wants to receive. |
| 110 | `have` is the list of commit IDs the client already has locally, allowing the |
| 111 | server to compute the minimal delta to send (analogous to Git's fetch |
| 112 | negotiation). |
| 113 | |
| 114 | **Response:** JSON `PackBundle` (see below). |
| 115 | |
| 116 | ### `POST {repo_url}/push` |
| 117 | |
| 118 | Receives commits, snapshots, and objects from the client. |
| 119 | |
| 120 | **Request body (JSON):** |
| 121 | |
| 122 | ```json |
| 123 | { |
| 124 | "bundle": { ... PackBundle ... }, |
| 125 | "branch": "main", |
| 126 | "force": false |
| 127 | } |
| 128 | ``` |
| 129 | |
| 130 | When `force` is `false`, MuseHub must verify the push is a fast-forward (the |
| 131 | current remote HEAD is an ancestor of the pushed commit). Return HTTP 409 to |
| 132 | reject a non-fast-forward push. |
| 133 | |
| 134 | **Response (JSON):** |
| 135 | |
| 136 | ```json |
| 137 | { |
| 138 | "ok": true, |
| 139 | "message": "ok", |
| 140 | "branch_heads": { "main": "<new_commit_id>" } |
| 141 | } |
| 142 | ``` |
| 143 | |
| 144 | **HTTP error codes:** |
| 145 | |
| 146 | | Code | Meaning | |
| 147 | |------|---------| |
| 148 | | 401 | Invalid or missing bearer token | |
| 149 | | 404 | Repository does not exist | |
| 150 | | 409 | Push rejected — non-fast-forward without `force: true` | |
| 151 | | 5xx | Server error | |
| 152 | |
| 153 | --- |
| 154 | |
| 155 | ## PackBundle Wire Format |
| 156 | |
| 157 | A `PackBundle` is a self-contained JSON object carrying everything needed to |
| 158 | reconstruct a slice of commit history: |
| 159 | |
| 160 | ```json |
| 161 | { |
| 162 | "commits": [ |
| 163 | { |
| 164 | "commit_id": "<sha256>", |
| 165 | "repo_id": "<uuid>", |
| 166 | "branch": "main", |
| 167 | "snapshot_id": "<sha256>", |
| 168 | "message": "Add verse", |
| 169 | "committed_at": "2026-03-18T12:00:00+00:00", |
| 170 | "parent_commit_id": "<sha256> | null", |
| 171 | "author": "gabriel", |
| 172 | ... |
| 173 | } |
| 174 | ], |
| 175 | "snapshots": [ |
| 176 | { |
| 177 | "snapshot_id": "<sha256>", |
| 178 | "manifest": { "tracks/drums.mid": "<sha256>", ... }, |
| 179 | "created_at": "2026-03-18T12:00:00+00:00" |
| 180 | } |
| 181 | ], |
| 182 | "objects": [ |
| 183 | { |
| 184 | "object_id": "<sha256>", |
| 185 | "content_b64": "<base64-encoded raw bytes>" |
| 186 | } |
| 187 | ], |
| 188 | "branch_heads": { |
| 189 | "main": "<commit_id>" |
| 190 | } |
| 191 | } |
| 192 | ``` |
| 193 | |
| 194 | Objects are the raw blob bytes stored in `.muse/objects/`, base64-encoded for |
| 195 | JSON transport. `apply_pack()` in `muse/core/pack.py` writes objects, then |
| 196 | snapshots, then commits — in dependency order. |
| 197 | |
| 198 | --- |
| 199 | |
| 200 | ## Authentication |
| 201 | |
| 202 | All MuseHub API calls include an `Authorization: Bearer <token>` header when a |
| 203 | token is configured. |
| 204 | |
| 205 | Store your token in `.muse/config.toml`: |
| 206 | |
| 207 | ```toml |
| 208 | [auth] |
| 209 | token = "your-hub-token-here" |
| 210 | ``` |
| 211 | |
| 212 | The token is read by `muse.cli.config.get_auth_token()` and is **never** |
| 213 | written to any log line. Add `.muse/config.toml` to `.gitignore` to prevent |
| 214 | accidental commit. |
| 215 | |
| 216 | --- |
| 217 | |
| 218 | ## Commands |
| 219 | |
| 220 | ### muse remote |
| 221 | |
| 222 | Manage named remote connections. Remote state is stored entirely in |
| 223 | `.muse/config.toml` and `.muse/remotes/<name>/<branch>`. No network calls. |
| 224 | |
| 225 | #### Subcommands |
| 226 | |
| 227 | ``` |
| 228 | muse remote add <name> <url> |
| 229 | ``` |
| 230 | Register a new named remote. |
| 231 | `<name>` — identifier (e.g. `origin`, `upstream`). |
| 232 | `<url>` — full MuseHub repository URL. |
| 233 | |
| 234 | ``` |
| 235 | muse remote remove <name> |
| 236 | ``` |
| 237 | Remove a remote and all its tracking refs under `.muse/remotes/<name>/`. |
| 238 | |
| 239 | ``` |
| 240 | muse remote rename <old> <new> |
| 241 | ``` |
| 242 | Rename a remote in config and move its tracking refs directory. |
| 243 | |
| 244 | ``` |
| 245 | muse remote [-v] |
| 246 | ``` |
| 247 | List configured remotes (the default when no subcommand is given). With |
| 248 | `-v` / `--verbose`, shows URL and upstream tracking branch for each remote. |
| 249 | |
| 250 | ``` |
| 251 | muse remote get-url <name> |
| 252 | ``` |
| 253 | Print the URL of a named remote. |
| 254 | |
| 255 | ``` |
| 256 | muse remote set-url <name> <url> |
| 257 | ``` |
| 258 | Update the URL of an existing remote. |
| 259 | |
| 260 | --- |
| 261 | |
| 262 | ### muse clone |
| 263 | |
| 264 | Clone a remote Muse repository into a new local directory. |
| 265 | |
| 266 | ``` |
| 267 | muse clone <url> [<directory>] |
| 268 | ``` |
| 269 | |
| 270 | **Options:** |
| 271 | |
| 272 | | Flag | Description | |
| 273 | |------|-------------| |
| 274 | | `--branch <b>` | Check out branch `<b>` instead of the remote default branch | |
| 275 | |
| 276 | **What clone does:** |
| 277 | |
| 278 | 1. Calls `GET <url>/refs` to discover the remote's repo_id, domain, and branch heads. |
| 279 | 2. Creates the target directory and initialises `.muse/` with the remote's `repo_id` and `domain`. |
| 280 | 3. Calls `POST <url>/fetch` with `want=all, have=[]` to download the complete history. |
| 281 | 4. Applies the `PackBundle` (objects → snapshots → commits). |
| 282 | 5. Sets `origin` remote and upstream tracking. |
| 283 | 6. Restores `state/` from the default branch HEAD snapshot. |
| 284 | |
| 285 | **Examples:** |
| 286 | |
| 287 | ```bash |
| 288 | muse clone https://hub.muse.io/repos/my-song |
| 289 | muse clone https://hub.muse.io/repos/my-song local-copy |
| 290 | muse clone https://hub.muse.io/repos/my-song --branch dev |
| 291 | ``` |
| 292 | |
| 293 | --- |
| 294 | |
| 295 | ### muse fetch |
| 296 | |
| 297 | Download commits, snapshots, and objects from a remote without merging. |
| 298 | |
| 299 | ``` |
| 300 | muse fetch [<remote>] [--branch <b>] |
| 301 | ``` |
| 302 | |
| 303 | **Options:** |
| 304 | |
| 305 | | Flag | Description | |
| 306 | |------|-------------| |
| 307 | | `<remote>` | Remote name (default: `origin`) | |
| 308 | | `--branch <b>` / `-b <b>` | Remote branch to fetch (default: tracked branch or current branch) | |
| 309 | |
| 310 | After fetch, the remote tracking pointer `.muse/remotes/<remote>/<branch>` is |
| 311 | updated. The local branch HEAD is **not** changed. Use `muse merge` or |
| 312 | `muse pull` to integrate the fetched commits. |
| 313 | |
| 314 | **Examples:** |
| 315 | |
| 316 | ```bash |
| 317 | muse fetch |
| 318 | muse fetch origin |
| 319 | muse fetch origin --branch dev |
| 320 | ``` |
| 321 | |
| 322 | --- |
| 323 | |
| 324 | ### muse pull |
| 325 | |
| 326 | Fetch from a remote and merge into the current branch. |
| 327 | |
| 328 | ``` |
| 329 | muse pull [<remote>] [options] |
| 330 | ``` |
| 331 | |
| 332 | **Options:** |
| 333 | |
| 334 | | Flag | Description | |
| 335 | |------|-------------| |
| 336 | | `<remote>` | Remote name (default: `origin`) | |
| 337 | | `--branch <b>` / `-b <b>` | Remote branch to pull (default: tracked branch or current branch) | |
| 338 | | `--no-merge` | Stop after fetch; do not merge | |
| 339 | | `-m <msg>` / `--message <msg>` | Override the merge commit message | |
| 340 | |
| 341 | **Merge behaviour:** |
| 342 | |
| 343 | - Fast-forward: if the remote HEAD is a direct descendant of local HEAD, the |
| 344 | local branch ref and `state/` are advanced without a merge commit. |
| 345 | - Three-way merge: delegates to the active domain plugin's `merge()` / |
| 346 | `merge_ops()` — identical to `muse merge`. |
| 347 | - Conflicts: MERGE_STATE.json is written; the user fixes conflicts then runs |
| 348 | `muse commit`. |
| 349 | |
| 350 | **Examples:** |
| 351 | |
| 352 | ```bash |
| 353 | muse pull |
| 354 | muse pull origin --branch dev |
| 355 | muse pull origin --no-merge # equivalent to muse fetch |
| 356 | ``` |
| 357 | |
| 358 | --- |
| 359 | |
| 360 | ### muse push |
| 361 | |
| 362 | Upload local commits, snapshots, and objects to a remote. |
| 363 | |
| 364 | ``` |
| 365 | muse push [<remote>] [options] |
| 366 | ``` |
| 367 | |
| 368 | **Options:** |
| 369 | |
| 370 | | Flag | Description | |
| 371 | |------|-------------| |
| 372 | | `<remote>` | Remote name (default: `origin`) | |
| 373 | | `--branch <b>` / `-b <b>` | Branch to push (default: tracked branch or current branch) | |
| 374 | | `-u` / `--set-upstream` | Record `<remote>/<branch>` as the upstream for this branch | |
| 375 | | `--force` | Force-push even if the remote has diverged | |
| 376 | |
| 377 | **Fast-forward enforcement:** by default, MuseHub rejects a push if its current |
| 378 | HEAD is not an ancestor of the local HEAD (HTTP 409). Pass `--force` to |
| 379 | override. Use `muse pull` first to integrate remote changes before pushing. |
| 380 | |
| 381 | **First push workflow:** |
| 382 | |
| 383 | ```bash |
| 384 | muse push origin -u # push + record origin/main as upstream |
| 385 | ``` |
| 386 | |
| 387 | Subsequent pushes on the same branch: |
| 388 | |
| 389 | ```bash |
| 390 | muse push # infers remote and branch from upstream config |
| 391 | ``` |
| 392 | |
| 393 | --- |
| 394 | |
| 395 | ### muse plumbing ls-remote |
| 396 | |
| 397 | List branch references on a remote repository. **Plumbing command** — no local |
| 398 | state is written. |
| 399 | |
| 400 | ``` |
| 401 | muse plumbing ls-remote [<remote-or-url>] [--json] |
| 402 | ``` |
| 403 | |
| 404 | **Options:** |
| 405 | |
| 406 | | Flag | Description | |
| 407 | |------|-------------| |
| 408 | | `<remote-or-url>` | Named remote or a full HTTPS URL (default: `origin`) | |
| 409 | | `--json` | Emit structured JSON instead of tab-delimited text | |
| 410 | |
| 411 | **Default output** (tab-delimited, `*` marks default branch): |
| 412 | |
| 413 | ``` |
| 414 | abc123def456... main * |
| 415 | 789012abc345... dev |
| 416 | ``` |
| 417 | |
| 418 | **JSON output** (for agent consumption): |
| 419 | |
| 420 | ```json |
| 421 | { |
| 422 | "repo_id": "<uuid>", |
| 423 | "domain": "midi", |
| 424 | "default_branch": "main", |
| 425 | "branches": { |
| 426 | "main": "<commit_id>", |
| 427 | "dev": "<commit_id>" |
| 428 | } |
| 429 | } |
| 430 | ``` |
| 431 | |
| 432 | **Examples:** |
| 433 | |
| 434 | ```bash |
| 435 | muse plumbing ls-remote |
| 436 | muse plumbing ls-remote upstream |
| 437 | muse plumbing ls-remote https://hub.muse.io/repos/r1 |
| 438 | muse plumbing ls-remote --json origin |
| 439 | ``` |
| 440 | |
| 441 | --- |
| 442 | |
| 443 | ## Tracking Branches |
| 444 | |
| 445 | Muse tracks the relationship between local branches and remote branches in two |
| 446 | places: |
| 447 | |
| 448 | 1. **Upstream config** — `.muse/config.toml`: |
| 449 | ```toml |
| 450 | [remotes.origin] |
| 451 | url = "https://hub.muse.io/repos/my-song" |
| 452 | branch = "main" # local branch "main" tracks origin/main |
| 453 | ``` |
| 454 | Written by `muse push -u` or `muse clone`. |
| 455 | |
| 456 | 2. **Remote tracking heads** — `.muse/remotes/<name>/<branch>`: |
| 457 | Plain-text files containing the last-known commit ID for each remote branch. |
| 458 | Written by `muse fetch`, `muse pull`, and `muse push`. |
| 459 | |
| 460 | When a tracking relationship is set, `muse push` and `muse pull` resolve the |
| 461 | remote and branch automatically without additional arguments. |
| 462 | |
| 463 | --- |
| 464 | |
| 465 | ## Authentication |
| 466 | |
| 467 | Muse stores bearer tokens in `~/.muse/identity.toml` — a machine-scoped, |
| 468 | `0o600` credential file that is never part of any repository snapshot. |
| 469 | Credentials are **not** stored in `.muse/config.toml`. |
| 470 | |
| 471 | See [`docs/reference/auth.md`](auth.md) for the complete reference: |
| 472 | |
| 473 | - `muse auth login` — store a token |
| 474 | - `muse auth whoami` — inspect stored identity |
| 475 | - `muse auth logout` — remove a token |
| 476 | - File format, permissions model, and security properties |
| 477 | |
| 478 | The bearer token is automatically picked up by `fetch`, `pull`, `push`, and |
| 479 | `ls-remote` via `muse.core.identity.resolve_token()`. The token value is |
| 480 | **never** written to any log line — only `"Bearer ***"` appears in debug logs. |
| 481 | |
| 482 | --- |
| 483 | |
| 484 | *See also:* |
| 485 | - [`docs/reference/auth.md`](auth.md) — identity management (`muse auth`) |
| 486 | - [`docs/reference/hub.md`](hub.md) — hub fabric connection (`muse hub`) |
| 487 | - [`docs/reference/security.md`](security.md) — full security architecture |
| 488 | - [`docs/reference/museignore.md`](museignore.md) — domain-scoped ignore rules |
| 489 | - [`docs/reference/muse-attributes.md`](muse-attributes.md) — merge strategy overrides |
| 490 | - [`docs/architecture/muse-vcs.md`](../architecture/muse-vcs.md) — system architecture |