remotes.md
markdown
| 1 | # Remotes — Muse Remote Sync Reference |
| 2 | |
| 3 | Muse supports synchronizing repositories with a remote host (e.g. MuseHub) |
| 4 | using a small set of commands modelled on Git's porcelain/plumbing separation. |
| 5 | The CLI is a **pure client** — no server runs inside `muse`. The server lives on |
| 6 | MuseHub. |
| 7 | |
| 8 | --- |
| 9 | |
| 10 | ## Table of Contents |
| 11 | |
| 12 | 1. [Overview](#overview) |
| 13 | 2. [Transport Architecture](#transport-architecture) |
| 14 | 3. [MuseHub API Contract](#musehub-api-contract) |
| 15 | 4. [PackBundle Wire Format](#packbundle-wire-format) |
| 16 | 5. [Authentication](#authentication) |
| 17 | 6. [Commands](#commands) |
| 18 | - [muse remote](#muse-remote) |
| 19 | - [muse clone](#muse-clone) |
| 20 | - [muse fetch](#muse-fetch) |
| 21 | - [muse pull](#muse-pull) |
| 22 | - [muse push](#muse-push) |
| 23 | - [muse plumbing ls-remote](#muse-plumbing-ls-remote) |
| 24 | 7. [Tracking Branches](#tracking-branches) |
| 25 | 8. [Token Lifecycle](#token-lifecycle) |
| 26 | |
| 27 | --- |
| 28 | |
| 29 | ## Overview |
| 30 | |
| 31 | The remote sync workflow mirrors Git's model: |
| 32 | |
| 33 | ``` |
| 34 | muse clone <url> # one-time: download a full copy of a remote repo |
| 35 | muse fetch [remote] # download new commits without merging |
| 36 | muse pull [remote] # fetch + three-way merge into current branch |
| 37 | muse push [remote] # upload local commits to the remote |
| 38 | muse remote add <n> <url> # manage named remotes |
| 39 | muse plumbing ls-remote [remote] # list remote branches (Tier 1 plumbing) |
| 40 | ``` |
| 41 | |
| 42 | Remotes are named connections to a MuseHub repository URL. The default remote |
| 43 | name is `origin` (set automatically by `muse clone`). |
| 44 | |
| 45 | --- |
| 46 | |
| 47 | ## Transport Architecture |
| 48 | |
| 49 | ``` |
| 50 | Muse CLI process MuseHub server |
| 51 | ───────────────── ────────────── |
| 52 | HttpTransport REST API |
| 53 | └─ urllib.request (stdlib) ────HTTPS──► GET /refs |
| 54 | POST /fetch |
| 55 | POST /push |
| 56 | ``` |
| 57 | |
| 58 | The `MuseTransport` Protocol in `muse/core/transport.py` is the seam between |
| 59 | CLI command code and the HTTP implementation. It is entirely synchronous — the |
| 60 | Muse CLI has no async code. The `HttpTransport` class uses Python's stdlib |
| 61 | `urllib.request` (HTTP/1.1 + TLS), requiring zero new dependencies. |
| 62 | |
| 63 | **Why HTTP/1.1?** |
| 64 | Each `muse` CLI invocation is a short-lived OS process making 2–3 requests. |
| 65 | HTTP/2 multiplexing benefits arise within a single long-lived connection. |
| 66 | Agent scale comes from MuseHub handling millions of concurrent HTTP/1.1 |
| 67 | connections via its load balancer — not from any individual CLI process. |
| 68 | When MuseHub is ready to upgrade, the `MuseTransport` Protocol seam means |
| 69 | only `HttpTransport` changes; all CLI command code stays the same. |
| 70 | |
| 71 | --- |
| 72 | |
| 73 | ## MuseHub API Contract |
| 74 | |
| 75 | MuseHub must implement these three endpoints under each repository URL: |
| 76 | |
| 77 | ### `GET {repo_url}/refs` |
| 78 | |
| 79 | Returns the current state of the repository. |
| 80 | |
| 81 | **Response (JSON):** |
| 82 | |
| 83 | ```json |
| 84 | { |
| 85 | "repo_id": "<uuid>", |
| 86 | "domain": "midi", |
| 87 | "default_branch": "main", |
| 88 | "branch_heads": { |
| 89 | "main": "<commit_id>", |
| 90 | "dev": "<commit_id>" |
| 91 | } |
| 92 | } |
| 93 | ``` |
| 94 | |
| 95 | ### `POST {repo_url}/fetch` |
| 96 | |
| 97 | Downloads commits, snapshots, and objects the client does not have. |
| 98 | |
| 99 | **Request body (JSON):** |
| 100 | |
| 101 | ```json |
| 102 | { |
| 103 | "want": ["<commit_id>", ...], |
| 104 | "have": ["<commit_id>", ...] |
| 105 | } |
| 106 | ``` |
| 107 | |
| 108 | `want` is the list of remote commit IDs the client wants to receive. |
| 109 | `have` is the list of commit IDs the client already has locally, allowing the |
| 110 | server to compute the minimal delta to send (analogous to Git's fetch |
| 111 | negotiation). |
| 112 | |
| 113 | **Response:** JSON `PackBundle` (see below). |
| 114 | |
| 115 | ### `POST {repo_url}/push` |
| 116 | |
| 117 | Receives commits, snapshots, and objects from the client. |
| 118 | |
| 119 | **Request body (JSON):** |
| 120 | |
| 121 | ```json |
| 122 | { |
| 123 | "bundle": { ... PackBundle ... }, |
| 124 | "branch": "main", |
| 125 | "force": false |
| 126 | } |
| 127 | ``` |
| 128 | |
| 129 | When `force` is `false`, MuseHub must verify the push is a fast-forward (the |
| 130 | current remote HEAD is an ancestor of the pushed commit). Return HTTP 409 to |
| 131 | reject a non-fast-forward push. |
| 132 | |
| 133 | **Response (JSON):** |
| 134 | |
| 135 | ```json |
| 136 | { |
| 137 | "ok": true, |
| 138 | "message": "ok", |
| 139 | "branch_heads": { "main": "<new_commit_id>" } |
| 140 | } |
| 141 | ``` |
| 142 | |
| 143 | **HTTP error codes:** |
| 144 | |
| 145 | | Code | Meaning | |
| 146 | |------|---------| |
| 147 | | 401 | Invalid or missing bearer token | |
| 148 | | 404 | Repository does not exist | |
| 149 | | 409 | Push rejected — non-fast-forward without `force: true` | |
| 150 | | 5xx | Server error | |
| 151 | |
| 152 | --- |
| 153 | |
| 154 | ## PackBundle Wire Format |
| 155 | |
| 156 | A `PackBundle` is a self-contained JSON object carrying everything needed to |
| 157 | reconstruct a slice of commit history: |
| 158 | |
| 159 | ```json |
| 160 | { |
| 161 | "commits": [ |
| 162 | { |
| 163 | "commit_id": "<sha256>", |
| 164 | "repo_id": "<uuid>", |
| 165 | "branch": "main", |
| 166 | "snapshot_id": "<sha256>", |
| 167 | "message": "Add verse", |
| 168 | "committed_at": "2026-03-18T12:00:00+00:00", |
| 169 | "parent_commit_id": "<sha256> | null", |
| 170 | "author": "gabriel", |
| 171 | ... |
| 172 | } |
| 173 | ], |
| 174 | "snapshots": [ |
| 175 | { |
| 176 | "snapshot_id": "<sha256>", |
| 177 | "manifest": { "tracks/drums.mid": "<sha256>", ... }, |
| 178 | "created_at": "2026-03-18T12:00:00+00:00" |
| 179 | } |
| 180 | ], |
| 181 | "objects": [ |
| 182 | { |
| 183 | "object_id": "<sha256>", |
| 184 | "content_b64": "<base64-encoded raw bytes>" |
| 185 | } |
| 186 | ], |
| 187 | "branch_heads": { |
| 188 | "main": "<commit_id>" |
| 189 | } |
| 190 | } |
| 191 | ``` |
| 192 | |
| 193 | Objects are the raw blob bytes stored in `.muse/objects/`, base64-encoded for |
| 194 | JSON transport. `apply_pack()` in `muse/core/pack.py` writes objects, then |
| 195 | snapshots, then commits — in dependency order. |
| 196 | |
| 197 | --- |
| 198 | |
| 199 | ## Authentication |
| 200 | |
| 201 | All MuseHub API calls include an `Authorization: Bearer <token>` header when a |
| 202 | token is configured. |
| 203 | |
| 204 | Store your token in `.muse/config.toml`: |
| 205 | |
| 206 | ```toml |
| 207 | [auth] |
| 208 | token = "your-hub-token-here" |
| 209 | ``` |
| 210 | |
| 211 | The token is read by `muse.cli.config.get_auth_token()` and is **never** |
| 212 | written to any log line. Add `.muse/config.toml` to `.gitignore` to prevent |
| 213 | accidental commit. |
| 214 | |
| 215 | --- |
| 216 | |
| 217 | ## Commands |
| 218 | |
| 219 | ### muse remote |
| 220 | |
| 221 | Manage named remote connections. Remote state is stored entirely in |
| 222 | `.muse/config.toml` and `.muse/remotes/<name>/<branch>`. No network calls. |
| 223 | |
| 224 | #### Subcommands |
| 225 | |
| 226 | ``` |
| 227 | muse remote add <name> <url> |
| 228 | ``` |
| 229 | Register a new named remote. |
| 230 | `<name>` — identifier (e.g. `origin`, `upstream`). |
| 231 | `<url>` — full MuseHub repository URL. |
| 232 | |
| 233 | ``` |
| 234 | muse remote remove <name> |
| 235 | ``` |
| 236 | Remove a remote and all its tracking refs under `.muse/remotes/<name>/`. |
| 237 | |
| 238 | ``` |
| 239 | muse remote rename <old> <new> |
| 240 | ``` |
| 241 | Rename a remote in config and move its tracking refs directory. |
| 242 | |
| 243 | ``` |
| 244 | muse remote list [-v] |
| 245 | ``` |
| 246 | List configured remotes. With `-v` / `--verbose`, shows URL and upstream |
| 247 | tracking branch for each remote. |
| 248 | |
| 249 | ``` |
| 250 | muse remote get-url <name> |
| 251 | ``` |
| 252 | Print the URL of a named remote. |
| 253 | |
| 254 | ``` |
| 255 | muse remote set-url <name> <url> |
| 256 | ``` |
| 257 | Update the URL of an existing remote. |
| 258 | |
| 259 | --- |
| 260 | |
| 261 | ### muse clone |
| 262 | |
| 263 | Clone a remote Muse repository into a new local directory. |
| 264 | |
| 265 | ``` |
| 266 | muse clone <url> [<directory>] |
| 267 | ``` |
| 268 | |
| 269 | **Options:** |
| 270 | |
| 271 | | Flag | Description | |
| 272 | |------|-------------| |
| 273 | | `--branch <b>` | Check out branch `<b>` instead of the remote default branch | |
| 274 | |
| 275 | **What clone does:** |
| 276 | |
| 277 | 1. Calls `GET <url>/refs` to discover the remote's repo_id, domain, and branch heads. |
| 278 | 2. Creates the target directory and initialises `.muse/` with the remote's `repo_id` and `domain`. |
| 279 | 3. Calls `POST <url>/fetch` with `want=all, have=[]` to download the complete history. |
| 280 | 4. Applies the `PackBundle` (objects → snapshots → commits). |
| 281 | 5. Sets `origin` remote and upstream tracking. |
| 282 | 6. Restores `muse-work/` from the default branch HEAD snapshot. |
| 283 | |
| 284 | **Examples:** |
| 285 | |
| 286 | ```bash |
| 287 | muse clone https://hub.muse.io/repos/my-song |
| 288 | muse clone https://hub.muse.io/repos/my-song local-copy |
| 289 | muse clone https://hub.muse.io/repos/my-song --branch dev |
| 290 | ``` |
| 291 | |
| 292 | --- |
| 293 | |
| 294 | ### muse fetch |
| 295 | |
| 296 | Download commits, snapshots, and objects from a remote without merging. |
| 297 | |
| 298 | ``` |
| 299 | muse fetch [<remote>] [--branch <b>] |
| 300 | ``` |
| 301 | |
| 302 | **Options:** |
| 303 | |
| 304 | | Flag | Description | |
| 305 | |------|-------------| |
| 306 | | `<remote>` | Remote name (default: `origin`) | |
| 307 | | `--branch <b>` / `-b <b>` | Remote branch to fetch (default: tracked branch or current branch) | |
| 308 | |
| 309 | After fetch, the remote tracking pointer `.muse/remotes/<remote>/<branch>` is |
| 310 | updated. The local branch HEAD is **not** changed. Use `muse merge` or |
| 311 | `muse pull` to integrate the fetched commits. |
| 312 | |
| 313 | **Examples:** |
| 314 | |
| 315 | ```bash |
| 316 | muse fetch |
| 317 | muse fetch origin |
| 318 | muse fetch origin --branch dev |
| 319 | ``` |
| 320 | |
| 321 | --- |
| 322 | |
| 323 | ### muse pull |
| 324 | |
| 325 | Fetch from a remote and merge into the current branch. |
| 326 | |
| 327 | ``` |
| 328 | muse pull [<remote>] [options] |
| 329 | ``` |
| 330 | |
| 331 | **Options:** |
| 332 | |
| 333 | | Flag | Description | |
| 334 | |------|-------------| |
| 335 | | `<remote>` | Remote name (default: `origin`) | |
| 336 | | `--branch <b>` / `-b <b>` | Remote branch to pull (default: tracked branch or current branch) | |
| 337 | | `--no-merge` | Stop after fetch; do not merge | |
| 338 | | `-m <msg>` / `--message <msg>` | Override the merge commit message | |
| 339 | |
| 340 | **Merge behaviour:** |
| 341 | |
| 342 | - Fast-forward: if the remote HEAD is a direct descendant of local HEAD, the |
| 343 | local branch ref and `muse-work/` are advanced without a merge commit. |
| 344 | - Three-way merge: delegates to the active domain plugin's `merge()` / |
| 345 | `merge_ops()` — identical to `muse merge`. |
| 346 | - Conflicts: MERGE_STATE.json is written; the user fixes conflicts then runs |
| 347 | `muse commit`. |
| 348 | |
| 349 | **Examples:** |
| 350 | |
| 351 | ```bash |
| 352 | muse pull |
| 353 | muse pull origin --branch dev |
| 354 | muse pull origin --no-merge # equivalent to muse fetch |
| 355 | ``` |
| 356 | |
| 357 | --- |
| 358 | |
| 359 | ### muse push |
| 360 | |
| 361 | Upload local commits, snapshots, and objects to a remote. |
| 362 | |
| 363 | ``` |
| 364 | muse push [<remote>] [options] |
| 365 | ``` |
| 366 | |
| 367 | **Options:** |
| 368 | |
| 369 | | Flag | Description | |
| 370 | |------|-------------| |
| 371 | | `<remote>` | Remote name (default: `origin`) | |
| 372 | | `--branch <b>` / `-b <b>` | Branch to push (default: tracked branch or current branch) | |
| 373 | | `-u` / `--set-upstream` | Record `<remote>/<branch>` as the upstream for this branch | |
| 374 | | `--force` | Force-push even if the remote has diverged | |
| 375 | |
| 376 | **Fast-forward enforcement:** by default, MuseHub rejects a push if its current |
| 377 | HEAD is not an ancestor of the local HEAD (HTTP 409). Pass `--force` to |
| 378 | override. Use `muse pull` first to integrate remote changes before pushing. |
| 379 | |
| 380 | **First push workflow:** |
| 381 | |
| 382 | ```bash |
| 383 | muse push origin -u # push + record origin/main as upstream |
| 384 | ``` |
| 385 | |
| 386 | Subsequent pushes on the same branch: |
| 387 | |
| 388 | ```bash |
| 389 | muse push # infers remote and branch from upstream config |
| 390 | ``` |
| 391 | |
| 392 | --- |
| 393 | |
| 394 | ### muse plumbing ls-remote |
| 395 | |
| 396 | List branch references on a remote repository. **Plumbing command** — no local |
| 397 | state is written. |
| 398 | |
| 399 | ``` |
| 400 | muse plumbing ls-remote [<remote-or-url>] [--json] |
| 401 | ``` |
| 402 | |
| 403 | **Options:** |
| 404 | |
| 405 | | Flag | Description | |
| 406 | |------|-------------| |
| 407 | | `<remote-or-url>` | Named remote or a full HTTPS URL (default: `origin`) | |
| 408 | | `--json` | Emit structured JSON instead of tab-delimited text | |
| 409 | |
| 410 | **Default output** (tab-delimited, `*` marks default branch): |
| 411 | |
| 412 | ``` |
| 413 | abc123def456... main * |
| 414 | 789012abc345... dev |
| 415 | ``` |
| 416 | |
| 417 | **JSON output** (for agent consumption): |
| 418 | |
| 419 | ```json |
| 420 | { |
| 421 | "repo_id": "<uuid>", |
| 422 | "domain": "midi", |
| 423 | "default_branch": "main", |
| 424 | "branches": { |
| 425 | "main": "<commit_id>", |
| 426 | "dev": "<commit_id>" |
| 427 | } |
| 428 | } |
| 429 | ``` |
| 430 | |
| 431 | **Examples:** |
| 432 | |
| 433 | ```bash |
| 434 | muse plumbing ls-remote |
| 435 | muse plumbing ls-remote upstream |
| 436 | muse plumbing ls-remote https://hub.muse.io/repos/r1 |
| 437 | muse plumbing ls-remote --json origin |
| 438 | ``` |
| 439 | |
| 440 | --- |
| 441 | |
| 442 | ## Tracking Branches |
| 443 | |
| 444 | Muse tracks the relationship between local branches and remote branches in two |
| 445 | places: |
| 446 | |
| 447 | 1. **Upstream config** — `.muse/config.toml`: |
| 448 | ```toml |
| 449 | [remotes.origin] |
| 450 | url = "https://hub.muse.io/repos/my-song" |
| 451 | branch = "main" # local branch "main" tracks origin/main |
| 452 | ``` |
| 453 | Written by `muse push -u` or `muse clone`. |
| 454 | |
| 455 | 2. **Remote tracking heads** — `.muse/remotes/<name>/<branch>`: |
| 456 | Plain-text files containing the last-known commit ID for each remote branch. |
| 457 | Written by `muse fetch`, `muse pull`, and `muse push`. |
| 458 | |
| 459 | When a tracking relationship is set, `muse push` and `muse pull` resolve the |
| 460 | remote and branch automatically without additional arguments. |
| 461 | |
| 462 | --- |
| 463 | |
| 464 | ## Token Lifecycle |
| 465 | |
| 466 | 1. Obtain a bearer token via MuseHub (`POST /auth/token` or the web UI). |
| 467 | 2. Store it in `.muse/config.toml`: |
| 468 | ```toml |
| 469 | [auth] |
| 470 | token = "muse_tok_..." |
| 471 | ``` |
| 472 | 3. Add `.muse/config.toml` to `.gitignore` and `.museignore`: |
| 473 | ```toml |
| 474 | # .museignore [global] |
| 475 | patterns = [".muse/config.toml"] |
| 476 | ``` |
| 477 | 4. The token is automatically picked up by `fetch`, `pull`, `push`, and |
| 478 | `ls-remote` via `muse.cli.config.get_auth_token()`. |
| 479 | 5. The token value is **never** written to any log line — only `"Bearer ***"` |
| 480 | appears in debug logs. |
| 481 | |
| 482 | --- |
| 483 | |
| 484 | *See also:* |
| 485 | - [`docs/reference/museignore.md`](museignore.md) — domain-scoped ignore rules |
| 486 | - [`docs/reference/muse-attributes.md`](muse-attributes.md) — merge strategy overrides |
| 487 | - [`docs/architecture/muse-vcs.md`](../architecture/muse-vcs.md) — system architecture |