gabriel / muse public
feat BREAKING wire task/w16-snapshot-delta #1 / 1
AI Agent gabriel · 152 days ago · Apr 23, 2026 · Diff

feat(wire): snapshot delta encoding in push stream

Wall 16 root cause: C frames sent full 1071-entry snapshot manifests for every commit (50 commits × 128KB = ~4MB per frame), exceeding CF's gRPC per-message limit of ~248KB.

Fix: delta-encode snapshots in _frame_generator. - First snapshot in stream: full manifest - Each subsequent: delta(added, removed) against previous - Frame size now proportional to what changed (2-5 files ≈ 1KB per delta) - Server reconstructs full manifests via stream-local _snap_manifest_cache

New: compute_snapshot_delta, apply_snapshot_delta in muse.core.mpack New: write_commit_pack accepts snapshot_deltas=[...] Tests: test_snapshot_delta.py (10), test_wire_snapshot_delta.py (8)

sha256:5a2df6e01f1482d1c058a7a5b12d1adad04c1985114ab930377398edb786640e sha
sha256:b12ec62424e850b9fd36ea129f3aa715801c2166773b5c4d0092de2ea3fb1023 parent
+35 ~8 −1 symbols
sha256:dc19dd1ca110866686432bdca8480a6dca90450aecfe47330ec04b8777ed6104 snapshot
+35
symbols added
~8
symbols modified
−1
symbol removed
0
dead code introduced
Semantic Changes 44 symbols
~ tests/test_snapshot_delta.py .py 12 symbols added
+ _make_manifest function function _make_manifest L20–25
+ annotations import import annotations L17–17
+ test_delta_add_one_file function function test_delta_add_one_file L44–51
+ test_delta_from_empty_base function function test_delta_from_empty_base L161–168
+ test_delta_identity_is_empty function function test_delta_identity_is_empty L32–37
+ test_delta_mixed_operations function function test_delta_mixed_operations L87–100
+ test_delta_modify_one_file function function test_delta_modify_one_file L58–66
+ test_delta_remove_one_file function function test_delta_remove_one_file L73–80
+ test_delta_roundtrip function function test_delta_roundtrip L107–120
+ test_delta_roundtrip_large_repo function function test_delta_roundtrip_large_repo L123–133
+ test_delta_smaller_than_full_for_typical_commit function function test_delta_smaller_than_full_for_typical_commit L140–154
+ test_delta_to_empty_manifest function function test_delta_to_empty_manifest L175–182
~ tests/test_wire_snapshot_delta.py .py 19 symbols added
+ GRPC_LIMIT variable variable GRPC_LIMIT L23–23
+ _build_linear_history function function _build_linear_history L85–108
+ _build_manifest function function _build_manifest L51–52
+ _decode_grpc_frames function function _decode_grpc_frames L26–37
+ _make_commit function function _make_commit L55–69
+ _make_snapshot function function _make_snapshot L72–82
+ _parse_mwp_payload function function _parse_mwp_payload L40–48
+ annotations import import annotations L12–12
+ msgpack import import msgpack L15–15
+ pytest import import pytest L16–16
+ struct import import struct L14–14
+ test_c_frames_under_grpc_limit_large_repo function function test_c_frames_under_grpc_limit_large_repo L156–172
+ test_delta_chain_full_reconstruction function function test_delta_chain_full_reconstruction L254–283
+ test_delta_chain_references_valid_base function function test_delta_chain_references_valid_base L179–197
+ test_first_snapshot_is_full function function test_first_snapshot_is_full L115–127
+ test_single_commit_sends_full_snapshot_no_deltas function function test_single_commit_sends_full_snapshot_no_deltas L204–215
+ test_subsequent_snapshots_are_deltas function function test_subsequent_snapshots_are_deltas L134–149
+ test_write_commit_pack_includes_snapshot_deltas function function test_write_commit_pack_includes_snapshot_deltas L222–239
+ test_write_commit_pack_no_deltas_key_when_empty function function test_write_commit_pack_no_deltas_key_when_empty L242–247
~ docs/protocol/musewire-performance.md .md 2 symbols added, 1 symbol removed, 3 symbols modified
~ code@L537
− Wall 16 — Large C frame (~4MB commits) truncated at ~248KB mid-stream 🔴 ACTIVE section Wall 16 — Large C frame (~4MB commits) truncated at ~248KB mid-stream 🔴 ACTIVE L532–606
+ Wall 16 — Large C frame (~4MB commits) truncated at ~248KB mid-stream ✅ FIXED section Wall 16 — Large C frame (~4MB commits) truncated at ~248KB mid-stream ✅ FIXED L532–634
+ code[python]@L598 variable variable code[python]@L598 L598–615
~ MuseWire Protocol — Explicit Frame Framing Roadmap
~ Wall log
~ muse/core/mpack.py .py 2 symbols added, 2 symbols modified
+ apply_snapshot_delta function function apply_snapshot_delta L140–154
+ compute_snapshot_delta function function compute_snapshot_delta L121–137
~ MPackStreamWriter
~ write_commit_pack
~ muse/core/transport.py .py 3 symbols modified
~ HttpTransport
~ push_stream
~ _frame_generator
← Older Oldest on task/w16-snapshot-delta
All commits
Newer → Latest on task/w16-snapshot-delta

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:5a2df6e01f1482d1c058a7a5b12d1adad04c1985114ab930377398edb786640e --body "your comment"