gabriel / muse public
feat BREAKING task/wire-frame-protocol #1 / 1
AI Agent gabriel · 152 days ago · Apr 22, 2026 · Diff

feat: implement explicit wire frame protocol with magic bytes and content-addressed envelopes

- Add MuseWireFrameWriter with magic b'muse', version 0x01, and envelope headers - Add WireFrameError and WIRE_CONTENT_TYPE to muse.core.mpack - Add iter_wire_frames async generator to transport for server-side frame parsing - Fix C frame always emitted even when commits=[] in both chunked and httpx paths - Add comprehensive wire framing tests (test_wire_framing.py) - Update musewire-performance.md with full protocol spec and wall log

sha256:f52a04f0f9bcdd7fbbaa7a47de9590c17781d629b9db8f6cbf399529f5ec0976 sha
sha256:2fa41bb1cc623a6667eeb7e747839309ee78cb0e596bf7f92927d538abf61f45 parent
+77 ~12 −35 symbols
sha256:717578082d3e6b6e4fa6e8fa925811a10fc45501d9352fbe6d997dcd9c2dce94 snapshot
+77
symbols added
~12
symbols modified
−35
symbols removed
0
dead code introduced
Semantic Changes 124 symbols
~ tests/test_wire_framing.py .py 37 symbols added
+ TestEnvelopeLogicalMismatch class class TestEnvelopeLogicalMismatch L280–311
+ test_can_detect_type_mismatch method method test_can_detect_type_mismatch L293–311
+ test_matching_types_are_consistent method method test_matching_types_are_consistent L283–291
+ TestHashMismatch class class TestHashMismatch L201–232
+ _tamper method method _tamper L204–206
+ test_tampered_frame_has_wrong_hash method method test_tampered_frame_has_wrong_hash L208–221
+ test_untampered_frame_hash_matches method method test_untampered_frame_hash_matches L223–232
+ TestSizeMismatch class class TestSizeMismatch L239–273
+ test_envelope_sz_matches_payload method method test_envelope_sz_matches_payload L242–250
+ test_sz_mismatch_detected method method test_sz_mismatch_detected L252–273
+ TestWall5Regression class class TestWall5Regression L318–379
+ _build_c_frame method method _build_c_frame L321–329
+ test_no_map32_from_truncation_at_various_offsets method method test_no_map32_from_truncation_at_various_offsets L352–379
+ test_truncation_at_magic_detected_cleanly method method test_truncation_at_magic_detected_cleanly L331–336
+ test_truncation_mid_payload_gives_wrong_length method method test_truncation_mid_payload_gives_wrong_length L338–350
+ TestWireFrameWriterRoundtrip class class TestWireFrameWriterRoundtrip L67–194
+ test_empty_payload method method test_empty_payload L147–157
+ test_import method method test_import L70–71
+ test_large_payload_size_field method method test_large_payload_size_field L135–145
+ test_total_frame_byte_length method method test_total_frame_byte_length L184–194
+ test_wire_content_type_import method method test_wire_content_type_import L76–78
+ test_wire_frame_error_import method method test_wire_frame_error_import L73–74
+ test_wrap_commit_pack_frame method method test_wrap_commit_pack_frame L112–121
+ test_wrap_end_frame method method test_wrap_end_frame L123–133
+ test_wrap_header_frame method method test_wrap_header_frame L80–95
+ test_wrap_object_frame method method test_wrap_object_frame L97–110
+ test_wrap_produces_exact_layout method method test_wrap_produces_exact_layout L159–182
+ _decode_envelope function function _decode_envelope L51–60
+ _make_envelope function function _make_envelope L36–48
+ _pack function function _pack L28–29
+ _sha256_oid function function _sha256_oid L24–25
+ _unpack function function _unpack L32–33
+ annotations import import annotations L12–12
+ hashlib import import hashlib L14–14
+ msgpack import import msgpack L17–17
+ pytest import import pytest L18–18
+ struct import import struct L15–15
~ docs/protocol/musewire-performance.md .md 34 symbols added, 35 symbols removed, 1 symbol modified
~ code@L404
− MuseWire Protocol — GitHub-Level Performance Checklist section MuseWire Protocol — GitHub-Level Performance Checklist L1–509
− py) section Checklist — Client side (muse/core/transport.py) L71–155
− Connection / TLS section Connection / TLS L136–145
− HTTP library section HTTP library L73–87
− Large-object presigned upload path section Large-object presigned upload path L145–155
− fetch_remote_info (refs) section fetch_remote_info (refs) L124–128
− fetch_stream section fetch_stream L109–119
− negotiate section negotiate L119–124
− presign_objects section presign_objects L128–132
− push_stream / _push_stream_httpx section push_stream / _push_stream_httpx L87–109
− push_tags, create_release, list_releases_remote, delete_* section push_tags, create_release, list_releases_remote, delete_* L132–136
− Checklist — Commands that touch MuseWire section Checklist — Commands that touch MuseWire L196–213
− table@L198 section table@L198 L198–208
− py) section Checklist — Server side (musehub/musehub/api/routes/wire.py) L155–196
− Cloudflare configuration section Cloudflare configuration L182–196
− fetch_stream endpoint section fetch_stream endpoint L169–177
− presign_objects endpoint section presign_objects endpoint L177–182
− push_stream endpoint section push_stream endpoint L157–169
− Current architecture diagram section Current architecture diagram L47–71
− code@L49 variable variable code@L49 L49–68
− Done criteria section Done criteria L281–296
− Implementation order section Implementation order L253–281
− Inviolable rules section Inviolable rules L23–47
− Known issue — Cloudflare connection staleness (RESOLVED in client) section Known issue — Cloudflare connection staleness (RESOLVED in client) L213–238
− code@L219 variable variable code@L219 L219–222
− Wall log — errors hit in order, TDD items attached section Wall log — errors hit in order, TDD items attached L296–509
− Wall 1 — SSLV3 during R2 presigned PUTs (FIXED) section Wall 1 — SSLV3 during R2 presigned PUTs (FIXED) L303–318
− Wall 2 — int is not allowed for map key when strict_map_key=True (INVALIDATED) section Wall 2 — int is not allowed for map key when strict_map_key=True (INVALIDATED) L318–336
− Wall 3 — SSLV3 on R2 PUT even with no keep-alive (ACTIVE) section Wall 3 — SSLV3 on R2 PUT even with no keep-alive (ACTIVE) L336–369
− Wall 4 — SSLV3 persists with fresh client per PUT; ALPN HTTP/2 hypothesis (ACTIVE) section Wall 4 — SSLV3 persists with fresh client per PUT; ALPN HTTP/2 hypothesis (ACTIVE) L369–400
− Wall 5 — stream read error: 2662457625 exceeds max_map_len(1073741824) (ACTIVE) section Wall 5 — stream read error: 2662457625 exceeds max_map_len(1073741824) (ACTIVE) L400–475
− Wall 6 — SSLV3ALERTBADRECORDMAC returns on R2 PUT #14 (ACTIVE) section Wall 6 — SSLV3ALERTBADRECORDMAC returns on R2 PUT #14 (ACTIVE) L475–509
− What GitHub actually does (the reference model) section What GitHub actually does (the reference model) L238–253
− table@L240 section table@L240 L240–250
− Why we keep churning on this section Why we keep churning on this L9–23
+ MuseWire Protocol — Explicit Frame Framing Roadmap section MuseWire Protocol — Explicit Frame Framing Roadmap L1–353
+ Architecture: Explicit Content-Addressed Frame Framing section Architecture: Explicit Content-Addressed Frame Framing L16–65
+ Envelope fields section Envelope fields L46–56
+ code[python]@L48 variable variable code[python]@L48 L48–55
+ Key invariants section Key invariants L56–65
+ The fix: one tight envelope around the existing protocol section The fix: one tight envelope around the existing protocol L26–31
+ The problem we are solving (Wall 5) section The problem we are solving (Wall 5) L18–26
+ Wire format section Wire format L31–46
+ code@L33 variable variable code@L33 L33–36
+ table@L37 section table@L37 L37–45
+ Current test status section Current test status L157–169
+ code@L159 variable variable code@L159 L159–166
+ Done criteria section Done criteria L344–353
+ Implementation phases section Implementation phases L65–157
+ Phase 1 — Client writer ✅ DONE section Phase 1 — Client writer ✅ DONE L67–75
+ Phase 2 — Client writer tests ✅ DONE section Phase 2 — Client writer tests ✅ DONE L75–85
+ Phase 3 — Client transport ✅ DONE section Phase 3 — Client transport ✅ DONE L85–96
+ Phase 4 — Server reader ✅ DONE section Phase 4 — Server reader ✅ DONE L96–107
+ Phase 5 — Server reader tests ✅ DONE section Phase 5 — Server reader tests ✅ DONE L107–118
+ Phase 6 — Integration tests (T1–T6) ✅ DONE section Phase 6 — Integration tests (T1–T6) ✅ DONE L118–129
+ Phase 7 — ASGI route tests (T7–T10) 🔴 BLOCKED (Wall 7) section Phase 7 — ASGI route tests (T7–T10) 🔴 BLOCKED (Wall 7) L129–139
+ Phase 8 — Commit and push both repos ⬜ PENDING section Phase 8 — Commit and push both repos ⬜ PENDING L139–148
+ Phase 9 — Deploy and verify ⬜ PENDING section Phase 9 — Deploy and verify ⬜ PENDING L148–157
+ Inviolable rules section Inviolable rules L169–182
+ Wall log section Wall log L182–344
+ Wall 1 — SSLV3 during R2 presigned PUTs (FIXED) section Wall 1 — SSLV3 during R2 presigned PUTs (FIXED) L189–203
+ Wall 2 — int is not allowed for map key when strict_map_key=True (INVALIDATED) section Wall 2 — int is not allowed for map key when strict_map_key=True (INVALIDATED) L203–214
+ Wall 3 — SSLV3 on R2 PUT even with no keep-alive (FIXED) section Wall 3 — SSLV3 on R2 PUT even with no keep-alive (FIXED) L214–232
+ Wall 4 — SSLV3 persists with fresh client per PUT (FIXED) section Wall 4 — SSLV3 persists with fresh client per PUT (FIXED) L232–247
+ Wall 5 — 2662457625 exceeds max_map_len(1073741824) (FIXED) section Wall 5 — 2662457625 exceeds max_map_len(1073741824) (FIXED) L247–278
+ Wall 6 — SSLV3 returns on R2 PUT #14 (PENDING RETRY) section Wall 6 — SSLV3 returns on R2 PUT #14 (PENDING RETRY) L278–294
+ Wall 7 — T7+ ASGI route tests hang (ACTIVE) section Wall 7 — T7+ ASGI route tests hang (ACTIVE) L294–344
+ code[python]@L323 variable variable code[python]@L323 L323–327
+ Why this doc exists section Why this doc exists L8–16
~ muse/core/mpack.py .py 5 symbols added
+ MuseWireFrameWriter class class MuseWireFrameWriter L758–798
+ wrap method method wrap L779–798
+ WIRE_CONTENT_TYPE variable variable WIRE_CONTENT_TYPE L105–105
+ WireFrameError class class WireFrameError L754–755
+ struct import import struct L94–94
~ muse/core/transport.py .py 3 symbols modified
~ HttpTransport
~ _push_stream_httpx
~ push_stream
~ tests/test_mpack_transport.py .py 1 symbol added, 8 symbols modified
+ WIRE_CONTENT_TYPE import import WIRE_CONTENT_TYPE L29–29
~ TestPushStreamContentType
~ test_push_stream_accept_uses_mpack_content_type
~ test_push_stream_uses_mpack_content_type
~ _decode_push_frames
~ MPACK_CONTENT_TYPE
~ MPACK_VERSION
~ MPackStreamReader
~ MPackStreamWriter
← Older Oldest on task/wire-frame-protocol
All commits
Newer → Latest on task/wire-frame-protocol

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:f52a04f0f9bcdd7fbbaa7a47de9590c17781d629b9db8f6cbf399529f5ec0976 --body "your comment"