gabriel / muse public
feat patch task/phase14-server-side-object-streaming #3 / 3
AI Agent gabriel · 151 days ago · Apr 23, 2026 · Diff

feat: Phase 14 plan + TDD — server-side object streaming (OC frames, no presign)

Spec out the architectural fix for Walls 1-4/18/21/23: eliminate all client-side R2 PUTs by streaming every object through the gRPC batch path. Server receives O/OC frames and writes to R2 internally.

Phase 14A: no presign for objects ≤ 290 KB (client-only change) Phase 14B: OC-frame wire protocol for arbitrarily large objects Phase 14C: delete presign path entirely from client + server

Added: - docs/protocol/musewire-performance.md: Phase 14 multiphase plan, updated inviolable rules (rule 3 now targets presign elimination) - tests/test_wire_phase14a.py: 6 failing TDD tests for P14A-0..3 - tests/test_wire_oc_frames.py: 22 failing TDD tests for P14B-0..6 - tests/test_wire_h2_wall22.py: 8 passing tests for Wall 22 fix

All 27 new Phase 14 tests fail correctly (red state confirmed).

sha256:d434f75fc4ce3b620a688a4469ba03b4adc78c4fb058c77dbf6074636be9b8b1 sha
sha256:88ee40ee9027a252c9a1de53da9e10cf5179569a35725ad09aa6553fe1401e9f parent
+109 ~43 symbols
sha256:d52743ce16f24889a0eff6d77f87f295b6bba5e914525f16d0742ca422b378de snapshot
+109
symbols added
~43
symbols modified
0
dead code introduced
Semantic Changes 152 symbols
~ tests/test_wire_h2_wall22.py .py 26 symbols added
+ TestW22A0RotationIntervalExported class class TestW22A0RotationIntervalExported L84–92
+ test_conn_rotation_interval_is_1 method method test_conn_rotation_interval_is_1 L90–92
+ test_conn_rotation_interval_is_positive_int method method test_conn_rotation_interval_is_positive_int L85–88
+ TestW22A1ClientRotationCount class class TestW22A1ClientRotationCount L100–167
+ _run_with_n_batches method method _run_with_n_batches L103–145
+ test_four_batches_opens_two_clients method method test_four_batches_opens_two_clients L154–157
+ test_nine_batches_opens_three_clients method method test_nine_batches_opens_three_clients L159–162
+ test_one_batch_opens_one_client method method test_one_batch_opens_one_client L147–148
+ test_ten_batches_opens_four_clients method method test_ten_batches_opens_four_clients L164–167
+ test_three_batches_opens_one_client method method test_three_batches_opens_one_client L150–152
+ TestW22A3FinalResult class class TestW22A3FinalResult L175–221
+ test_result_is_from_last_batch method method test_result_is_from_last_batch L176–221
+ _grpc_frame function function _grpc_frame L42–44
+ _make_batch function function _make_batch L75–76
+ _mock_client function function _mock_client L67–72
+ _mock_stream_response function function _mock_stream_response L52–64
+ _result_frame function function _result_frame L47–49
+ AsyncMock import import AsyncMock L29–29
+ MagicMock import import MagicMock L29–29
+ PushResult import import PushResult L34–34
+ annotations import import annotations L26–26
+ call import import call L29–29
+ math import import math L28–28
+ msgpack import import msgpack L31–31
+ patch import import patch L29–29
+ pytest import import pytest L32–32
~ tests/test_wire_oc_frames.py .py 35 symbols added
+ TestP14B0ChunkSizeExported class class TestP14B0ChunkSizeExported L37–45
+ test_oc_chunk_size_equals_290kb method method test_oc_chunk_size_equals_290kb L43–45
+ test_oc_chunk_size_is_positive_int method method test_oc_chunk_size_is_positive_int L38–41
+ TestP14B1ExactSizeNoChunking class class TestP14B1ExactSizeNoChunking L52–79
+ test_object_at_chunk_size_is_single_o_frame method method test_object_at_chunk_size_is_single_o_frame L53–66
+ test_object_below_chunk_size_is_single_o_frame method method test_object_below_chunk_size_is_single_o_frame L68–79
+ TestP14B2TwoChunks class class TestP14B2TwoChunks L86–121
+ _get_frames method method _get_frames L87–97
+ test_all_frames_are_oc_type method method test_all_frames_are_oc_type L104–108
+ test_each_frame_content_le_chunk_size method method test_each_frame_content_le_chunk_size L110–114
+ test_frame_content_sums_to_original method method test_frame_content_sums_to_original L116–121
+ test_one_byte_over_chunk_size_produces_two_frames method method test_one_byte_over_chunk_size_produces_two_frames L99–102
+ TestP14B3ThreeChunks class class TestP14B3ThreeChunks L128–186
+ test_chunk_indices_are_sequential method method test_chunk_indices_are_sequential L173–186
+ test_three_chunk_size_object_produces_three_frames method method test_three_chunk_size_object_produces_three_frames L129–141
+ test_three_chunks_reassemble_correctly method method test_three_chunks_reassemble_correctly L143–156
+ test_total_chunks_field_is_correct method method test_total_chunks_field_is_correct L158–171
+ TestP14B4MetadataOnFirstChunkOnly class class TestP14B4MetadataOnFirstChunkOnly L193–233
+ _frames_for_large_object method method _frames_for_large_object L194–204
+ test_first_chunk_has_encoding method method test_first_chunk_has_encoding L206–208
+ test_first_chunk_has_path method method test_first_chunk_has_path L210–212
+ test_first_chunk_has_sz method method test_first_chunk_has_sz L214–218
+ test_subsequent_chunks_have_no_encoding method method test_subsequent_chunks_have_no_encoding L220–223
+ test_subsequent_chunks_have_no_path method method test_subsequent_chunks_have_no_path L225–228
+ test_subsequent_chunks_have_no_sz method method test_subsequent_chunks_have_no_sz L230–233
+ TestP14B5FunctionExported class class TestP14B5FunctionExported L240–256
+ test_function_is_callable method method test_function_is_callable L241–243
+ test_returns_list_of_dicts method method test_returns_list_of_dicts L245–256
+ TestP14B6FrameFields class class TestP14B6FrameFields L263–297
+ test_all_oc_frames_have_required_fields method method test_all_oc_frames_have_required_fields L264–282
+ test_last_frame_chunk_index_equals_total_minus_one method method test_last_frame_chunk_index_equals_total_minus_one L284–297
+ Any import import Any L28–28
+ annotations import import annotations L25–25
+ math import import math L27–27
+ pytest import import pytest L30–30
~ tests/test_wire_phase14a.py .py 24 symbols added
+ TestP14A0NoPresignForSmallObjects class class TestP14A0NoPresignForSmallObjects L70–121
+ _push_objects method method _push_objects L73–105
+ test_1kb_objects_no_presign method method test_1kb_objects_no_presign L107–110
+ test_290kb_objects_no_presign method method test_290kb_objects_no_presign L117–121
+ test_50kb_objects_no_presign method method test_50kb_objects_no_presign L112–115
+ TestP14A1FiveHundredSmallObjects class class TestP14A1FiveHundredSmallObjects L128–165
+ test_500_small_objects_no_presign_result_ok method method test_500_small_objects_no_presign_result_ok L129–165
+ TestP14A2HundredKBObjects class class TestP14A2HundredKBObjects L172–207
+ test_multiple_batches_no_presign method method test_multiple_batches_no_presign L173–207
+ TestP14A3BatchByteBoundary class class TestP14A3BatchByteBoundary L214–251
+ test_object_at_batch_boundary_goes_inline method method test_object_at_batch_boundary_goes_inline L217–251
+ _make_objects function function _make_objects L45–56
+ _mock_transport function function _mock_transport L59–63
+ _ok_result function function _ok_result L41–42
+ AsyncMock import import AsyncMock L29–29
+ MagicMock import import MagicMock L29–29
+ PushResult import import PushResult L34–34
+ annotations import import annotations L25–25
+ asyncio import import asyncio L27–27
+ call import import call L29–29
+ msgpack import import msgpack L31–31
+ os import import os L28–28
+ patch import import patch L29–29
+ pytest import import pytest L32–32
~ docs/protocol/musewire-performance.md .md 14 symbols added, 37 symbols modified
~ code[python]@L213
~ code@L198
~ table@L202
~ code@L409
~ table@L170
~ code@L113
~ code[bash]@L133
~ code[bash]@L148
~ code[bash]@L160
~ table@L67
~ code[bash]@L89
~ code@L37
~ code@L612
~ code@L634
~ code@L656
~ code@L662
~ code@L695
~ code@L714
~ code@L735
~ code@L770
~ code[python]@L831
~ code@L871
~ code@L878
~ code@L888
~ code@L899
~ code@L928
~ code@L1013
~ code@L1048
~ code@L1076
~ code[bash]@L1108
~ code@L511
~ code@L558
+ Phase 14 — Server-Side Object Streaming (GitHub Architecture Parity) section Phase 14 — Server-Side Object Streaming (GitHub Architecture Parity) L186–343
+ Phase 14A — Inline streaming: no presign for objects ≤ 290 KB section Phase 14A — Inline streaming: no presign for objects ≤ 290 KB L232–261
+ table@L251 section table@L251 L251–258
+ Phase 14B — OC-frame protocol: chunked large objects through the stream section Phase 14B — OC-frame protocol: chunked large objects through the stream L261–318
+ code@L267 variable variable code@L267 L267–279
+ table@L302 section table@L302 L302–315
+ Phase 14C — Verify and clean up section Phase 14C — Verify and clean up L318–343
+ table@L322 section table@L322 L322–329
+ The fix section The fix L208–232
+ The problem section The problem L188–208
+ code@L1098 variable variable code@L1098 L1098–1101
+ Wall 22 — H2 RST_STREAM after ~4 batches on shared AsyncClient section Wall 22 — H2 RST_STREAM after ~4 batches on shared AsyncClient L1324–1358
+ code@L1328 variable variable code@L1328 L1328–1331
+ Wall 23 — SSLV3_ALERT_BAD_RECORD_MAC on 2nd batch of 2nd rotation group section Wall 23 — SSLV3_ALERT_BAD_RECORD_MAC on 2nd batch of 2nd rotation group L1358–1402
~ MuseWire Protocol — Explicit Frame Framing Roadmap
~ Inviolable rules
~ Test Pyramid — The Only Way Forward
~ Current status
~ Wall log
~ muse/cli/commands/push.py .py 9 symbols added, 5 symbols modified
+ CONN_ROTATION_INTERVAL variable variable CONN_ROTATION_INTERVAL L118–118
+ _make_push_ssl_ctx function function _make_push_ssl_ctx L127–130
+ _run_batches_with_rotation function async_function _run_batches_with_rotation L133–192
+ _PUSH_TIMEOUT import import _PUSH_TIMEOUT L123–123
+ _asyncio import import _asyncio L120–120
+ _httpx import import _httpx L122–122
+ _ignore_sigpipe import import _ignore_sigpipe L103–103
+ _ssl import import _ssl L121–121
+ _time_mod import import _time_mod L124–124
~ _push_stream
~ MuseTransport
~ SigningIdentity
~ TransportError
~ make_transport
~ muse/core/transport.py .py 1 symbol added, 1 symbol modified
+ push_stream_coro method async_method push_stream_coro L1148–1253
~ HttpTransport
← Older Oldest on task/phase14-server-side-object-streaming
All commits Newer feat: Phase 14A/B — inline object streaming, OC-frame protocol sha256:e87b350357f084c4e4fcfc74abc557a4809ff56259424df7c013ad7fd8b0dac1

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:d434f75fc4ce3b620a688a4469ba03b4adc78c4fb058c77dbf6074636be9b8b1 --body "your comment"