gabriel / muse public
feat BREAKING mpack task/mpack-protocol #2 / 4
AI Agent gabriel · 163 days ago · Apr 17, 2026 · Diff

feat(mpack): true chunked HTTP/1.1 streaming for push_stream — eliminate b''.join bottleneck

- Add _open_chunked_connection() module-level function as the testable seam for http.client.HTTPConnection/HTTPSConnection (mirrors _open_url pattern) - Add _send_chunked_frame() and _send_terminal_chunk() helpers that encode each frame as '{size_hex}\r\n{data}\r\n' and '0\r\n\r\n' respectively - Refactor HttpTransport.push_stream to stream frames incrementally: · H frame built first; Authorization signed over H frame bytes only · O frames compressed and sent one at a time (peak memory O(1 object)) · Transfer-Encoding: chunked, no Content-Length, no b''.join · Server can validate auth + start writing objects while later frames arrive · Error handling via http.client.HTTPException / OSError (not urllib) · Connection always closed in finally block - MSign signing upgrade: Authorization covers H frame bytes (not full body) · H frame carries embedded Ed25519 over canonical intent · Content-addressed objects self-verify (sha256: prefix) · E frame counts cross-validate stream integrity - tests/test_mpack_chunked_push.py: 30 TDD tests covering Transfer-Encoding, no Content-Length, H-frame signing, per-frame send(), memory boundedness, HTTP vs HTTPS connection selection, connection lifecycle, R/X/P/4xx response - tests/test_mpack_transport.py: updated _capture_push_request helper to use _open_chunked_connection seam instead of _open_url

sha256:262a0a1858ebb8abcff6171d534916c898feb05d265ff3504681581f6eb6fe89 sha
sha256:67412443a7c7cb446b2b87fb467f126a68a1e4b5e33867dd1280ad33e4cb3d6a parent
+81 ~6 symbols
sha256:54a3535088e54b87f0037440aa7a9327a1398975986dfaab373f7fc35313e527 snapshot
+81
symbols added
~6
symbols modified
0
dead code introduced
Semantic Changes 87 symbols
~ tests/test_mpack_chunked_push.py .py 67 symbols added
+ MockHTTPConnection class class MockHTTPConnection L92–125
+ __init__ method method __init__ L95–104
+ close method method close L124–125
+ endheaders method method endheaders L115–116
+ getresponse method method getresponse L121–122
+ putheader method method putheader L112–113
+ putrequest method method putrequest L106–110
+ send method method send L118–119
+ MockHTTPResponse class class MockHTTPResponse L71–89
+ __init__ method method __init__ L74–77
+ getheader method method getheader L88–89
+ read method method read L79–86
+ TestChunkedConnectionType class class TestChunkedConnectionType L490–559
+ _run_with_url method method _run_with_url L491–525
+ test_connection_closed_after_error method method test_connection_closed_after_error L553–559
+ test_connection_closed_after_success method method test_connection_closed_after_success L547–551
+ test_host_extracted_correctly method method test_host_extracted_correctly L537–540
+ test_http_url_uses_use_ssl_false method method test_http_url_uses_use_ssl_false L527–530
+ test_https_url_uses_use_ssl_true method method test_https_url_uses_use_ssl_true L532–535
+ test_port_extracted_correctly method method test_port_extracted_correctly L542–545
+ TestChunkedFrameSequence class class TestChunkedFrameSequence L355–427
+ test_each_object_is_separate_send method method test_each_object_is_separate_send L401–411
+ test_empty_push_has_h_c_e_terminal method method test_empty_push_has_h_c_e_terminal L375–384
+ test_h_frame_first_send method method test_h_frame_first_send L356–364
+ test_o_frames_between_h_and_c method method test_o_frames_between_h_and_c L386–399
+ test_send_count_matches_frame_count method method test_send_count_matches_frame_count L413–427
+ test_terminal_chunk_is_last_send method method test_terminal_chunk_is_last_send L366–373
+ TestChunkedHeaders class class TestChunkedHeaders L207–249
+ test_accept_is_mpack method method test_accept_is_mpack L231–235
+ test_content_type_is_mpack method method test_content_type_is_mpack L225–229
+ test_no_content_length method method test_no_content_length L216–223
+ test_path_ends_with_push_stream method method test_path_ends_with_push_stream L243–249
+ test_transfer_encoding_chunked method method test_transfer_encoding_chunked L208–214
+ test_uses_post_method method method test_uses_post_method L237–241
+ TestChunkedMemoryBounded class class TestChunkedMemoryBounded L435–482
+ test_no_single_send_contains_all_frames method method test_no_single_send_contains_all_frames L436–459
+ test_peak_send_size_bounded_per_object method method test_peak_send_size_bounded_per_object L461–482
+ TestChunkedResponseParsing class class TestChunkedResponseParsing L567–675
+ test_http_4xx_raises_transport_error method method test_http_4xx_raises_transport_error L624–647
+ test_no_result_frame_raises_transport_error method method test_no_result_frame_raises_transport_error L649–675
+ test_p_frame_does_not_raise method method test_p_frame_does_not_raise L612–622
+ test_r_frame_message_preserved method method test_r_frame_message_preserved L577–583
+ test_r_frame_ok_returns_push_result method method test_r_frame_ok_returns_push_result L568–575
+ test_x_frame_raises_transport_error method method test_x_frame_raises_transport_error L585–610
+ TestChunkedSigning class class TestChunkedSigning L257–347
+ test_auth_header_present_when_signed method method test_auth_header_present_when_signed L264–271
+ test_auth_not_signed_over_full_body method method test_auth_not_signed_over_full_body L311–347
+ test_auth_signed_over_h_frame_bytes method method test_auth_signed_over_h_frame_bytes L273–309
+ test_no_auth_header_when_unsigned method method test_no_auth_header_when_unsigned L258–262
+ _decode_mpack_frames function function _decode_mpack_frames L189–199
+ _decode_sends function function _decode_sends L175–186
+ _make_r_frame function function _make_r_frame L66–68
+ _make_signing function function _make_signing L55–57
+ _parse_chunk function function _parse_chunk L60–63
+ _run_push function function _run_push L128–172
+ BytesIO import import BytesIO L37–37
+ HttpTransport import import HttpTransport L47–47
+ MPACK_CONTENT_TYPE import import MPACK_CONTENT_TYPE L42–42
+ MPACK_VERSION import import MPACK_VERSION L42–42
+ MPackStreamWriter import import MPackStreamWriter L42–42
+ SigningIdentity import import SigningIdentity L47–47
+ TransportError import import TransportError L47–47
+ annotations import import annotations L33–33
+ hashlib import import hashlib L35–35
+ msgpack import import msgpack L39–39
+ pytest import import pytest L40–40
+ mock import import unittest.mock L36–36
~ muse/core/transport.py .py 3 symbols added, 2 symbols modified
+ _open_chunked_connection function function _open_chunked_connection L437–464
+ _send_chunked_frame function function _send_chunked_frame L467–480
+ _send_terminal_chunk function function _send_terminal_chunk L483–492
~ HttpTransport
~ push_stream
~ tests/test_mpack_transport.py .py 11 symbols added, 4 symbols modified
+ test_error_frame_status_code method method test_error_frame_status_code L506–529
+ _MockPushConn class class _MockPushConn L67–109
+ __init__ method method __init__ L70–74
+ close method method close L108–109
+ endheaders method method endheaders L82–83
+ getresponse method method getresponse L88–106
+ putheader method method putheader L79–80
+ putrequest method method putrequest L76–77
+ send method method send L85–86
+ _decode_push_frames function function _decode_push_frames L112–128
+ _run_push_via_chunked function function _run_push_via_chunked L185–223
~ TestPushStreamResponseParsing
~ test_error_frame_raises_transport_error
~ test_result_frame_ok_returns_push_result
~ _capture_push_request
Older feat(mpack): wire MPackStreamWriter/Reader into transport push_s… sha256:67412443a7c7cb446b2b87fb467f126a68a1e4b5e33867dd1280ad33e4cb3d6a All commits Newer feat(phase-9): pre-compression fix + e2e integration tests sha256:1c08072d0d612e8ee8687845d596d81b6d4472704a004a32b7d39c348b3f8787

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:262a0a1858ebb8abcff6171d534916c898feb05d265ff3504681581f6eb6fe89 --body "your comment"