gabriel / musehub public
feat patch dev
AI Agent gabriel · 147 days ago · Apr 29, 2026 · Diff

feat: presigned R2 push path for large repos

Add wire_push_presign() service function and POST /push/presign endpoint. For pushes above threshold (500 objects or 50 MB), client gets presigned R2 PUT URLs, uploads directly to R2 bypassing Cloudflare's ~100s timeout, then sends a zero-O-frame push/stream to finalize the commit. The server's existing referential integrity check (Phase C) accepts objects already in storage. LocalBackend returns stream_these for local dev fallback.

Tests: P1-P7 covering LocalBackend, mocked S3, empty list, zero-O-frame push/stream success and failure, full round-trip, and URL key format.

sha256:52b9750b58326265a6b16b945e58ce31ace2cde7a8ad6552085c1bcfd2f20274 sha
sha256:3caced6de13d175d2041ae1c6a53e753dda8ca6b2cc52b8193b6a18ebbe46af6 parent
+45 ~10 symbols
sha256:91840af951fd920ba74bbb25f4a58ca24f0b6fd79fa3af7793c9a98d488cdb10 snapshot
+45
symbols added
~10
symbols modified
0
dead code introduced
Semantic Changes 55 symbols
~ tests/test_wire_presign_push.py .py 42 symbols added
+ _body_iter function async_function _body_iter L192–193
+ _collect_frames function async_function _collect_frames L182–189
+ _commit_pack_frame function function _commit_pack_frame L105–110
+ _end_frame function function _end_frame L97–102
+ _fw variable variable _fw L59–59
+ _header_frame function function _header_frame L82–94
+ _make_commit_and_snapshot function async_function _make_commit_and_snapshot L148–179
+ _now function function _now L66–67
+ _o_frame function function _o_frame L113–121
+ _pack function function _pack L74–75
+ _store_object function async_function _store_object L124–145
+ _uid function function _uid L70–71
+ _wrap function function _wrap L78–79
+ Any import import Any L40–40
+ AsyncMock import import AsyncMock L41–41
+ AsyncSession import import AsyncSession L46–46
+ MagicMock import import MagicMock L41–41
+ MuseWireFrameWriter import import MuseWireFrameWriter L56–56
+ SFRAME_COMMIT_PACK import import SFRAME_COMMIT_PACK L50–50
+ SFRAME_END import import SFRAME_END L50–50
+ SFRAME_HEADER import import SFRAME_HEADER L50–50
+ SFRAME_OBJECT import import SFRAME_OBJECT L50–50
+ annotations import import annotations L35–35
+ asyncio import import asyncio L37–37
+ blob_id import import blob_id L48–48
+ create_repo import import create_repo L57–57
+ datetime import import datetime L39–39
+ db import import db L49–49
+ fake_id import import fake_id L48–48
+ msgpack import import msgpack L43–43
+ patch import import patch L41–41
+ pg_insert import import pg_insert L45–45
+ pytest import import pytest L44–44
+ timezone import import timezone L39–39
+ zlib import import zlib L38–38
+ test_p1_local_backend_missing_goes_to_stream_these function async_function test_p1_local_backend_missing_goes_to_stream_these L201–230
+ test_p2_s3_backend_missing_gets_presigned_url function async_function test_p2_s3_backend_missing_gets_presigned_url L238–264
+ test_p3_empty_object_list function async_function test_p3_empty_object_list L272–281
+ test_p4_zero_o_frames_all_prestored_commit_finalises function async_function test_p4_zero_o_frames_all_prestored_commit_finalises L289–340
+ test_p5_zero_o_frames_missing_objects_yields_422 function async_function test_p5_zero_o_frames_missing_objects_yields_422 L348–391
+ test_p6_full_round_trip_presign_store_push function async_function test_p6_full_round_trip_presign_store_push L399–459
+ test_p7_s3_backend_presign_batch_url_format function async_function test_p7_s3_backend_presign_batch_url_format L467–494
~ musehub/api/routes/wire.py .py 2 symbols added, 10 symbols modified
+ wire_push_presign import import wire_push_presign L49–49
+ push_presign function async_function push_presign L429–465
~ ObjectHashMismatch
~ PolyglotObjectError
~ wire_fetch_objects
~ wire_fetch_stream
~ wire_negotiate
~ wire_push_object_pack
~ wire_push_stream
~ wire_refs
~ wire_repair_object
~ wire_repair_snapshot
~ musehub/services/musehub_wire.py .py 1 symbol added
+ wire_push_presign function async_function wire_push_presign L1212–1254
← Older Oldest on dev
All commits
Newer → Latest on dev

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:52b9750b58326265a6b16b945e58ce31ace2cde7a8ad6552085c1bcfd2f20274 --body "your comment"