gabriel / musehub public
feat BREAKING task/fetch-presign #1 / 1
AI Agent gabriel · 146 days ago · Apr 30, 2026 · Diff

feat: presigned fetch path for large repos

- Add wire_fetch_presign() service function — BFS walk, bundle via wire_fetch_stream frames, SHA-256 content-addressed upload to R2, presigned GET URL returned - Add POST /{owner}/{slug}/fetch/presign route (optional auth, public repo accessible) - Add supports_presign: bool to LocalBackend (False) and S3Backend (True) - Fix ghost guard sequential R2 HEAD requests — skip entirely when _objects_in_push is empty (presign path verifies at confirm time); run parallel with semaphore(50) for inline pushes — eliminates Cloudflare timeout on large presigned push - Add constants FETCH_PRESIGN_OBJECT_THRESHOLD=500, FETCH_PRESIGN_SIZE_THRESHOLD=50MB - Add FetchPresignResult TypedDict - 11 TDD tests covering below-threshold, above-threshold, have-set exclusion, LocalBackend fallback, empty want, commit/object counts, route 404/200/private

sha256:68a22a4c5483413ee020a79b09772ae301352b9af89ea9a2d24bca64d5e47862 sha
sha256:991c735af464517d231ac3124044509d1c8f72b644442c4a5ab6bf32ca140d1b parent
+42 ~2 symbols
sha256:9be07a80d86ec7b2ba2d5e273fec6952d0a4a057cae8967236c7738ac9096299 snapshot
+42
symbols added
~2
symbols modified
0
dead code introduced
Semantic Changes 44 symbols
~ tests/test_wire_fetch_presign.py .py 37 symbols added
+ _make_commit function async_function _make_commit L112–143
+ _now function function _now L74–75
+ _parse_wire_frames function async_function _parse_wire_frames L146–149
+ _store_object function async_function _store_object L82–109
+ _uid function function _uid L78–79
+ AsyncIterator import import AsyncIterator L50–50
+ AsyncMock import import AsyncMock L52–52
+ AsyncSession import import AsyncSession L57–57
+ FETCH_PRESIGN_OBJECT_THRESHOLD import import FETCH_PRESIGN_OBJECT_THRESHOLD L62–62
+ FETCH_PRESIGN_SIZE_THRESHOLD import import FETCH_PRESIGN_SIZE_THRESHOLD L62–62
+ JSONObject import import JSONObject L67–67
+ MagicMock import import MagicMock L52–52
+ WireFetchRequest import import WireFetchRequest L61–61
+ annotations import import annotations L47–47
+ blob_id import import blob_id L59–59
+ create_repo import import create_repo L68–68
+ datetime import import datetime L51–51
+ db import import db L60–60
+ fake_id import import fake_id L59–59
+ msgpack import import msgpack L54–54
+ patch import import patch L52–52
+ pg_insert import import pg_insert L56–56
+ pytest import import pytest L55–55
+ timezone import import timezone L51–51
+ wire_fetch_presign import import wire_fetch_presign L62–62
+ zlib import import zlib L49–49
+ test_fp0_below_threshold_returns_presign_false function async_function test_fp0_below_threshold_returns_presign_false L157–169
+ test_fp10_bundle_bytes_are_valid_wire_frames function async_function test_fp10_bundle_bytes_are_valid_wire_frames L419–460
+ test_fp1_above_object_threshold_s3_presigns function async_function test_fp1_above_object_threshold_s3_presigns L177–211
+ test_fp2_above_size_threshold_presigns function async_function test_fp2_above_size_threshold_presigns L219–248
+ test_fp3_have_set_excludes_known_commits function async_function test_fp3_have_set_excludes_known_commits L256–279
+ test_fp4_local_backend_never_presigns function async_function test_fp4_local_backend_never_presigns L287–303
+ test_fp5_empty_want_returns_presign_false function async_function test_fp5_empty_want_returns_presign_false L311–318
+ test_fp6_counts_are_accurate function async_function test_fp6_counts_are_accurate L326–352
+ test_fp7_route_404_missing_repo function async_function test_fp7_route_404_missing_repo L360–366
+ test_fp8_route_small_public_repo function async_function test_fp8_route_small_public_repo L374–393
+ test_fp9_route_private_repo_non_owner_gets_404 function async_function test_fp9_route_private_repo_non_owner_gets_404 L401–411
~ musehub/api/routes/wire.py .py 1 symbol added
+ fetch_presign function async_function fetch_presign L644–677
~ musehub/services/musehub_wire.py .py 4 symbols added
+ FETCH_PRESIGN_OBJECT_THRESHOLD variable variable FETCH_PRESIGN_OBJECT_THRESHOLD L82–82
+ FETCH_PRESIGN_SIZE_THRESHOLD variable variable FETCH_PRESIGN_SIZE_THRESHOLD L83–83
+ FetchPresignResult class class FetchPresignResult L106–114
+ wire_fetch_presign function async_function wire_fetch_presign L1213–1367
~ musehub/storage/backends.py .py 2 symbols modified
~ LocalBackend
~ S3Backend
← Older Oldest on task/fetch-presign
All commits
Newer → Latest on task/fetch-presign

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:68a22a4c5483413ee020a79b09772ae301352b9af89ea9a2d24bca64d5e47862 --body "your comment"