gabriel / musehub public
perf patch task/push-scale-testing #1 / 1
AI Agent gabriel · 126 days ago · May 10, 2026 · Diff

perf: eliminate ghost checks from presign and flush_batch — trust content-addressed DB

write-path invariant: MinIO PUT succeeds before DB INSERT is recorded. musehub_objects presence is sufficient proof — no MinIO HEAD verification needed.

- wire_push_presign: remove parallel Semaphore(64) HEAD loop; DB-present → already_stored - _flush_batch: remove sequential HEAD loop; storage_confirmed = db_present - wire_push_confirm: unchanged — client R2 upload verification is correct and necessary - Phase-C (_check_missing_objects): already DB-only, no change - test_push_presign_ghost_guard: rewrite to document new invariant (DB=truth, no HEAD) - test_push_presign_scale: new — S1/S2/S3 prove O(1) query, zero MinIO calls, <200ms

sha256:4be57596d4cb4efa4ec8be3df1875a5c715c173b499d6343c03019bd30fed75c sha
sha256:c01e97c20a4343f0696a8733cfe73b4b9b915998697f7e378e1f08a8ab4cc028 parent
+20 ~2 −9 symbols
sha256:0a65772a2afdf3f63777ccae880097afa90c111860156da5d3895c05eafecc26 snapshot
+20
symbols added
~2
symbols modified
−9
symbols removed
0
dead code introduced
Semantic Changes 31 symbols
~ tests/test_push_presign_scale.py .py 15 symbols added
+ _bulk_register function async_function _bulk_register L24–41
+ AsyncMock import import AsyncMock L13–13
+ AsyncSession import import AsyncSession L17–17
+ MagicMock import import MagicMock L13–13
+ annotations import import annotations L10–10
+ create_repo import import create_repo L21–21
+ db import import db L20–20
+ fake_id import import fake_id L19–19
+ patch import import patch L13–13
+ pg_insert import import pg_insert L16–16
+ pytest import import pytest L15–15
+ time import import time L12–12
+ test_S1_presign_1000_registered_objects function async_function test_S1_presign_1000_registered_objects L49–75
+ test_S2_no_minio_head_for_registered_objects function async_function test_S2_no_minio_head_for_registered_objects L83–103
+ test_S3_presign_1000_new_objects function async_function test_S3_presign_1000_new_objects L111–138
~ musehub/services/musehub_wire.py .py 2 symbols modified
~ wire_push_presign
~ wire_push_stream
~ tests/test_push_presign_ghost_guard.py .py 5 symbols added, 9 symbols removed
− _insert_ghost_row function async_function _insert_ghost_row L134–143
− func function import func L30–30
− select import import select L30–30
− test_P5_1_presign_ghost_guard_demotes_ghost function async_function test_P5_1_presign_ghost_guard_demotes_ghost L151–179
− test_P5_2_fixed_presign_demotes_ghost_to_upload function async_function test_P5_2_fixed_presign_demotes_ghost_to_upload L187–211
− test_P5_3_fixed_presign_keeps_present_object function async_function test_P5_3_fixed_presign_keeps_present_object L219–239
− test_P5_4_fixed_presign_mixed_batch function async_function test_P5_4_fixed_presign_mixed_batch L247–279
− test_P5_5_push_with_all_genuine_objects_passes function async_function test_P5_5_push_with_all_genuine_objects_passes L287–307
− test_P5_6_never_committed_ghost_rejected_by_phase_c function async_function test_P5_6_never_committed_ghost_rejected_by_phase_c L315–349
+ _register_object function async_function _register_object L127–137
+ test_P5_3_db_present_object_is_already_stored function async_function test_P5_3_db_present_object_is_already_stored L145–166
+ test_P5_5_push_with_registered_objects_passes function async_function test_P5_5_push_with_registered_objects_passes L174–193
+ test_P5_7_db_absent_object_is_upload_path function async_function test_P5_7_db_absent_object_is_upload_path L201–222
+ test_P5_8_mixed_batch_split function async_function test_P5_8_mixed_batch_split L230–258
← Older Oldest on task/push-scale-testing
All commits
Newer → Latest on task/push-scale-testing

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:4be57596d4cb4efa4ec8be3df1875a5c715c173b499d6343c03019bd30fed75c --body "your comment"