gabriel / musehub public
feat patch task/bundle-index-objects #1 / 1
AI Agent gabriel · 124 days ago · May 11, 2026 · Diff

feat: bundle.index job writes objects to MinIO and registers refs

The job was indexing snapshots and commits but never writing object blobs to MinIO or inserting musehub_objects / musehub_object_refs rows. This caused the ghost guard to fail on any stream push after a bundle push and made fetch/clone unable to serve file content.

Changes: - Extract raw_objects from bundle, decompress zstd in-memory - Parallel MinIO writes capped by _get_blob_put_sem() (100 slots) - Bulk INSERT musehub_objects (on_conflict_do_nothing — idempotent) - _upsert_object_refs for all objects in bundle - Extend return dict: objects_written, object_refs_written, elapsed_ms

No per-object sha256 re-verification — sha256(wire_bytes) == bundle_key already authenticated every byte. Checking again is checking the work of the hash.

TDD: test_bundle_index_job.py — 6 tests green.

sha256:638a778fa91d16b2db87440016d98abc40e0fc7ac598279fe4c665a166265169 sha
sha256:9076f2e667fe458121f908717e4d396f07e4b29a1fefb082ec0bda6cd45109b3 parent
+44 ~1 symbols
sha256:eacd2e0f9460bc584f8d190fda146de4c39718550ce9e9f31368fb4b91dc4345 snapshot
+44
symbols added
~1
symbol modified
0
dead code introduced
Semantic Changes 45 symbols
~ tests/test_bundle_index_job.py .py 44 symbols added
+ _AUTH_CTX variable variable _AUTH_CTX L48–53
+ _BLOB_SIZE variable variable _BLOB_SIZE L59–59
+ _FILES_CHANGED variable variable _FILES_CHANGED L58–58
+ _N_COMMITS variable variable _N_COMMITS L57–57
+ _N_FILES variable variable _N_FILES L56–56
+ _make_repo function function _make_repo L94–152
+ _push_bundle function async_function _push_bundle L155–201
+ client function async_function client L65–79
+ ASGITransport import import ASGITransport L25–25
+ AsyncClient import import AsyncClient L25–25
+ AsyncSession import import AsyncSession L27–27
+ CommitRecord import import CommitRecord L38–38
+ MSignContext import import MSignContext L29–29
+ SnapshotRecord import import SnapshotRecord L38–38
+ annotations import import annotations L16–16
+ app import import app L32–32
+ blob_id import import blob_id L45–45
+ build_mpack import import build_mpack L35–35
+ compute_commit_id import import compute_commit_id L37–37
+ compute_snapshot_id import import compute_snapshot_id L37–37
+ datetime import import datetime L18–18
+ db import import db L30–30
+ func function import func L26–26
+ get_db import import get_db L31–31
+ hashlib import import hashlib L19–19
+ msgpack import import msgpack L22–22
+ muse_dir import import muse_dir L36–36
+ optional_signed_request import import optional_signed_request L29–29
+ pathlib import import pathlib L20–20
+ pytest import import pytest L23–23
+ pytest_asyncio import import pytest_asyncio L24–24
+ require_signed_request import import require_signed_request L29–29
+ select import import select L26–26
+ write_branch_ref import import write_branch_ref L38–38
+ write_commit import import write_commit L38–38
+ write_object import import write_object L34–34
+ write_snapshot import import write_snapshot L38–38
+ repo function async_function repo L83–91
+ test_job_is_idempotent function async_function test_job_is_idempotent L339–371
+ test_musehub_objects_rows_after_job function async_function test_musehub_objects_rows_after_job L241–262
+ test_no_per_object_sha256_verification function async_function test_no_per_object_sha256_verification L295–335
+ test_object_refs_after_job function async_function test_object_refs_after_job L266–291
+ test_objects_written_to_minio_after_job function async_function test_objects_written_to_minio_after_job L207–237
+ test_return_dict_includes_counts_and_timing function async_function test_return_dict_includes_counts_and_timing L375–391
~ musehub/services/musehub_wire.py .py 1 symbol modified
~ process_bundle_index_job
← Older Oldest on task/bundle-index-objects
All commits
Newer → Latest on task/bundle-index-objects

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:638a778fa91d16b2db87440016d98abc40e0fc7ac598279fe4c665a166265169 --body "your comment"