gabriel / musehub public
perf patch task/mpack-phase3-synchronous-push #1 / 1
AI Agent gabriel · 121 days ago · May 17, 2026 · Diff

perf: make push fully synchronous — remove background job and size threshold (Phase 3 issue #69)

wire_push_unpack_mpack now processes ALL mpacks inline regardless of size. The mpack_content_cache_max_bytes threshold is removed as a gate. No mpack.index background job is enqueued — push is fully synchronous.

storage_uri is set to mpack://{mpack_key} immediately on the push path. read_object_bytes now handles mpack:// URIs by extracting from the covering mpack on demand.

Deletes test_mpack_fetch_readiness.py and test_unpack_mpack_skips_msgpack.py which encoded the old background-job / content_cache / threshold architecture. Replaces them with P3-1, P3-2, P3-3 in test_mpack_phase3.py.

sha256:ce4a0610eda998f083cff4ba6d056353ca1d03f73ceb120e48420d1d1e847d76 sha
sha256:469672af9a6ebe85d6427a36c60abef318cad2be30ba91a070a474eef7969701 parent
+34 ~2 −79 symbols
sha256:c8db1e4804c8467043f2799df136ebba8af2bb0103795c613d62c184cf346cd7 snapshot
+34
symbols added
~2
symbols modified
−79
symbols removed
0
dead code introduced
Semantic Changes 115 symbols
~ tests/test_mpack_phase3.py .py 34 symbols added
+ _FakeBackend class class _FakeBackend L62–99
+ __init__ method method __init__ L65–68
+ delete method async_method delete L87–88
+ exists method async_method exists L84–85
+ get method async_method get L75–76
+ get_mpack method async_method get_mpack L78–79
+ presign_get method async_method presign_get L90–91
+ put method async_method put L70–73
+ put_mpack method async_method put_mpack L81–82
+ quarantine_mpack method async_method quarantine_mpack L93–94
+ uri_for method method uri_for L96–97
+ _build_push_mpack function function _build_push_mpack L49–59
+ _mpack_id function function _mpack_id L45–46
+ _now function function _now L41–42
+ AsyncSession import import AsyncSession L28–28 ← moved from tests/test_unpack_mpack_skips_msgpack.py
+ MusehubBackgroundJob import import MusehubBackgroundJob L33–33 ← moved from tests/test_mpack_fetch_readiness.py
+ annotations import import annotations L19–19 ← moved from tests/test_unpack_mpack_skips_msgpack.py
+ blob_id import import blob_id L31–31
+ create_repo import import create_repo L38–38
+ datetime import import datetime L22–22
+ db import import db L32–32
+ fake_id import import fake_id L31–31
+ hashlib import import hashlib L21–21 ← moved from tests/test_unpack_mpack_skips_msgpack.py
+ msgpack import import msgpack L24–24 ← moved from tests/test_unpack_mpack_skips_msgpack.py
+ pg_insert import import pg_insert L27–27
+ pytest import import pytest L25–25 ← moved from tests/test_unpack_mpack_skips_msgpack.py
+ select import import select L29–29 ← moved from tests/test_mpack_fetch_readiness.py
+ timezone import import timezone L22–22
+ wire_fetch_mpack import import wire_fetch_mpack L34–34
+ wire_push_unpack_mpack import import wire_push_unpack_mpack L34–34
+ zstandard import import zstandard L26–26
+ test_p3_1_no_mpack_index_job_enqueued function async_function test_p3_1_no_mpack_index_job_enqueued L105–140
+ test_p3_2_storage_uri_is_mpack_uri_after_push function async_function test_p3_2_storage_uri_is_mpack_uri_after_push L146–181
+ test_p3_3_immediate_fetch_after_push function async_function test_p3_3_immediate_fetch_after_push L187–276
~ tests/test_mpack_fetch_readiness.py .py 43 symbols removed
− _AUTH_CTX variable variable _AUTH_CTX L49–54
− _BLOB_SIZE variable variable _BLOB_SIZE L60–60
− _FILES_CHANGED variable variable _FILES_CHANGED L59–59
− _N_COMMITS variable variable _N_COMMITS L58–58
− _N_FILES variable variable _N_FILES L57–57
− _make_small_repo function function _make_small_repo L95–153
− _push_mpack function async_function _push_mpack L156–202
− client function async_function client L66–80
− ASGITransport import import ASGITransport L23–23
− AsyncClient import import AsyncClient L23–23
− AsyncSession import import AsyncSession L25–25
− CommitRecord import import CommitRecord L38–38
− JSONObject import import JSONObject L46–46
− MSignContext import import MSignContext L28–28
− MusehubBackgroundJob import import MusehubBackgroundJob L29–29 → moved to tests/test_mpack_phase3.py
− MusehubObject import import MusehubObject L30–30
− SnapshotRecord import import SnapshotRecord L38–38
− annotations import import annotations L14–14
− app import import app L32–32
− blob_id import import blob_id L45–45
− build_mpack import import build_mpack L35–35
− compute_commit_id import import compute_commit_id L37–37
− compute_snapshot_id import import compute_snapshot_id L37–37
− datetime import import datetime L16–16
− get_db import import get_db L31–31
− hashlib import import hashlib L17–17
− msgpack import import msgpack L20–20
− muse_dir import import muse_dir L36–36
− optional_signed_request import import optional_signed_request L28–28
− patch import import patch L26–26
− pathlib import import pathlib L18–18
− pytest import import pytest L21–21
− pytest_asyncio import import pytest_asyncio L22–22
− require_signed_request import import require_signed_request L28–28
− select import import select L24–24 → moved to tests/test_mpack_phase3.py
− write_branch_ref import import write_branch_ref L38–38
− write_commit import import write_commit L38–38
− write_object import import write_object L34–34
− write_snapshot import import write_snapshot L38–38
− repo function async_function repo L84–92
− test_content_cache_cleared_after_job function async_function test_content_cache_cleared_after_job L249–307
− test_fetch_object_before_worker_returns_bytes function async_function test_fetch_object_before_worker_returns_bytes L208–245
− test_large_mpack_skips_content_cache function async_function test_large_mpack_skips_content_cache L311–348
~ tests/test_unpack_mpack_skips_msgpack.py .py 36 symbols removed
− _AUTH_CTX variable variable _AUTH_CTX L52–57
− _BLOB_SIZE variable variable _BLOB_SIZE L63–63
− _FILES_CHANGED variable variable _FILES_CHANGED L62–62
− _N_COMMITS variable variable _N_COMMITS L61–61
− _N_FILES variable variable _N_FILES L60–60
− _make_repo function function _make_repo L96–152
− client function async_function client L67–81
− ASGITransport import import ASGITransport L31–31
− AsyncClient import import AsyncClient L31–31
− AsyncSession import import AsyncSession L32–32 → moved to tests/test_mpack_phase3.py
− CommitRecord import import CommitRecord L42–42
− MSignContext import import MSignContext L34–34
− SnapshotRecord import import SnapshotRecord L42–42
− annotations import import annotations L21–21 → moved to tests/test_mpack_phase3.py
− app import import app L36–36
− blob_id import import blob_id L49–49
− build_mpack import import build_mpack L39–39
− compute_commit_id import import compute_commit_id L41–41
− compute_snapshot_id import import compute_snapshot_id L41–41
− datetime import import datetime L23–23
− get_db import import get_db L35–35
− hashlib import import hashlib L24–24 → moved to tests/test_mpack_phase3.py
− msgpack import import msgpack L28–28 → moved to tests/test_mpack_phase3.py
− muse_dir import import muse_dir L40–40
− optional_signed_request import import optional_signed_request L34–34
− patch import import patch L26–26
− pathlib import import pathlib L25–25
− pytest import import pytest L29–29 → moved to tests/test_mpack_phase3.py
− pytest_asyncio import import pytest_asyncio L30–30
− require_signed_request import import require_signed_request L34–34
− write_branch_ref import import write_branch_ref L42–42
− write_commit import import write_commit L42–42
− write_object import import write_object L38–38
− write_snapshot import import write_snapshot L42–42
− repo function async_function repo L85–93
− test_unpack_mpack_does_not_call_msgpack_unpackb function async_function test_unpack_mpack_does_not_call_msgpack_unpackb L156–229
~ musehub/services/musehub_wire.py .py 1 symbol modified
~ wire_push_unpack_mpack
~ musehub/storage/backends.py .py 1 symbol modified
~ read_object_bytes
← Older Oldest on task/mpack-phase3-synchronous-push
All commits
Newer → Latest on task/mpack-phase3-synchronous-push

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:ce4a0610eda998f083cff4ba6d056353ca1d03f73ceb120e48420d1d1e847d76 --body "your comment"