gabriel / musehub public
fix patch task/phase1-per-batch-commit #1 / 1
AI Agent gabriel · 118 days ago · May 10, 2026 · Diff

fix: per-batch session.commit() in wire_push_stream to prevent connection timeout

- Add await session.commit() inside _flush_batch after each batch's DB writes (both the all-skipped early-return path and the new-objects path) - Fold background job enqueuing into the branch-update transaction so the final phase makes exactly one commit instead of two - Add TDD test file tests/test_push_per_batch_commit.py (P1-1 through P1-6) verifying commit count per batch, durability under mid-push failure, and idempotency on second push

Root cause: single asyncpg transaction held open for entire push duration; for large repos (4000+ objects, ~2 min) asyncpg reaps the idle connection during MinIO PUTs and the final commit() hits ConnectionDoesNotExistError.

sha256:af0265f72a2cc8625621c96e405f2d3cbf5ebd9ac157717e643ea92fbc790ddf sha
sha256:acae5ece080b5e6151f70550bdf176881cb9b3760e0dadb58cdab1c35998af5a parent
+52 ~1 symbols
sha256:8736004f167ee493a634a5099aa5a1cc0246f09df2e39d19767ef1da650567fd snapshot
+52
symbols added
~1
symbol modified
0
dead code introduced
Semantic Changes 53 symbols
~ tests/test_push_per_batch_commit.py .py 52 symbols added
+ BATCH variable variable BATCH L68–68
+ _collect function async_function _collect L130–134
+ _commit_pack function function _commit_pack L98–103
+ _end function function _end L106–107
+ _frames_of_type function function _frames_of_type L137–138
+ _fw variable variable _fw L66–66
+ _header function function _header L83–88
+ _make_backend function function _make_backend L492–504
+ _make_commit function function _make_commit L114–123
+ _make_objects function function _make_objects L110–111
+ _make_repo function async_function _make_repo L145–161
+ _make_snapshot function function _make_snapshot L126–127
+ _object_frame function function _object_frame L91–95
+ _pack function function _pack L75–76
+ _push function async_function _push L185–205
+ _stub_backend function function _stub_backend L164–182
+ _wrap function function _wrap L79–80
+ Any import import Any L44–44
+ AsyncIterator import import AsyncIterator L42–42
+ AsyncMock import import AsyncMock L45–45
+ AsyncSession import import AsyncSession L50–50
+ JSONObject import import JSONObject L64–64
+ JSONValue import import JSONValue L64–64
+ MagicMock import import MagicMock L45–45
+ Mapping import import Mapping L42–42
+ MuseWireFrameWriter import import MuseWireFrameWriter L53–53
+ SFRAME_COMMIT_PACK import import SFRAME_COMMIT_PACK L55–55
+ SFRAME_END import import SFRAME_END L55–55
+ SFRAME_ERROR import import SFRAME_ERROR L55–55
+ SFRAME_HEADER import import SFRAME_HEADER L55–55
+ SFRAME_OBJECT import import SFRAME_OBJECT L55–55
+ SFRAME_RESULT import import SFRAME_RESULT L55–55
+ STREAM_OBJECT_BATCH_SIZE import import STREAM_OBJECT_BATCH_SIZE L55–55
+ annotations import import annotations L38–38
+ blob_id import import blob_id L52–52
+ call import import call L45–45
+ datetime import import datetime L43–43
+ db import import db L54–54
+ math import import math L40–40
+ msgpack import import msgpack L47–47
+ now_utc_iso import import now_utc_iso L52–52
+ patch import import patch L45–45
+ pytest import import pytest L48–48
+ secrets import import secrets L41–41
+ select import import select L49–49
+ timezone import import timezone L43–43
+ test_p1_1_commit_called_per_batch function async_function test_p1_1_commit_called_per_batch L213–249
+ test_p1_2_one_full_batch_two_commits function async_function test_p1_2_one_full_batch_two_commits L257–287
+ test_p1_3_partial_second_batch_three_commits function async_function test_p1_3_partial_second_batch_three_commits L295–325
+ test_p1_4_batch_0_durable_after_batch_1_fails function async_function test_p1_4_batch_0_durable_after_batch_1_fails L333–398
+ test_p1_5_second_push_idempotent function async_function test_p1_5_second_push_idempotent L406–436
+ test_p1_6_no_minio_io_in_final_commit_transaction function async_function test_p1_6_no_minio_io_in_final_commit_transaction L444–489
~ musehub/services/musehub_wire.py .py 1 symbol modified
~ wire_push_stream
← Older Oldest on task/phase1-per-batch-commit
All commits
Newer → Latest on task/phase1-per-batch-commit

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:af0265f72a2cc8625621c96e405f2d3cbf5ebd9ac157717e643ea92fbc790ddf --body "your comment"