gabriel / muse public
feat patch task/mpack-phase1-pack-store #1 / 1
AI Agent gabriel · 114 days ago · May 18, 2026 · Diff

feat: MPack local pack store (issue #70 Phase 1)

New muse/core/pack_store.py implements the MPack binary format: - .muse/objects/pack/sha256/<64hex>.mpack — pack data file - .muse/objects/pack/sha256/<64hex>.idx — sorted seek index

write_pack(): 2 atomic file writes regardless of object count. read_object_from_packs(): binary-search index, seek to content, O(1). has_object_in_packs(): index lookup, no decode. verify_pack(): re-hashes both files against their SHA-256 footers.

object_store.read_object() and has_object() fall through to pack store when the object is absent from loose storage — transparent to all callers.

packs_dir() added to paths.py with algorithm canonical in the path, mirroring the loose object store convention.

sha256:a9e14ae2c66b46d5b2d17e3a53e1486b1ee89c587e88944cf1869e96ed456210 sha
sha256:d894a5e5355c79db89afd3086c95306c082fc58ccbb2f290552c3b23bdfc06f3 parent
+83 ~2 symbols
sha256:2aef619eefc10f9c8b8d982f61899dea2da2853cb7a180a1bed568dc01ed19fd snapshot
+83
symbols added
~2
symbols modified
0
dead code introduced
Semantic Changes 85 symbols
~ muse/core/pack_store.py .py 34 symbols added
+ _FOOTER_BYTES variable variable _FOOTER_BYTES L77–77
+ _IDX_ENTRY_BYTES variable variable _IDX_ENTRY_BYTES L80–80
+ _IDX_HEADER_BYTES variable variable _IDX_HEADER_BYTES L79–79
+ _IDX_MAGIC variable variable _IDX_MAGIC L74–74
+ _OID_BYTES variable variable _OID_BYTES L76–76
+ _PACK_HEADER_BYTES variable variable _PACK_HEADER_BYTES L78–78
+ _PACK_MAGIC variable variable _PACK_MAGIC L73–73
+ _VERSION variable variable _VERSION L75–75
+ _all_idx_paths function function _all_idx_paths L264–269
+ _atomic_write function function _atomic_write L96–109
+ _binary_search function function _binary_search L255–261
+ _build_idx function function _build_idx L148–178
+ _build_pack function function _build_pack L125–145
+ _idx_path function function _idx_path L116–117
+ _load_idx function function _load_idx L226–252
+ _pack_path function function _pack_path L112–113
+ _sha256_file function function _sha256_file L88–93
+ has_object_in_packs function function has_object_in_packs L277–286
+ DEFAULT_HASH_ALGO import import DEFAULT_HASH_ALGO L63–63
+ TYPE_CHECKING import import TYPE_CHECKING L60–60
+ annotations import import annotations L52–52
+ bisect import import bisect L54–54
+ hashlib import import hashlib L55–55
+ os import import os L56–56
+ packs_dir import import packs_dir L62–62
+ pathlib import import pathlib L57–57
+ split_id import import split_id L63–63
+ struct import import struct L58–58
+ tempfile import import tempfile L59–59
+ validate_object_id import import validate_object_id L64–64
+ list_packs function function list_packs L323–328
+ read_object_from_packs function function read_object_from_packs L289–320
+ verify_pack function function verify_pack L331–363
+ write_pack function function write_pack L186–218
~ tests/test_pack_store.py .py 48 symbols added
+ TestHasObjectInPacks class class TestHasObjectInPacks L209–234
+ test_finds_objects_across_multiple_packs method method test_finds_objects_across_multiple_packs L227–234
+ test_finds_written_object method method test_finds_written_object L210–215
+ test_returns_false_for_missing method method test_returns_false_for_missing L217–221
+ test_returns_false_when_no_packs_exist method method test_returns_false_when_no_packs_exist L223–225
+ TestListPacks class class TestListPacks L242–252
+ test_empty_when_no_packs method method test_empty_when_no_packs L243–245
+ test_returns_all_pack_ids method method test_returns_all_pack_ids L247–252
+ TestObjectStoreFallthrough class class TestObjectStoreFallthrough L294–324
+ test_has_object_finds_pack_objects method method test_has_object_finds_pack_objects L303–309
+ test_loose_object_takes_priority_over_pack method method test_loose_object_takes_priority_over_pack L311–319
+ test_read_object_finds_pack_objects method method test_read_object_finds_pack_objects L295–301
+ test_read_object_returns_none_when_absent_everywhere method method test_read_object_returns_none_when_absent_everywhere L321–324
+ TestPacksDir class class TestPacksDir L57–66
+ test_packs_dir_algo_subdir_is_sha256 method method test_packs_dir_algo_subdir_is_sha256 L63–66
+ test_packs_dir_is_under_objects method method test_packs_dir_is_under_objects L58–61
+ TestReadObjectFromPacks class class TestReadObjectFromPacks L157–201
+ test_binary_search_finds_object_in_large_pack method method test_binary_search_finds_object_in_large_pack L183–190
+ test_byte_exact_round_trip method method test_byte_exact_round_trip L176–181
+ test_reads_across_multiple_packs method method test_reads_across_multiple_packs L192–201
+ test_returns_correct_bytes method method test_returns_correct_bytes L158–163
+ test_returns_none_for_missing method method test_returns_none_for_missing L165–169
+ test_returns_none_when_no_packs_exist method method test_returns_none_when_no_packs_exist L171–174
+ TestVerifyPack class class TestVerifyPack L260–286
+ test_passes_on_valid_pack method method test_passes_on_valid_pack L261–264
+ test_raises_on_corrupted_idx method method test_raises_on_corrupted_idx L277–286
+ test_raises_on_corrupted_mpack method method test_raises_on_corrupted_mpack L266–275
+ TestWritePack class class TestWritePack L74–149
+ test_idx_file_starts_with_musi_magic method method test_idx_file_starts_with_musi_magic L109–115
+ test_pack_file_encodes_object_count method method test_pack_file_encodes_object_count L117–126
+ test_pack_file_starts_with_muse_magic method method test_pack_file_starts_with_muse_magic L101–107
+ test_pack_id_is_sha256_of_pack_content method method test_pack_id_is_sha256_of_pack_content L92–99
+ test_pack_path_contains_sha256_algo_dir method method test_pack_path_contains_sha256_algo_dir L84–90
+ test_write_pack_creates_mpack_and_idx method method test_write_pack_creates_mpack_and_idx L75–82
+ test_write_pack_empty_object_list_is_noop method method test_write_pack_empty_object_list_is_noop L128–132
+ test_write_pack_idempotent method method test_write_pack_idempotent L141–149
+ test_write_pack_returns_prefixed_pack_id method method test_write_pack_returns_prefixed_pack_id L134–139
+ _make_objects function function _make_objects L42–49
+ annotations import import annotations L10–10
+ blob_id import import blob_id L20–20
+ hashlib import import hashlib L12–12
+ json import import json L13–13
+ muse_dir import import muse_dir L19–19
+ packs_dir import import packs_dir L19–19
+ pathlib import import pathlib L14–14
+ pytest import import pytest L17–17
+ struct import import struct L15–15
+ repo function function repo L29–39
~ muse/core/object_store.py .py 2 symbols modified
~ has_object
~ read_object
~ muse/core/paths.py .py 1 symbol added
+ packs_dir function function packs_dir L34–42
← Older Oldest on task/mpack-phase1-pack-store
All commits
Newer → Latest on task/mpack-phase1-pack-store

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:a9e14ae2c66b46d5b2d17e3a53e1486b1ee89c587e88944cf1869e96ed456210 --body "your comment"