gabriel / musehub public
feat BREAKING task/cursor-pagination-snapshot-rewrite #1 / 1
AI Agent gabriel · 169 days ago · Apr 10, 2026 · Diff

feat: platform-wide cursor pagination + snapshot manifest_blob rewrite

- Add cursor-based keyset pagination across all list endpoints (snapshots, commits, proposals, issues, releases, webhooks, repos, tags, labels) - Rewrite snapshot service: eliminate musehub_snapshot_entries read paths; all reads now use manifest_blob (msgpack BYTEA) for O(1) access - Add entry_count column to musehub_snapshots for O(1) count reads without blob decoding; Alembic migration 0040 back-fills and promotes NOT NULL - Remove all legacy fallback paths, backward-compat shims, and deprecated code - Fix ORM PK column name: MusehubProposalReview.id (was .review_id) - Fix ReleaseListResponse iteration in releases route and tags endpoint - Update all tests: cursor params, corrected URL prefix /api/repos, manifest_blob seeding in _make_snapshot and _push_branch_with_snapshot - mypy: zero errors; 166 tests passing

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

sha256:357e8b27797c1728d32d4d29b8c30f0905b9c2fe05a226a23ad5fc26e7e5b806 sha
+78 ~155 −128 symbols
sha256:5660a0c397a8690995fa52d37370bb0a12ee8e015a1716a2acf9d2f46627cd37 snapshot
+78
symbols added
~155
symbols modified
−128
symbols removed
0
dead code introduced
Semantic Changes 361 symbols
+ _EMPTY_MSGPACK_DICT variable variable _EMPTY_MSGPACK_DICT L29–29
+ branch_labels variable variable branch_labels L25–25
+ depends_on variable variable depends_on L26–26
+ down_revision variable variable down_revision L24–24
+ downgrade function function downgrade L63–70
+ annotations import import annotations L18–18
+ op import import op L20–20
+ sa import import sa L21–21
+ revision variable variable revision L23–23
+ upgrade function function upgrade L32–60
~ musehub/api/routes/api/snapshots.py .py 2 symbols added, 1 symbol removed, 2 symbols modified
− build_link_header import import build_link_header L53–53
+ PaginationParams import import PaginationParams L53–53
+ build_cursor_link_header import import build_cursor_link_header L53–53
~ musehub/api/routes/musehub/issues.py .py 1 symbol added, 2 symbols removed, 3 symbols modified
− build_link_header import import build_link_header L31–31
− paginate_list import import paginate_list L31–31
+ build_cursor_link_header import import build_cursor_link_header L31–31
~ musehub/api/routes/musehub/pagination.py .py 6 symbols removed, 4 symbols modified
− T variable variable T L33–33
− build_link_header function function build_link_header L60–94
− TypeVar import import TypeVar L28–28
− math import import math L27–27
− urljoin import import urljoin L29–29
− paginate_list function function paginate_list L119–137
~ musehub/api/routes/musehub/proposals.py .py 1 symbol added, 2 symbols removed, 4 symbols modified
− build_link_header import import build_link_header L29–29
− paginate_list import import paginate_list L29–29
+ build_cursor_link_header import import build_cursor_link_header L29–29
~ musehub/api/routes/musehub/releases.py .py 4 symbols added, 7 symbols modified
+ PaginationParams import import PaginationParams L30–30
+ Request import import Request L26–26
+ Response import import Response L26–26
+ build_cursor_link_header import import build_cursor_link_header L30–30
~ status
~ musehub/api/routes/musehub/repos.py .py 1 symbol removed, 2 symbols modified
− build_link_header import import build_link_header L30–30
~ musehub/api/routes/musehub/webhooks.py .py 4 symbols added, 7 symbols modified
+ PaginationParams import import PaginationParams L21–21
+ Request import import Request L17–17
+ Response import import Response L17–17
+ build_cursor_link_header import import build_cursor_link_header L21–21
~ Query
~ status
~ musehub/services/musehub_issues.py .py 1 symbol added, 5 symbols modified
~ musehub/services/musehub_releases.py .py 2 symbols added, 4 symbols modified
+ datetime import import datetime L22–22
+ func function import func L24–24
~ select
~ musehub/services/musehub_snapshot.py .py 2 symbols added, 5 symbols removed, 17 symbols modified
− AggByIdDict variable variable AggByIdDict L52–52
− _to_entry_response function function _to_entry_response L74–80
− IntDict import import IntDict L39–39
− func function import func L33–33 → moved to musehub/services/musehub_webhook_dispatcher.py
− selectinload import import selectinload L36–36
+ _EMPTY_BLOB variable variable _EMPTY_BLOB L72–72
+ _entry_response function function _entry_response L98–105
~ select
~ musehub/services/musehub_webhook_dispatcher.py .py 3 symbols added, 7 symbols modified
+ WebhookDeliveryListResponse import import WebhookDeliveryListResponse L38–38
+ WebhookListResponse import import WebhookListResponse L38–38
+ func function import func L32–32 ← moved from musehub/services/musehub_snapshot.py
~ select
~ tests/test_api_performance_section62.py .py 4 symbols added, 5 symbols removed, 2 symbols modified
− test_build_link_header_emits_rfc8288 function function test_build_link_header_emits_rfc8288 L53–65
− test_paginate_list_last_page_shorter function function test_paginate_list_last_page_shorter L45–50
− test_paginate_list_slices_correctly function function test_paginate_list_slices_correctly L37–42
− test_per_page_is_bounded function function test_per_page_is_bounded L68–77
− test_ui_commits_uses_db_level_limit_offset function function test_ui_commits_uses_db_level_limit_offset L84–88
+ test_build_cursor_link_header_emits_rel_next function function test_build_cursor_link_header_emits_rel_next L46–61
+ test_limit_is_bounded function function test_limit_is_bounded L64–72
+ test_pagination_params_defaults function function test_pagination_params_defaults L38–43
+ test_ui_commits_uses_db_level_limit function function test_ui_commits_uses_db_level_limit L79–82
~ tests/test_api_snapshots.py .py 3 symbols added, 3 symbols removed, 47 symbols modified
− test_per_page_capped_at_500 method async_method test_per_page_capped_at_500 L526–542
− test_per_page_capped_at_100 method async_method test_per_page_capped_at_100 L257–271
− MusehubSnapshotEntry import import MusehubSnapshotEntry L32–32
+ test_limit_capped_at_200_for_entries method async_method test_limit_capped_at_200_for_entries L522–538
+ test_limit_capped_at_200 method async_method test_limit_capped_at_200 L241–255
+ msgpack import import msgpack L32–32
~ tests/test_musehub_pagination.py .py 11 symbols added, 22 symbols removed, 4 symbols modified
− AsyncSession import import AsyncSession L19–19
− build_link_header import import build_link_header L23–23
− paginate_list import import paginate_list L23–23
− test_build_cursor_link_header_emits_next_only function function test_build_cursor_link_header_emits_next_only L139–147
− test_build_link_header_first_of_many function function test_build_link_header_first_of_many L98–107
− test_build_link_header_last_page function function test_build_link_header_last_page L122–129
− test_build_link_header_middle_page function function test_build_link_header_middle_page L110–119
− test_build_link_header_preserves_existing_query_params function function test_build_link_header_preserves_existing_query_params L132–136
− test_build_link_header_single_page function function test_build_link_header_single_page L88–95
− test_list_commits_link_header_with_per_page function async_function test_list_commits_link_header_with_per_page L340–372
− test_list_commits_no_link_header_without_per_page function async_function test_list_commits_no_link_header_without_per_page L376–388
− test_list_issues_default_page_returns_all_when_small function async_function test_list_issues_default_page_returns_all_when_small L243–259
− test_list_issues_last_page_no_next function async_function test_list_issues_last_page_no_next L223–239
− test_list_issues_link_header_present function async_function test_list_issues_link_header_present L181–199
− test_list_issues_total_field_returned function async_function test_list_issues_total_field_returned L203–219
− test_list_proposals_link_header_present function async_function test_list_proposals_link_header_present L269–299
− test_list_proposals_total_field_returned function async_function test_list_proposals_total_field_returned L303–331
− test_paginate_list_beyond_last_page_returns_empty function function test_paginate_list_beyond_last_page_returns_empty L72–77
− test_paginate_list_empty_input function function test_paginate_list_empty_input L80–85
− test_paginate_list_first_page function function test_paginate_list_first_page L48–53
− test_paginate_list_last_partial_page function function test_paginate_list_last_partial_page L64–69
− test_paginate_list_middle_page function function test_paginate_list_middle_page L56–61
+ test_build_cursor_link_header_overrides_existing_cursor function function test_build_cursor_link_header_overrides_existing_cursor L113–118
+ test_build_cursor_link_header_preserves_existing_params function function test_build_cursor_link_header_preserves_existing_params L104–110
+ test_build_cursor_link_header_shape function function test_build_cursor_link_header_shape L95–101
+ test_list_issues_cursor_advances_to_next_page function async_function test_list_issues_cursor_advances_to_next_page L188–217
+ test_list_issues_empty_repo_has_no_cursor function async_function test_list_issues_empty_repo_has_no_cursor L127–142
+ test_list_issues_returns_next_cursor_when_more_exist function async_function test_list_issues_returns_next_cursor_when_more_exist L167–184
+ test_list_issues_single_page_no_next_cursor function async_function test_list_issues_single_page_no_next_cursor L146–163
+ test_list_issues_total_stable_across_pages function async_function test_list_issues_total_stable_across_pages L221–235
+ test_list_proposals_empty_repo_has_no_cursor function async_function test_list_proposals_empty_repo_has_no_cursor L244–258
+ test_pagination_params_custom function function test_pagination_params_custom L83–87
+ test_pagination_params_defaults function function test_pagination_params_defaults L76–80
~ tests/test_musehub_proposals.py .py 1 symbol added, 1 symbol modified
+ msgpack import import msgpack L26–26
~ tests/test_pagination_section39.py .py 28 symbols added, 81 symbols removed, 2 symbols modified
− TestBuildCursorLinkHeaderEdgeCases class class TestBuildCursorLinkHeaderEdgeCases L245–271
− test_cursor_with_special_chars_is_percent_encoded method method test_cursor_with_special_chars_is_percent_encoded L248–254
− test_existing_query_params_preserved method method test_existing_query_params_preserved L256–260
− test_limit_encoded_correctly method method test_limit_encoded_correctly L262–265
− test_no_prev_first_last_in_cursor_header method method test_no_prev_first_last_in_cursor_header L267–271
− TestBuildLinkHeaderEdgeCases class class TestBuildLinkHeaderEdgeCases L193–242
− test_exact_multiple_last_page_no_next method method test_exact_multiple_last_page_no_next L204–209
− test_first_page_url_always_page_1 method method test_first_page_url_always_page_1 L237–242
− test_last_page_number_correct_for_exact_multiple method method test_last_page_number_correct_for_exact_multiple L211–215
− test_last_page_number_correct_for_partial_final_page method method test_last_page_number_correct_for_partial_final_page L217–221
− test_link_urls_are_valid_format method method test_link_urls_are_valid_format L229–235
− test_per_page_1_total_5_last_is_5 method method test_per_page_1_total_5_last_is_5 L223–227
− test_zero_total_returns_first_and_last_only method method test_zero_total_returns_first_and_last_only L196–202
− TestPaginateListEdgeCases class class TestPaginateListEdgeCases L136–190
− test_exact_multiple_last_page_full method method test_exact_multiple_last_page_full L146–152
− test_page_2_starts_after_page_1_ends method method test_page_2_starts_after_page_1_ends L177–182
− test_page_far_beyond_end_returns_empty method method test_page_far_beyond_end_returns_empty L159–163
− test_per_page_larger_than_list method method test_per_page_larger_than_list L165–170
− test_per_page_one_returns_single_item_pages method method test_per_page_one_returns_single_item_pages L139–144
− test_preserves_item_order method method test_preserves_item_order L172–175
− test_single_item_list method method test_single_item_list L154–157
− test_total_unchanged_across_pages method method test_total_unchanged_across_pages L184–190
− TestPaginationDataIntegrity class class TestPaginationDataIntegrity L688–799
− test_full_page_through_no_items_skipped method method test_full_page_through_no_items_skipped L691–703
− test_http_total_stable_across_page_requests method async_method test_http_total_stable_across_page_requests L737–767
− test_last_exact_page_is_full method method test_last_exact_page_is_full L731–735
− test_last_page_is_remainder_not_full method method test_last_page_is_remainder_not_full L724–729
− test_no_duplicates_across_pages method method test_no_duplicates_across_pages L705–712
− test_no_http_items_skipped_across_pages method async_method test_no_http_items_skipped_across_pages L769–799
− test_total_constant_across_all_pages method method test_total_constant_across_all_pages L714–722
− TestPaginationE2E class class TestPaginationE2E L423–579
− test_commits_page_based_link_header method async_method test_commits_page_based_link_header L538–563
− test_issues_last_page_no_next_link method async_method test_issues_last_page_no_next_link L443–457
− test_issues_page1_link_header_has_next method async_method test_issues_page1_link_header_has_next L426–441
− test_issues_page2_contains_correct_slice method async_method test_issues_page2_contains_correct_slice L473–511
− test_issues_total_field_matches_seeded_count method async_method test_issues_total_field_matches_seeded_count L459–471
− test_my_repos_cursor_link_header_absent_on_last_page method async_method test_my_repos_cursor_link_header_absent_on_last_page L565–579
− test_proposals_pagination_link_header method async_method test_proposals_pagination_link_header L513–536
− TestPaginationIntegration class class TestPaginationIntegration L279–415
− test_commits_empty_repo_returns_empty method async_method test_commits_empty_repo_returns_empty L396–415
− test_empty_repo_issues_returns_empty_list_not_404 method async_method test_empty_repo_issues_returns_empty_list_not_404 L282–295
− test_filter_plus_pagination_correct_total method async_method test_filter_plus_pagination_correct_total L315–345
− test_page_beyond_total_returns_empty_slice method async_method test_page_beyond_total_returns_empty_slice L297–313
− test_proposals_empty_repo_returns_empty method async_method test_proposals_empty_repo_returns_empty L382–394
− test_sort_stability_across_pages method async_method test_sort_stability_across_pages L347–380
− TestPaginationParamsDefaults class class TestPaginationParamsDefaults L98–133
− test_class_is_instantiable_with_defaults class method test_class_is_instantiable_with_defaults L110–124
− test_cursor_default_is_none method method test_cursor_default_is_none L126–133
− test_defaults_are_sane method method test_defaults_are_sane L101–108
− TestPaginationPerformance class class TestPaginationPerformance L902–991
− test_build_cursor_link_header_under_5ms_repeated_100x method method test_build_cursor_link_header_under_5ms_repeated_100x L921–927
− test_build_link_header_under_5ms_repeated_100x method method test_build_link_header_under_5ms_repeated_100x L913–919
− test_paginate_list_10k_items_under_10ms method method test_paginate_list_10k_items_under_10ms L905–911
− test_paginated_issue_list_http_under_300ms method async_method test_paginated_issue_list_http_under_300ms L929–961
− test_paginated_proposals_http_under_300ms method async_method test_paginated_proposals_http_under_300ms L963–991
− TestPaginationSecurity class class TestPaginationSecurity L807–894
− test_cursor_sql_injection_does_not_500 method async_method test_cursor_sql_injection_does_not_500 L862–872
− test_huge_page_number_does_not_500 method async_method test_huge_page_number_does_not_500 L874–887
− test_link_header_next_url_does_not_expose_internal_paths method method test_link_header_next_url_does_not_expose_internal_paths L889–894
− test_negative_page_rejected_422 method async_method test_negative_page_rejected_422 L810–821
− test_non_integer_page_rejected_422 method async_method test_non_integer_page_rejected_422 L849–860
− test_per_page_above_max_rejected_422 method async_method test_per_page_above_max_rejected_422 L823–834
− test_per_page_zero_rejected_422 method async_method test_per_page_zero_rejected_422 L836–847
− TestPaginationStress class class TestPaginationStress L587–680
− test_build_link_header_large_total method method test_build_link_header_large_total L609–613
− test_issue_list_with_50_items_under_500ms method async_method test_issue_list_with_50_items_under_500ms L648–680
− test_paginate_list_100_items_full_page_through method method test_paginate_list_100_items_full_page_through L590–599
− test_paginate_list_large_dataset method method test_paginate_list_large_dataset L601–607
− test_sequential_issue_page_through method async_method test_sequential_issue_page_through L615–646
− _make_issue function async_function _make_issue L79–90
− _make_repo function async_function _make_repo L65–76
− _req function function _req L53–62
− MusehubCommit import import MusehubCommit L45–45
− MusehubIssue import import MusehubIssue L45–45
− MusehubProposal import import MusehubProposal L45–45
− MusehubRepo import import MusehubRepo L45–45
− build_link_header import import build_link_header L39–39
− datetime import import datetime L31–31
− paginate_list import import paginate_list L39–39
− timedelta import import timedelta L31–31
− timezone import import timezone L31–31
+ _create_issue function async_function _create_issue L69–82
+ _create_repo function async_function _create_repo L57–66
+ _make_request function function _make_request L46–54
+ _walk_all_issues function async_function _walk_all_issues L85–109
+ issues_svc import import issues_svc L38–38
+ test_build_cursor_link_header_contains_rel_next function function test_build_cursor_link_header_contains_rel_next L136–140
+ test_build_cursor_link_header_encodes_special_chars function function test_build_cursor_link_header_encodes_special_chars L143–150
+ test_build_cursor_link_header_overrides_existing_cursor function function test_build_cursor_link_header_overrides_existing_cursor L153–159
+ test_build_cursor_link_header_preserves_filter_params function function test_build_cursor_link_header_preserves_filter_params L162–167
+ test_e2e_cursor_in_link_header_matches_body function async_function test_e2e_cursor_in_link_header_matches_body L317–332
+ test_e2e_link_header_present_only_when_more_pages function async_function test_e2e_link_header_present_only_when_more_pages L289–313
+ test_e2e_total_consistent_across_pages function async_function test_e2e_total_consistent_across_pages L263–285
+ test_empty_repo_returns_empty_list_not_404 function async_function test_empty_repo_returns_empty_list_not_404 L176–187
+ test_integrity_filter_preserved_across_pages function async_function test_integrity_filter_preserved_across_pages L420–444
+ test_integrity_no_duplicates_across_pages function async_function test_integrity_no_duplicates_across_pages L405–416
+ test_integrity_no_items_skipped_across_pages function async_function test_integrity_no_items_skipped_across_pages L390–401
+ test_label_filter_combined_with_pagination function async_function test_label_filter_combined_with_pagination L191–210
+ test_pagination_params_defaults function function test_pagination_params_defaults L117–121
+ test_pagination_params_max_limit function function test_pagination_params_max_limit L130–133
+ test_pagination_params_min_limit function function test_pagination_params_min_limit L124–127
+ test_performance_issue_list_50_items_under_500ms function async_function test_performance_issue_list_50_items_under_500ms L495–520
+ test_security_cursor_with_special_chars_accepted function async_function test_security_cursor_with_special_chars_accepted L475–486
+ test_security_limit_over_max_rejected function async_function test_security_limit_over_max_rejected L464–471
+ test_security_limit_zero_rejected function async_function test_security_limit_zero_rejected L453–460
+ test_sort_stability_ascending_by_number function async_function test_sort_stability_ascending_by_number L214–225
+ test_state_filter_combined_with_pagination function async_function test_state_filter_combined_with_pagination L229–254
+ test_stress_100_item_full_cursor_walk function async_function test_stress_100_item_full_cursor_walk L341–359
+ test_stress_last_page_has_correct_remainder function async_function test_stress_last_page_has_correct_remainder L363–381
← Older Oldest on task/cursor-pagination-snapshot-rewrite
All commits
Newer → Latest on task/cursor-pagination-snapshot-rewrite

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:357e8b27797c1728d32d4d29b8c30f0905b9c2fe05a226a23ad5fc26e7e5b806 --body "your comment"