gabriel / musehub public
feat BREAKING intel task/intel-always-on #3 / 3
AI Agent gabriel · 3 days ago · Nov 13, 2026 · Diff

feat(intel): always-on code intelligence for every repository (universal gate, prebuild priority, reconciler, full-set rebuild)

- MUSEHUB_INTEL_ALL_REPOS=1 (exactly) makes every repository eligible in a deployment whose identity is known and matches MUSE_ENV; local still needs a local DB. Off by default; never a wildcard in the allowlist. Persisted for hosted environments in deploy/secrets.sh. - Prebuild priority in the shared scheduler: non-intel work and prebuilds are claimed before intel; a running intel job yields to a waiting prebuild (bounded: at most MAX_INTEL_YIELDS, not an attempt); an aged intel job gets one turn per slice so it cannot starve. - Reconciler (event-driven path stays primary): enqueues exactly the missing/stale/skipped/retry-eligible providers of the registry per repository, bounded (backlog, per-pass cap, resumable cursor, bounded retries with back-off, dead-index holds dependents, exhausted pending jobs failed). Also the historical backfill. - The rebuild route enqueues the index AND every provider; provider state gains queued and not_applicable; aggregate fleet endpoint /api/intel/fleet. - Full symbol history load shares repeated strings (memory); hosted child limits sized 1024 MiB / 1800 s (a cold 1.7k-commit index died at 768 MiB on production). Tests: tests/test_intel_universal.py (56); updated rebuild-route, serial-barrier and health-status tests for the deliberate semantic changes. Focused regression: 840+ intel/job tests pass.

sha256:545266da3f09ffbfe65a21bfc233bb91fe37f83ab714dd2d07db1fee98c3988b sha
+125 ~25 −1 symbols
sha256:e72f8181132339905e774eaa58326b71eb9b0a0d4e3adb1d981df62db4387d87 snapshot
+125
symbols added
~25
symbols modified
−1
symbol removed
0
dead code introduced
Semantic Changes 151 symbols
+ FAILURE_RETRY_LIMIT variable variable FAILURE_RETRY_LIMIT L40–40
+ MAX_BACKLOG variable variable MAX_BACKLOG L39–39
+ MAX_REPOS_PER_PASS variable variable MAX_REPOS_PER_PASS L38–38
+ PAGE variable variable PAGE L37–37
+ RETRY_BACKOFF_SECONDS variable variable RETRY_BACKOFF_SECONDS L41–41
+ ReconcileReport class class ReconcileReport L47–58
+ _FAILED_JOB variable variable _FAILED_JOB L43–43
+ _NEEDS_WORK variable variable _NEEDS_WORK L42–42
+ _backlog function async_function _backlog L65–70
+ _before function function _before L158–162
+ _fail_exhausted_pending function async_function _fail_exhausted_pending L73–81
+ _utc_now function function _utc_now L61–62
+ fleet_summary function async_function fleet_summary L189–244
+ AsyncSession import import AsyncSession L26–26
+ Counter import import Counter L21–21
+ IntelStatus import import IntelStatus L32–32
+ MusehubBackgroundJob import import MusehubBackgroundJob L29–29
+ MusehubBranch import import MusehubBranch L30–30
+ MusehubDomain import import MusehubDomain L28–28
+ MusehubRepo import import MusehubRepo L30–30
+ SNAPSHOT_JOB import import SNAPSHOT_JOB L32–32
+ _MAX_ATTEMPTS import import _MAX_ATTEMPTS L33–33
+ annotations import import annotations L18–18
+ dataclass class import dataclass L22–22
+ datetime import import datetime L23–23
+ enqueue_push_intel import import enqueue_push_intel L33–33
+ field import import field L22–22
+ intel_execution_allowed import import intel_execution_allowed L31–31
+ load_intel_status import import load_intel_status L32–32
+ logging import import logging L20–20
+ required_job_types import import required_job_types L32–32
+ sa import import sa L25–25
+ timedelta import import timedelta L23–23
+ timezone import import timezone L23–23
+ logger variable variable logger L35–35
+ missing_providers function function missing_providers L165–186
+ reconcile_pass function async_function reconcile_pass L93–155
+ retry_eligible function function retry_eligible L84–90
~ tests/test_intel_universal.py .py 63 symbols added
+ H variable variable H L38–38
+ LOCAL_DB variable variable LOCAL_DB L36–36
+ REMOTE_DB variable variable REMOTE_DB L37–37
+ REPO variable variable REPO L35–35
+ _n variable variable _n L115–115
+ _st function function _st L299–301
+ add_job function async_function add_job L118–130
+ env function function env L46–57
+ AsyncSession import import AsyncSession L23–23
+ IntelStatus import import IntelStatus L31–31
+ MusehubBackgroundJob import import MusehubBackgroundJob L25–25
+ MusehubBranch import import MusehubBranch L26–26
+ REQUIRED_JOB_TYPES import import REQUIRED_JOB_TYPES L31–31
+ annotations import import annotations L15–15
+ asyncio import import asyncio L17–17
+ create_repo import import create_repo L32–32
+ datetime import import datetime L19–19
+ gate import import gate L27–27
+ job_types_for_push import import job_types_for_push L30–30
+ jobs import import jobs L29–29
+ load_intel_status import import load_intel_status L31–31
+ pytest import import pytest L21–21
+ rec import import rec L28–28
+ required_job_types import import required_job_types L31–31
+ secrets import import secrets L18–18
+ select import import select L22–22
+ timedelta import import timedelta L19–19
+ timezone import import timezone L19–19
+ update import import update L22–22
+ worker_db import import worker_db L33–33
+ listed function function listed L152–157
+ repo_with_head function async_function repo_with_head L133–137
+ row function async_function row L140–142
+ test_f1_the_fleet_summary_reports_aggregates_only function async_function test_f1_the_fleet_summary_reports_aggregates_only L485–493
+ test_m1_full_history_entries_share_repeated_strings_and_are_unchanged function async_function test_m1_full_history_entries_share_repeated_strings_and_are_unchanged L501–518
+ test_r1_a_repository_with_no_intel_gets_the_whole_registry_set_exactly_once function async_function test_r1_a_repository_with_no_intel_gets_the_whole_registry_set_exactly_once L332–341
+ test_r1_a_repository_without_a_default_head_and_a_non_code_domain_are_left_alone function async_function test_r1_a_repository_without_a_default_head_and_a_non_code_domain_are_left_alone L362–370
+ test_r1_missing_providers_returns_exactly_the_work_that_is_not_current_or_in_flight function function test_r1_missing_providers_returns_exactly_the_work_that_is_not_current_or_in_flight L304–309
+ test_r1_only_the_missing_providers_are_enqueued function async_function test_r1_only_the_missing_providers_are_enqueued L345–358
+ test_r2_a_dead_index_holds_back_its_dependents function function test_r2_a_dead_index_holds_back_its_dependents L322–328
+ test_r2_a_pending_job_with_no_attempts_left_is_failed_not_left_pending_forever function async_function test_r2_a_pending_job_with_no_attempts_left_is_failed_not_left_pending_forever L417–424
+ test_r2_failed_work_is_retried_in_bounds_and_then_stays_terminal function async_function test_r2_failed_work_is_retried_in_bounds_and_then_stays_terminal L388–413
+ test_r2_retry_is_bounded_with_exponential_backoff function function test_r2_retry_is_bounded_with_exponential_backoff L312–319
+ test_r3_a_full_backlog_holds_the_pass function async_function test_r3_a_full_backlog_holds_the_pass L428–436
+ test_r3_allowlist_mode_reconciles_only_listed_repositories function async_function test_r3_allowlist_mode_reconciles_only_listed_repositories L461–470
+ test_r3_nothing_is_enqueued_in_an_unidentified_deployment function async_function test_r3_nothing_is_enqueued_in_an_unidentified_deployment L474–481
+ test_r3_the_pass_is_capped_and_resumable function async_function test_r3_the_pass_is_capped_and_resumable L440–457
+ test_s1_a_prebuild_is_claimed_before_older_intel_work function async_function test_s1_a_prebuild_is_claimed_before_older_intel_work L165–172
+ test_s1_a_young_intel_job_does_not_jump_the_queue function async_function test_s1_a_young_intel_job_does_not_jump_the_queue L202–207
+ test_s1_an_aged_intel_job_goes_first_once_per_slice_so_it_cannot_starve function async_function test_s1_an_aged_intel_job_goes_first_once_per_slice_so_it_cannot_starve L185–198
+ test_s1_non_intel_work_keeps_its_place_ahead_of_intel function async_function test_s1_non_intel_work_keeps_its_place_ahead_of_intel L176–181
+ test_s2_prebuild_waiting_mirrors_the_claim_eligibility function async_function test_s2_prebuild_waiting_mirrors_the_claim_eligibility L211–222
+ test_s2_yield_job_is_fenced_is_not_an_attempt_and_counts_yields function async_function test_s2_yield_job_is_fenced_is_not_an_attempt_and_counts_yields L226–239
+ test_s3_a_job_that_is_not_asked_to_yield_runs_to_completion function async_function test_s3_a_job_that_is_not_asked_to_yield_runs_to_completion L275–292
+ test_s3_a_running_execution_ends_with_reason_yield_when_asked function async_function test_s3_a_running_execution_ends_with_reason_yield_when_asked L243–271
+ test_t1_status_distinguishes_queued_from_analyzing_and_not_applicable function async_function test_t1_status_distinguishes_queued_from_analyzing_and_not_applicable L374–384
+ test_u1_a_wildcard_in_the_allowlist_is_still_not_a_wildcard function function test_u1_a_wildcard_in_the_allowlist_is_still_not_a_wildcard L87–91
+ test_u1_every_repository_is_eligible_in_a_correctly_identified_deployment function function test_u1_every_repository_is_eligible_in_a_correctly_identified_deployment L62–67
+ test_u1_hosted_environments_carry_the_flag_from_secrets_sh_and_local_from_env_example function function test_u1_hosted_environments_carry_the_flag_from_secrets_sh_and_local_from_env_example L101–108
+ test_u1_only_exactly_one_turns_it_on function function test_u1_only_exactly_one_turns_it_on L71–74
+ test_u1_the_flag_is_reread_on_every_call function function test_u1_the_flag_is_reread_on_every_call L94–98
+ test_u1_unknown_or_mismatched_identity_refuses_whatever_the_flag_says function function test_u1_unknown_or_mismatched_identity_refuses_whatever_the_flag_says L81–84
+ types_for function async_function types_for L145–148
~ .env.example .example
~ deploy/secrets.sh .sh 3 symbols added
+ INTEL_ALL_REPOS variable variable INTEL_ALL_REPOS L133–133
+ INTEL_CHILD_MEMORY_MIB variable variable INTEL_CHILD_MEMORY_MIB L134–134
+ INTEL_CHILD_WALL_SECONDS variable variable INTEL_CHILD_WALL_SECONDS L134–134
~ musehub/main.py .py 1 symbol added
+ intel_fleet function async_function intel_fleet L380–398
~ musehub/services/musehub_intel_gate.py .py 2 symbols added, 1 symbol modified
+ ALL_REPOS_ENV variable variable ALL_REPOS_ENV L30–30
+ intel_all_repos function function intel_all_repos L60–67
~ musehub/services/musehub_intel_status.py .py 2 symbols added, 4 symbols modified
+ MusehubDomain import import MusehubDomain L49–49
+ required_job_types function function required_job_types L63–65
~ musehub/services/musehub_jobs.py .py 8 symbols added, 8 symbols modified
+ INTEL_SLICE_SECONDS variable variable INTEL_SLICE_SECONDS L83–83
+ INTEL_STARVATION_SECONDS variable variable INTEL_STARVATION_SECONDS L82–82
+ MAX_INTEL_YIELDS variable variable MAX_INTEL_YIELDS L84–84
+ _PREBUILD_JOB variable variable _PREBUILD_JOB L85–85
+ case import import case L43–43
+ prebuild_waiting function async_function prebuild_waiting L518–530
+ yield_job function async_function yield_job L541–556
+ yields_of function function yields_of L533–538
~ and_
~ exists
~ func
~ or_
~ select
~ update
~ musehub/worker.py .py 6 symbols added, 4 symbols modified
+ _RECONCILE_BUSY_SECONDS variable variable _RECONCILE_BUSY_SECONDS L547–547
+ _RECONCILE_IDLE_SECONDS variable variable _RECONCILE_IDLE_SECONDS L546–546
+ _RECONCILE_LOCK_KEY variable variable _RECONCILE_LOCK_KEY L548–548
+ _YIELD_POLL_SECONDS variable variable _YIELD_POLL_SECONDS L83–83
+ _reconcile_tick function async_function _reconcile_tick L551–576
+ Callable import import Callable L59–59
~ run
~ tests/test_intel_rebuild_route.py .py 1 symbol added, 1 symbol modified
+ test_rebuild_enqueues_the_index_and_every_registry_provider_once function async_function test_rebuild_enqueues_the_index_and_every_registry_provider_once L87–96
~ jobs
~ tests/test_intel_serial_barrier.py .py 1 symbol added, 1 symbol removed
− test_b3_the_next_serial_job_is_claimable_once_nothing_is_running_oldest_first function async_function test_b3_the_next_serial_job_is_claimable_once_nothing_is_running_oldest_first L62–73
+ test_b3_the_next_serial_job_is_claimable_once_nothing_is_running_prebuild_first_then_oldest function async_function test_b3_the_next_serial_job_is_claimable_once_nothing_is_running_prebuild_first_then_oldest L62–81

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:545266da3f09ffbfe65a21bfc233bb91fe37f83ab714dd2d07db1fee98c3988b --body "your comment"