gabriel / musehub public
fix BREAKING worker task/intel-code-experiment #9 / 11
AI Agent gabriel · 4 days ago · Nov 13, 2026 · Diff

fix(worker): truthful job states, intel dependency barriers, no overlap, stale-write guard (#8 C1-C4)

C1: a job type the worker does not execute now ends 'skipped' with a reason (it fell through to 'done' with a NULL result, so 'done' meant 'did nothing'); intel outcomes are recorded in result (computed / empty / already_current), a superseded revision is 'skipped', a missing prerequisite is 'blocked', exceptions retry up to _MAX_ATTEMPTS then 'failed'. Production behaviour is unchanged: intel.* jobs execute only for a repository listed in MUSEHUB_INTEL_WORKER_REPOS on a local database (musehub_intel_gate); nothing lists any.

C2: claim_next_job enforces intel.code before the eight index-dependent providers (coupling, entangle, dead, blast_risk, stable, velocity, clones, gravity), refuses to start intel.code while one of them reads the index (no torn reads), and require_index_current blocks a dependent whose revision intel.code has not published. The six snapshot providers do not wait. The created_at stagger stays as a hint only.

C3: one running job per (repository, intel type) across workers; a push that arrives while one runs queues a new pending job instead of being dropped; a pending intel job is refreshed to the newest push instead of keeping a stale head.

C4: persist_intel_results locks the stored row and rejects a result whose revision is a strict ancestor of the stored one (ancestry, never timestamps); divergent revisions have no order and the later completion wins. It returns the types written.

The revision helpers move out of the indexer into musehub_intel_revision so the guard and the indexer share one ancestry implementation.

sha256:1d6f4203c8aefabfff4a51b02e93f64e7b44858566bbcbfc6ce9d5c791691174 sha
+78 ~10 −6 symbols
sha256:4cfaf9d4430aef0bbedec4cb253d20827f134e5b38f0c68a7534f4c74f762554 snapshot
+78
symbols added
~10
symbols modified
−6
symbols removed
0
dead code introduced
Semantic Changes 94 symbols
+ ALLOWLIST_ENV variable variable ALLOWLIST_ENV L18–18
+ LOCAL_DB_HOSTS variable variable LOCAL_DB_HOSTS L20–20 ← moved from musehub/intel_experiment.py
+ annotations import import annotations L13–13
+ os import import os L15–15
+ urlsplit import import urlsplit L16–16
+ intel_execution_allowed function function intel_execution_allowed L29–34
+ is_local_database function function is_local_database L23–26
+ ParentMap variable variable ParentMap L19–19
+ causal_order function function causal_order L45–69
+ closure function function closure L32–42
+ AsyncSession import import AsyncSession L15–15
+ MusehubCommit import import MusehubCommit L17–17
+ MusehubCommitRef import import MusehubCommitRef L17–17
+ annotations import import annotations L8–8
+ datetime import import datetime L12–12
+ defaultdict import import defaultdict L11–11
+ heapq import import heapq L10–10
+ select import import select L14–14
+ timezone import import timezone L12–12
+ is_strict_ancestor function async_function is_strict_ancestor L79–84
+ load_parent_map function async_function load_parent_map L22–29
+ sort_ts function function sort_ts L72–74
~ tests/test_intel_job_state_machine.py .py 40 symbols added
+ ALLOW_ENV variable variable ALLOW_ENV L36–36
+ _seq variable variable _seq L55–55
+ add_job function async_function add_job L58–69
+ AsyncGenerator import import AsyncGenerator L21–21
+ AsyncSession import import AsyncSession L27–27
+ Dag import import Dag L34–34
+ MusehubBackgroundJob import import MusehubBackgroundJob L31–31
+ MusehubIntelResult import import MusehubIntelResult L30–30
+ MusehubSymbolIntel import import MusehubSymbolIntel L30–30
+ annotations import import annotations L19–19
+ cid import import cid L34–34
+ claim_next_job import import claim_next_job L32–32
+ create_repo import import create_repo L33–33
+ database import import database L29–29
+ datetime import import datetime L22–22
+ enqueue_push_intel import import enqueue_push_intel L32–32
+ ins import import ins L34–34
+ pytest import import pytest L24–24
+ pytest_asyncio import import pytest_asyncio L25–25
+ select import import select L26–26
+ standard_dag import import standard_dag L34–34
+ timedelta import import timedelta L22–22
+ timezone import import timezone L22–22
+ job function async_function job L72–74
+ repo_with_history function async_function repo_with_history L77–82
+ test_a_dependent_job_for_a_superseded_revision_is_skipped_as_stale function async_function test_a_dependent_job_for_a_superseded_revision_is_skipped_as_stale L283–294
+ test_a_dependent_provider_without_a_current_index_is_blocked_not_computed function async_function test_a_dependent_provider_without_a_current_index_is_blocked_not_computed L170–179
+ test_a_failing_job_is_retried_a_bounded_number_of_times_then_fails_leaving_nothing function async_function test_a_failing_job_is_retried_a_bounded_number_of_times_then_fails_leaving_nothing L145–162
+ test_a_job_type_the_worker_does_not_execute_ends_skipped_never_done function async_function test_a_job_type_the_worker_does_not_execute_ends_skipped_never_done L91–99
+ test_a_newer_revision_replaces_an_older_one_and_a_divergent_one_is_accepted function async_function test_a_newer_revision_replaces_an_older_one_and_a_divergent_one_is_accepted L269–279
+ test_a_push_during_a_running_job_is_queued_not_dropped_and_a_pending_job_is_refreshed function async_function test_a_push_during_a_running_job_is_queued_not_dropped_and_a_pending_job_is_refreshed L231–249
+ test_an_allowlisted_intel_code_job_runs_and_records_what_it_computed function async_function test_an_allowlisted_intel_code_job_runs_and_records_what_it_computed L103–114
+ test_completions_in_reversed_order_leave_the_newest_revision_published function async_function test_completions_in_reversed_order_leave_the_newest_revision_published L257–265
+ test_dependents_wait_for_a_pending_index_even_when_they_are_older function async_function test_dependents_wait_for_a_pending_index_even_when_they_are_older L183–192
+ test_empty_input_is_reported_as_empty_not_as_a_success_with_nothing function async_function test_empty_input_is_reported_as_empty_not_as_a_success_with_nothing L118–130
+ test_one_running_job_per_repository_and_type_but_other_repositories_proceed function async_function test_one_running_job_per_repository_and_type_but_other_repositories_proceed L219–227
+ test_snapshot_providers_do_not_wait_for_the_index function async_function test_snapshot_providers_do_not_wait_for_the_index L205–211
+ test_the_allowlist_never_applies_to_a_non_local_database function async_function test_the_allowlist_never_applies_to_a_non_local_database L134–141
+ test_the_index_never_starts_while_a_dependent_is_reading_it function async_function test_the_index_never_starts_while_a_dependent_is_reading_it L196–201
+ worker_db function async_function worker_db L40–52
~ musehub/intel_experiment.py .py 2 symbols added, 1 symbol removed, 3 symbols modified
− LOCAL_DB_HOSTS variable variable LOCAL_DB_HOSTS L64–64 → moved to musehub/services/musehub_intel_gate.py
+ LOCAL_DB_HOSTS import import LOCAL_DB_HOSTS L56–56
+ is_local_database import import is_local_database L56–56
~ musehub/services/musehub_intel_providers.py .py 4 symbols added, 1 symbol modified
+ INDEX_DEPENDENT_JOB_TYPES variable variable INDEX_DEPENDENT_JOB_TYPES L106–109
+ PrerequisiteNotMet class class PrerequisiteNotMet L112–113
+ SupersededRevision class class SupersededRevision L116–117
+ require_index_current function async_function require_index_current L120–139
~ musehub/services/musehub_jobs.py .py 4 symbols added, 3 symbols modified
+ _INDEX_JOB variable variable _INDEX_JOB L67–67
+ block_job function async_function block_job L535–541
+ INDEX_DEPENDENT_JOB_TYPES import import INDEX_DEPENDENT_JOB_TYPES L52–52
+ skip_job function async_function skip_job L526–532
~ musehub/services/musehub_symbol_indexer.py .py 4 symbols added, 5 symbols removed, 2 symbols modified
− _ParentMap variable variable _ParentMap L274–274
− _causal_order function function _causal_order L300–326
− _closure function function _closure L287–297
− _load_parent_map function async_function _load_parent_map L277–284
− _sort_ts function function _sort_ts L329–331
+ causal_order import import causal_order L53–53
+ closure import import closure L53–53
+ load_parent_map import import load_parent_map L53–53
+ sort_ts import import sort_ts L53–53
~ musehub/worker.py .py 2 symbols added, 1 symbol modified
+ _EXECUTED_JOB_TYPES variable variable _EXECUTED_JOB_TYPES L179–179
+ _run_intel_job function async_function _run_intel_job L182–219

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:1d6f4203c8aefabfff4a51b02e93f64e7b44858566bbcbfc6ce9d5c791691174 --body "your comment"