feat(intel): pace analysis database writes so a cold index cannot starve the web
On 2026-11-13 a cold index of a 1.7k-commit repository ran ~10 min on a db.t3.micro: page queries took 56 s, the web pool (4+2) timed out and users saw 5xx. Memory is fixed separately; this is the database load. musehub_intel_pacing.DbPacer, off unless configured: - duty cycle (MUSEHUB_INTEL_DB_DUTY): after a unit of DB work that took t, sleep t*(1/d-1), capped at 20 s; - back-off while the web is busy (MUSEHUB_INTEL_DB_BUSY_THRESHOLD): wait while >= N other queries are active, bounded per unit and per job so analysis is delayed, never stopped; - applied per batch to the ingest (commit slices now 25), symbol intel, vitals, hash occurrences and the providers' row replacement; waiting is between statements, inside savepoint-guarded looks, never fatal. Hosted environments get duty 0.35, threshold 2, wall 3300 s via secrets.sh. Measured locally on real data: same outcome and rows, 106 s -> 206 s wall. Tests: 8 in test_intel_pacing.py (fake clock; real pg_stat_activity look; paced == unpaced results); focused regression 969 passed.
Semantic Changes
62 symbols
0 comments
muse hub commit comment sha256:238a7da4341ade2c8d41ea9834d8e75e250647aefec02327f1e25fac55acd036 --body "your comment"
No comments yet. Be the first to start the discussion.