gabriel / musehub public
feat BREAKING dev
AI Agent gabriel · 124 days ago · May 11, 2026 · Diff

feat: content scanning and DMCA takedown (issue #49 phase 3)

Adds the known-hash blocklist check (3a), content.scan job infrastructure (3b), and DMCA takedown endpoint (3c) to the bundle push pipeline.

3a — Blocklist check in process_bundle_index_job: - musehub_blocked_hashes table (MusehubBlockedHash model) - Check runs after Phase 2 validation, before any MinIO writes - BundleValidationError raised on match → job quarantined - Reports all blocked IDs in the error message

3b — content.scan job infrastructure: - After successful bundle.index, a content.scan job is enqueued per object - Job type='content.scan', payload={object_id, repo_id, bundle_key} - Stub processor always returns clean; CSAM API drops in when ready

3c — DMCA takedown endpoint: - POST /api/admin/takedown (admin-only, 403 for non-admin) - Adds object_ids to musehub_blocked_hashes (idempotent) - Moves existing MinIO objects to quarantine/ prefix - Sets dmca_hold=True on specified repo_ids - dmca_hold column on musehub_repos - quarantine_object() on BlobBackend

10 new tests in test_bundle_content_scanning_phase3.py

sha256:7039004cb9fd3d423a5be2e56258d8814ee3f52aed30f15cd9ea7d9eb799d75f sha
sha256:7189fd2a116a24aa69ce8efd79893929ac75aa42aeb6308678df94d00246a9c7 parent
+72 ~3 symbols
sha256:c02a69333bc122ee8d90f1bee8fb56ce2c97f5608a97c7f93fe673ca5a8c378f snapshot
+72
symbols added
~3
symbols modified
0
dead code introduced
Semantic Changes 75 symbols
~ musehub/api/routes/api/admin.py .py 19 symbols added
+ TakedownRequest class class TakedownRequest L22–25
+ TakedownResponse class class TakedownResponse L28–31
+ admin_takedown function async_function admin_takedown L35–89
+ APIRouter import import APIRouter L9–9
+ AsyncSession import import AsyncSession L13–13
+ BaseModel import import BaseModel L10–10
+ Depends import import Depends L9–9
+ HTTPException import import HTTPException L9–9
+ MSignContext import import MSignContext L15–15
+ Sequence import import Sequence L7–7
+ annotations import import annotations L5–5
+ db import import db L16–16
+ get_db import import get_db L17–17
+ pg_insert import import pg_insert L12–12
+ require_signed_request import import require_signed_request L15–15
+ select import import select L11–11
+ status import import status L9–9
+ update import import update L11–11
+ router variable variable router L19–19
~ tests/test_bundle_content_scanning_phase3.py .py 50 symbols added
+ _AUTH_CTX variable variable _AUTH_CTX L45–50
+ _BLOB_SIZE variable variable _BLOB_SIZE L62–62
+ _FILES_CHANGED variable variable _FILES_CHANGED L61–61
+ _NON_ADMIN_CTX variable variable _NON_ADMIN_CTX L52–57
+ _N_COMMITS variable variable _N_COMMITS L60–60
+ _N_FILES variable variable _N_FILES L59–59
+ _make_repo function function _make_repo L115–172
+ _push_bundle function async_function _push_bundle L175–206
+ client function async_function client L68–82
+ ASGITransport import import ASGITransport L21–21
+ AsyncClient import import AsyncClient L21–21
+ AsyncSession import import AsyncSession L23–23
+ CommitRecord import import CommitRecord L35–35
+ MSignContext import import MSignContext L26–26
+ SnapshotRecord import import SnapshotRecord L35–35
+ annotations import import annotations L12–12
+ app import import app L29–29
+ blob_id import import blob_id L42–42
+ build_mpack import import build_mpack L32–32
+ compute_commit_id import import compute_commit_id L34–34
+ compute_snapshot_id import import compute_snapshot_id L34–34
+ datetime import import datetime L14–14
+ db import import db L27–27
+ get_db import import get_db L28–28
+ hashlib import import hashlib L15–15
+ msgpack import import msgpack L18–18
+ muse_dir import import muse_dir L33–33
+ optional_signed_request import import optional_signed_request L26–26
+ patch import import patch L24–24
+ pathlib import import pathlib L16–16
+ pytest import import pytest L19–19
+ pytest_asyncio import import pytest_asyncio L20–20
+ require_signed_request import import require_signed_request L26–26
+ select import import select L22–22
+ write_branch_ref import import write_branch_ref L35–35
+ write_commit import import write_commit L35–35
+ write_object import import write_object L31–31
+ write_snapshot import import write_snapshot L35–35
+ non_admin_client function async_function non_admin_client L86–100
+ repo function async_function repo L104–112
+ test_blocked_check_fires_before_minio_puts function async_function test_blocked_check_fires_before_minio_puts L251–284
+ test_blocked_object_quarantines_bundle function async_function test_blocked_object_quarantines_bundle L212–247
+ test_clean_bundle_bypasses_blocklist_unimpeded function async_function test_clean_bundle_bypasses_blocklist_unimpeded L317–339
+ test_content_scan_jobs_enqueued_after_indexing function async_function test_content_scan_jobs_enqueued_after_indexing L489–521
+ test_dmca_takedown_adds_hashes_to_blocklist function async_function test_dmca_takedown_adds_hashes_to_blocklist L345–373
+ test_dmca_takedown_idempotent function async_function test_dmca_takedown_idempotent L462–483
+ test_dmca_takedown_marks_repos_dmca_hold function async_function test_dmca_takedown_marks_repos_dmca_hold L377–404
+ test_dmca_takedown_quarantines_existing_minio_objects function async_function test_dmca_takedown_quarantines_existing_minio_objects L408–440
+ test_dmca_takedown_requires_admin function async_function test_dmca_takedown_requires_admin L444–458
+ test_multiple_blocked_objects_all_reported function async_function test_multiple_blocked_objects_all_reported L288–313
~ musehub/db/musehub_models.py .py 1 symbol added, 1 symbol modified
+ MusehubBlockedHash class class MusehubBlockedHash L2147–2162
~ MusehubRepo
~ musehub/main.py .py 1 symbol added
+ api_admin_router import import api_admin_router L68–68
~ musehub/services/musehub_wire.py .py 1 symbol modified
~ process_bundle_index_job
~ musehub/storage/backends.py .py 1 symbol added, 1 symbol modified
+ quarantine_object method async_method quarantine_object L426–448
~ BlobBackend
← Older Oldest on dev
All commits
Newer → Latest on dev

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:7039004cb9fd3d423a5be2e56258d8814ee3f52aed30f15cd9ea7d9eb799d75f --body "your comment"