Commit Graph

29 Commits

Author SHA1 Message Date
m17hr1l
367f17a013 stage-topo-b topology-export: /federation/topology endpoint + CORS cache 2026-06-07 01:56:09 +02:00
m17hr1l
a8216d00ef stage-topo-a topology-export: sanitized public docker snapshot module + tests 2026-06-07 01:55:49 +02:00
m17hr1l
cef3bcb1ed merge explore: public transparent federation explorer with cross-jump 2026-06-07 01:19:56 +02:00
m17hr1l
9ab3271bc8 stage-exp-f explore: tests 2026-06-07 01:17:11 +02:00
m17hr1l
56466c334d stage-exp-a explore: public payload builder + tests 2026-06-07 01:11:17 +02:00
m17hr1l
351e16c3ce inference: openai-compatible mode + bearer auth (for api.neuronetz.ai etc.) 2026-06-07 01:09:19 +02:00
m17hr1l
0d9baef4c8 stage-netd-f network detail: tests for admin enrichment (stats/corroboration/timeline) 2026-06-07 01:00:39 +02:00
m17hr1l
865be2e239 stage-net-f network view: tests 2026-06-07 00:42:11 +02:00
m17hr1l
a53aacfdd8 merge auto-response: severity/quorum/local-only gated execution
# Conflicts:
#	src/psyc/db.py
2026-06-06 21:17:20 +02:00
m17hr1l
53ba537ce8 merge vouching+translog: web-of-trust + signed merkle audit log
# Conflicts:
#	src/psyc/_federation_cli.py
#	src/psyc/cockpit/federation_routes.py
2026-06-06 21:15:11 +02:00
m17hr1l
c5472b3134 stage-auto-e pulse: tests for auto-response gating
Cover the auto-fire decision matrix:
- _severity_rank ordering
- mode != auto-execute → never fires (auto-propose, manual)
- below-threshold action is skipped + audited
- federation case + no quorum → skipped + audited "no quorum"
- federation case + quorum met → fires
- local case + quorum required + local-only on → still fires
- local case + quorum required + local-only off → still fires
- quorum gating disabled → federation cases fire too
- kill switch armed → tick() skips everything
- pulse_audit records both auto-fire and skip rows
- audit_count_since returns the per-action counts the cockpit needs
- config round-trips through pulse_settings

Tests patch federation.is_quorum_met (raising=False so the sibling
agent can ship the real function later without breaking these), and
swap respond.execute_action for a counter so no SOAR sink call escapes.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 21:12:02 +02:00
m17hr1l
f4148d86a6 stage-vouch-e federation: tests for vouching + quorum gate
test_vouching covers the contract auto-response and other agents will
gate on:

- issue_vouch round-trip (sign + verify under our own pubkey)
- accept_vouch rejects expired vouches
- accept_vouch rejects mismatched signatures
- accept_vouch rejects vouchers whose peers.status != "trusted"
- accept_vouch happy path
- is_vouched needs DISTINCT vouchers (two upserts from one peer == 1)
- is_vouched clears threshold with two distinct trusted vouchers
- is_quorum_met counts only listening-eligible peers (untrusted +
  duplicate rows don't count)
- quorum_config defaults + pulse_settings persistence
- import_signed_feed rejects unknown peer ("not trusted")
- import_signed_feed accepts directly-trusted peer
- import_signed_feed accepts a peer made eligible via two vouches
- import_signed_feed stores vouches embedded in a trusted peer's feed

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 21:11:18 +02:00
m17hr1l
eadd1aea3b stage-vouch-c federation: import gate + translog hook (stage-trans-b)
import_signed_feed now refuses any feed whose declared fingerprint isn't
peer_is_listening_eligible (directly trusted OR vouched in), returning
Err("peer not trusted: …") before any signal lands.

For every case/IOC it does record, it also appends a "signal" entry to
the transparency log (best-effort — logger warns but doesn't abort
ingest if the append fails). This is the stage-trans-b hook: the
import path is the chokepoint, so attaching the chain there gives
us coverage of every peer-originated signal we've ever accepted.

build_signed_feed now includes our_vouches() in the feed body so vouches
propagate. On import we accept_vouch each one — but only if the embedded
voucher_fingerprint matches the peer we just authenticated, so a peer
can't forge vouches "from" someone else through us.

test_federation: the long-standing round-trip test now first registers
the synthetic peer as trusted so the gate lets it through.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 21:10:36 +02:00
m17hr1l
7a510c7acf stage-trans-a translog: append-only signed merkle chain + tests
translog.append computes
sha256(canonical({prev_hash, entry_type, entry_data, timestamp})) and
writes one row per call; the first entry uses prev_hash = "0"*64.
verify_chain walks rows in id order, re-hashes each, and returns
Err("broken at id=X expected=... got=...") on the first mismatch — so
tampering with either entry_data or prev_hash invalidates every
downstream row. recent / entries_after / head support peer sync and UI.

Tests cover: genesis prev_hash, chained prev_hash, full-chain verify,
tampered-data detection, tampered-prev_hash detection, slicing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 21:09:32 +02:00
m17hr1l
ff88aba569 stage-disc-e discovery: tests 2026-06-06 21:08:15 +02:00
m17hr1l
de5ff09815 merge federation: ed25519 identity + signed feeds
# Conflicts:
#	src/psyc/db.py
2026-06-06 16:13:36 +02:00
m17hr1l
d4229dd264 stage-fed-g federation: tests 2026-06-06 16:10:31 +02:00
m17hr1l
a7c59c9faa stage-33e pulse: tests 2026-06-06 16:06:54 +02:00
m17hr1l
76a0b0b636 stage-31: Newsline polish — featured case, time buckets, severity accents, generated visuals
The home page goes from a flat event stream to something that reads like
a news blog:

- Featured-case hero card at the top, picked as the highest-severity
  case (CRITICAL > HIGH, tie-break recency) from the last 7 days. Wide,
  with a procedurally generated SVG hero behind a gradient overlay that
  carries title + severity + TLP + feed + ingest time.
- Recent activity is now grouped under Today / Yesterday / Earlier this
  week / Older bucket headers.
- Each item gets a left-border severity accent (red CRITICAL, amber
  HIGH, muted MEDIUM/LOW) so the page is scannable at a glance.

Images: new cockpit/case_visuals.py generates SVGs from case data —
zero external image gen, zero curated assets. Every visual is
deterministic from case_id (so a case keeps its identity across
sessions) and themed to its severity:

- case_hero_svg() — 880x220 hero with severity radial glow, a faint
  scan grid, a particle constellation with auto-connecting lines, HUD
  corner brackets, and the case id whispered in the bottom-right.
- case_glyph_svg() — small mirror-symmetric identicon (5-grid),
  severity-colored, shown beside each case news item in place of an
  emoji icon. Two case_ids → two distinct glyphs; same id → same glyph.

7 news tests pass; visual sanity print confirms hero is deterministic
and uses the right severity accent.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 19:02:22 +02:00
m17hr1l
16cf873044 stage-30: home page (Newsline digest) + PWA + mobile pass
NEW / start page replaces the redirect-to-/cases:
- KPI strip (cases, IOCs, +24h, high/critical,  enforced 24h, ledger total) —
  clickable, responsive grid (2 cols mobile, 3 mid, 6 desktop).
- Recent activity feed: ledger events (enforced/submitted/rejected/failed) +
  newest case ingests, interleaved newest-first, with severity badges, icons,
  case links. Sources via lines/news.py.
- Feed health sidebar: per-feed count + last ingest time.

PWA:
- /static/manifest.json declares a standalone install with theme colors.
- /static/sw.js — cache-first for static, network-first for HTML/API, with a
  graceful offline page. Registered from / scope via a dedicated /sw.js route
  that sets Service-Worker-Allowed: /.
- viewport + apple-touch-icon + theme-color meta tags in base.html.

Mobile pass on the chrome:
- Topbar wraps; nav horizontally scrolls instead of crowding; brand-sub hides.
- Tables (cases, ledger) scroll horizontally on narrow screens instead of
  exploding the layout.
- Hero / KPI / news-list layouts collapse cleanly at < 720px.

4 news tests; verified locally — home page renders, /sw.js serves with
Service-Worker-Allowed: /, manifest is valid JSON.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:18:40 +02:00
m17hr1l
b51a88d502 stage-26b: Docker topology in /admin — read-only socket-proxy + graph
New tecnativa/docker-socket-proxy sidecar exposes only GET on
containers/networks/info/ping; POST and DELETE are blocked. The cockpit
queries it over the backend network — /var/run/docker.sock is never
mounted into a web-facing container.

cockpit/docker_view.py normalizes the daemon view: containers carry
per-network IP/MAC + published_ports; networks carry subnet/gateway from
IPAM; host_info pulls /info (degrades gracefully). topology() returns
the combined snapshot.

/admin/docker (admin-gated): a force-directed graph (pure SVG +
vanilla JS, ~280 lines) renders the complete setup — a host node,
switch nodes with subnet labels colored by driver, container nodes
colored by state, member wires labeled with the container's IP on that
network, uplinks from non-internal switches to the host labeled with
the gateway, and dashed publish-edges from containers to the host for
their published ports. Drag to rearrange, scroll to zoom, re-settle
kicks the physics. Below the graph: containers table + grouped network
cards as a textual mirror. 12 docker_view tests; verified live (32
containers, 11 switches, real subnets + gateways).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 03:08:39 +02:00
m17hr1l
cb7bef4e40 stage-27: per-member TOTP enrollment + individual revocation
Replaces the single shared admin secret with named per-member
enrollments — no more "one key for everyone". First visit bootstraps an
'owner' slot; further members are added from inside the admin panel,
each scanning their own QR. Login accepts a code matching any active
member and records who got in. Offboarding is a per-member revoke: that
person's codes stop immediately, everyone else is unaffected, nobody
re-enrolls. Old single-secret state migrates to an 'owner' member.

Admin panel gains an Access Control table (member, enrolled, last used,
revoke) + add-member form that shows the new QR once. 7 tests including
revocation isolation; verified the full lifecycle live (bootstrap → add
→ authenticate → revoke → rejected while owner persists).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 00:46:45 +02:00
m17hr1l
73a932d8be stage-25: response actions — human-gated enforcement + the disco
Closes the loop: intel -> decision -> enforcement -> audit. High/critical
cases propose response actions (alert SOC, push IOCs to perimeter
firewall+DNS). Nothing fires automatically — each sits PROPOSED until a
human approves, then it's POSTed to the enforcement sink (PSYC_SOAR_URL,
default mock-cert /soar/enforce) and written to the ledger as ACTIONED.

- models: ActionType / ActionStatus / ResponseAction
- db: response_actions table
- lines/respond.py: propose_for_case (idempotent, sev-gated), execute_action
  (fire + ledger + mark), reject_action; mock SOAR endpoint in mock_cert
- cockpit /response page: proposed/enforced/declined tabs,  Enforce +
  decline, and the disco — a full-screen strobe + "ENFORCED" + IOC-scatter
  animation that fires on approval (respects prefers-reduced-motion)
- cli: respond / actions / act-approve / act-reject
- 8 tests; verified the full loop live (propose -> enforce -> disco ->
  SOAR receipt -> ledger ACTIONED row)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 00:24:31 +02:00
m17hr1l
9a2a31ec9a stage-23: IOC index + lookup — the actionable keystone
New iocs table (value, type, case_id, feed, severity, first_seen) +
lines/lookup.py: normalize() (CVE upper, rest lower), reindex() to
rebuild from the corpus, lookup() (normalization-insensitive, scans all
types), export_blocklist() (deduped, min-severity filter).

CLI: psyc reindex / lookup <indicator> / export-blocklist --type --min-severity.

Verified on the live corpus: 1288 IOCs from 598 cases; lookup of a real
IP/CVE resolves to its case+feed+severity; 8.8.8.8 correctly misses;
blocklist export yields 148 IPs / 289 domains / 150 URLs / 514 hashes /
108 CVEs. This primitive backs the upcoming search UI, asset matching,
and watchlist alerting.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 23:39:05 +02:00
m17hr1l
f6fa52839f stage-20: defanging pipeline for IOC-extraction augmentation
Real CTI prose defangs IOCs (1[.]2[.]3[.]4, hxxp://, evil[dot]com) so they
don't auto-link in email/chat. A model trained only on canonical inputs
will fail to extract them.

New lines/defang.py: defang_ip, defang_domain, defang_url, defang_text —
four dot-styles ([.], (.), [dot], {.}) plus protocol defanging
(http→hxxp, https→hxxps). Each occurrence picks its style independently
since real advisories don't keep one style across paragraphs.

train.BuildOptions adds defang_frac (default 0.0) and seed; build()
threads options + a seeded Random through the example builders so
the augmentation is reproducible. Only _ex_ioc_extraction reads it
today — output stays canonical so the model learns messy→canonical.

CLI: train-build and train-build-all gain --defang-frac and --seed.
8 new tests including a frac=1.0 / output-canonical integration check.
The pipeline runs but is dormant at defang_frac=0.0 — psyc-v5 dataset
build will set 0.5 once OTX cases land.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 22:33:52 +02:00
m17hr1l
85830be9fa stage-19-fix: ThreatFox + MalwareBazaar — real API shape
Live test against abuse.ch revealed two issues with the stage-19 wiring:

- ThreatFox returns `ioc` (not `ioc_value`) and `first_seen` (not
  `first_seen_utc`) — older field names from stale docs. Parser now reads
  the real names and falls back to the old aliases defensively. Also
  captures `malware_malpedia` (per-family writeup URL) and
  `threat_type_desc` for richer downstream prose.
- MalwareBazaar's API expects form-encoded bodies, unlike ThreatFox's
  JSON. Extended _http with form_body=; MB fetcher switched to it.

Verified live: 10 ThreatFox cases landed with mixed botnet/malware
classification (4/6 split from threat_type signal — first real
incident-type diversity from a single feed). 10 MalwareBazaar cases
landed with sha256+sha1 hash observables and exe/file_type metadata.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 22:25:56 +02:00
m17hr1l
d87bd710bb stage-19: ThreatFox + MalwareBazaar + OTX Scoutline sources
Three new feeds — biggest near-term data-diversity win. ThreatFox brings
multi-malware IOCs with threat_type signal (botnet_cc → BOTNET,
payload_delivery → MALWARE, phishing → PHISHING). MalwareBazaar brings
file-hash samples with signatures. OTX brings curated multi-source pulses
with paragraph-form descriptions — by far the richest real-prose source.

Auth: THREATFOX_AUTH_KEY (one abuse.ch key covers ThreatFox + MalwareBazaar)
and OTX_API_KEY. fetch-all skips keyed feeds cleanly with where-to-get-it
guidance instead of tracebacking. Proofline reliability table extended;
abuse.ch sources rated B/2, OTX rated C/3 (community-driven).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 22:14:18 +02:00
m17hr1l
994a5c642f stage-18: approval queue — human gate before evidence leaves
CERT-Bund (authority) requires_approval by default; PSYC_REQUIRE_APPROVAL=1
forces every routable submission through the queue. Courier branches at
execute_routes: approval-required → freeze payload + enqueue, no HTTP; else
submit directly as before. Approve dispatches the frozen payload to mock-cert
and writes the ledger row (detail=approved_by=…); reject writes a ledger row
with the reviewer's reason. CLI: queue / approve / reject. Cockpit /queue
page with POST approve / reject and counts.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 21:42:08 +02:00
m17hr1l
e504b3dbcf stage-14: pytest test suite over the worker lines
38 tests covering the pure worker-line logic: Classifyline rules, Routeline
TLP/country/incident-type gates, Sealine seal/unseal round-trip, Proofline
confidence scoring, Mapline CVEResolver escalation, Trainline dataset
well-posedness (the v1/v3 input-signal bugs are now regression-guarded), and
the Scoutline feed parsers. pytest added as a dev extra.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 23:36:41 +02:00