Commit Graph

80 Commits

Author SHA1 Message Date
m17hr1l
7a510c7acf stage-trans-a translog: append-only signed merkle chain + tests
translog.append computes
sha256(canonical({prev_hash, entry_type, entry_data, timestamp})) and
writes one row per call; the first entry uses prev_hash = "0"*64.
verify_chain walks rows in id order, re-hashes each, and returns
Err("broken at id=X expected=... got=...") on the first mismatch — so
tampering with either entry_data or prev_hash invalidates every
downstream row. recent / entries_after / head support peer sync and UI.

Tests cover: genesis prev_hash, chained prev_hash, full-chain verify,
tampered-data detection, tampered-prev_hash detection, slicing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 21:09:32 +02:00
m17hr1l
4a9f6ceb7f stage-vouch-a federation: vouches table + DB helpers
Add `vouches` (voucher_fp, target_fp, issued_at, expires_at, signature)
with unique (voucher, target) and target index, plus `translog` for the
append-only signed merkle chain (id, prev_hash, entry_type, entry_data,
timestamp, entry_hash). Also surface `setting_get` / `setting_set`
helpers on pulse_settings so the quorum config has a place to live.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 21:09:25 +02:00
m17hr1l
1675a2326e stage-33 wire pulse + federation: cockpit routes, CLI, nav links, SW bump 2026-06-06 16:15:48 +02:00
m17hr1l
de5ff09815 merge federation: ed25519 identity + signed feeds
# Conflicts:
#	src/psyc/db.py
2026-06-06 16:13:36 +02:00
m17hr1l
02ce6d791c merge pulse: scheduler line + autonomy dial 2026-06-06 16:11:17 +02:00
m17hr1l
d4229dd264 stage-fed-g federation: tests 2026-06-06 16:10:31 +02:00
m17hr1l
2ef0448165 stage-fed-f federation: CLI commands 2026-06-06 16:10:26 +02:00
m17hr1l
17b94acf6b stage-fed-e federation: cockpit admin page + public feed routes 2026-06-06 16:10:19 +02:00
m17hr1l
55ffd9da3d stage-fed-d federation: signed feed export + verified import 2026-06-06 16:09:53 +02:00
m17hr1l
63e3ff2777 stage-fed-c federation: db tables for peers + signal buffer 2026-06-06 16:08:36 +02:00
m17hr1l
50158f7fa8 stage-fed-b federation: dns record format 2026-06-06 16:08:31 +02:00
m17hr1l
4c35aad2bb stage-fed-a federation: ed25519 keypair + fingerprint 2026-06-06 16:08:03 +02:00
m17hr1l
a7c59c9faa stage-33e pulse: tests 2026-06-06 16:06:54 +02:00
m17hr1l
e071f289f2 stage-33d pulse: CLI commands 2026-06-06 16:05:14 +02:00
m17hr1l
26fbe08b65 stage-33c pulse: admin cockpit page 2026-06-06 16:04:39 +02:00
m17hr1l
4d67605371 stage-33b pulse: db tables + persistence 2026-06-06 16:03:30 +02:00
m17hr1l
e710be6ebd stage-33a pulse: scheduler module with pipeline registry 2026-06-06 16:03:22 +02:00
m17hr1l
6356c5535b stage-32: add Umami analytics tracking script in base.html
Inserted the analytics.neuronetz.ai tracking script tag in the <head>
of base.html with the project's website-id. defer attribute means it
never blocks page render. Cookieless / privacy-respecting (Umami).
Applies to every page that extends base — the admin gate's standalone
template intentionally skips it (no point tracking the login screen).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 20:09:54 +02:00
m17hr1l
43c7c199c3 stage-31 polish: featured hero — particles sync with sweep, stat chips, radar emblem
1. Particles now blink in time with the scan column.
   Each <circle>/<rect> in the case hero SVG carries class="hero-particle"
   and a CSS animation-delay equal to -(arrival-time-of-sweep-at-its-x),
   so as the sweep marches left-to-right the particles light up in
   sequence — a real "the scanner just touched this point" effect. The
   sweep is now linear / non-alternating on a 12s cycle so the phase
   stays predictable. Drop-shadow + opacity flash at the peak.

2. Stat chips added under the featured title.
   Up to five compact chips showing incident type, total IOC count with
   per-type breakdown (U/D/I/H/C), victim country if mapped, confidence
   level, malware family. Each chip is color-coded.

3. Radar emblem in the top-right of the hero.
   Pure SVG: three concentric range rings, cross-hairs, a translucent
   sweep arm that spins every 4.5s, a center dot. The whole emblem
   inherits the cycling-hero hue cycle so it changes color with the
   grid. Smaller + dimmer on mobile; honors prefers-reduced-motion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 19:38:31 +02:00
m17hr1l
977c3670f3 stage-31 polish: featured hero — full-bleed animated grid cycling colors
The hero is now layered:
  base   — radial accent + dark gradient (truly full-width, min 240px /
           up to 320px tall, no aspect-ratio cap)
  grid   — full-bleed CSS grid (38px cells), line color cycles through
           cyan → red → gold → green → violet over 18s, plus a slow
           grid-drift so the lattice subtly slides; a glowing column
           sweeps left-to-right across the hero every 12s for depth
  particles — the existing case-specific SVG sits on top at lower
           opacity (mix-blend-mode: screen) so the per-case identity
           is preserved as a constellation glittering over the grid
  overlay — title + Open-case CTA pinned to the bottom

Severity just nudges the grid opacity (CRITICAL/HIGH get a bit denser);
the color cycle stays on for all severities. Honors prefers-reduced-motion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 19:32:27 +02:00
m17hr1l
3f1f7cc420 stage-31 polish: featured-threat banner — uniform tint, full width
The banner gradient was a horizontal accent → dark fade, so visually
only the left ~30% read as a banner; the right looked unstyled. Swapped
for a uniform severity-tinted background (subtle top-to-bottom gradient
for depth, but no horizontal falloff). Border + inset shadow on the
bottom edge give it a real divider between the banner and the hero
beneath. Per-severity tints (CRITICAL red, HIGH amber) get the same
treatment — full-width, uniform.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 19:28:26 +02:00
m17hr1l
04e0d3323f stage-31 fix: featured card — display:block on the new <a> wrapper
When I refactored the featured card from <section>→<a> for full-card
clicking, I forgot the <a> defaults to display:inline. That collapsed
the hero+overlay layout — the title rendered word-per-line down the
left edge instead of spanning the hero. display:block (+ inherit
color, no text-decoration) restores the layout.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 19:15:48 +02:00
m17hr1l
5cf7cb5655 stage-31 polish: featured banner header + clickable news cards w/ hover
Two enhancements from the screenshot annotations:

1. The featured card now has a proper banner header above the hero —
   gradient strip with "⌖ Featured Threat" label, a pulsing severity
   dot, severity/TLP badges, source + ingest time. The header is
   sev-themed (red border + glow for CRITICAL, amber for HIGH), and an
   "Open case →" CTA at the bottom of the hero with an arrow that
   nudges right on hover.

2. News items are now full-card clickable (transparent overlay link)
   with rich severity-tinted hover transitions:
   - Lifts (translateY -1px), tinted background + border that matches
     severity (red CRITICAL, amber HIGH, pale MEDIUM/LOW), per-kind
     accents for enforced/submitted/rejected/failed items.
   - A right-side arrow (→) slides in on hover signaling "click to open".
   - Case glyph SVG gets a subtle drop-shadow + 1.04x scale on hover.
   - 180ms ease transitions across background/border/transform/shadow.
   - Respects prefers-reduced-motion (turns animations off).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 19:12:55 +02:00
m17hr1l
f51e672ad3 stage-31 fix: home page 500 — replace bad jinja sum with precomputed count
Template had `{{ buckets|sum(attribute='items')|length }}` which tried
to sum lists starting at 0 → TypeError. Switched to a Python-side
`total_items = sum(len(b.items) for b in buckets)` passed through the
context and rendered directly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 19:05:25 +02:00
m17hr1l
76a0b0b636 stage-31: Newsline polish — featured case, time buckets, severity accents, generated visuals
The home page goes from a flat event stream to something that reads like
a news blog:

- Featured-case hero card at the top, picked as the highest-severity
  case (CRITICAL > HIGH, tie-break recency) from the last 7 days. Wide,
  with a procedurally generated SVG hero behind a gradient overlay that
  carries title + severity + TLP + feed + ingest time.
- Recent activity is now grouped under Today / Yesterday / Earlier this
  week / Older bucket headers.
- Each item gets a left-border severity accent (red CRITICAL, amber
  HIGH, muted MEDIUM/LOW) so the page is scannable at a glance.

Images: new cockpit/case_visuals.py generates SVGs from case data —
zero external image gen, zero curated assets. Every visual is
deterministic from case_id (so a case keeps its identity across
sessions) and themed to its severity:

- case_hero_svg() — 880x220 hero with severity radial glow, a faint
  scan grid, a particle constellation with auto-connecting lines, HUD
  corner brackets, and the case id whispered in the bottom-right.
- case_glyph_svg() — small mirror-symmetric identicon (5-grid),
  severity-colored, shown beside each case news item in place of an
  emoji icon. Two case_ids → two distinct glyphs; same id → same glyph.

7 news tests pass; visual sanity print confirms hero is deterministic
and uses the right severity accent.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 19:02:22 +02:00
m17hr1l
4d36db90f1 stage-30 fix: SW cache strategy — bump version + stale-while-revalidate
User reported mobile nav still looked unresponsive. Verified the new
nav-toggle CSS and markup are deployed; the browser's service worker
was serving the prior cockpit.css cache-first from psyc-v1.

Two fixes so this can't recur:
- CACHE_VERSION bumped to psyc-v2 → the activate handler now wipes
  the v1 cache on the user's next visit.
- Static-asset strategy changed from cache-first to
  stale-while-revalidate: serve from cache for instant render, then
  refresh in the background so the *next* load picks up the new CSS/JS
  with no manual cache-busting. Falls back to network if cold.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:36:08 +02:00
m17hr1l
88e4fb1dcd stage-30 fix: proper responsive nav (hamburger drawer) + cases-pipeline fix
Replaced the horizontal-scroll hack with a real mobile pattern:
- Hamburger button (☰) appears below 760px width, animates into an X when
  open. Tap to slide the full nav into a drawer below the topbar, each
  link a tap-target-sized row.
- Nav links auto-close the drawer on click so navigating dismisses it.
- aria-expanded / aria-controls wired up so screen readers track state.
- Below 420px the model status chip + family icon hide to keep the
  topbar compact; brand-sub already hidden by the earlier mobile pass.

Discovered while debugging: prod had 377 unclassified cases (only
fetch-all was run, not classify-all/prove-all/reindex), so the journey
page's Classifier beat skipped the live model verdict entirely. Ran
the full pipeline on prod — the psyc-v5 model chip now renders on
case journeys. Worth noting for the deploy runbook: after psyc fetch-all
on a new env, always also run classify-all + prove-all + reindex to
prime the pipeline state.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:27:11 +02:00
m17hr1l
16cf873044 stage-30: home page (Newsline digest) + PWA + mobile pass
NEW / start page replaces the redirect-to-/cases:
- KPI strip (cases, IOCs, +24h, high/critical,  enforced 24h, ledger total) —
  clickable, responsive grid (2 cols mobile, 3 mid, 6 desktop).
- Recent activity feed: ledger events (enforced/submitted/rejected/failed) +
  newest case ingests, interleaved newest-first, with severity badges, icons,
  case links. Sources via lines/news.py.
- Feed health sidebar: per-feed count + last ingest time.

PWA:
- /static/manifest.json declares a standalone install with theme colors.
- /static/sw.js — cache-first for static, network-first for HTML/API, with a
  graceful offline page. Registered from / scope via a dedicated /sw.js route
  that sets Service-Worker-Allowed: /.
- viewport + apple-touch-icon + theme-color meta tags in base.html.

Mobile pass on the chrome:
- Topbar wraps; nav horizontally scrolls instead of crowding; brand-sub hides.
- Tables (cases, ledger) scroll horizontally on narrow screens instead of
  exploding the layout.
- Hero / KPI / news-list layouts collapse cleanly at < 720px.

4 news tests; verified locally — home page renders, /sw.js serves with
Service-Worker-Allowed: /, manifest is valid JSON.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:18:40 +02:00
m17hr1l
7a57a7390a stage-29 fix: inference service — wire build: directive in compose
The inference service declared 'image: psyc-trainer' but no build: stanza,
so 'docker compose build inference' was a silent no-op and 'compose up
inference' tried to pull from a registry that doesn't exist (denied).
Added build context pointing at Dockerfile.train so future production
deploys can rebuild it via the compose lifecycle instead of needing a
manual 'docker build' on the side.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:08:50 +02:00
m17hr1l
d7999150b3 stage-29: fetch-all resilience + Mozilla-compatible UA for CISA
Two production-discovered fixes after first deploy:

- CISA's CDN was 403'ing the "psyc/0.1 (defensive CTI; hackathon
  prototype)" User-Agent from the cloud.neuronetz.ai exit IP. Switched
  to a Mozilla-compatible UA that identifies us honestly while passing
  the CDN's UA filters. Overridable via PSYC_HTTP_USER_AGENT.
- fetch-all aborted on the first HTTPStatusError, so a CISA hiccup
  killed the threatfox/malware-bazaar/otx legs that come after. The
  outer loop now catches any exception per-source, logs a skip, and
  moves on. Single-source failures no longer poison the rest of the
  pull.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 16:56:27 +02:00
m17hr1l
fad7ad0d49 stage-28: make the proxy docker network name configurable per environment
On AnD0R the reverse-proxy lives on the 'backend' docker network; on
cloud.neuronetz.ai it's 'neuronetz_default'. With a hardcoded name the
cockpit ended up on a network the prod proxy couldn't see and routing
silently dropped. Network is now overridable via PSYC_PROXY_NETWORK in
.env (default 'backend' keeps dev working).

On prod, set PSYC_PROXY_NETWORK=neuronetz_default in .env before the
next compose up.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 16:45:47 +02:00
m17hr1l
92f754e012 stage-28: wire LETSENCRYPT_HOST + LETSENCRYPT_EMAIL on the cockpit service
Adds the two env vars nginxproxy/acme-companion looks for to issue +
auto-renew the TLS cert for psyc.neuronetz.ai. LETSENCRYPT_EMAIL is
interpolated from the prod .env (LETSENCRYPT_EMAIL=...) with a sensible
fallback so dev / local deploys don't fail on the variable being unset.
.env.example documents the var.

Requires the proxy stack to (a) have acme-companion alongside
nginx-proxy with shared certs/vhost.d/html volumes and (b) publish :443.
psyc-side change only — no app code touched.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 16:42:46 +02:00
m17hr1l
9c3447723a stage-28 fix: deploy.sh — auto-trust Gitea host (TOFU), never touch identity keys
Reinstating the auto known_hosts entry on first deploy. Clear scope:
host trust (TOFU known_hosts entry) is automated — same as
'ssh -o StrictHostKeyChecking=accept-new' would do; identity keypairs
(~/.ssh/id_*) are never generated/copied/modified by deploy.sh.
PSYC_SKIP_HOST_TRUST=1 disables the auto-trust step if you'd rather
verify fingerprints manually.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 16:36:18 +02:00
m17hr1l
9edd56e28b stage-28 fix: deploy.sh — read-only SSH preflight, no key/known_hosts edits
User asked the script not to touch their SSH config. Reverted the
auto-ssh-keyscan; the script now only READS ~/.ssh/known_hosts (via
ssh-keygen -F) and, when the entry is missing, exits with explicit
manual instructions for verifying the host key and registering an
identity key in Gitea. Identical behavior on the happy path; clearer
diagnostics on the unhappy path; zero modification of ~/.ssh anywhere.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 15:39:06 +02:00
m17hr1l
2c2ead6149 stage-28 fix: deploy.sh pre-trusts the Gitea SSH host key (first-clone)
A fresh prod box has never SSH'd to gitea.neuronetz.ai before, so the
first 'git clone' failed with 'Host key verification failed'. The
script now parses the git remote URL to extract host+port, and on the
prod box does an ssh-keyscan into ~/.ssh/known_hosts before the clone
when the entry is missing. TOFU — if you want to verify the fingerprint
out-of-band, pre-populate known_hosts manually and the script will see
the entry and skip the scan.

Also: if the clone still fails after the host key is trusted (likely a
missing SSH key on Gitea side), the script now prints a clear hint
pointing at where to register it. Supports both ssh://user@host:port/
and user@host: URL forms.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 15:32:44 +02:00
m17hr1l
61b7b8ef20 stage-28: deploy.sh — idempotent remote deploy + health probe
scripts/deploy.sh pushes the current branch to origin, ssh's into the
prod box (neuronetz@cloud.neuronetz.ai:/home/neuronetz/docker-public/
neuro-psyc by default — overridable via env vars), clones-or-pulls,
ensures the external 'backend' docker network exists, runs docker
compose up -d --build (+ --profile gpu if PSYC_PROD_GPU=1), and then
verifies the cockpit is healthy both on prod-internal :8767 and at the
public URL — so the script ends knowing whether the page is up.

Refuses to touch prod's .env (warns + copies .env.example if missing,
so you can edit it manually). Never transfers data/ or adapters
(gitignored; prod fetches its own corpus). Color output, idempotent,
safe to re-run.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 14:51:47 +02:00
m17hr1l
494755ec4f stage-26d: click any topology node → structured spec panel below
Click a container, switch, or the host in the graph: the node gets a
pulsing accent highlight and a detail panel below renders its full
logical overview — for a container, image/status/per-network IPv4+MAC+
gateway + published ports + all ports; for a switch, driver/scope/
subnet/gateway/internal + attached-container table; for the host, OS/
CPUs/container counts. Cross-jumps in the spec tables click through to
the related node (the graph re-selects too). Click the × button or any
empty graph area to deselect.

Drag is distinguished from click by a 4px move threshold so dragging
nodes doesn't accidentally open the panel. HTML escaped end-to-end.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 12:25:15 +02:00
m17hr1l
ef88cd9d5d stage-26c: topology layout views, traffic flow, full-width page
Three layout modes (Force/Hierarchical/Radial) switch the topology graph
between organic force-directed, host→switches→containers tiers, and a
radial host-in-center wheel. Animated traffic flow on edges via
stroke-dashoffset marching: wires to running containers flow cyan,
host↔switch uplinks pulse slower in violet, container→host publish
edges flash fastest in amber-dashed; dead edges (exited containers) fade.
"traffic flow" checkbox kills the animation globally; layout choice
auto-fixes nodes (drag still overrides). Respects prefers-reduced-motion.

/admin/docker now opts into a wide layout via a new body_class block on
base.html — content area uncaps, SVG sized min(74vh, 880px), network
cards get wider min columns. Other pages keep the 1280px reading cap.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 12:20:18 +02:00
m17hr1l
b51a88d502 stage-26b: Docker topology in /admin — read-only socket-proxy + graph
New tecnativa/docker-socket-proxy sidecar exposes only GET on
containers/networks/info/ping; POST and DELETE are blocked. The cockpit
queries it over the backend network — /var/run/docker.sock is never
mounted into a web-facing container.

cockpit/docker_view.py normalizes the daemon view: containers carry
per-network IP/MAC + published_ports; networks carry subnet/gateway from
IPAM; host_info pulls /info (degrades gracefully). topology() returns
the combined snapshot.

/admin/docker (admin-gated): a force-directed graph (pure SVG +
vanilla JS, ~280 lines) renders the complete setup — a host node,
switch nodes with subnet labels colored by driver, container nodes
colored by state, member wires labeled with the container's IP on that
network, uplinks from non-internal switches to the host labeled with
the gateway, and dashed publish-edges from containers to the host for
their published ports. Drag to rearrange, scroll to zoom, re-settle
kicks the physics. Below the graph: containers table + grouped network
cards as a textual mirror. 12 docker_view tests; verified live (32
containers, 11 switches, real subnets + gateways).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 03:08:39 +02:00
m17hr1l
eaca27be26 stage-27 polish: admin presence announces itself in the topbar
Once you're signed in, the topbar gains:
- An "Admin" nav link (cyan-accented, with a small shield-lock SVG)
  appearing only when an admin session exists.
- A glowing chip on the right showing "● ADMIN · <who>" with a
  separated lock button (⏻) for sign-out. Pops in with a scale-bounce
  entrance, then a slow box-shadow pulse so it stays noticeable without
  being annoying. Respects prefers-reduced-motion.
- Both rendered via request.session in base.html, so every page gets
  them automatically with no per-route plumbing.

Removed the now-redundant in-panel "lock ⏻" from admin.html — the
topbar owns logout, the admin page header shows enrolled count instead.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 01:14:58 +02:00
m17hr1l
cb7bef4e40 stage-27: per-member TOTP enrollment + individual revocation
Replaces the single shared admin secret with named per-member
enrollments — no more "one key for everyone". First visit bootstraps an
'owner' slot; further members are added from inside the admin panel,
each scanning their own QR. Login accepts a code matching any active
member and records who got in. Offboarding is a per-member revoke: that
person's codes stop immediately, everyone else is unaffected, nobody
re-enrolls. Old single-secret state migrates to an 'owner' member.

Admin panel gains an Access Control table (member, enrolled, last used,
revoke) + add-member form that shows the new QR once. 7 tests including
revocation isolation; verified the full lifecycle live (bootstrap → add
→ authenticate → revoke → rejected while owner persists).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 00:46:45 +02:00
m17hr1l
4a832964a3 stage-26 polish: restyle the /admin gate as a secure console
Replaced the plain login box with a proper restricted-zone screen:
ambient breathing radial backdrop, frosted glass card with a glowing
top accent + animated HUD corner brackets, an SVG shield-lock emblem in
a pulsing glow ring, a SECURE CHANNEL status line with a blinking dot,
the QR in a white frame with a sweeping cyan scanline, a large
OTP-style monospace code field, and a gradient UNLOCK button. Honors
prefers-reduced-motion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 00:38:47 +02:00
m17hr1l
abdf5e7747 stage-26: hidden /admin gated by TOTP (authenticator-app 2FA)
A hidden /admin path (not in nav) protected by a TOTP secret you enroll
by scanning a QR into Google Authenticator / Authy, then entering the
rotating 6-digit code. adminauth.py persists the secret + session key
under DATA_DIR (gitignored); the QR only renders until first successful
verification so the provisioning secret isn't perpetually exposed.
SessionMiddleware carries a 60-min admin session. This becomes the
secured control center the rest of the system gets built into.

Verified end-to-end: gate renders QR, the live code authenticates and
sets the session, the dashboard renders only with the session cookie,
a wrong code is rejected, and an unauthenticated request never leaks
the dashboard. Deps: pyotp, qrcode[pil], itsdangerous.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 00:35:02 +02:00
m17hr1l
73a932d8be stage-25: response actions — human-gated enforcement + the disco
Closes the loop: intel -> decision -> enforcement -> audit. High/critical
cases propose response actions (alert SOC, push IOCs to perimeter
firewall+DNS). Nothing fires automatically — each sits PROPOSED until a
human approves, then it's POSTed to the enforcement sink (PSYC_SOAR_URL,
default mock-cert /soar/enforce) and written to the ledger as ACTIONED.

- models: ActionType / ActionStatus / ResponseAction
- db: response_actions table
- lines/respond.py: propose_for_case (idempotent, sev-gated), execute_action
  (fire + ledger + mark), reject_action; mock SOAR endpoint in mock_cert
- cockpit /response page: proposed/enforced/declined tabs,  Enforce +
  decline, and the disco — a full-screen strobe + "ENFORCED" + IOC-scatter
  animation that fires on approval (respects prefers-reduced-motion)
- cli: respond / actions / act-approve / act-reject
- 8 tests; verified the full loop live (propose -> enforce -> disco ->
  SOAR receipt -> ledger ACTIONED row)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 00:24:31 +02:00
m17hr1l
d0a71d0226 stage-24: indicator lookup page + blocklist download in cockpit
Surfaces the stage-23 index in the UI. New /lookup page: paste any
indicator (IP/domain/URL/hash/CVE) → red KNOWN-BAD banner with the
matching cases/feeds/severities, or green clean banner. New
/export/blocklist endpoint returns deduplicated plain-text indicator
lists (all or high+ severity) for firewall/DNS/SIEM ingestion, linked
from a download table on the lookup page. Lookup added to topbar nav.

Verified live: lookup of a real corpus IP returns the OTX case;
8.8.8.8 returns clean; blocklist endpoint emits 26 high-severity IPs
with a descriptive header line.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 23:55:50 +02:00
m17hr1l
9a2a31ec9a stage-23: IOC index + lookup — the actionable keystone
New iocs table (value, type, case_id, feed, severity, first_seen) +
lines/lookup.py: normalize() (CVE upper, rest lower), reindex() to
rebuild from the corpus, lookup() (normalization-insensitive, scans all
types), export_blocklist() (deduped, min-severity filter).

CLI: psyc reindex / lookup <indicator> / export-blocklist --type --min-severity.

Verified on the live corpus: 1288 IOCs from 598 cases; lookup of a real
IP/CVE resolves to its case+feed+severity; 8.8.8.8 correctly misses;
blocklist export yields 148 IPs / 289 domains / 150 URLs / 514 hashes /
108 CVEs. This primitive backs the upcoming search UI, asset matching,
and watchlist alerting.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 23:39:05 +02:00
m17hr1l
f88db2fdf7 stage-22: cockpit reflects the live adapter + all six feeds
The Worker Mesh Classifier badge was hardcoded "psyc-v4" — it kept
claiming the old model even after the v5 swap, making it look like
nothing had changed. Now derived dynamically from the inference
server's reported adapter via inference.adapter_name(); shows the real
version (psyc-v5) or "rules" when the server is down, so it can't go
stale on the next swap.

Also refreshed the cases page intro/help text to name all six feeds
(URLhaus, CISA KEV, Feodo, ThreatFox, MalwareBazaar, OTX) instead of
the original three.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 23:33:19 +02:00
m17hr1l
ee387abcd4 stage-21: swap inference server to psyc-v5 adapter
v5 trained on 598 ex/task (20× v4's 30), with --defang-frac 0.5 over
the new ThreatFox + MalwareBazaar + OTX corpus. Final train_loss 0.3225
vs v4's 0.7397 (56% reduction), 60m20s wall clock on a 3090.

Live eval before swap:
- severity (botnet, ONLINE): v4 high / v5 high — tied, both correct
- ioc_extraction with defanged input (hxxps://, [.], (.), [dot]):
    v4 kept hxxps:// in output (failed canonicalization)
    v5 returned canonical https:// — defang training paid off
- ioc_extraction on real OTX-style prose (never trained on this shape):
    v5 cleanly extracted 2 domains + 1 IP + 1 SHA256 + 1 CVE

Cockpit /api/inference-status confirms the swap:
  {"online":true,"adapter":"/data/adapters/psyc-v5/final"}

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 23:55:47 +02:00
m17hr1l
376c5b6f4a stage-19-fix2: OTX — narrow by modified_since, longer timeout
The /pulses/subscribed endpoint enumerates every curated feed a fresh
account is auto-subscribed to. On its own that's enough to 504 from
OTX's backend regardless of client timeout. Narrowing by
modified_since=now-7d brings the response back to a single-second fetch.

Also: _http now accepts params + per-call timeout overrides (OTX uses
120s). The CLI --limit still slices post-fetch.

Verified live: 10 OTX pulse-cases ingested, each carrying real
paragraph-form descriptions (Mirai, macOS Stealer, FlowerStorm PhaaS,
Vidar v1.5, manufacturing intrusion) — exactly the real-prose source
the IOC extractor's been missing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 22:39:24 +02:00
m17hr1l
f6fa52839f stage-20: defanging pipeline for IOC-extraction augmentation
Real CTI prose defangs IOCs (1[.]2[.]3[.]4, hxxp://, evil[dot]com) so they
don't auto-link in email/chat. A model trained only on canonical inputs
will fail to extract them.

New lines/defang.py: defang_ip, defang_domain, defang_url, defang_text —
four dot-styles ([.], (.), [dot], {.}) plus protocol defanging
(http→hxxp, https→hxxps). Each occurrence picks its style independently
since real advisories don't keep one style across paragraphs.

train.BuildOptions adds defang_frac (default 0.0) and seed; build()
threads options + a seeded Random through the example builders so
the augmentation is reproducible. Only _ex_ioc_extraction reads it
today — output stays canonical so the model learns messy→canonical.

CLI: train-build and train-build-all gain --defang-frac and --seed.
8 new tests including a frac=1.0 / output-canonical integration check.
The pipeline runs but is dormant at defang_frac=0.0 — psyc-v5 dataset
build will set 0.5 once OTX cases land.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 22:33:52 +02:00