m17hr1l/psyc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

stage-9: consolidate into one compose stack behind nginx-proxy

Browse Source 
psyc now runs as a single docker compose stack — cockpit + mock-cert +
(gpu-profile) inference — on the shared external `backend` network, fronted
by nginx-proxy as psyc.neuronetz.ai. Replaces the venv processes + one-off
docker run. MOCK_CERT_BASE and INFERENCE_URL are now env-configurable
(PSYC_MOCK_CERT_URL / PSYC_INFERENCE_URL) so the cockpit reaches the other
services by compose service name. Restart policies + healthchecks. deploy.md
rewritten to match.

Verified: cockpit serves directly and via the proxy; the full
scout→…→courier→ledger chain runs over the compose network.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
m17hr1l
2026-05-18 22:57:33 +02:00
parent e54242178f
commit 372ee72353
4 changed files with 94 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files

72
docker-compose.yml
View File

@@ -1,37 +1,71 @@
# psyc — company-network deployment (cockpit + mock destination receiver). # psyc — neuronetz.ai deployment stack.
# #
# docker compose up -d --build # docker compose up -d --build # cockpit + mock-cert (no GPU)
# docker compose --profile gpu up -d --build # + the live model (needs an NVIDIA GPU)
# #
# WARNING: psyc has no built-in authentication. The cockpit exposes cases, the # The cockpit is fronted by the external `backend` network's nginx-proxy as
# ledger, and sealed-package metadata to anyone who can reach port 8767. Deploy # psyc.neuronetz.ai (point DNS for that name at the proxy host). mock-cert and
# behind the company reverse proxy / SSO / VPN, or firewall the ports to the # the inference server stay internal — no VIRTUAL_HOST, reachable only inside
# SOC subnet. See docs/deploy.md. # `backend` by service name.
#
# WARNING: psyc has no built-in authentication. The reverse proxy / network
# perimeter is the security boundary. See docs/deploy.md.
services: services:
cockpit: cockpit:
build: . build: .
image: psyc:latest image: psyc:latest
command: ["psyc", "serve", "--host", "0.0.0.0", "--port", "8767"] command: ["psyc", "serve", "--host", "0.0.0.0", "--port", "8767"]
environment:
VIRTUAL_HOST: psyc.neuronetz.ai
VIRTUAL_PORT: "8767"
PSYC_MOCK_CERT_URL: http://mock-cert:8770
PSYC_INFERENCE_URL: http://inference:8771
ports: ports:
- "8767:8767" - "8767:8767" # direct/debug access; the proxy serves psyc.neuronetz.ai on :80
volumes: volumes:
- psyc-data:/data - ./data:/data
networks: [backend]
restart: unless-stopped restart: unless-stopped
# Behind a company egress proxy, uncomment and set: healthcheck:
# environment: test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8767/healthz')"]
# HTTPS_PROXY: http://proxy.corp:3128 interval: 30s
# HTTP_PROXY: http://proxy.corp:3128 timeout: 5s
retries: 3
mock-cert: mock-cert:
image: psyc:latest image: psyc:latest
command: ["psyc", "mock-cert", "--host", "0.0.0.0", "--port", "8770"] command: ["psyc", "mock-cert", "--host", "0.0.0.0", "--port", "8770"]
ports:
- "8770:8770"
volumes: volumes:
- psyc-data:/data - ./data:/data
depends_on: networks: [backend]
- cockpit
restart: unless-stopped restart: unless-stopped
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8770/healthz')"]
interval: 30s
timeout: 5s
retries: 3
volumes: # The live fine-tuned model behind the Classifier bot. GPU-only — opt in with
psyc-data: # `--profile gpu`. Uses the psyc-trainer image (built from Dockerfile.train).
inference:
image: psyc-trainer
command: ["/scripts/serve_model.py", "--adapter", "/data/adapters/psyc-v4/final", "--host", "0.0.0.0", "--port", "8771"]
volumes:
- ./data:/data
- ./scripts:/scripts
networks: [backend]
restart: unless-stopped
profiles: ["gpu"]
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
networks:
backend:
name: backend
external: true

75
docs/deploy.md
View File

@@ -1,57 +1,59 @@
# psyc — deployment # psyc — deployment
Deploying the psyc platform (cockpit + workers) as Docker containers — e.g. on a psyc deploys as a three-service Docker Compose stack, fronted by an existing
Proxmox-hosted VM in the company network. `jwilder/nginx-proxy` on the shared external `backend` network and served as
`psyc.neuronetz.ai`.
## Read this before deploying ## Read this before deploying
- **No built-in authentication.** The cockpit exposes cases, the ledger, and - **No built-in authentication.** The cockpit exposes cases, the ledger, and
sealed-package metadata to anyone who can reach port 8767. Deploy it **behind sealed-package metadata to anyone who can reach it. The reverse proxy / network
the company reverse proxy / SSO / VPN**, or firewall the ports to the SOC perimeter **is** the security boundary — keep `psyc.neuronetz.ai` on the
subnet. Do not expose 8767 to the open network. (If you want in-app auth internal network or behind SSO. nginx-proxy can add per-vhost basic auth via a
instead of relying on the perimeter, that's a feature to add — not present today.) mounted `htpasswd` file if you need a quick gate. (In-app auth is not built.)
- **The live model is separate.** This image has no GPU and no torch. The - **The live model is GPU-only.** The `inference` service needs an NVIDIA GPU
fine-tuned-model bot needs `serve_model.py` running in the CUDA container on a and the `psyc-trainer` image (`docker build -f Dockerfile.train`). It is gated
GPU host (Proxmox GPU passthrough to a VM). Without it the Classifier bot behind the `gpu` compose profile. Without it the Classifier bot falls back to
falls back to rules — the platform works fine, just rules-only. rules — the platform runs fine.
- **Outbound network.** Scoutline (URLhaus / CISA KEV / Feodo) and Mapline - **Outbound network.** Scoutline (URLhaus / CISA KEV / Feodo) and Mapline
(ip-api.com) make outbound HTTPS. Behind a company egress proxy, set (ip-api.com) make outbound HTTPS. Behind a company egress proxy, set
`HTTPS_PROXY` / `HTTP_PROXY` in the container environment (see the commented `HTTPS_PROXY` / `HTTP_PROXY` on the `cockpit` service.
block in `docker-compose.yml`). - **mock-cert is a stand-in** for real destinations — wire real CERT / MISP /
- **mock-cert is a stand-in.** It accepts submissions for testing — it is not a abuse endpoints (and credentials, per `docs/dossier.md` §18) before relying on
real destination. Wire real CERT / MISP / abuse endpoints (and their routing in production.
credentials, per `docs/dossier.md` §18) before relying on routing in production.
## Proxmox ## Prerequisites
Docker is not native to Proxmox. Run it inside a Proxmox **VM** (recommended — - A Docker host (on Proxmox: a VM running Docker — cleanest; or a privileged LXC).
clean isolation, simplest Docker support) or a privileged LXC. Install Docker + - The external `backend` network and an `nginx-proxy` on it (the shared
the Compose plugin in that guest, give it outbound network for the feeds, then reverse-proxy stack). psyc joins that network; the proxy auto-discovers the
deploy as below. The GPU inference server, if used, needs a separate VM with cockpit by its `VIRTUAL_HOST`.
GPU passthrough. - DNS: point `psyc.neuronetz.ai` at the proxy host.
- For the live model: an NVIDIA GPU + the NVIDIA container runtime, and the
`psyc-trainer` image built.
## Deploy ## Deploy
```bash ```bash
git clone ssh://git@gitea.neuronetz.ai:222/m17hr1l/psyc.git git clone ssh://git@gitea.neuronetz.ai:222/m17hr1l/psyc.git
cd psyc cd psyc
docker compose up -d --build
docker compose up -d --build # cockpit + mock-cert
docker compose --profile gpu up -d --build # + the live model (GPU host)
``` ```
Starts two containers from one `psyc:latest` image: | Service | Exposure | Role |
| Service | Port | Role |
|---|---|---| |---|---|---|
| `cockpit` | 8767 | operator UI | | `cockpit` | `psyc.neuronetz.ai` via the proxy (+ `:8767` direct, debug) | operator UI |
| `mock-cert` | 8770 | stand-in destination receiver (testing) | | `mock-cert` | internal to `backend` only | stand-in destination receiver |
| `inference` | internal to `backend` only · `gpu` profile | live fine-tuned model |
The sqlite db, sealed packages, and recipient keys persist in the `psyc-data` Data (sqlite db, sealed packages, recipient keys, model adapters) is bind-mounted
named volume — they survive container restarts and rebuilds. from `./data` and persists across restarts and rebuilds.
## First run ## First run
The schema is created on cockpit startup, but there are no cases until you The schema is created on cockpit startup; ingest to populate it:
ingest. Run inside the container:
```bash ```bash
docker compose exec cockpit psyc fetch-all docker compose exec cockpit psyc fetch-all
@@ -59,21 +61,18 @@ docker compose exec cockpit psyc classify-all
docker compose exec cockpit psyc map-all docker compose exec cockpit psyc map-all
``` ```
Keep it ingesting by scheduling `fetch-all` — a host cron entry calling Schedule `fetch-all` (host cron → `docker compose exec`) to keep ingesting.
`docker compose exec cockpit psyc fetch-all`, e.g. hourly.
## Updating ## Updating
```bash ```bash
git pull git pull
docker compose up -d --build docker compose --profile gpu up -d --build
``` ```
The `psyc-data` volume is preserved across updates.
## Health ## Health
```bash ```bash
curl http://<host>:8767/healthz # cockpit curl -H 'Host: psyc.neuronetz.ai' http://<proxy-host>/healthz
curl http://<host>:8770/healthz # mock-cert docker compose ps
``` ```

3
src/psyc/cockpit/inference.py
View File

@@ -8,6 +8,7 @@ gracefully — if the server is down, callers get None and fall back to rules.
from __future__ import annotations from __future__ import annotations
import json import json
import os
from typing import Optional from typing import Optional
import httpx import httpx
@@ -17,7 +18,7 @@ from psyc.lines.train import SEVERITY_INSTRUCTION, severity_features
from psyc.models import Case from psyc.models import Case
INFERENCE_URL = "http://127.0.0.1:8771" INFERENCE_URL = os.environ.get("PSYC_INFERENCE_URL", "http://127.0.0.1:8771")
_log = log.get(__name__) _log = log.get(__name__)

3
src/psyc/lines/route.py
View File

@@ -2,6 +2,7 @@
from __future__ import annotations from __future__ import annotations
import os
from typing import Dict, List, Optional, Tuple from typing import Dict, List, Optional, Tuple
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
@@ -15,7 +16,7 @@ _log = log.get(__name__)
_TLP_RANK: Dict[TLP, int] = {TLP.CLEAR: 0, TLP.GREEN: 1, TLP.AMBER: 2, TLP.RED: 3} _TLP_RANK: Dict[TLP, int] = {TLP.CLEAR: 0, TLP.GREEN: 1, TLP.AMBER: 2, TLP.RED: 3}
MOCK_CERT_BASE = "http://127.0.0.1:8770" MOCK_CERT_BASE = os.environ.get("PSYC_MOCK_CERT_URL", "http://127.0.0.1:8770")
ENDPOINTS: Dict[str, str] = { ENDPOINTS: Dict[str, str] = {
"CERT-Bund": f"{MOCK_CERT_BASE}/cert-bund/submit", "CERT-Bund": f"{MOCK_CERT_BASE}/cert-bund/submit",