m17hr1l 2a9c0bf34a stage-6: model inference server ...

scripts/serve_model.py — FastAPI in the CUDA container, loads base Qwen3.5-4B
+ a psyc adapter once and serves POST /infer. Lets the cockpit (no torch in
its venv) put a real fine-tuned model behind a Worker Mesh bot over HTTP.
Dockerfile.train gains a fastapi + uvicorn layer.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-18 21:05:16 +02:00

1.4 KiB

Raw Blame History

View Raw