stage-3c: working QLoRA training + eval — pytorch base, Qwen3.5 slug, SFTConfig

Training and eval now run clean on the unsloth 2026.5.2 / transformers v5 /
torch 2.10 stack. Fixes: pytorch/pytorch base image (sidesteps the nvidia/cuda
apt-signature failure and the torch download), correct base-model slug
unsloth/Qwen3.5-4B, TRL SFTConfig API. Adds scripts/eval_adapter.py — runs
dataset rows through base+adapter with structured (transformers-v5) message
content and Qwen3.5 thinking-mode stripping.

First v1 adapter: loss 2.10 -> 0.32 over 3 epochs. Eval surfaced an ill-posed
ioc_extraction dataset (output URL not present in input) — to be fixed in the
ExampleBuilder before the next training run.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
m17hr1l
2026-05-17 14:16:22 +02:00
parent f1ab11f89d
commit b95e3e02bd
4 changed files with 121 additions and 35 deletions

View File

@@ -121,7 +121,7 @@ To fine-tune Qwen3.5-4B with QLoRA in an NVIDIA Docker container:
# 1. build datasets (one-off; re-run after ingestion changes)
.venv/bin/psyc train-build-all
# 2. build the training image (CUDA 12.4 + unsloth + Qwen3.5)
# 2. build the training image (pytorch 2.6/CUDA 12.4 base + unsloth + Qwen3.5)
docker build -t psyc-trainer -f Dockerfile.train .
# 3. fine-tune (mount host data/ so adapters land there)
@@ -135,14 +135,25 @@ docker run --gpus all --rm \
--output /data/adapters/psyc-v1
```
Defaults target a 24 GB consumer GPU (3090/4090): Qwen3.5-4B-Instruct at 4-bit,
Defaults target a 24 GB consumer GPU (3090/4090): `unsloth/Qwen3.5-4B` at 4-bit,
LoRA `r=16`/`alpha=16`, bf16, 3 epochs, effective batch size 8. For A100-40/80
bump `--base-model unsloth/Qwen3.5-9B-Instruct-bnb-4bit` and raise
`--batch-size` + `--max-seq-length`.
bump `--base-model unsloth/Qwen3.5-9B` and raise `--batch-size` +
`--max-seq-length`.
Output: `data/adapters/psyc-v1/final/` (adapter weights) + `training_meta.json`
(base model, hyperparameters, dataset list).
Evaluate the adapter against held-out dataset rows:
```bash
docker run --gpus all --rm \
--entrypoint python \
-v $(pwd)/data:/data -v $(pwd)/scripts:/scripts \
psyc-trainer /scripts/eval_adapter.py \
--adapter /data/adapters/psyc-v1/final \
--dataset /data/datasets/ioc_extraction-v1.jsonl --n 5
```
## Status
Day 2 of a 48h build. Shipped: Scoutline (URLhaus) → Classifyline → Mapline