stage-3c: working QLoRA training + eval — pytorch base, Qwen3.5 slug, SFTConfig
Training and eval now run clean on the unsloth 2026.5.2 / transformers v5 / torch 2.10 stack. Fixes: pytorch/pytorch base image (sidesteps the nvidia/cuda apt-signature failure and the torch download), correct base-model slug unsloth/Qwen3.5-4B, TRL SFTConfig API. Adds scripts/eval_adapter.py — runs dataset rows through base+adapter with structured (transformers-v5) message content and Qwen3.5 thinking-mode stripping. First v1 adapter: loss 2.10 -> 0.32 over 3 epochs. Eval surfaced an ill-posed ioc_extraction dataset (output URL not present in input) — to be fixed in the ExampleBuilder before the next training run. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
19
README.md
19
README.md
@@ -121,7 +121,7 @@ To fine-tune Qwen3.5-4B with QLoRA in an NVIDIA Docker container:
|
||||
# 1. build datasets (one-off; re-run after ingestion changes)
|
||||
.venv/bin/psyc train-build-all
|
||||
|
||||
# 2. build the training image (CUDA 12.4 + unsloth + Qwen3.5)
|
||||
# 2. build the training image (pytorch 2.6/CUDA 12.4 base + unsloth + Qwen3.5)
|
||||
docker build -t psyc-trainer -f Dockerfile.train .
|
||||
|
||||
# 3. fine-tune (mount host data/ so adapters land there)
|
||||
@@ -135,14 +135,25 @@ docker run --gpus all --rm \
|
||||
--output /data/adapters/psyc-v1
|
||||
```
|
||||
|
||||
Defaults target a 24 GB consumer GPU (3090/4090): Qwen3.5-4B-Instruct at 4-bit,
|
||||
Defaults target a 24 GB consumer GPU (3090/4090): `unsloth/Qwen3.5-4B` at 4-bit,
|
||||
LoRA `r=16`/`alpha=16`, bf16, 3 epochs, effective batch size 8. For A100-40/80
|
||||
bump `--base-model unsloth/Qwen3.5-9B-Instruct-bnb-4bit` and raise
|
||||
`--batch-size` + `--max-seq-length`.
|
||||
bump `--base-model unsloth/Qwen3.5-9B` and raise `--batch-size` +
|
||||
`--max-seq-length`.
|
||||
|
||||
Output: `data/adapters/psyc-v1/final/` (adapter weights) + `training_meta.json`
|
||||
(base model, hyperparameters, dataset list).
|
||||
|
||||
Evaluate the adapter against held-out dataset rows:
|
||||
|
||||
```bash
|
||||
docker run --gpus all --rm \
|
||||
--entrypoint python \
|
||||
-v $(pwd)/data:/data -v $(pwd)/scripts:/scripts \
|
||||
psyc-trainer /scripts/eval_adapter.py \
|
||||
--adapter /data/adapters/psyc-v1/final \
|
||||
--dataset /data/datasets/ioc_extraction-v1.jsonl --n 5
|
||||
```
|
||||
|
||||
## Status
|
||||
|
||||
Day 2 of a 48h build. Shipped: Scoutline (URLhaus) → Classifyline → Mapline
|
||||
|
||||
Reference in New Issue
Block a user