stage-3c: working QLoRA training + eval — pytorch base, Qwen3.5 slug, SFTConfig

Training and eval now run clean on the unsloth 2026.5.2 / transformers v5 /
torch 2.10 stack. Fixes: pytorch/pytorch base image (sidesteps the nvidia/cuda
apt-signature failure and the torch download), correct base-model slug
unsloth/Qwen3.5-4B, TRL SFTConfig API. Adds scripts/eval_adapter.py — runs
dataset rows through base+adapter with structured (transformers-v5) message
content and Qwen3.5 thinking-mode stripping.

First v1 adapter: loss 2.10 -> 0.32 over 3 epochs. Eval surfaced an ill-posed
ioc_extraction dataset (output URL not present in input) — to be fixed in the
ExampleBuilder before the next training run.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
m17hr1l
2026-05-17 14:16:22 +02:00
parent f1ab11f89d
commit b95e3e02bd
4 changed files with 121 additions and 35 deletions

View File

@@ -12,36 +12,19 @@
# --dataset /data/datasets/routing_decision-v1.jsonl \
# --dataset /data/datasets/tlp_assignment-v1.jsonl \
# --output /data/adapters/psyc-v1
#
# Base image already ships Python 3.11 + torch 2.6 + CUDA 12.4 + cuDNN9, so
# there is no apt step and no torch download. Qwen3.5 needs transformers v5 —
# unsloth pulls it automatically.
FROM nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04
FROM pytorch/pytorch:2.6.0-cuda12.4-cudnn9-devel
ENV DEBIAN_FRONTEND=noninteractive \
PYTHONUNBUFFERED=1 \
ENV PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1 \
HF_HOME=/data/.hf-cache
RUN apt-get update && apt-get install -y --no-install-recommends \
python3.11 python3.11-venv python3-pip \
git curl ca-certificates \
&& rm -rf /var/lib/apt/lists/* \
&& ln -sf /usr/bin/python3.11 /usr/local/bin/python \
&& ln -sf /usr/bin/python3.11 /usr/local/bin/python3
RUN python -m pip install --upgrade pip wheel setuptools && \
python -m pip install \
torch==2.5.1 \
--index-url https://download.pytorch.org/whl/cu124
RUN python -m pip install \
"unsloth @ git+https://github.com/unslothai/unsloth.git" \
transformers>=4.46 \
datasets>=3.0 \
peft>=0.13 \
trl>=0.12 \
accelerate>=1.1 \
bitsandbytes>=0.44 \
sentencepiece \
protobuf
RUN pip install --upgrade pip && \
pip install unsloth unsloth_zoo trl datasets
WORKDIR /workspace
COPY scripts/train_qlora.py /workspace/train_qlora.py