stage-3c: working QLoRA training + eval — pytorch base, Qwen3.5 slug, SFTConfig

Training and eval now run clean on the unsloth 2026.5.2 / transformers v5 /
torch 2.10 stack. Fixes: pytorch/pytorch base image (sidesteps the nvidia/cuda
apt-signature failure and the torch download), correct base-model slug
unsloth/Qwen3.5-4B, TRL SFTConfig API. Adds scripts/eval_adapter.py — runs
dataset rows through base+adapter with structured (transformers-v5) message
content and Qwen3.5 thinking-mode stripping.

First v1 adapter: loss 2.10 -> 0.32 over 3 epochs. Eval surfaced an ill-posed
ioc_extraction dataset (output URL not present in input) — to be fixed in the
ExampleBuilder before the next training run.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
m17hr1l
2026-05-17 14:16:22 +02:00
parent f1ab11f89d
commit b95e3e02bd
4 changed files with 121 additions and 35 deletions

View File

@@ -22,8 +22,7 @@ from pathlib import Path
from typing import Dict, List
from datasets import Dataset
from transformers import TrainingArguments
from trl import SFTTrainer
from trl import SFTConfig, SFTTrainer
def load_examples(paths: List[Path]) -> List[Dict[str, str]]:
@@ -44,7 +43,7 @@ def load_examples(paths: List[Path]) -> List[Dict[str, str]]:
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--dataset", action="append", required=True, help="JSONL path (repeatable)")
parser.add_argument("--base-model", default="unsloth/Qwen3.5-4B-Instruct-bnb-4bit")
parser.add_argument("--base-model", default="unsloth/Qwen3.5-4B")
parser.add_argument("--output", default="/data/adapters/psyc-v1")
parser.add_argument("--epochs", type=int, default=3)
parser.add_argument("--lr", type=float, default=2e-4)
@@ -96,9 +95,9 @@ def main() -> None:
model=model,
tokenizer=tokenizer,
train_dataset=dataset,
dataset_text_field="text",
max_seq_length=args.max_seq_length,
args=TrainingArguments(
args=SFTConfig(
dataset_text_field="text",
max_seq_length=args.max_seq_length,
per_device_train_batch_size=args.batch_size,
gradient_accumulation_steps=args.grad_accum,
warmup_steps=5,