Fine-tuning & adaptation

Most production LLM features need three capabilities: correct facts, consistent format, and appropriate tone. These map to different adaptation layers. Facts that change weekly belong in a retrieval index, not in memorized weights. Stable JSON schemas and classification boundaries are fine-tuning sweet spots when prompt engineering plateaus on your golden set.

Four adaptation paths

Decision matrix

Score each incoming use case against these signals. If policy or catalog freshness is critical, RAG wins regardless of team preference. If the task is “always return this JSON shape for support tickets,” try structured output + prompt first; fine-tune only when eval shows persistent format drift across hundreds of real examples.

Decision flowchart

flowchart TD
  A[New LLM feature request] --> B{Does answer require
citing live documents?}
  B -->|Yes| C[Use RAG]
  B -->|No| D{Is behavior stable
12+ months?}
  D -->|No| E[Prompt + tools + routing]
  D -->|Yes| F{Prompt + structured output
passes eval?}
  F -->|Yes| G[Ship prompt path — no FT]
  F -->|No| H{Have 500+ quality
labeled examples?}
  H -->|No| I[Collect data — improve prompt/RAG first]
  H -->|Yes| J{Before/after eval
shows FT lift?}
  J -->|No| K[Do not fine-tune]
  J -->|Yes| L[Fine-tune with CI eval gate]
  C --> M[Track chunk freshness SLA]
  L --> N[Register model ID in gateway + flags]

Prerequisite context: RAG explained, system prompts, and LLM gateway for routing fine-tuned model IDs alongside base models.

Comparison table

Hosted vs self-hosted

OpenAI fine-tuning is managed instruction tuning: you upload JSONL, they train, you receive ft:gpt-4o-mini:org:... model IDs. You do not choose LoRA rank or learning rate on the public API. AWS Bedrock offers custom model import (Continued Pre-training, fine-tuning on supported base models) with S3-stored output artifacts and provisioned throughput options. Self-hosted (Hugging Face PEFT, Axolotl, LLaMA-Factory) gives full control over LoRA/QLoRA hyperparameters and merge-to-single-checkpoint for vLLM or TGI serving.

from peft import LoraConfig, get_peft_model

lora_config = LoraConfig(
    r=16,                    # rank — higher = more capacity, more VRAM
    lora_alpha=32,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)
model = get_peft_model(base_model, lora_config)
# Train only ~0.1–1% of parameters; save adapter dir, not full 7B weights

// Self-hosted LoRA typically runs via Python training jobs.
// Java serving layer loads base + adapter metadata from model registry:

public record AdapterManifest(
    String baseModelId,      // e.g. meta-llama/Llama-3.1-8B
    String adapterPath,      // s3://models/support-intent-lora-v3
    int rank,
    String taskType
) {}

// Gateway resolves: tenant → adapter manifest → inference endpoint

When each approach wins

• Hosted OpenAI FT — fast path for gpt-4o-mini classification/extraction; no GPU ops team
• Bedrock custom model — enterprise AWS footprint, IAM-bound artifacts, private VPC inference
• LoRA self-hosted — multiple tenant-specific adapters on one GPU pool; air-gapped deploy
• Full FT — rare in prod; consider only small specialist models (<3B) or embedding retrievers

Volume guidelines

JSONL format (OpenAI chat fine-tuning)

Each line is one JSON object with a messages array. Roles: system, user, assistant. Optional weight on assistant messages for DPO-style preference data on supported pipelines. Validate with provider CLI before upload.

# data/ft_support_intent.jsonl — one JSON object per line
EXAMPLE_CLASSIFY = {
    "messages": [
        {"role": "system", "content": "Classify support tickets. Reply with JSON: {\"intent\": \"...\", \"priority\": \"...\"}"},
        {"role": "user", "content": "My card was charged twice for order ORD-8821. Need refund ASAP."},
        {"role": "assistant", "content": "{\"intent\": \"billing_duplicate_charge\", \"priority\": \"high\"}"},
    ]
}

EXAMPLE_EXTRACT = {
    "messages": [
        {"role": "system", "content": "Extract refund fields as JSON. Use null for missing."},
        {"role": "user", "content": "Customer Jane Doe, order ORD-4420, wants partial refund $24.50 for damaged item."},
        {"role": "assistant", "content": "{\"order_id\": \"ORD-4420\", \"customer_name\": \"Jane Doe\", \"amount_usd\": 24.50, \"reason\": \"damaged_item\"}"},
    ]
}

import json
from pathlib import Path

def write_jsonl(rows: list[dict], path: Path) -> None:
    with path.open("w", encoding="utf-8") as f:
        for row in rows:
            f.write(json.dumps(row, ensure_ascii=False) + "\n")

// Jackson writes one JSON object per line for OpenAI upload
record ChatMessage(String role, String content) {}
record FineTuneRow(List<ChatMessage> messages) {}

void writeJsonl(List<FineTuneRow> rows, Path path) throws IOException {
    var mapper = new ObjectMapper();
    try (var writer = Files.newBufferedWriter(path)) {
        for (FineTuneRow row : rows) {
            writer.write(mapper.writeValueAsString(row));
            writer.newLine();
        }
    }
}

// Example row — intent classification
var classify = new FineTuneRow(List.of(
    new ChatMessage("system", "Classify support tickets as JSON intent + priority."),
    new ChatMessage("user", "Charged twice on ORD-8821, need refund."),
    new ChatMessage("assistant", "{\"intent\":\"billing_duplicate_charge\",\"priority\":\"high\"}")
));

Data hygiene checklist

• Deduplicate — near-duplicate tickets overweight frequent phrasing
• Stratify — hold out 15–20% validation split never seen in training
• Tag provenance — source: human|synthetic|prod_sample per row
• Version datasets — append-only git LFS or S3 with semver (ft-v1.3.0.jsonl)
• PII scrub — replace names/emails with placeholders consistent at train and eval time

Augmentation (careful)

Safe augmentations: paraphrase user text while preserving gold label; swap entity values (order IDs, amounts) in extraction tasks; translate to secondary locale if prod is multilingual. Unsafe: LLM-generated assistant responses without review; mixing contradictory labels for the same input; augmenting policy answers that should come from RAG.

import copy, random, re

ORDER_RE = re.compile(r"ORD-\d{4}")

def augment_order_ids(row: dict, n: int = 3) -> list[dict]:
    """Swap order IDs in user text; assistant JSON updated to match."""
    out = []
    base_user = row["messages"][1]["content"]
    base_asst = json.loads(row["messages"][2]["content"])
    for _ in range(n):
        new_id = f"ORD-{random.randint(1000, 9999)}"
        new_row = copy.deepcopy(row)
        new_row["messages"][1]["content"] = ORDER_RE.sub(new_id, base_user, count=1)
        asst = copy.deepcopy(base_asst)
        asst["order_id"] = new_id
        new_row["messages"][2]["content"] = json.dumps(asst)
        new_row["metadata"] = {"augmentation": "entity_swap"}
        out.append(new_row)
    return out

List<FineTuneRow> augmentOrderIds(FineTuneRow base, int n, Random rng) {
    var results = new ArrayList<FineTuneRow>();
    var userText = base.messages().get(1).content();
    for (int i = 0; i < n; i++) {
        String newId = "ORD-" + (1000 + rng.nextInt(9000));
        String newUser = userText.replaceFirst("ORD-\\d{4}", newId);
        // Parse assistant JSON, swap order_id, re-serialize — same pattern as Python
        results.add(/* rebuilt row */);
    }
    return results;
}

Supported models (typical)

Eligible base models change with provider releases. Common production choices:

Check OpenAI docs for current supported_models before pipeline automation.

Job lifecycle

1. validating_files — schema and token count checks
2. queued — waiting for capacity
3. running — training in progress (minutes to hours)
4. succeeded — fine_tuned_model ID available
5. failed — inspect error object; fix data, retry
6. cancelled — user or timeout abort

from openai import OpenAI
import time

client = OpenAI()

# 1. Upload training file
with open("data/ft_support_intent.jsonl", "rb") as f:
    train_file = client.files.create(file=f, purpose="fine-tune")

# 2. Create job
job = client.fine_tuning.jobs.create(
    training_file=train_file.id,
    model="gpt-4o-mini-2024-07-18",
    suffix="support-intent-v1",
    hyperparameters={"n_epochs": 3},  # start conservative; watch overfit on small sets
)

# 3. Poll until complete
while job.status not in ("succeeded", "failed", "cancelled"):
    time.sleep(30)
    job = client.fine_tuning.jobs.retrieve(job.id)
    print(job.status, job.trained_tokens)

if job.status != "succeeded":
    raise RuntimeError(job.error)

fine_tuned_model = job.fine_tuned_model  # ft:gpt-4o-mini:org:support-intent-v1:abc123

# 4. Smoke inference
resp = client.chat.completions.create(
    model=fine_tuned_model,
    messages=[{"role": "user", "content": "Double charge on ORD-9912"}],
)
print(resp.choices[0].message.content)

// Spring AI / OpenAI Java SDK pattern
OpenAiApi api = new OpenAiApi(System.getenv("OPENAI_API_KEY"));

Resource trainingFile = new FileSystemResource("data/ft_support_intent.jsonl");
String fileId = api.uploadFile(trainingFile, "fine-tune").getId();

FineTuningJobRequest req = FineTuningJobRequest.builder()
    .trainingFile(fileId)
    .model("gpt-4o-mini-2024-07-18")
    .suffix("support-intent-v1")
    .hyperparameters(Map.of("n_epochs", 3))
    .build();

FineTuningJob job = api.createFineTuningJob(req);
while (!List.of("succeeded", "failed", "cancelled").contains(job.getStatus())) {
    Thread.sleep(30_000);
    job = api.retrieveFineTuningJob(job.getId());
}

String fineTunedModel = job.getFineTunedModel();
// Route via gateway: model_registry.put("support-intent", fineTunedModel);

Cost estimation

Estimate training tokens ≈ sum of all message tokens × epochs. Example: 800 rows × ~400 tokens/row × 3 epochs ≈ 960K training tokens → ~$7.68 at $8/1M. Add 20% buffer for validation files and failed retries. Inference on fine-tuned models uses the same per-token inference rates as the base model tier—FT does not reduce inference cost, only improves task accuracy per call.

AWS Bedrock custom models

For teams standardized on Bedrock, custom model import and model customization jobs store artifacts in S3 and expose inference through Bedrock runtime with IAM boundaries. Flow parallels OpenAI: prepare JSONL/JSON → upload to S3 → create customization job → deploy custom model ARN in gateway routing table alongside foundation model IDs.

import boto3

bedrock = boto3.client("bedrock")

# Exact API varies by base model family — consult Bedrock model customization docs
job = bedrock.create_model_customization_job(
    jobName="support-intent-v1",
    customModelName="support-intent-classifier",
    roleArn="arn:aws:iam::123456789012:role/BedrockCustomizationRole",
    baseModelIdentifier="anthropic.claude-3-haiku-20240307-v1:0",
    trainingDataConfig={
        "s3Uri": "s3://ml-datasets/ft/support-intent-v1.jsonl"
    },
    outputDataConfig={
        "s3Uri": "s3://ml-models/custom/support-intent-v1/"
    },
    hyperParameters={"epochCount": "3", "batchSize": "8"},
)

# Poll describe_model_customization_job → status Complete
# Register customModelArn in llm-gateway bedrock routing

Reuse Track 5 infrastructure: golden JSONL, scorers, CI thresholds from evaluation explained, CI evaluation, and LLM-as-judge patterns. Fine-tune eval is not a separate discipline—it is another model variant in the same regression matrix as prompt v2 or RAG index v3.

Comparison protocol

1. Freeze golden set — held-out rows never in training JSONL
2. Run base model — record scores, latencies, token usage per case ID
3. Run fine-tuned model — identical prompts, temperature, max_tokens
4. Diff per bucket — intent, locale, risk tier; block if any bucket drops >5%
5. Human review sample — 20 regressions where FT scored lower than base
6. Sign-off artifact — JSON report committed or attached to deploy PR

"""scripts/compare_ft_models.py — run before any FT deploy."""
from dataclasses import dataclass
from pathlib import Path
import json

@dataclass
class EvalRow:
    id: str
    input: str
    expected: dict
    tags: list[str]

@dataclass
class CaseResult:
    row_id: str
    model: str
    score: float
    output: str
    latency_ms: float

def load_golden(path: Path) -> list[EvalRow]:
    return [EvalRow(**json.loads(line)) for line in path.read_text().splitlines() if line.strip()]

def score_exact_json(output: str, expected: dict) -> float:
    try:
        parsed = json.loads(output)
        return 1.0 if parsed == expected else 0.0
    except json.JSONDecodeError:
        return 0.0

def run_suite(rows: list[EvalRow], model_id: str, complete_fn) -> list[CaseResult]:
    results = []
    for row in rows:
        out, latency = complete_fn(model_id, row.input)
        results.append(CaseResult(row.id, model_id, score_exact_json(out, row.expected), out, latency))
    return results

def compare(base: list[CaseResult], ft: list[CaseResult], min_lift: float = 0.03) -> dict:
    by_id_base = {r.row_id: r for r in base}
    by_id_ft = {r.row_id: r for r in ft}
    lifts = [by_id_ft[i].score - by_id_base[i].score for i in by_id_base]
    mean_lift = sum(lifts) / len(lifts)
    regressions = [i for i in by_id_base if by_id_ft[i].score < by_id_base[i].score]
    return {
        "mean_lift": mean_lift,
        "pass": mean_lift >= min_lift and len(regressions) / len(lifts) < 0.15,
        "regression_ids": regressions[:20],
    }

# Usage:
# base_results = run_suite(golden, "gpt-4o-mini", gateway_complete)
# ft_results = run_suite(golden, "ft:gpt-4o-mini:org:...", gateway_complete)
# report = compare(base_results, ft_results)
# assert report["pass"], report

public record EvalRow(String id, String input, Map<String, Object> expected, List<String> tags) {}
public record CaseResult(String rowId, String model, double score, String output, long latencyMs) {}

public CompareReport compare(List<CaseResult> base, List<CaseResult> ft, double minLift) {
    Map<String, CaseResult> baseMap = base.stream()
        .collect(Collectors.toMap(CaseResult::rowId, r -> r));
    List<Double> lifts = new ArrayList<>();
    List<String> regressions = new ArrayList<>();
    for (CaseResult ftRow : ft) {
        CaseResult b = baseMap.get(ftRow.rowId());
        double lift = ftRow.score() - b.score();
        lifts.add(lift);
        if (lift < 0) regressions.add(ftRow.rowId());
    }
    double meanLift = lifts.stream().mapToDouble(d -> d).average().orElse(0);
    boolean pass = meanLift >= minLift && (double) regressions.size() / lifts.size() < 0.15;
    return new CompareReport(meanLift, pass, regressions.stream().limit(20).toList());
}

CI integration sketch

PR opens (dataset or FT job completes)
  → evals/compare_ft.yaml workflow
  → download golden held-out v2.1
  → run base + candidate ft model
  → upload compare.json artifact
  → PR comment: mean_lift + regression table
  → merge blocked if pass=false

Pre-deploy checklist

• Decision doc — why FT vs prompt/RAG; link to path comparison eval
• Dataset semver — training JSONL hash in model registry
• Held-out eval — base vs FT report attached; all buckets green
• Safety suite — no regression on adversarial golden
• Gateway entry — model_alias: support-intent → ft:… in routing config
• Feature flag — ft_support_intent_v1 default off; staged % rollout
• Cost projection — inference tokens unchanged; training cost amortized in FinOps tag
• Rollback runbook — flip flag to base model; no redeploy required
• Retrain trigger — backlog threshold + eval plateau documented
• On-call — ML owner paged on FT latency/error rate anomalies

# config/models.yaml
MODELS = {
    "support-intent": {
        "default": "gpt-4o-mini-2024-07-18",
        "variants": {
            "ft_v1": {
                "model_id": "ft:gpt-4o-mini:org:support-intent-v1:abc123",
                "dataset": "ft-support-intent-v1.2.0",
                "dataset_sha256": "a1b2c3…",
                "eval_report": "reports/ft_v1_compare.json",
                "flag": "ft_support_intent_v1",
            }
        },
    }
}

def resolve_model(feature: str, flags: dict) -> str:
    cfg = MODELS[feature]
    if flags.get("ft_support_intent_v1"):
        return cfg["variants"]["ft_v1"]["model_id"]
    return cfg["default"]

@ConfigurationProperties("llm.models")
public class ModelRegistry {
    Map<String, FeatureModels> features;

    public String resolve(String feature, FeatureFlags flags) {
        var f = features.get(feature);
        if (flags.isEnabled("ft_support_intent_v1")) {
            return f.variants().get("ft_v1").modelId();
        }
        return f.defaultModel();
    }
}