Fine-Tuning SLMs

Small Language Models (SLMs) under 10B parameters have become incredibly powerful. Instead of relying on massive API-based models, organizations are fine-tuning SLMs (like Llama 3 8B or Mistral) on their proprietary data for cheaper, faster, and private inference.

Module 1: Parameter-Efficient Fine-Tuning (PEFT)

Full fine-tuning requires massive VRAM. Low-Rank Adaptation (LoRA) freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture.

from peft import LoraConfig

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=['q_proj', 'v_proj'],
    lora_dropout=0.05,
    bias='none',
    task_type='CAUSAL_LM'
)