Llm Finetuning

• Configure LoRA with appropriate rank (r=8 to 64), alpha (typically 2x rank), and target modules (q_proj, v_proj for attention, or all linear layers for broader adaptation)
• Use QLoRA for memory-constrained setups: load the base model in 4-bit NormalFloat quantization, attach LoRA adapters in fp16/bf16, and train with paged optimizers to handle memory spikes
• Format datasets as instruction-response pairs with consistent templates; include a system field for persona or context, an instruction field for the task, and a response field for the expected output
• Apply the PEFT library workflow: load base model, create LoRA config, get_peft_model(), train with the Hugging Face Trainer or a custom loop, then save and load adapters independently
• Set training hyperparameters carefully: learning rate between 1e-5 and 2e-4 with cosine schedule, 1-5 epochs (watch for overfitting), warmup ratio of 0.03-0.1, and gradient accumulation to simulate larger batch sizes
• Evaluate with multiple signals: validation loss for overfitting detection, task-specific metrics (ROUGE for summarization, exact match for QA), and structured human evaluation on a held-out set

Common Patterns

• Domain Adaptation: Fine-tune on domain-specific text (legal, medical, financial) to teach the model terminology, reasoning patterns, and output formats unique to that field
• Instruction Following: Train on diverse instruction-response pairs to improve the model's ability to follow complex multi-step instructions and produce structured outputs
• Adapter Merging: After training, merge the LoRA adapter weights back into the base model with merge_and_unload() for inference without the PEFT overhead
• Multi-task Training: Mix datasets from different tasks (summarization, classification, extraction) in a single fine-tuning run to create a versatile adapter

Pitfalls to Avoid

• Do not fine-tune on data that contains personally identifiable information, copyrighted content, or harmful material without proper review and filtering
• Do not train for too many epochs on a small dataset; language models memorize quickly, and overfitting manifests as repetitive, templated outputs that lack generalization
• Do not skip decontamination between training and evaluation sets; if evaluation examples appear in training data, metrics will be artificially inflated
• Do not assume a single set of hyperparameters works across base models; different architectures and sizes respond differently to learning rates, LoRA ranks, and batch sizes