Train high-performance medical LLMs on consumer GPUs using parameter-efficient fine-tuning
ID: 215
Category: AI/ML Research
Language: Python
Framework: PyTorch + PEFT (LoRA/QLoRA) + Transformers
Based on Parameter-Efficient Fine-Tuning (PEFT) technology, trains high-performance medical domain large language models on consumer-grade GPUs or single A100. Supports advanced fine-tuning methods such as LoRA, QLoRA, optimized for medical text understanding and generation tasks.
# Core dependencies
pip install torch transformers datasets accelerate peft bitsandbytes
# Optional for training optimization
pip install flash-attn --no-build-isolation
pip install wandb tensorboard
# Medical NLP utilities
pip install scispacy scikit-learn
from skills.low_resource_ai_researcher.scripts.main import MedicalPEFTTrainer
# Initialize trainer
trainer = MedicalPEFTTrainer(
model_name="meta-llama/Llama-2-7b-hf",
task="medical_qa"
)
# Train with LoRA
trainer.train(
output_dir="./medical_lora_model",
num_epochs=3,
batch_size=4,
use_qlora=True # 4-bit quantization
)
| Profile | GPU Memory | Quantization | Max Model Size | Batch Size |
|---|---|---|---|---|
| consumer-24g | 24GB (RTX 3090/4090) | QLoRA 4-bit | 70B | 1-2 |
| a100-40g | 40GB (A100) | LoRA 8-bit | 70B | 4-8 |
| a100-80g | 80GB (A100) | LoRA 16-bit | 70B | 8-16 |
| multi-gpu | 2x A100 | LoRA 16-bit | 70B+ | 16+ |