Fine-Tuning LLMs — LoRA, QLoRA and DPO in Practice

Why choose this training?

Advanced training on fine-tuning language models. Covers when to fine-tune vs RAG vs prompt engineering, training data preparation, Supervised Fine-Tuning (SFT), Parameter-Efficient Fine-Tuning (PEFT) with LoRA and QLoRA, RLHF and DPO (Direct Preference Optimization), quantization (GPTQ, AWQ, GGUF), fine-tuned model evaluation, and deployment and serving. This training combines theoretical knowledge with intensive hands-on exercises, enabling participants to immediately apply their skills in daily work. The program is designed and delivered by practitioners with real-world experience.

What you will learn

You will gain comprehensive knowledge and practical skills in fine-tuning llms. The program covers all key aspects from foundational concepts through advanced techniques to real-world implementation patterns.

Through hands-on exercises and realistic scenarios, you will develop the ability to apply learned concepts in your organization. After completing the training, you will have actionable knowledge that translates directly into improved capabilities for your team and organization.

Fine-tuning vs RAG vs Prompt Engineering — when to use which

A critical decision in any LLM project is choosing the right approach:

Approach	When to use	Cost	Latency	Accuracy gain
Prompt Engineering	Quick PoC, simple tasks	$0	None	Limited
Few-shot prompting	Small examples, structured output	$0	None	Moderate
RAG (Retrieval Augmented Generation)	Domain knowledge, frequently changing data	Low ($)	+500ms	High for facts
Fine-tuning (LoRA)	Task-specific behavior, brand voice, format	Medium ($$)	None	High for style
Full fine-tuning	Massive domain shift, performance critical	High ($$$$)	None	Highest

Decision tree: Start with prompt engineering. If insufficient, add RAG (cheaper than fine-tuning). Fine-tune only when style/format/behavior must change permanently or when running at scale (>1M requests/month) where prompt token costs add up.

LoRA, QLoRA, and PEFT — efficient fine-tuning

Full fine-tuning of a 70B parameter model requires 280+ GB GPU memory ($10-50k cluster). Prohibitive for most organizations. PEFT (Parameter-Efficient Fine-Tuning) revolutionized this:

LoRA (Low-Rank Adaptation)

Instead of updating all model weights, freeze base model and add small “adapter” layers (~0.1% of original parameters). Train only adapters. Result: 99% less memory, similar performance.

QLoRA (Quantized LoRA)

Combine LoRA with 4-bit quantization of frozen base model. Fine-tune 70B on a single A100 (80GB) — practical for individual practitioners.

Hardware requirements 2026:

• 7B model + LoRA: single RTX 4090 (24GB) — $1500
• 13B + QLoRA: single A6000 (48GB) — $5000
• 70B + QLoRA: single A100 (80GB) — $15000
• 70B + full fine-tune: 8x H100 (640GB) — $300k+

RLHF and DPO — preference optimization

Beyond supervised fine-tuning (SFT), preference optimization aligns models with human values:

• RLHF (Reinforcement Learning from Human Feedback) — used to train ChatGPT-style models. Complex: needs reward model + PPO. Expensive.
• DPO (Direct Preference Optimization, 2023) — simpler alternative. Skip the reward model. Just need preference pairs (chosen vs rejected). 90% of RLHF results with 50% complexity.

DPO has become the de facto standard for preference tuning in 2024-2026. We’ll implement both in hands-on labs.

Quantization for deployment

Production inference cost scales with model size. Quantization (reducing precision from FP16 to INT8/INT4) reduces costs 4-8x with minimal accuracy loss:

• GPTQ — accurate post-training quantization
• AWQ — Activation-aware quantization, faster
• GGUF — for llama.cpp, runs on CPUs and consumer GPUs

We’ll quantize fine-tuned models and benchmark accuracy/speed tradeoffs.

Real-world fine-tuning use cases

Hands-on projects we’ll complete:

1. Customer support chatbot — fine-tune Mistral-7B on company FAQ + tone of voice
2. Code generation — fine-tune CodeLlama on internal codebase patterns
3. Document summarization — DPO-tune Llama-3 on company-style executive summaries
4. SQL generation — fine-tune for specific database schema (text-to-SQL)
5. Multilingual support — fine-tune for Polish/Swedish business contexts