Ai Engineering

For detailed framework comparison, see frameworks.md

2. ML vs DL Decision

Dataset size < 10k rows?
├─ Yes → Traditional ML (scikit-learn)
└─ No → Consider deep learning

Tabular data?
├─ Yes → XGBoost, LightGBM, CatBoost
└─ No (images, text, audio) → Deep learning

Need interpretability?
├─ Yes → Decision trees, linear models
└─ No → Deep learning acceptable

Computational resources limited?
├─ Yes → Traditional ML or small neural networks
└─ No → Large deep learning models

3. Architecture Selection

For architecture details and variants, see architectures.md

4. Training Workflow

Standard Training Pipeline

1. Data Preparation

   • Load and explore data
   • Split: 70% train, 15% val, 15% test
   • Normalize/standardize
   • Augmentation (if needed)

2. Model Selection

   • Start simple (baseline model)
   • Choose architecture from table above
   • Consider transfer learning

3. Training

   • Initialize with good hyperparameters
   • Monitor train + validation metrics
   • Use callbacks (early stopping, lr scheduling)

4. Evaluation

   • Test set performance
   • Error analysis
   • Compare to baseline

5. Optimization (if needed)

   • Hyperparameter tuning
   • Architecture search
   • Ensemble methods

Hyperparameter Tuning Strategy

5. Reinforcement Learning

RL vs Supervised Learning

Use RL when:

• No labeled data, only reward signals
• Sequential decision making
• Agent-environment interaction
• Need to learn optimal policy

Algorithm Selection

For RL/DRL implementation details, see rl-drl.md

6. Model Evaluation

Metrics by Task Type

Validation Strategy

Data size < 1000 samples?
├─ Yes → K-fold cross-validation (k=5 or 10)
└─ No → Single train/val/test split

Time series data?
├─ Yes → Time-based splits (no shuffle!)
└─ No → Random or stratified split

Imbalanced classes?
├─ Yes → Stratified split
└─ No → Random split

7. Deployment Decision Tree

Where will model run?
├─ Cloud → API serving (TF Serving, TorchServe)
├─ Edge/Mobile → Model compression + TFLite/ONNX
├─ Browser → TensorFlow.js
└─ Batch → Scheduled jobs

Latency requirements?
├─ Real-time (< 100ms) → Optimize model, use caching
├─ Interactive (< 1s) → Standard serving
└─ Batch (minutes/hours) → Batch processing

Scale?
├─ High traffic → Kubernetes + auto-scaling
├─ Medium → Cloud Run, Lambda
└─ Low → Simple API server

Model Optimization

Before deployment, consider:

• Quantization: 4x smaller, 2x faster (INT8)
• Pruning: Remove redundant weights
• Distillation: Smaller student model
• ONNX: Cross-framework deployment

For MLOps and production deployment details, see mlops.md

8. Common Decision Points

Transfer Learning vs Train from Scratch

Use transfer learning when:

• ✅ Similar domain (ImageNet → custom images)
• ✅ Limited data (< 10k samples)
• ✅ Limited compute

Train from scratch when:

• ❌ Very different domain
• ❌ Huge dataset (millions of samples)
• ❌ Need maximum performance

Data Augmentation

Vision:

• Always: Resize/crop, normalization
• Usually: Random flip, color jitter
• Sometimes: Mixup, CutMix, AutoAugment

NLP:

• Back-translation
• Synonym replacement
• Contextual word embeddings

Time Series:

• Window slicing
• Noise injection
• Time warping

9. Quick Start Checklist

• Define problem (classification, regression, generation, RL?)
• Choose framework (see section 1)
• Prepare data (clean, split, augment)
• Select baseline architecture (see section 3)
• Train with default hyperparameters
• Evaluate on validation set
• Analyze errors
• Iterate (tune, improve)
• Evaluate on test set
• Plan deployment (see section 7)

10. Common Pitfalls

Philosophy: Start simple, measure everything, iterate based on data. The best model is the one that solves the problem with minimum complexity.

References

• frameworks.md - Deep dive into PyTorch, TensorFlow, JAX, scikit-learn
• architectures.md - CNN, RNN, Transformer, GAN details
• rl-drl.md - Reinforcement learning algorithms and implementation
• mlops.md - Production ML systems and deployment