Genai Dac Specialist

Decision Framework

When to Use DACs vs On-Demand

Use Dedicated AI Clusters when:

- Data isolation required (private GPUs)
- Predictable, high-volume workloads
- Fine-tuning with proprietary data
- SLA requirements (guaranteed performance)
- Multi-model deployment (up to 50 endpoints)
- Regulatory compliance needs

Use On-Demand when:

- Development and experimentation
- Low-volume, unpredictable usage
- Testing before production commitment
- Quick prototyping

Model Selection Guide

┌─────────────────────────────────────────────────────────────────┐
│                     MODEL SELECTION MATRIX                       │
├──────────────────┬─────────────┬─────────────┬─────────────────┤
│ Use Case         │ Recommended │ Alternative │ Why             │
├──────────────────┼─────────────┼─────────────┼─────────────────┤
│ Complex reasoning│ Command R+  │ Llama 405B  │ Best reasoning  │
│ General chat     │ Command R   │ Llama 70B   │ Good balance    │
│ Simple tasks     │ Command     │ Llama 8B    │ Cost efficient  │
│ High volume      │ Command Light│ Llama 8B   │ Fast, cheap     │
│ Embeddings/RAG   │ Cohere Embed│ -           │ Purpose-built   │
│ Multi-modal      │ Llama 3.2   │ -           │ Vision support  │
└──────────────────┴─────────────┴─────────────┴─────────────────┘

Cluster Sizing

Hosting Cluster Sizing

Traffic Estimate → Units Needed:

Light (< 10 req/sec):     2-5 units
Medium (10-50 req/sec):   5-15 units
Heavy (50-200 req/sec):   15-30 units
Enterprise (200+ req/sec): 30-50 units

Each unit = 1 endpoint slot
Cluster max = 50 units (50 endpoints)

Fine-Tuning Cluster Sizing

Dataset Size → Cluster Recommendation:

Small (< 10K examples):    2 units, ~2-4 hours
Medium (10K-100K):         4 units, ~4-8 hours
Large (100K-1M):           8 units, ~8-24 hours

Fine-tuning is batch - pay for duration

Terraform Templates

Basic Hosting Cluster

resource "oci_generative_ai_dedicated_ai_cluster" "hosting" {
  compartment_id = var.compartment_id
  type           = "HOSTING"

  unit_count     = var.hosting_units
  unit_shape     = var.model_family  # "LARGE_COHERE" or "LARGE_GENERIC"

  display_name   = "${var.project}-hosting-cluster"

  freeform_tags = {
    Environment = var.environment
    Project     = var.project
  }
}

resource "oci_generative_ai_endpoint" "primary" {
  compartment_id          = var.compartment_id
  dedicated_ai_cluster_id = oci_generative_ai_dedicated_ai_cluster.hosting.id
  model_id                = var.model_id

  display_name = "${var.project}-endpoint"

  content_moderation_config {
    is_enabled = var.enable_moderation
  }
}

Fine-Tuning Workflow

# Fine-tuning cluster
resource "oci_generative_ai_dedicated_ai_cluster" "finetuning" {
  compartment_id = var.compartment_id
  type           = "FINE_TUNING"

  unit_count     = 4
  unit_shape     = "LARGE_COHERE"

  display_name   = "${var.project}-finetuning-cluster"
}

# Training dataset in Object Storage
resource "oci_objectstorage_bucket" "training_data" {
  compartment_id = var.compartment_id
  namespace      = data.oci_objectstorage_namespace.ns.namespace
  name           = "${var.project}-training-data"

  access_type    = "NoPublicAccess"
}

Fine-Tuning Best Practices

Data Preparation

// training_data.jsonl format
{"prompt": "Your custom prompt here", "completion": "Expected response"}
{"prompt": "Another example", "completion": "Another response"}

Quality Guidelines

1. QUANTITY
   - Minimum: 100 high-quality examples
   - Recommended: 500-2000 examples
   - More isn't always better - quality > quantity

2. DIVERSITY
   - Cover all expected use cases
   - Include edge cases
   - Vary prompt styles

3. CONSISTENCY
   - Same format throughout
   - Consistent tone and style
   - Clear completion boundaries

4. VALIDATION
   - Hold out 10-20% for testing
   - Review samples manually
   - Test before full training

Hyperparameter Recommendations

# Conservative (start here)
learning_rate: 0.0001