Multiclass Classification

When to use this skill

• The target has > 2 discrete classes (sentiment 1-5, product category A-Z, intent buckets, fault types)
• The classes are mutually exclusive (each example belongs to exactly one class — multilabel is a different skill)
• The features are tabular
• You have at least a few hundred examples per class

When NOT to use this skill

• Binary target → see binary-classification
• Multiple labels per example → see multilabel-classification
• Classes have natural ordering (rating 1-5) → consider an ordinal regression model (not XGBoost) — XGBoost will work but ignores the order
• Very many classes (> 100) and you only care about top-K → consider hierarchical softmax or learning-to-rank approaches

Project layout

<project>/
├── data/                # input parquet/csv
├── src/
│   ├── train.py         # Pipeline + XGBClassifier(multi:softprob) + MLflow
│   ├── predict.py       # reload, return top-K predictions per row
│   └── plots.py         # confusion matrix, per-class metrics, ROC OvR, SHAP
├── notebooks/
│   └── demo.py          # marimo walkthrough
└── mlruns/

Data access — ibis at the source

Same pattern as the other tabular bundles. Use ibis to read; materialize once with .execute() for sklearn:

import ibis

table = ibis.duckdb.connect().read_parquet("data/train.parquet")
feature_cols = [c for c in table.columns if c.startswith("feature_")]
data = (
    table
    .select(*feature_cols, "target")
    .execute()
)
X = data[feature_cols]
y = data["target"].astype(int)
n_classes = int(y.max()) + 1

The pipeline

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from xgboost import XGBClassifier

def build_pipeline(feature_cols, n_classes, seed):
    return Pipeline([
        ("preprocess", ColumnTransformer([("num", StandardScaler(), feature_cols)])),
        ("clf", XGBClassifier(
            n_estimators=300,
            max_depth=5,
            learning_rate=0.05,
            subsample=0.8,
            colsample_bytree=0.8,
            reg_lambda=1.0,
            objective="multi:softprob",
            num_class=n_classes,
            eval_metric="mlogloss",
            random_state=seed,
            n_jobs=-1,
        )),
    ])

The only changes from binary are objective, num_class, and eval_metric.

The five things that separate this from a tutorial

1. Per-class metrics — never just accuracy

Accuracy on a 5-class problem can be 80% while the model completely fails on one class. You need per-class precision, recall, F1, and