OLS Regression Skill

The Gauss-Markov Assumptions

1. Linearity in parameters
2. Random sampling
3. No perfect multicollinearity
4. Zero conditional mean of errors: E(ε|X) = 0
5. Homoskedasticity: Var(ε|X) = σ²
6. (For inference) Normally distributed errors

Violation of assumptions 4–5 does not bias OLS but affects standard errors. Violation of assumption 4 (endogeneity) biases estimates — recommend IV methods.

Standard Error Options

• Default OLS SE: valid only under homoskedasticity
• HC robust SE (White): use when heteroskedasticity is suspected; always safe for cross-section data
• Clustered SE: use when observations are grouped (e.g., by firm, region, year)
• Newey-West SE: use for time series with autocorrelation

Quick Code Templates

Python (statsmodels)

import statsmodels.api as sm
import statsmodels.formula.api as smf

# With robust standard errors
model = smf.ols('y ~ x1 + x2 + x3', data=df).fit(cov_type='HC3')
print(model.summary())

R

library(lmtest)
library(sandwich)

model <- lm(y ~ x1 + x2 + x3, data = df)
coeftest(model, vcov = vcovHC(model, type = "HC3"))

Stata

reg y x1 x2 x3, robust

Diagnostics Checklist

Run all diagnostics after fitting. See references/ols-reference.md for full test details.

Reporting Standards (Academic)

• Report coefficients with standard errors in parentheses (or t-stats)
• Use asterisks for significance: * p<0.10, ** p<0.05, *** p<0.01
• Always state which standard errors are used (robust, clustered, etc.)
• Report R², adjusted R², N, and F-statistic
• Describe the identification strategy and potential endogeneity concerns

For detailed test formulas, code, and extended examples, see references/ols-reference.md.

Common Pitfalls

• Claiming causality without identification: OLS with controls does not establish causality — use IV, DID, or RDD for causal claims
• Using default SE with clustered data: Always cluster SE at the group level when observations are grouped
• Including "bad controls": Don't control for post-treatment variables (mediators) — they introduce collider bias
• Log-transforming variables with zeros: ln(0) is undefined; use asinh(x) or ln(x+1) with appropriate interpretation
• Reporting R² as evidence of a good model: High R² does not mean the model is correctly specified or causal