Data Analyst

Data Cleaning

• Handle missing values deliberately: drop rows, fill with mean/median/mode, or interpolate — choose based on the data context.
• Standardize formats: consistent date parsing (pd.to_datetime), string normalization (.str.lower().str.strip()).
• Remove or flag duplicates with df.duplicated().
• Convert data types appropriately: categories to pd.Categorical, IDs to strings, amounts to float.
• Document every cleaning step so the analysis is reproducible.

Visualization Best Practices

• Every chart needs a title, labeled axes, and appropriate units.
• Use color intentionally — highlight the key insight, not every category.
• Avoid 3D charts, pie charts with many slices, and truncated y-axes that exaggerate differences.
• Use figsize to ensure charts are readable. Export at high DPI for reports.
• Annotate key data points or thresholds directly on the chart.

Statistical Analysis

• Report measures of central tendency (mean, median) and spread (std, IQR) together.
• Use hypothesis tests when comparing groups: t-test for means, chi-square for proportions, Mann-Whitney for non-parametric.
• Always report effect size and confidence intervals, not just p-values.
• Check assumptions: normality, homoscedasticity, independence before applying parametric tests.

Pitfalls to Avoid

• Do not draw causal conclusions from correlations alone.
• Do not ignore sample size — small samples produce unreliable statistics.
• Do not cherry-pick results — report what the data shows, including inconvenient findings.
• Avoid aggregating data at the wrong granularity — Simpson's paradox can reverse observed trends.