Name: Data Analyst
Author: lastdays03

Data Analyst Workflow

Python 생태계(Jupyter, Pandas, Scikit-learn)를 활용하여 데이터에서 심층적인 인사이트를 도출하는 전문 분석 워크플로우입니다. OSEMN 방법론과 SKILL.md의 표준을 따릅니다.

0단계: 기본 원칙 (Core Principles)

Explanation First (선 설명 후 코드): 코드를 작성하기 전에 무엇을, 왜 분석하는지 Markdown으로 서술합니다.
Interpretation Mandatory (결과 해석 필수): 모든 코드 셀(Code Cell)의 출력 하단에는 반드시 Markdown 셀을 추가하여 통계적 수치나 그래프가 의미하는 바를 한글로 상세히 해석합니다.
- Bad: 그래프만 덩그러니 있음.
- Good: "히스토그램의 꼬리가 우측으로 긴 것으로 보아(Skewed > 1), 로그 변환이 필요함을 시사합니다."

1단계: 분석 환경 및 목표 정의 (Environment & Goal)

Context Loading:
- this document를 로드하여 **'Core Principles'**를 확인합니다.
- Methodology Screening (방법론 스크리닝):
  - SKILL.md의 **'Methodology Master List'**를 스캔합니다.

Data Analyst Workflow

0단계: 기본 원칙 (Core Principles)

Explanation First (선 설명 후 코드): 코드를 작성하기 전에 무엇을, 왜 분석하는지 Markdown으로 서술합니다.
Interpretation Mandatory (결과 해석 필수): 모든 코드 셀(Code Cell)의 출력 하단에는 반드시 Markdown 셀을 추가하여 통계적 수치나 그래프가 의미하는 바를 한글로 상세히 해석합니다.
- Bad: 그래프만 덩그러니 있음.
- Good: "히스토그램의 꼬리가 우측으로 긴 것으로 보아(Skewed > 1), 로그 변환이 필요함을 시사합니다."

1단계: 분석 환경 및 목표 정의 (Environment & Goal)

Context Loading:
- this document를 로드하여 **'Core Principles'**를 확인합니다.
- Methodology Screening (방법론 스크리닝):
  - SKILL.md의 **'Methodology Master List'**를 스캔합니다.

Methodology	Usage / Purpose	Data Constraints
Simple Imputation	Missing Value Imputation (Simple Replacement)	Mean/Median (Numeric), Mode (Categorical)
KNN Imputation	Missing Value Imputation (Similarity-based)	Mainly Numeric, useful when correlations exist
Iterative Imputation	Missing Value Imputation (Model-based)	High variable correlation, assumes MAR
One-Hot Encoding	Categorical to Numeric	Nominal data, Low Cardinality
Label Encoding	Categorical to Numeric	Ordinal data
Target Encoding	Categorical to Numeric	High Cardinality features, Risk of Overfitting
Standard Scaler	Scaling (Standardization)	Sensitive to outliers, assumes Gaussian distribution
MinMax Scaler	Scaling (Normalization)	Bounded data, distribution agnostic
Robust Scaler	Scaling (Robust to Outliers)	Data with many outliers (Uses Median/IQR)
SMOTE	Oversampling (Imbalanced Data)	Synthesize minority class samples (Training set ONLY)
PCA	Dimensionality Reduction, Multicollinearity Removal	Continuous variables, assumes linear relationships

Methodology	Type	Usage / Purpose	Constraints / Notes
Linear Regression	Regression	Baseline for regression	Linear relationship assumption
Logistic Regression	Classification	Baseline for classification	Linear separation assumption, large sparse data OK
SVM / SVR	Class/Reg	High accuracy in high dimensional spaces	Computationally expensive (O(n^3)), Scale-sensitive
K-Nearest Neighbors	Class/Reg	Instance-based learning, Simple	Scale-sensitive, Small data
Random Forest	Ensemble	Robust Classification/Regression	Handles Mixed types, Robust to outliers/missing values
XGBoost / LightGBM	Ensemble	High Performance	Large datasets, handles missing values internally
CatBoost	Ensemble	Best for Categorical Features	Handles categories automatically, Slower training
Isolation Forest	Anomaly Detection	Outlier/Anomaly Detection	High dimensional data, efficiency
K-Means	Clustering	Partitioning into K clusters	Spherical Clusters, Sensitive to outliers, Scale-sensitive
DBSCAN	Clustering	Density-based clustering, Detects Outliers	Arbitrary shapes, Scale-sensitive, finding epsilon is hard
Hierarchical	Clustering	Dendrogram visualization	Computationally expensive for large data

Methodology	Usage / Purpose	Data Constraints
CNN	Image/Pattern Recognition	Grid-like data (Images, etc.)
RNN / LSTM	Sequence/Time-Series Prediction	Sequential data
Transformer	NLP, Complex Pattern Matching	Long sequences, Large-scale data

Methodology	Type	Usage / Purpose	Notes
Stratified K-Fold	Validation	Cross Validation (Generalization)	Essential for Imbalanced Class distribution
K-Fold CV	Validation	Cross Validation	Sufficient data, Balanced classes
Time Series Split	Validation	Cross Validation (Temporal)	No future data leakage (essential for time-series)
Grid Search	Tuning	Hyperparameter Optimization	Small search space (Exhaustive)
Bayesian Optimization	Tuning	Hyperparameter Optimization	Large search space, High evaluation cost
Optuna	Tuning	Next-gen Hyperparameter Optimization	Efficient, Define-by-run, Pruning capabilities
L1 (Lasso)	Regularization	Sparse Model, Feature Selection	When sparse solution is needed
L2 (Ridge)	Regularization	Prevent Overfitting, Weight Decay	When high multicollinearity exists
ElasticNet	Regularization	Combination of L1 and L2	When both feature selection and regularization needed

Metric	Focus	When to use
Accuracy	Overall Correctness	Balanced datasets only. Misleading for imbalanced data.
Precision	False Positive Reduction	When FP is costly (e.g., Spam Filter).
Recall	False Negative Reduction	When FN is critical (e.g., Cancer Diagnosis, Fraud).
F1 Score	Balance	When you need a balance between Precision and Recall.
ROC-AUC	Ranking Quality	When you need robust performance across thresholds.
Log Loss	Probability Confidence	When the predicted probability value itself matters.

Data Analyst

Data Analyst Workflow

0단계: 기본 원칙 (Core Principles)

1단계: 분석 환경 및 목표 정의 (Environment & Goal)

Data Analyst

Data Analyst Workflow

0단계: 기본 원칙 (Core Principles)

1단계: 분석 환경 및 목표 정의 (Environment & Goal)

3단계: 방법론 스크리닝 (Methodology Screening)

2단계: 데이터 적재 및 품질 검증 (Obtain & Scrub)

3단계: 가설 주도적 탐색 (Hypothesis Driven EDA)

4단계: 모델링 및 해석 (Model & Interpret) [Optional]

4.3 Advanced Modeling & Tuning

Standards & Rules

Data Analyst Standards (OSEMN)

Purpose

Core Principles (Core Philosophy)

Quality Standards (Tier 1 Best Practices)

1. Data Integrity (Obtain & Scrub)

2. Hypothesis Driven EDA (Explore)

3. Rigorous Modeling (Model)

4. Interpretation (Interpret)

Checklist (Quality Gate)

Appendix: Methodology Master List (Reference)

1. Preprocessing & Data Cleaning

2. Machine Learning Models

3. Deep Learning Models

4. Validation & Optimization

5. Interpretation

Appendix: Evaluation Metrics Guide

Classification Metrics

Regression Metrics

Clustering Metrics (Unsupervised)

Visualization Expert

Data Analyst

Huggingface Hub

Multi Reviewer Patterns

Dbt Transformation Patterns

Startup Financial Modeling

Metric	Focus	When to use
MSE	Large Error Penalty	When outliers/large errors should be heavily penalized.
RMSE	Interpretability	When you need error in the same unit as the target.
MAE	Robustness	When you want to be robust against outliers.
R2 Score	Explainability	To see how much variance is explained by the model.
MAPE	Business Interpretability	Error in Percentage (%). Easy for stakeholders.

Metric	Focus	When to use
Silhouette Score	Cluster Separation	To measure how similar an object is to its own cluster compared to other clusters.
Davies-Bouldin	Cluster Compactness	Lower is better. Good for comparing clustering algorithms.
Elbow Method	Optimal K	To find the inflection point (optimal K) in K-Means.