Data Analysis

3. Create an exploratory data analysis (EDA) file (01 eda.r). This will include steps for summarizing the data, creating visualizations, and identifying any patterns or trends in the data.

This file outputs a full correaltion matrix for all variables (and sub matrices if it helps interpretation). Histograms for all variables. Checks for outliers, etc. Output everything as a series of plots to a subfolder called eda.

4. Create a final analysis file. This will include steps for conducting any statistical analyses, creating final visualizations, and writing up the results. This will be a qmd file.

This should have all the final analyses and visualizations, and should be written up in a clear and concise manner. Make sure to include all relevant information, such as the sample size, the statistical tests used, and the results of those tests. DO NOT use cat to show results, instead use r results to show results in the quarto document. This will make it easier to read and understand the results. Make the code hideable Make sure the code is runnable interactively or from the quarto document. This will allow you to easily test and debug your code, and also make it easier for others to understand and reproduce your analysis. For example:

# switch for data load.
is_running_in_quarto <- function() {
  # Check if a Quarto-specific environment variable is set
  return(Sys.getenv("QUARTO_DOCUMENT_PATH") != "")}

# Example usage:
if (is_running_in_quarto()) {
  dat <- read_parquet("../data/processed/analysis_data.parquet")
  fig_path <- "../output/figures/"
  tab_path <- "../output/tables/"
} else {
 dat <- read_parquet("./data/processed/analysis_data.parquet")
 fig_path <- "./output/figures/"
 tab_path <- "./output/tables/"
}

Before submitting your work, always read the rendered report to make sure your reporting makes sense: for example if you wrote A and B were positively correlated (r = 0.023, p = 0.523), then you need to fix your write up.

Some rules:

• Always use the same file structure for each project. This will make it easier to find files and keep things organized.
• Use clear and descriptive names for files and variables. This will make it easier to understand what each file and variable is for.
• Document your code thoroughly. This will make it easier for others (or yourself in the future) to understand what you did and why you did it.
• Use version control (e.g., Git) to track changes to your code and collaborate with others. This will help you keep track of changes and avoid losing work.
• Always test your code to ensure it works as expected. This will help you catch any errors or bugs before they become a problem. Make sure you run the quarto document to make sure everything is working.
• Absolutely NO MAGIC NUMBERS. Don't hard code any statistics, instead calculate them from the data. This will ensure that your analysis is reproducible and that you can easily update your results if you need to make changes to your data or analysis.