ESP32 Development with ESP-IDF

1. esp32_utils.py: Shared utilities for all scripts
2. Auto environment setup: Scripts handle ESP-IDF sourcing automatically
3. PySerial support: Monitor works in non-interactive terminals
4. Better diagnostics: Detailed error messages and troubleshooting
5. Platform compatibility: Cross-platform serial port detection

Configuration

First-Time Setup

Before using the skill, run the configuration wizard to set up your ESP-IDF environment:

python scripts/config_manager.py --setup

The wizard will:

1. Detect ESP-IDF installations and versions
2. Let you select which ESP-IDF installation to use
3. Set default ESP32 chip target (esp32, esp32s3, esp32c3, etc.)
4. Configure default serial port (optional)

Configuration File

Settings are saved to ~/.esp32-developer-config.json. You can:

• View config: python scripts/config_manager.py --show
• Reset config: python scripts/config_manager.py --reset
• Get IDF_PATH: python scripts/config_manager.py --path

Using Configuration in Scripts

All scripts (create_esp32_project.py, build_esp32_project.py, etc.) will:

• Use configured IDF_PATH automatically
• Automatically setup ESP-IDF environment without manual sourcing
• Use default target chip if not specified
• Use default serial port if not specified (with auto-detection fallback)
• Prompt for setup if not configured (with --setup option to force wizard)
• Provide better error messages and troubleshooting tips

Quick Start

Prerequisites

1. Install ESP-IDF (see ESP-IDF Setup Guide)
2. Optional: Set up environment manually: source $IDF_PATH/export.sh (skill scripts handle this automatically)
3. Connect ESP32 board via USB
4. Install pyserial for non-interactive monitoring: pip install pyserial (optional, but recommended)

Common Workflows

Create and run a Hello World project (recommended):

# Configure ESP-IDF first (if not already configured)
python scripts/config_manager.py --setup

# Create a new project
python scripts/create_esp32_project.py my_project --target esp32

cd my_project

# Build the project
python scripts/build_esp32_project.py

# Flash to board (auto-detects port)
python scripts/flash_esp32.py

# Monitor output (non-interactive)
python scripts/monitor_serial.py --method pyserial

Quick workflow with skill scripts:

# One-liner: build, flash, and monitor
python scripts/build_esp32_project.py && \
python scripts/flash_esp32.py --monitor

# For non-interactive environments:
python scripts/flash_esp32.py && \
python scripts/monitor_serial.py --method pyserial --timeout 10

Or use ESP-IDF commands directly:

# Copy example
cp -r $IDF_PATH/examples/get-started/hello_world my_project
cd my_project

# Build and flash
idf.py set-target esp32
idf.py build
idf.py flash monitor

Creating ESP32 Projects

Using the Project Creation Script

The scripts/create_esp32_project.py script creates a new ESP32 project from the official Hello World example with interactive configuration support:

python scripts/create_esp32_project.py <project_name> [--target TARGET] [--path PATH] [--setup]

Features:

• Uses configured IDF_PATH from ~/.esp32-developer-config.json
• If no target specified, uses default target from config
• Prompts for ESP-IDF setup if not configured (or use --setup to force wizard)
• Detects ESP-IDF version automatically

Examples:

# Create project with interactive setup (if not configured)
python scripts/create_esp32_project.py my_app

# Run setup wizard first, then create project
python scripts/create_esp32_project.py my_app --setup

# Create ESP32-S3 project in specific directory
python scripts/create_esp32_project.py my_s3_app --target esp32s3 --path ~/projects

# Create project using default target from config
python scripts/create_esp32_project.py my_app  # Uses configured default target

Manual Project Creation

1. Copy from ESP-IDF examples: cp -r $IDF_PATH/examples/get-started/hello_world my_project
2. Update CMakeLists.txt for your target chip
3. Modify main.c for your application

Project Template

A template project is available in assets/esp32_hello_world/ containing:

• Basic CMakeLists.txt files
• Hello World main.c with LED blinking
• README with instructions

Building and Compiling

Using the Build Script

The scripts/build_esp32_project.py script handles building with proper environment checks and configuration support:

python scripts/build_esp32_project.py [--path PROJECT_PATH] [--target TARGET] [--clean] [--setup] [--verbose]

Enhanced Features:

• Automatic ESP-IDF environment setup without manual sourcing
• Better error handling with detailed troubleshooting tips
• Project validation to ensure it's a valid ESP-IDF project
• Progress feedback and build summary with binary sizes
• Verbose mode for detailed output

Examples:

# Build current directory project (uses default target from config)
python scripts/build_esp32_project.py

# Run setup wizard first, then build
python scripts/build_esp32_project.py --setup

# Build specific project with target
python scripts/build_esp32_project.py --path ~/projects/my_app --target esp32s3

# Clean build with verbose output
python scripts/build_esp32_project.py --clean --verbose

# Build with detailed error reporting
python scripts/build_esp32_project.py --verbose

Manual Building

# Set target (if not default esp32)
idf.py set-target <chip>

# Configure (optional)
idf.py menuconfig

# Build
idf.py build

# Parallel build for speed
idf.py -j $(nproc) build

Build Output

• Binaries: build/bin/ (.bin, .elf files)
• Map file: build/<project_name>.map
• Size report: idf.py size

Flashing to Development Board

Using the Flash Script

The scripts/flash_esp32.py script handles flashing with automatic port detection and configuration support:

python scripts/flash_esp32.py [--port SERIAL_PORT] [--baud BAUD_RATE] [--path PROJECT_PATH] 
                   [--monitor] [--setup] [--verbose] [--no-reset]

Enhanced Features:

• Platform-specific serial port detection (Linux, macOS, Windows)
• Better error handling with detailed troubleshooting tips
• Project validation to ensure it's built before flashing
• Flash verification tips and post-flash instructions
• No-reset option for boards with problematic auto-reset

Examples:

# Flash with auto-detected port
python scripts/flash_esp32.py

# Run setup wizard first, then flash
python scripts/flash_esp32.py --setup

# Specify port and baud rate
python scripts/flash_esp32.py --port /dev/ttyUSB0 --baud 460800

# Flash with verbose output
python scripts/flash_esp32.py --port /dev/ttyUSB0 --verbose

# Flash without auto-reset (press BOOT button manually)
python scripts/flash_esp32.py --port /dev/ttyUSB0 --no-reset

# Windows example
python scripts/flash_esp32.py --port COM3 --baud 115200

Manual Flashing

# Auto-detect port
idf.py flash

# Specify port
idf.py -p /dev/ttyUSB0 flash

# Specify baud rate
idf.py -p /dev/ttyUSB0 -b 460800 flash

# Flash and monitor
idf.py flash monitor

Board-Specific Notes

• ESP32-DevKitC: May require BOOT button press for flashing
• ESP32-S3/C3-DevKitC: USB-CDC, auto-reset usually works
• Third-party boards: Check schematics for correct GPIO0/EN connections

See ESP32 Boards Reference for details.

Serial Monitoring and Debugging

Using the Monitor Script

The scripts/monitor_serial.py script provides enhanced serial monitoring with filtering, logging, and configuration support:

python scripts/monitor_serial.py [--port SERIAL_PORT] [--baud BAUD_RATE] [--path PROJECT_PATH] 
                      [--filter PATTERN] [--save LOG_FILE] [--no-color] [--setup]
                      [--method {idf,pyserial}] [--timeout SECONDS] [--exit-on ERROR_PATTERN]

Enhanced Features:

• Dual monitoring methods: idf.py monitor (interactive) or pyserial (non-interactive)
• Platform-specific serial port detection
• Real-time filtering with regex patterns
• Log saving to file with timestamp
• Timeout and exit-on-pattern options
• Works in non-interactive environments (like opencode)

Examples:

# Basic monitor with auto-detected port (uses pyserial by default)
python scripts/monitor_serial.py

# Run setup wizard first, then monitor
python scripts/monitor_serial.py --setup

# Monitor with error filtering
python scripts/monitor_serial.py --filter "error|warn|fail"

# Save log to file with timeout
python scripts/monitor_serial.py --save debug.log --timeout 30

# Monitor specific port with pyserial method
python scripts/monitor_serial.py --port /dev/ttyUSB0 --baud 115200 --method pyserial

# Monitor and exit on panic
python scripts/monitor_serial.py --exit-on "panic|abort"

# Use idf.py monitor (interactive terminals only)
python scripts/monitor_serial.py --method idf --no-color

Manual Monitoring

# Start monitor
idf.py monitor

# Specify port and baud rate
idf.py -p /dev/ttyUSB0 -b 115200 monitor

# Exit monitor: Ctrl+]

Log Analysis

• Default baud rate: 115200 (application), 74880 (boot messages)
• For non-interactive environments: Use --method pyserial option
• Use ESP_LOGI(), ESP_LOGW(), ESP_LOGE() for structured logging
• Enable verbose logging: esp_log_level_set("*", ESP_LOG_VERBOSE)
• Filtering: Use --filter option to show only specific messages (errors, warnings, etc.)
• Log saving: Use --save option to capture logs to file for later analysis

Troubleshooting and Common Issues

Quick Diagnostics

1. Check ESP-IDF environment: idf.py --version
2. Verify serial port: ls /dev/ttyUSB* /dev/ttyACM*
3. Check board power LED
4. Monitor boot messages

Common Problems

• Flashing timeout: Press BOOT button during flash, use lower baud rate, try --no-reset option
• Permission denied: Add user to dialout group: sudo usermod -a -G dialout $USER
• Build errors: Clean build: idf.py fullclean, check CMakeLists.txt
• No serial output: Check baud rate (115200 for app, 74880 for boot), verify code has print statements
• idf.py monitor fails in non-interactive environment: Use --method pyserial option
• Serial port not detected: Check cable, try different port, ensure drivers installed

Debug Techniques

1. Enable core dumps in menuconfig
2. Use JTAG debugging with OpenOCD
3. Add diagnostic prints: printf("Debug: value=%d\n", variable);
4. Check free heap: esp_get_free_heap_size()

See Troubleshooting Guide for detailed solutions.

Reference Materials

Included References

Load these references as needed for specific tasks:

1. ESP-IDF Setup Guide - Installation and environment setup
2. ESP-IDF Command Reference - Complete idf.py command list
3. ESP32 Boards Reference - Development board specifications
4. Troubleshooting Guide - Common issues and solutions

ESP-IDF Resources

• Official Documentation
• API Reference
• Examples Directory
• ESP32 Forum

Scripts Overview

scripts/config_manager.py

Configuration manager for ESP-IDF setup. Handles ESP-IDF installation detection, version selection, and user preferences. Run python scripts/config_manager.py --setup for interactive configuration wizard.

scripts/esp32_utils.py

New: Shared utility module providing common functions for ESP32 development:

• Automatic ESP-IDF environment setup
• Platform-specific serial port detection
• Project validation and information
• Unified idf.py command execution
• Formatting helpers and utilities

scripts/create_esp32_project.py

Creates new ESP32 projects from Hello World example with target chip configuration and interactive setup support.

scripts/build_esp32_project.py (Enhanced)

Builds ESP32 projects with enhanced features:

• Automatic environment setup
• Better error handling with troubleshooting tips
• Project validation
• Progress feedback and build summary

scripts/flash_esp32.py (Enhanced)

Flashes firmware to ESP32 boards with enhanced features:

• Platform-specific port detection
• Better error handling
• Flash verification tips
• No-reset option support

scripts/monitor_serial.py (Enhanced)

Monitors serial output with enhanced features:

• Dual monitoring methods (idf.py or pyserial)
• Works in non-interactive environments
• Filtering, logging, timeout options
• Exit-on-pattern capability

Using Bundled Resources

Templates

• assets/esp32_hello_world/ - Minimal Hello World project template
• Copy and customize for new projects

References

• Load reference files when specific information is needed
• Use grep patterns to find relevant sections

Direct ESP-IDF Usage

When scripts aren't needed, use standard ESP-IDF commands:

# Complete workflow
cp -r $IDF_PATH/examples/get-started/hello_world my_project
cd my_project
idf.py set-target esp32
idf.py menuconfig
idf.py build
idf.py flash monitor

When to use skill scripts vs direct commands:

• Use skill scripts: For consistent environment setup, better error handling, non-interactive environments
• Use direct commands: When you need full control, interactive development, or advanced features
• Special cases: Use --method pyserial when idf.py monitor fails in non-interactive terminals

Configuration Management and Git Best Practices

Based on real-world ESP32 project experience (OpenClaw Monitor), follow these practices for secure and maintainable projects:

Secure Configuration Management

ESP-IDF projects should never commit sensitive data (WiFi credentials, API keys) to version control. Use a layered configuration strategy:

1. sdkconfig.defaults (committed): Non-sensitive default values, template for new setups

   # Example: Define placeholder values
   CONFIG_WIFI_SSID="your_wifi_ssid"
   CONFIG_WIFI_PASSWORD="your_wifi_password"
   CONFIG_GATEWAY_HOST="192.168.2.195"
   

2. sdkconfig.ci (committed): CI/CD configuration with environment variable placeholders

   # Use environment variables for CI/CD pipelines
   CONFIG_WIFI_SSID="$WIFI_SSID"
   CONFIG_WIFI_PASSWORD="$WIFI_PASSWORD"
   CONFIG_GATEWAY_HOST="$GATEWAY_HOST"
   

3. sdkconfig (.gitignore): Local configuration with real values - never commit

   # Generated by menuconfig or manually edited
   CONFIG_WIFI_SSID="MyHomeWiFi"
   CONFIG_WIFI_PASSWORD="secure_password"
   CONFIG_GATEWAY_HOST="192.168.2.195"
   

4. Chip-specific defaults: sdkconfig.defaults.esp32s2 for target-specific settings

Initialization workflow:

# First-time setup: Copy defaults template
cp sdkconfig.defaults sdkconfig

# Configure via menuconfig
idf.py menuconfig

# Or edit directly (for CI/CD)
idf.py -D SDKCONFIG_DEFAULTS="sdkconfig.ci" build

Git Repository Management

1. Branch strategy:

   • main: Production-ready releases
   • develop: Active development branch
   • Feature branches: feature/, fix/, refactor/

2. .gitignore essentials for ESP-IDF:

   # Build artifacts
   build/
   sdkconfig
   sdkconfig.old
   
   # IDE and editor files
   .vscode/
   .idea/
   *.swp
   
   # Auto-generated components
   managed_components/
   
   # Dependency locks
   dependencies.lock
   

3. Sensitive information protection:

   • Never commit sdkconfig with real credentials
   • Use environment variables in CI/CD (sdkconfig.ci)
   • Rotate credentials if accidentally committed

Automated Testing Framework

ESP-IDF provides comprehensive testing support:

1. Unit Testing (Unity framework):

   // components/my_component/test/test_my_component.c
   #include "unity.h"
   #include "my_component.h"
   
   TEST_CASE("Test function returns expected value", "[my_component]")
   {
       TEST_ASSERT_EQUAL(42, my_function());
   }
   

2. Test project structure:

   project/
   ├── components/my_component/
   │   ├── my_component.c
   │   └── test/                    # Component tests
   │       ├── CMakeLists.txt
   │       └── test_my_component.c
   └── test/                        # Test runner project
       ├── CMakeLists.txt          # Sets TEST_COMPONENTS
       └── main/test_runner.c      # unity_run_all_tests()
   

3. Integration Testing (pytest-embedded):

   # pytest_project.py
   import pytest
   from pytest_embedded import Dut
   
   @pytest.mark.esp32s2
   def test_wifi_connection(dut: Dut):
       dut.expect('Connecting to WiFi...')
       dut.expect('WiFi connected')
   

4. CI/CD Testing Commands:

   # Build test project
   cd test
   idf.py set-target esp32s2
   idf.py build
   
   # Run unit tests (hardware optional for mocked tests)
   idf.py flash monitor  # Runs registered tests
   
   # Run pytest tests
   pytest pytest_project.py --target esp32s2
   

Continuous Integration Setup

Example GitHub Actions workflow: