Organize downloaded FEMA eBFE/BLE model files into standardized 4-folder structure. Use when working with eBFE downloads that need organization into: - HMS Model/ (HEC-HMS hydrologic models) - RAS Model/ (HEC-RAS hydraulic models) - Spatial Data/ (GIS, terrain, geodatabases) - Documentation/ (reports, PDFs, metadata) Handles variable archive structures, nested zips, and mixed content through intelligent file analysis and recursive extraction.
When the user asks to organize eBFE or BLE downloads, use this skill to transform downloaded FEMA eBFE/BLE study area files into a standardized 4-folder structure, regardless of the original archive organization patterns.
Every organized eBFE study area should have this structure:
{StudyAreaName}_{HUC8}/
├── HMS Model/ # HEC-HMS hydrologic models
│ ├── {ProjectName}.hms
│ ├── {ProjectName}.basin
│ ├── *.dss
│ └── ... (all HMS-related files)
├── RAS Model/ # HEC-RAS hydraulic models
│ ├── {Model1}/
│ │ ├── {Model1}.prj
│ │ ├── {Model1}.g##
│ │ ├── {Model1}.p##
│ │ └── ... (all RAS project files)
│ └── {Model2}/...
├── Spatial Data/ # GIS, terrain, geodatabases
│ ├── Terrain/
│ ├── *.tif, *.hdf
│ ├── *.gdb/
│ └── ... (all spatial data)
└── Documentation/ # Reports, metadata, inventories
├── *.pdf
├── *.xlsx (inventories)
├── *_metadata.xml
└── ... (all documentation)
Handle these downloaded archive patterns:
Pattern 1: Multiple 1D models in variable wrapper folders (80-200 MB) Pattern 2: ModelURLs.txt links file (1 KB) Pattern 3: Single 2D model in nested zip (5-15 GB) Pattern 4: Compound HMS + RAS in nested zips (8+ GB)
See feature_dev_notes/eBFE_Integration/RESEARCH_FINDINGS.md for complete pattern documentation.
Include files with these extensions or patterns:
.hms - HEC-HMS project.basin - Basin model file.met - Meteorology file.control - Control specifications.run - Run configuration.results - HMS results.dss in Hydrology/ path.log, .out in HMS foldersPath indicators: Hydrology/, HMS/
Include files with these extensions or patterns:
.prj - HEC-RAS project (validate: contains "Proj Title=", "Geom File=").g## - Geometry file.p## - Plan file.f## - Flow file.u## - Unsteady flow file.c## - Sediment file.b## - Bridge/culvert file.bco## - Boundary condition override.IC.O## - Initial conditions override.x## - Cross section index.rasmap - RAS Mapper project.dsc - DSS catalog (if in RAS folder).dss (if in RAS folder, not HMS folder)Exclude: .prj files in Features/, Shp/, gis/ (shapefiles, not HEC-RAS)
Organization within RAS Model/:
Include files with these extensions or patterns:
.tif - GeoTIFF raster.hdf - Terrain HDF (if NOT .g##.hdf or .p##.hdf).vrt - Virtual raster.gdb/ - File geodatabase.shp, .shx, .dbf, .prj - Shapefiles (in Features/, Shp/, gis/ folders)Terrain/ (entire folder)Spatial/, GIS/, Shp/, Features/Exception: .g##.hdf (geometry preprocessor) and .p##.hdf (results) stay with RAS Model/
Include files with these extensions or patterns:
.pdf - Reports, guides.xlsx, .xls - Inventories, metadata spreadsheets.xml - MetadataREADME* filesREQUIRED for Every Organization:
agent/model_log.md - Agent work logMANIFEST.md - Organization manifestorganize_{modelname}() function in feature_dev_notes/eBFE_Integration/generated_functions/# Inspect the downloaded archive
# - List all files
# - Identify nested zips
# - Detect pattern (1-4)
# - Count .prj files (RAS) and .hms files (HMS)
# Extract Models.zip
# Find nested .zip files (_Final.zip, RAS_Submittal.zip, etc.)
# Extract nested zips in place
# Preserve directory structure during extraction
# Recursively walk extracted content
# For each file:
# - Determine classification (HMS/RAS/Spatial/Docs)
# - Move to appropriate standardized folder
# - For RAS models with multiple projects:
# - Group files by .prj stem
# - Create subfolder per model
# CRITICAL INTEGRATIONS (to create runnable HEC-RAS models):
# 1. Move Output/ folder INTO HEC-RAS project folder
# eBFE separates Output/ (pre-run HDF files) from Input/ (project)
# HEC-RAS expects HDF files in project folder
for model_folder in ras_model.glob('**/Input'):
output_folder = model_folder.parent / 'Output'
if output_folder.exists():
# Move all Output/*.hdf and other files INTO Input/
for output_file in output_folder.rglob('*'):
if output_file.is_file():
shutil.copy2(output_file, model_folder / output_file.name)
# 2. Move Terrain/ INTO HEC-RAS project folder
# eBFE places Terrain/ as sibling to Input/ (breaks .rasmap references)
# HEC-RAS expects Terrain/ in project folder
for model_folder in ras_model.glob('**/Input'):
terrain_folder = model_folder.parent / 'Terrain'
if terrain_folder.exists():
# Move Terrain/ INTO Input/Terrain/
shutil.copytree(terrain_folder, model_folder / 'Terrain', dirs_exist_ok=True)
# Verify each folder:
# - HMS Model/: Check for .hms files
# - RAS Model/: Verify .prj files are valid HEC-RAS projects
# - Spatial Data/: Check for terrain files
# - Documentation/: Ensure reports are present
#
# Report summary:
# - X HMS projects found
# - Y RAS projects found
# - Z terrain files found
# - N documents organized
# CRITICAL: Fix DSS file path references in HEC-RAS files AND verify files exist
# eBFE models have:
# - Absolute paths from original system: C:\eBFE\... (doesn't exist on user system)
# - Wrong relative paths: DSS\Input\file.dss (file is actually in same folder)
# These cause "DSS path needs correction" GUI popups that break automation
import re
# Step 1: Find all DSS files that ACTUALLY EXIST in organized structure
dss_files = list(ras_model_folder.glob('**/*.dss'))
dss_lookup = {dss.name: dss for dss in dss_files} # Lookup by filename
print(f"Found {len(dss_files)} DSS file(s):")
for dss in dss_files:
print(f" - {dss.relative_to(ras_model_folder)}")
# Step 2: Find HEC-RAS files that reference DSS (.u##, .prj, .p##)
hecras_files = []
hecras_files.extend(ras_model_folder.glob('**/*.u[0-9]*')) # Unsteady flow files
hecras_files.extend(ras_model_folder.glob('**/*.p[0-9][0-9]')) # Plan files
hecras_files.extend(ras_model_folder.glob('**/*.prj')) # Project files
# Step 3: For each HEC-RAS file, validate and correct DSS references
for hecras_file in hecras_files:
content = hecras_file.read_text(encoding='utf-8', errors='ignore')
modified = False
# Find all "DSS File=" lines
dss_file_pattern = re.compile(r'DSS File=(.+?)(?:\n|$)', re.IGNORECASE)
for match in dss_file_pattern.finditer(content):
old_path = match.group(1).strip()
dss_filename = Path(old_path).name # Extract just the filename
# Check if this DSS file exists in our organized structure
if dss_filename in dss_lookup:
actual_dss_path = dss_lookup[dss_filename]
# CRITICAL: Verify the DSS file actually exists
if actual_dss_path.exists():
# Calculate correct relative path from HEC-RAS file to actual DSS location
rel_path = actual_dss_path.relative_to(hecras_file.parent)
rel_path_str = str(rel_path).replace('\\', '/')
# Only replace if current path is wrong
if old_path != rel_path_str:
old_full = f"DSS File={old_path}"
new_full = f"DSS File={rel_path_str}"
content = content.replace(old_full, new_full)
modified = True
print(f" Corrected in {hecras_file.name}:")
print(f" Old: {old_path}")
print(f" New: {rel_path_str} (✓ verified exists)")
else:
print(f" ⚠️ WARNING: DSS not found: {dss_filename}")
else:
print(f" ⚠️ WARNING: DSS not in organized structure: {dss_filename}")
if modified:
hecras_file.write_text(content, encoding='utf-8')
# Result: Model opens without GUI popups, all DSS paths verified
# Handles cases like:
# DSS\Input\UPG_precip.dss → UPG_precip.dss (if file is in same folder)
# C:\eBFE\...\file.dss → ../DSS/file.dss (if file is in subdirectory)
# Validate DSS pathname contents (not just file paths)
from ras_commander.dss import RasDss
# Find all .dss files in RAS Model/
dss_files = list(ras_model_folder.glob('**/*.dss'))
for dss_file in dss_files:
# Get catalog
catalog = RasDss.get_catalog(dss_file)
# Check each pathname
for pathname in catalog['pathname']:
result = RasDss.check_pathname(dss_file, pathname)
if not result.is_valid:
# Document invalid pathnames in model_log.md
pass
# If RAS model organized, suggest optional compute test to user
# This validates terrain, land use, and DSS files are correct
print("=" * 80)
print("OPTIONAL VALIDATION: Compute Test")
print("=" * 80)
print("To verify terrain, land use, and DSS files are correct,")
print("run at least one plan:")
print()
print(" from ras_commander import init_ras_project, RasCmdr")
print(f" init_ras_project(r'{ras_model_folder}', '{version}')")
print(" RasCmdr.compute_plan('01', num_cores=2)")
print()
print("If the plan executes successfully, terrain/DSS files are valid.")
print("=" * 80)
# After compute test completes (if user runs it), check results for errors
from ras_commander.hdf import HdfResultsPlan
# Launch haiku subagent to check results
Task(
subagent_type="notebook-output-auditor",
model="haiku",
description="Check HEC-RAS results for errors",
prompt=f"""
Check HEC-RAS results for errors in: {hdf_file}
Extract compute messages and check for:
- Error messages
- Warning messages
- Convergence issues
- Numerical instabilities
- Missing data
Write findings to: {agent_folder}/compute_test_results.md
"""
)
REQUIRED: After organizing each model, generate a Python function for future use.
# Generate organize_{modelname}() function
# Write to: feature_dev_notes/eBFE_Integration/generated_functions/{studyarea}_{huc8}.py
def organize_{clean_name}(
downloaded_folder: Path,
output_folder: Optional[Path] = None
) -> Path:
"""
Organize {StudyArea} ({HUC8}) eBFE model.
Pattern {X}: {Pattern description}
Generated by ebfe_organize_models agent on {date}.
"""
# Specific organization steps discovered by agent
# Include pattern detection, extraction, classification
# Use lessons learned from this organization
pass
Deliverable: Write generated function to feature_dev_notes/eBFE_Integration/generated_functions/organize_{studyarea}_{huc8}.py
# Create organization manifest: MANIFEST.md (in project root)
# Document:
# - Original archive pattern detected
# - Files organized by category
# - RAS project inventory (model names, types, sizes)
# - HMS project inventory (if any)
# - Terrain files inventory
# - Document inventory
# - DSS validation results
# - Suggested compute test command
# Create agent work log: agent/model_log.md (in project root)
# **REQUIRED**: Every organized model MUST have agent/model_log.md
# Document:
# - Agent actions taken
# - Files classified and moved
# - Decisions made during organization
# - Issues encountered and resolutions
# - DSS validation results
# - Generated function location
# - Timestamps for all operations
CRITICAL: Every organized eBFE model MUST have agent/model_log.md documenting agent work.
Location: {StudyAreaName}_{HUC8}/agent/model_log.md
Template:
# Agent Work Log - {StudyAreaName}
**Agent**: ebfe_organize_models
**Date**: {YYYY-MM-DD HH:MM:SS}
**Study Area**: {Name} ({HUC8})
**Pattern**: {Pattern 1/2/3/4}
## Actions Taken
### Archive Extraction
- Extracted: {source_archive}
- Nested zips: {list any nested zips extracted}
- Total files extracted: {count}
### File Classification
{For each file category, list files moved}
#### HMS Model (X files, Y MB)
- {list key files}
#### RAS Model (X files, Y GB)
- {list models and key files}
#### Spatial Data (X files, Y MB)
- {list terrain, shapefiles, etc.}
#### Documentation (X files, Y MB)
- {list reports, inventories}
### Decisions Made
- {Document any classification decisions}
- {Document handling of ambiguous files}
- {Document terrain handling approach}
### Issues Encountered
- {List any problems}
- {Document resolutions}
### Validation Checks
- [x] All files classified
- [x] RAS .prj files validated (not shapefiles)
- [x] Terrain location documented
- [x] Manifest created
## Next Steps for User
{Provide clear next steps, e.g., testing with ras-commander}
# eBFE Model Organization Manifest
**Study Area**: {Name}
**HUC8**: {HUC8}
**Original Pattern**: {Pattern 1/2/3/4}
**Organized Date**: {YYYY-MM-DD}
## HMS Model
{X} HEC-HMS projects found:
- {ProjectName}.hms - {StormFrequencies}
## RAS Model
{Y} HEC-RAS projects found:
### {ModelName1}
- Type: {1D/2D}
- Plans: {#}
- Size: {MB/GB}
- Terrain: {Self-contained/External/Not needed}
### {ModelName2}
...
## Spatial Data
{Z} terrain/spatial files:
- {TerrainFile1} - {Size}
- Geodatabase: {.gdb folders}
- Shapefiles: {count}
## Documentation
{N} documents:
- {Report1}.pdf - {Size}
- {Inventory}.xlsx
- {Metadata}.xml
## Notes
{Any special observations or issues encountered}
# Organize a downloaded study area
from pathlib import Path
# Input: Raw downloaded and extracted files
source = Path("D:/eBFE/raw_downloads/12040102_Spring_Models_extracted/")
# Output: Organized into 4 folders
target = Path("D:/eBFE/organized/SpringCreek_12040102/")
# Agent organizes the files
# Result: target/ now has HMS Model/, RAS Model/, Spatial Data/, Documentation/
.prj in Features/, Shp/, gis/ → Spatial Data/.prj with "Proj Title=" content → RAS Model/For generated function templates, DSS validation examples, and compute test workflows, see references/templates-and-validation.md.
Skills (related workflows):
ebfe_validate_models -- Use downstream to validate organized modelsAgents (delegate when needed):
ebfe-organizer -- Delegate for complex multi-model eBFE organizationPrimary sources:
ras_commander/ebfe_models.py -- RasEbfeModels implementationdocs/ebfe_models.md -- Complete eBFE documentation