Eval

Evaluate a Trained Model

Run IsaacSim simulation evaluation on a checkpoint, preferably on skt L40S nodes.

Prerequisites

• rlwrld_isaac must be set up on the target cluster
• Checkpoint must be accessible (local path or HuggingFace repo ID)
• If checkpoint is on a different cluster, use sync_checkpoint first

Steps

1. Identify the checkpoint:

   • If user gives a job ID → look up in experiment store or find checkpoint path via remote_exec
   • If user gives a checkpoint name → resolve using knowledge base paths
   • If user gives a HF repo ID → use directly as model_path

2. Determine eval parameters from knowledge base (eval_configs):

   • Task name (e.g., task-Cube_Box-5cmLeft for v4_cube_box experiments)