Robolab Taskgen

When the user invokes this skill, display the following message verbatim:

I'll help you generate a RoboLab task file. I need a few things:

1. Scene file — Either:
   • A filename (e.g., banana_bowl.usda) if the scene is in robolab/assets/scenes/
   • A full path to the .usda file if it's elsewhere
2. Task instruction — What should the robot do? (e.g., "Pick up the banana and place it in the bowl"). This becomes the default instruction; I'll generate vague and specific variants automatically unless you provide them.
3. Episode length — How long should the robot have to complete the task, in seconds? (e.g., 50 for a simple pick-and-place, 90-120 for multi-step tasks)
4. Output directory — Where should I save the task file?
   • robolab/tasks/benchmark/
   • robolab/tasks/<name>/ — give me a folder name and I'll create it
   • Or specify another path

Here's an example of what I'll generate — a file called banana_in_bowl_task.py:

# banana_in_bowl_task.py

@configclass
class BananaInBowlTerminations:
    time_out = DoneTerm(func=mdp.time_out, time_out=True)
    success = DoneTerm(
        func=object_in_container,
        params={"object": "banana", "container": "bowl", ...},
    )

@dataclass
class BananaInBowlTask(Task):
    contact_object_list = ["banana", "bowl", "table"]
    scene = import_scene("banana_bowl.usda", contact_object_list)
    terminations = BananaInBowlTerminations
    instruction = {
        "default": "Pick up the banana and place it in the bowl",
        "vague": "Put the fruit in the bowl",
        "specific": "Grasp the yellow banana and place it inside the red bowl on the table",
    }
    episode_length_s: int = 50
    subtasks = [pick_and_place(object=["banana"], container="bowl", logical="all", score=1.0)]

After the User Provides Information

After receiving the user's input:

1. Check for duplicate task names. Search for existing task classes with the same name across robolab/tasks/ and the user's output directory. If a task with the same class name already exists, warn the user and ask them to choose a different name. Do not overwrite existing task files.

2. Always ask the user where to save the task on the first invocation. Do not assume a default — you must wait for the user to confirm a directory before writing any file. Offer these options:

   • robolab/tasks/benchmark/ (existing benchmark tasks)
   • robolab/tasks/<name>/ (a new folder — ask the user for <name>)
   • Another path of their choosing

   Once the user has chosen a directory, reuse it for all subsequent tasks in the same session without asking again.

3. State the termination criterion explicitly before generating the file. For example: "Based on your description, I'm using object_in_container as the success condition — this checks that the object is inside the container and the gripper has released it."

4. Use the user's instruction text as the "default" instruction variant. Generate "vague" and "specific" variants automatically unless the user explicitly provides them.

5. Extract the object list from the scene file and the user's description.

6. Proceed with the Step-by-Step Generation Workflow.

Information to Gather

Before generating a task, collect the following:

Intent Routing

• User wants to create a new task from scratch --> Display the When Invoked message, then follow the Step-by-Step Workflow
• User needs help choosing a conditional function --> Read references/conditionals.md
• User wants to see examples --> Read references/examples.md
• User wants to validate an existing task --> Validation
• User wants to organize tasks into a library --> See docs/task_libraries.md
• User wants to register and run tasks --> See docs/environment_registration.md

Task File Template

import os
from dataclasses import dataclass

import isaaclab.envs.mdp as mdp
from isaaclab.managers import TerminationTermCfg as DoneTerm
from isaaclab.utils import configclass

from robolab.core.scenes.utils import import_scene
from robolab.core.task.conditionals import <CONDITIONAL_IMPORTS>
from robolab.core.task.task import Task

SCENE_DIR = os.path.join(os.path.dirname(__file__), "..", "scenes")


@configclass
class <TaskName>Terminations:
    time_out = DoneTerm(func=mdp.time_out, time_out=True)
    success = DoneTerm(
        func=<conditional_function>,
        params={<params_dict>},
    )


@dataclass
class <TaskName>Task(Task):
    contact_object_list = [<all_object_names>]
    scene = import_scene(os.path.join(SCENE_DIR, "<scene_file>.usda"), contact_object_list)
    terminations = <TaskName>Terminations
    instruction = {
        "default": "<clear, natural instruction>",
        "vague": "<ambiguous version>",
        "specific": "<detailed version with colors, sizes, exact locations>",
    }
    episode_length_s: int = <seconds>
    attributes = [<attribute_tags>]
    subtasks = [<optional_subtasks>]

Importing Scenes

For scenes inside the RoboLab repo, pass just the filename (auto-resolved):

scene = import_scene("banana_bowl.usda", contact_object_list)

For scenes in your own repository, use an absolute path:

SCENE_DIR = os.path.join(os.path.dirname(__file__), "..", "scenes")
scene = import_scene(os.path.join(SCENE_DIR, "my_scene.usda"), contact_object_list)

To auto-extract the contact object list from a scene:

from robolab.core.scenes.utils import import_scene_and_contact_object_list
MyScene, contact_object_list = import_scene_and_contact_object_list("/path/to/scene.usda")

Step-by-Step Generation Workflow

1. Identify the goal and objects. List all objects the robot may touch, including surfaces like "table".

2. Choose the termination conditional. Match the success condition to a function (see references/conditionals.md for the full list):

   • "Put X in Y" --> object_in_container
   • "Put X on Y" --> object_on_top
   • "Stack X on Y on Z" --> stacked
   • "Move X left of Y" --> object_left_of
   • "Sort X into Y and Z" --> object_groups_in_containers
   • "Take X out of Y" --> object_outside_of

3. Write the terminations class. Always include time_out and success. Set require_gripper_detached=True for any placement condition.

4. Write instruction variants. Default (clear), vague (ambiguous), specific (detailed). See Instruction Variants.

5. Decompose into subtasks if multi-step. Use pick_and_place for standard pick-and-place; use Subtask with partial for custom conditions. See Subtasks.

6. Set episode length. Simple tasks: 30-50s. Multi-object: 60-90s. Complex sorting/stacking: 90-120s.

7. Choose attributes. Tag based on the skills required. See Attribute Tags.

8. Assemble the task file. Follow the template above.

Instruction Variants

Every task should define three instruction variants:

• default: Clear, natural language instruction. This is the primary instruction used during evaluation.
  • Example: "Pick up the banana and place it in the bowl"
• vague: Ambiguous version that tests semantic understanding. Omit specific object names or use general terms.
  • Example: "Put the fruit in the bowl"
• specific: Highly detailed with colors, sizes, materials, and exact locations.
  • Example: "Grasp the yellow banana and place it inside the red bowl on the wooden table"

When using a dict for instructions, omit the type annotation to avoid dataclass mutable-default errors:

# Correct:
instruction = {"default": "...", "vague": "...", "specific": "..."}

# Wrong (will cause dataclass error):