Installation

Requirements

The benchmark builds on ManiSkill and SAPIEN. You need a Python 3.11 environment, a uv installation, and a Vulkan-capable GPU driver for rendering. All commands below run from the repository root.

Install the package

Installation only requires a few pip installs plus a Vulkan setup for rendering. The repository uses uv to manage the environment:

# install the package
uv sync --active

# activate the uv environment
source .venv/bin/activate

# patch a known lerobot bug
curl -sSL https://raw.githubusercontent.com/huggingface/lerobot/0e81a275fcdbf03d74f78aa69eaa28c172a9f256/src/lerobot/datasets/lerobot_dataset.py -o .venv/lib/python3.11/site-packages/lerobot/datasets/lerobot_dataset.py

Vulkan rendering

Rendering is done with Vulkan. Follow the ManiSkill installation instructions to set up the Vulkan driver and the SAPIEN renderer on your platform.

Downloading assets

Scene objects and datasets are downloaded into ~/.maniskill/data. The standard YCB assets are fetched with the built-in downloader:

IG-10K Assets (one-shot)

The recommended path is to download the shared asset package into ~/.maniskill/data. It includes the assets used by the benchmark; the individual ManiSkill download commands below remain useful when you only need one namespace.

mkdir -p ~/.maniskill/data
hf download imitator-game/IG-10K-Assets --repo-type dataset --local-dir ~/.maniskill/data
# ModelScope alternative:
# modelscope download --dataset Zhouxunzhe/IG-10K-Assets --local_dir ~/.maniskill/data

for f in ~/.maniskill/data/*.tar.zst; do
  tar --use-compress-program=unzstd -xf "$f" -C ~/.maniskill/data
done
rm ~/.maniskill/data/*.tar.zst

# download assets for a specific environment id
python -m mani_skill.utils.download_asset ${ENV_ID}

# standard YCB assets
python -m mani_skill.utils.download_asset ycb

RoboTwin objects & background textures

RoboTwin objects and the background textures used by random_background=True (see the domain randomization guide) come from the RoboTwin asset set:

mkdir -p ~/.maniskill/data/robotwin
python -m mani_skill.utils.download_robotwin
cd ~/.maniskill/data/robotwin
unzip objects.zip && rm -rf objects.zip
unzip background_texture.zip && rm -rf background_texture.zip

PartNet-Mobility

PartNet-object assets (microwave, cabinet, and other articulated objects) follow the download scripts' instructions:

# PartNet-mobility dataset
python -m mani_skill.utils.download_partnet
mkdir -p ~/.maniskill/data/partnet_mobility/dataset
mv ~/Downloads/downloads/* ~/.maniskill/data/partnet_mobility/dataset/
cd ~/.maniskill/data/partnet_mobility/dataset && find . -name "*.zip" -execdir unzip -o {} \;

Scene datasets (RoboCasa / AI2THOR)

python -m mani_skill.utils.download_asset RoboCasa
python -m mani_skill.utils.download_asset AI2THOR

ReplicaCAD's link in download_asset.py is expired; download the dataset from this google drive link and unpack it manually:

pip install gdown
mkdir -p ~/.maniskill/data/scene_datasets/replica_cad_dataset
gdown "https://drive.google.com/uc?id=1mqImztNX1LYZFBzt9z895C814RsyGe4N" -O replica_cad_dataset.zip
unzip -o replica_cad_dataset.zip -d ~/.maniskill/data/scene_datasets/replica_cad_dataset && rm replica_cad_dataset.zip

External GLB assets

External GLB assets (Sketchfab models, Hunyuan-generated meshes, etc.) are downloaded to ~/.maniskill/data/sketchfab/objects/{object_key}/model.glb, registered in mani_skill/assets/sketchfab_registry.json, and loaded in tasks with create_sketchfab_actor(...). See mani_skill/assets/sketchfab_README.md for the full pipeline. Three L3 environments also require the balance_scale and weight_on_scale GLBs from the shared GLB asset folder:

~/.maniskill/data/sketchfab/objects/balance_scale/model.glb
~/.maniskill/data/sketchfab/objects/weight_on_scale/model.glb

Asset namespaces

The benchmark uses four asset namespaces. MS_ASSET_DIR can override the default ~/.maniskill root; the paths below show the default layout.

Verify the install

Quick sanity check: create and step one of the smallest benchmark environments. You should see a success-rate tensor from evaluate():

import gymnasium as gym
import mani_skill.envs  # Registers all benchmark environments.

from mani_skill.envs.tasks.tabletop.utils.dual_task_camera_utils import (
    configure_dual_task_level,
)

configure_dual_task_level("L0")
env = gym.make(
    "TwoRobotStirSpoon-v1",
    obs_mode="state",
    control_mode="pd_joint_pos",
    reward_mode="dense",
    sim_backend="physx_cpu",
)

obs, info = env.reset(seed=0)
action = env.action_space.sample()
obs, reward, terminated, truncated, info = env.step(action)

evaluation = env.unwrapped.evaluate()
print(evaluation["success"])
env.close()