Get started

Installation

Install the package with uv, set up Vulkan for rendering, and download the scene assets the benchmark environments depend on.

Requirements

The benchmark builds on ManiSkill and SAPIEN. You need a Python 3.11 environment, a uv installation, and a Vulkan-capable GPU driver for rendering. All commands below run from the repository root.

Install the package

Installation only requires a few pip installs plus a Vulkan setup for rendering. The repository uses uv to manage the environment:

bash

# install the package
uv sync --active

# activate the uv environment
source .venv/bin/activate

# patch a known lerobot bug
curl -sSL https://raw.githubusercontent.com/huggingface/lerobot/0e81a275fcdbf03d74f78aa69eaa28c172a9f256/src/lerobot/datasets/lerobot_dataset.py -o .venv/lib/python3.11/site-packages/lerobot/datasets/lerobot_dataset.py

Use the vendored ManiSkill

Always use the mani_skill package inside this repository, not an independently installed copy. The benchmark extends it with 50 dual-arm tabletop tasks, the four scene-mismatch levels, motion-planning solutions, and the shared level utilities.

Vulkan rendering

Rendering is done with Vulkan. Follow the ManiSkill installation instructions to set up the Vulkan driver and the SAPIEN renderer on your platform.

Downloading assets

IG-10K Assets (one-click download)

The recommended path is to download the shared asset package into ~/.maniskill/data. It includes the assets used by the benchmark.

bash

mkdir -p ~/.maniskill/data
hf download imitator-game/IG-10K-Assets --repo-type dataset --local-dir ~/.maniskill/data
# ModelScope alternative:
# modelscope download --dataset Zhouxunzhe/IG-10K-Assets --local_dir ~/.maniskill/data

for f in ~/.maniskill/data/*.tar.zst; do
  tar --use-compress-program=unzstd -xf "$f" -C ~/.maniskill/data
done
rm ~/.maniskill/data/*.tar.zst

Asset namespaces

The benchmark uses four asset namespaces. MS_ASSET_DIR can override the default ~/.maniskill root; the paths below show the default layout.

Asset set	Required location
ManiSkill YCB	`~/.maniskill/data/assets/mani_skill2_ycb/`
RoboTwin objects	`~/.maniskill/data/robotwin/objects/`
RoboTwin background textures	`~/.maniskill/data/robotwin/background_texture/`
PartNet-Mobility	`~/.maniskill/data/partnet_mobility/dataset/`
External GLB objects	`~/.maniskill/data/sketchfab/objects/{object_key}/model.glb`

Verify the install

Quick sanity check: create and step one of the smallest benchmark environments. You should see a success-rate tensor from evaluate():

python

import gymnasium as gym
import mani_skill.envs  # Registers all benchmark environments.

from mani_skill.envs.tasks.tabletop.utils.dual_task_camera_utils import (
    configure_dual_task_level,
)

configure_dual_task_level("L0")
env = gym.make(
    "TwoRobotStirSpoon-v1",
    obs_mode="state",
    control_mode="pd_joint_pos",
    reward_mode="dense",
    sim_backend="physx_cpu",
)

obs, info = env.reset(seed=0)
action = env.action_space.sample()
obs, reward, terminated, truncated, info = env.step(action)

evaluation = env.unwrapped.evaluate()
print(evaluation["success"])
env.close()

Next steps

Once installed, continue with the quickstart to run your first baseline, or jump straight to the domain randomization tutorial.