We present CALVIN (Composing Actions from Language and Vision), an open-source simulated benchmark to learn long-horizon language-conditioned tasks. Our aim is to make it possible to develop agents that can solve many robotic manipulation tasks over a long horizon, from onboard sensors, and specified only via human language. CALVIN tasks are more complex in terms of sequence length, action space, and language than existing vision-and-language task datasets and supports flexible specification of sensor suites.
Leaderboard
Train D → Test D
Train A, B, C, D → Test D
Train A, B, C → Test D
Videos
Code
The Calvin environments, baselines and benchmarks can be found in our GitHub repository for academic usage and is released under the MIT license.
Publications
CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks Oier Mees, Lukas Hermann, Erick Rosete, Wolfram Burgard IEEE Robotics and Automation Letters (RAL), 2022 Pdf BibTeX @article{mees2022calvin, author = {Oier Mees and Lukas Hermann and Erick Rosete-Beas and Wolfram Burgard}, title = {CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks}, journal={IEEE Robotics and Automation Letters (RA-L)}, volume={7}, number={3}, pages={7327-7334}, year={2022} }
CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks Oier Mees, Lukas Hermann, Erick Rosete, Wolfram Burgard IEEE Robotics and Automation Letters (RAL), 2022
@article{mees2022calvin, author = {Oier Mees and Lukas Hermann and Erick Rosete-Beas and Wolfram Burgard}, title = {CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks}, journal={IEEE Robotics and Automation Letters (RA-L)}, volume={7}, number={3}, pages={7327-7334}, year={2022} }