RESEARCH PAPERYear 2023

PaLM-E: An Embodied Multimodal Language Model

Danny Driess; Fei Xia; Mehdi S. M. Sajjadi; Corey Lynch; Aakanksha Chowdhery; Brian Ichter; Ayzaan Wahid; Jonathan Tompson; Quan Vuong; Tianhe Yu; Wenlong Huang; Yevgen Chebotar; Pierre Sermanet; Daniel Duckworth; Sergey Levine; Vincent Vanhoucke; Karol Hausman; Marc Toussaint; Klaus Greff; Andy Zeng; Igor Mordatch; Pete Florence

Read paper PDF Project page

Classification

View four quadrants

Major category: Components of WAMs
Quadrant: Not applicable
Architecture: Not applicable
Prediction paradigm: Not applicable
Subcategories: Language & vision-language backbones Language & VLM backbones
Source review status: Verified from primary sources

Category review. PaLM-E supplies a reusable embodied multimodal language backbone by interleaving encoded sensor observations with text tokens. It produces text/high-level skill instructions executed by separate controllers; its component role is language/vision-language conditioning, not a complete world-action backbone. Reading evidence

AT A GLANCE

Contribution

A contribution summary has not been added yet.

Abstract

An abstract has not been added yet.

Affiliations

Not listed in the collection.

BibTeX

@inproceedings{palme,
  author       = {Danny Driess and
                  Fei Xia and
                  Mehdi S. M. Sajjadi and
                  Corey Lynch and
                  Aakanksha Chowdhery and
                  Brian Ichter and
                  Ayzaan Wahid and
                  Jonathan Tompson and
                  Quan Vuong and
                  Tianhe Yu and
                  Wenlong Huang and
                  Yevgen Chebotar and
                  Pierre Sermanet and
                  Daniel Duckworth and
                  Sergey Levine and
                  Vincent Vanhoucke and
                  Karol Hausman and
                  Marc Toussaint and
                  Klaus Greff and
                  Andy Zeng and
                  Igor Mordatch and
                  Pete Florence},
  title        = {PaLM-E: An Embodied Multimodal Language Model},
  booktitle    = {ICML},
  pages        = {8469--8488},
  year         = {2023},
}

Classification

Contribution

Abstract

Affiliations

BibTeX

More in this category

V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning

DINOv3

World Simulation with Video Foundation Models for Physical AI