RESEARCH PAPERYear 2020

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Colin Raffel; Noam Shazeer; Adam Roberts; Katherine Lee; Sharan Narang; Michael Matena; Yanqi Zhou; Wei Li; Peter J. Liu

Read paper PDF

Classification

View four quadrants

Major category: Components of WAMs
Quadrant: Not applicable
Architecture: Not applicable
Prediction paradigm: Not applicable
Subcategories: Language & vision-language backbones Language & VLM backbones
Source review status: Verified from primary sources

Category review. T5 supplies a pretrained sequence encoder and conditional language decoder learned through span-corruption and transfer. The manuscript specifically uses the dense token-level encoder as a language-conditioning component. Reading evidence

AT A GLANCE

Contribution

A contribution summary has not been added yet.

Abstract

An abstract has not been added yet.

Affiliations

Not listed in the collection.

BibTeX

@article{t5,
  author       = {Colin Raffel and
                  Noam Shazeer and
                  Adam Roberts and
                  Katherine Lee and
                  Sharan Narang and
                  Michael Matena and
                  Yanqi Zhou and
                  Wei Li and
                  Peter J. Liu},
  title        = {Exploring the Limits of Transfer Learning with a Unified Text-to-Text
                  Transformer},
  journal      = {JMLR},
  volume       = {21},
  pages        = {140:1--140:67},
  year         = {2020},
  
}

Classification

Contribution

Abstract

Affiliations

BibTeX

More in this category

V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning

DINOv3

World Simulation with Video Foundation Models for Physical AI