About This Document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Timo Schick
- sl:arxiv_num : 2302.04761
- sl:arxiv_published : 2023-02-09T16:49:57Z
- sl:arxiv_summary : Language models (LMs) exhibit remarkable abilities to solve new tasks from
just a few examples or textual instructions, especially at scale. They also,
paradoxically, struggle with basic functionality, such as arithmetic or factual
lookup, where much simpler and smaller models excel. In this paper, we show
that LMs can teach themselves to use external tools via simple APIs and achieve
the best of both worlds. We introduce Toolformer, a model trained to decide
which APIs to call, when to call them, what arguments to pass, and how to best
incorporate the results into future token prediction. This is done in a
self-supervised way, requiring nothing more than a handful of demonstrations
for each API. We incorporate a range of tools, including a calculator, a Q\&A
system, two different search engines, a translation system, and a calendar.
Toolformer achieves substantially improved zero-shot performance across a
variety of downstream tasks, often competitive with much larger models, without
sacrificing its core language modeling abilities.@en
- sl:arxiv_title : Toolformer: Language Models Can Teach Themselves to Use Tools@en
- sl:arxiv_updated : 2023-02-09T16:49:57Z
- sl:bookmarkOf : https://arxiv.org/abs/2302.04761
- sl:creationDate : 2023-02-13
- sl:creationTime : 2023-02-13T15:18:25Z
- sl:relatedDoc : http://www.semanlink.net/doc/2023/02/jay_hack_sur_twitter_my_thou
Documents with similar tags (experimental)
Preferences...