Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Sinong Wang
- sl:arxiv_num : 2104.14690
- sl:arxiv_published : 2021-04-29T22:52:26Z
- sl:arxiv_summary : Large pre-trained language models (LMs) have demonstrated remarkable ability
as few-shot learners. However, their success hinges largely on scaling model
parameters to a degree that makes it challenging to train and serve. In this
paper, we propose a new approach, named as EFL, that can turn small LMs into
better few-shot learners. The key idea of this approach is to reformulate
potential NLP task into an entailment one, and then fine-tune the model with as
little as 8 examples. We further demonstrate our proposed method can be: (i)
naturally combined with an unsupervised contrastive learning-based data
augmentation method; (ii) easily extended to multilingual few-shot learning. A
systematic evaluation on 18 standard NLP tasks demonstrates that this approach
improves the various existing SOTA few-shot learning methods by 12\%, and
yields competitive few-shot performance with 500 times larger models, such as
GPT-3.@en
- sl:arxiv_title : Entailment as Few-Shot Learner@en
- sl:arxiv_updated : 2021-04-29T22:52:26Z
- sl:bookmarkOf : https://arxiv.org/abs/2104.14690
- sl:creationDate : 2021-05-03
- sl:creationTime : 2021-05-03T23:05:39Z
Documents with similar tags (experimental)
Préférences...