About This Document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Xavier Garcia
- sl:arxiv_num : 2302.01398
- sl:arxiv_published : 2023-02-02T20:19:46Z
- sl:arxiv_summary : We demonstrate the potential of few-shot translation systems, trained with
unpaired language data, for both high and low-resource language pairs. We show
that with only 5 examples of high-quality translation data shown at inference,
a transformer decoder-only model trained solely with self-supervised learning,
is able to match specialized supervised state-of-the-art models as well as more
general commercial translation systems. In particular, we outperform the best
performing system on the WMT'21 English - Chinese news translation task by only
using five examples of English - Chinese parallel data at inference. Moreover,
our approach in building these models does not necessitate joint multilingual
training or back-translation, is conceptually simple and shows the potential to
extend to the multilingual setting. Furthermore, the resulting models are two
orders of magnitude smaller than state-of-the-art language models. We then
analyze the factors which impact the performance of few-shot translation
systems, and highlight that the quality of the few-shot demonstrations heavily
determines the quality of the translations generated by our models. Finally, we
show that the few-shot paradigm also provides a way to control certain
attributes of the translation -- we show that we are able to control for
regional varieties and formality using only a five examples at inference,
paving the way towards controllable machine translation systems.@en
- sl:arxiv_title : The unreasonable effectiveness of few-shot learning for machine translation@en
- sl:arxiv_updated : 2023-02-02T20:19:46Z
- sl:bookmarkOf : https://arxiv.org/abs/2302.01398
- sl:creationDate : 2023-02-07
- sl:creationTime : 2023-02-07T18:49:52Z
Documents with similar tags (experimental)
Preferences...