Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Moussa Kamal Eddine
- sl:arxiv_num : 2010.12321
- sl:arxiv_published : 2020-10-23T11:57:33Z
- sl:arxiv_summary : Inductive transfer learning has taken the entire NLP field by storm, with
models such as BERT and BART setting new state of the art on countless NLU
tasks. However, most of the available models and research have been conducted
for English. In this work, we introduce BARThez, the first large-scale
pretrained seq2seq model for French. Being based on BART, BARThez is
particularly well-suited for generative tasks. We evaluate BARThez on five
discriminative tasks from the FLUE benchmark and two generative tasks from a
novel summarization dataset, OrangeSum, that we created for this research. We
show BARThez to be very competitive with state-of-the-art BERT-based French
language models such as CamemBERT and FlauBERT. We also continue the
pretraining of a multilingual BART on BARThez' corpus, and show our resulting
model, mBARThez, to significantly boost BARThez' generative performance. Code,
data and models are publicly available.@en
- sl:arxiv_title : BARThez: a Skilled Pretrained French Sequence-to-Sequence Model@en
- sl:arxiv_updated : 2021-02-09T09:31:57Z
- sl:bookmarkOf : https://arxiv.org/abs/2010.12321
- sl:creationDate : 2021-03-31
- sl:creationTime : 2021-03-31T19:08:05Z
- sl:relatedDoc :
Préférences...