Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Nils Reimers
- sl:arxiv_num : 1908.10084
- sl:arxiv_published : 2019-08-27T08:50:17Z
- sl:arxiv_summary : BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) has set a new
state-of-the-art performance on sentence-pair regression tasks like semantic
textual similarity (STS). However, it requires that both sentences are fed into
the network, which causes a massive computational overhead: Finding the most
similar pair in a collection of 10,000 sentences requires about 50 million
inference computations (~65 hours) with BERT. The construction of BERT makes it
unsuitable for semantic similarity search as well as for unsupervised tasks
like clustering.
In this publication, we present Sentence-BERT (SBERT), a modification of the
pretrained BERT network that use siamese and triplet network structures to
derive semantically meaningful sentence embeddings that can be compared using
cosine-similarity. This reduces the effort for finding the most similar pair
from 65 hours with BERT / RoBERTa to about 5 seconds with SBERT, while
maintaining the accuracy from BERT.
We evaluate SBERT and SRoBERTa on common STS tasks and transfer learning
tasks, where it outperforms other state-of-the-art sentence embeddings methods.@en
- sl:arxiv_title : Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks@en
- sl:arxiv_updated : 2019-08-27T08:50:17Z
- sl:bookmarkOf : https://arxiv.org/abs/1908.10084
- sl:creationDate : 2019-08-28
- sl:creationTime : 2019-08-28T22:41:55Z
- sl:relatedDoc :
Documents with similar tags (experimental)
Préférences...