Semanlink - UKPLab/sentence-transformers: Sentence Embeddings with BERT & XLNet

Tags:

Au sujet de ce document

sl:bookmarkOf : https://github.com/UKPLab/sentence-transformers
sl:creationDate : 2020-07-14
sl:creationTime : 2020-07-14T19:08:40Z
sl:mainDoc : http://www.semanlink.net/doc/2019/08/_1908_10084_sentence_bert_sen
sl:relatedDoc : http://www.semanlink.net/doc/2019/08/_1908_10084_sentence_bert_sen

Infos sur le fichier

Bookmark of: https://github.com/UKPLab/sentence-transformers

Linked From

[2104.06979] TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning

Tags:

> The most
successful previous approaches like InferSent (Conneau
et al., 2017), Universial Sentence Encoder
(USE) (Cer et al., 2018) and SBERT (Reimers and
Gurevych, 2019) heavily relied on labeled data to
train sentence embedding models.
>
> TSDAE can
achieve up to 93.1% of the performance of indomain
supervised approaches. Further, we
show that TSDAE is **a strong domain adaptation
and pre-training method for sentence
embeddings**, significantly outperforming other
approaches like Masked Language Model.

> During training, TSDAE
encodes corrupted sentences into fixed-sized
vectors and requires the decoder to reconstruct the
original sentences from this sentence embedding.

- <https://www.sbert.net/examples/unsupervised_learning/TSDAE/README.html>
- [github](https://github.com/UKPLab/sentence-transformers/tree/master/examples/unsupervised_learning/TSDAE)
- [UKPLab/sentence-transformers: Sentence Embeddings with BERT & XLNet](doc:2020/07/ukplab_sentence_transformers_s)
- [twitter](https://twitter.com/KexinWang2049/status/1433361957579538432):

> **TSDAE can learn domain-specific sentence embeddings with unlabeled sentences**
>
> Most importantly, instead of STS (Semantic Textual Similarity), **we suggest evaluating unsupervised sentence embeddings on the domain-specific tasks&datasets, which is the real use case for them**. Actually, STS scores do not correlate with performance on specific tasks.

2021-09-01 A propos

How to use BERT for finding similar sentences or similar news? · Issue #876 · huggingface/transformers

Tags:

2020-07-12 A propos

[1908.10084] Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Tags: