Semanlink - Unsupervised Sentence Embedding Learning

Γονείς:

Unsupervised Sentence Embedding Learning

Σχετικές ετικέτες:

Sentence-BERT: Domain adaptation

Απόγονοι

4 Έγγραφα (Όλα τα έγγραφα)

Jerry Liu sur X : "One major way to improve your RAG system is to fine-tune your embedding model"

Tags:

2023-08-25 About

Unsupervised Learning — Sentence-Transformers documentation

Tags:

2022-08-20 About

Unsupervised Training for Sentence Transformers | Pinecone

Tags:

2021-11-24 About

[2104.06979] TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning

Tags:

> The most
successful previous approaches like InferSent (Conneau
et al., 2017), Universial Sentence Encoder
(USE) (Cer et al., 2018) and SBERT (Reimers and
Gurevych, 2019) heavily relied on labeled data to
train sentence embedding models.
>
> TSDAE can
achieve up to 93.1% of the performance of indomain
supervised approaches. Further, we
show that TSDAE is **a strong domain adaptation
and pre-training method for sentence
embeddings**, significantly outperforming other
approaches like Masked Language Model.

> During training, TSDAE
encodes corrupted sentences into fixed-sized
vectors and requires the decoder to reconstruct the
original sentences from this sentence embedding.

- <https://www.sbert.net/examples/unsupervised_learning/TSDAE/README.html>
- [github](https://github.com/UKPLab/sentence-transformers/tree/master/examples/unsupervised_learning/TSDAE)
- [UKPLab/sentence-transformers: Sentence Embeddings with BERT & XLNet](doc:2020/07/ukplab_sentence_transformers_s)
- [twitter](https://twitter.com/KexinWang2049/status/1433361957579538432):

> **TSDAE can learn domain-specific sentence embeddings with unlabeled sentences**
>
> Most importantly, instead of STS (Semantic Textual Similarity), **we suggest evaluating unsupervised sentence embeddings on the domain-specific tasks&datasets, which is the real use case for them**. Actually, STS scores do not correlate with performance on specific tasks.

2021-09-01 About

Ιδιότητες

sl:creationDate : 2021-09-02
sl:creationTime : 2021-09-02T00:58:56Z
rdf:type : sl:Tag
skos:prefLabel : Unsupervised Sentence Embedding Learning