Semanlink - NLP: Text Representation

Γονείς:

NLP techniques

NLP: Text Representation

Σχετικές ετικέτες:

Text: similarity, embeddings...

Απόγονοι

Bag-of-words
- Neural Bag of Words
  - DANs (Deep Averaging Neural Networks)
Document embeddings
- Doc2Vec
- DSSM (Deep Semantic Similarity Model)
- Representation of text + metadata
Graph-based Text Representations
Long documents
Text Embeddings
- E5
- Geometry of language embeddings
- MTEB
- OpenAI Embeddings
- Representation of text + metadata
- Sentence Embeddings
  - A La Carte Embedding
  - Doc2Vec
  - DSSM (Deep Semantic Similarity Model)
  - E5
  - Sent2Vec
  - Sentence embeddings: Domain adaptation
    - Sentence-BERT: Domain adaptation
      - GPL (Generative Pseudo Labeling)
        
        Domain Adaptation with Generative Pseudo-Labeling (GPL) | Pinecone
        
        Tags:
        
        GPL (Generative Pseudo Labeling)
        Pinecone
        
        2023-04-09 About
        
        [2205.11498] Domain Adaptation for Memory-Efficient Dense Retrieval
        
        Tags:
        
        Arxiv Doc
        Dense retriever
        Domain adaptation
        GPL (Generative Pseudo Labeling)
        Nils Reimers
        
        Refers to [Binary Passage Retriever (BPR)](doc:2021/06/2106_00882_efficient_passage_)
        
        2022-09-26 About
        
        Domain transfer with GGPL: German Generative Pseudo Labeling 🥨 | by Matthias Richter | Jun, 2022 | ML6team
        
        Tags:
        
        GPL (Generative Pseudo Labeling)
        Multilingual NLP
        
        2022-06-02 About
        
        Nils Reimers sur Twitter : "GPL goes multi-lingual..."
        
        Tags:
        
        GPL (Generative Pseudo Labeling)
        Nils Reimers
        Tweet
        
        [Domain transfer with GGPL: German Generative Pseudo Labeling](doc:2022/06/domain_transfer_with_ggpl_germ)
        
        2022-06-01 About
        
        Ramsri Goutham Golla sur Twitter : "Hi @Nils_Reimers For GPL you used "msmarco-distilbert-base-tas-b" model and ..."
        
        Tags:
        
        GPL (Generative Pseudo Labeling)
        Nils Reimers
        Tweet
        
        2022-04-27 About
        
        Domain Adaptation — Sentence-Transformers documentation
        
        Tags:
        
        Documentation
        Domain adaptation (NLP)
        GPL (Generative Pseudo Labeling)
        Sentence-BERT: Domain adaptation
        TSDAE
        
        2022-03-31 About
        
        NAVER LABS Europe : "@Nils_Reimers of @huggingface on 'Unsupervised domain adaptation for neural search'"
        
        Tags:
        
        GPL (Generative Pseudo Labeling)
        J'y étais
        Neural Search
        Nils Reimers
        Unsupervised Domain Adaptation (NLP)
        
        2022-03-09 About
        
        [2112.07577] GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval
        
        Tags:
        
        Arxiv Doc
        Dense Passage Retrieval
        GPL (Generative Pseudo Labeling)
        Nils Reimers
        Synthetic QA data
        Unsupervised Domain Adaptation (NLP)
        
        An unsupervised domain adaptation technique for dense retrieval models 1. synthetic queries are generated for each passage from the target corpus (using an existing pre-trained [T5](tag:text_to_text_transfer_transformer) encoder-decoder) 2. the generated queries are used for mining negative passages (retrieving the most similar paragraphs using an existing dense retrieval model == hard negatives!) 3. the query-passage pairs are labeled by a cross-encoder and used to train the domain-adapted dense retriever (using method described in [Hofstätter et al., 2020](doc:2021/12/2010_02666_improving_efficien)) [Nils Reimers sur Twitter](doc:2021/12/nils_reimers_sur_twitter_do_), [GitHub](https://github.com/UKPLab/gpl), by the author of [TSDAE](doc:2021/09/2104_06979_tsdae_using_trans) Claims to improve "Doc2Query" [Document Expansion by Query Prediction](doc:2022/01/1904_08375_document_expansion): ([src](https://twitter.com/KexinWang2049/status/1471435779415150598)) > - GPL: Uses doc2query to construct synthetic data and does knowledge distillation (i.e. training) on that data. > - Doc2query: Generates queries to extend the documents and use BM25 on top of them w/o training.
        
        2021-12-15 About
      - sBERT fine-tuning
        SetFit (SBERT fine-tuning)
        
        UKP Lab sur X : "a lightweight solution for few-shot domain-specific sentence classification: AdaSent!..."
        
        Tags:
        
        Adapter modules (LM finetuning)
        EMNLP 2023
        Few-shot text classification
        Sentence-BERT: Domain adaptation
        Sentence embeddings: Domain adaptation
        SetFit (SBERT fine-tuning)
        Tweet
        
        AdaSent is an approach to creating domain-specialized sentence encoders for few-shot sentence classification > Reusable general sentence adapter across domains > AdaSent decouples DAPT (Domain Adaptative Pre-Training) & SEPT (Sentence Embedding Pre-Training) **by storing the sentence encoding abilities into an adapter**, which is trained only once in the general domain and plugged into various DAPT-ed PLMs [Github](https://github.com/UKPLab/AdaSent)
        
        2023-12-09 About
        
        Daniel Vila Suero sur Twitter : "Data quality is key for LLMs, but we're building Open Source LLMs with data of "unknown" quality... Introducing Alpaca GarbageCollector..."
        
        Tags:
        
        Alpaca
        Dataset quality
        SetFit (SBERT fine-tuning)
        Tweet
        
        > a cross-lingual SetFit model to identify potential bad instructions in Alpaca-like datasets
        
        2023-04-05 About
        
        ValueError "invalid literal for int() with base 10" in trainer.evaluate (dataset created from pandas) · Issue #228 · huggingface/setfit
        
        Tags:
        
        fps' post
        SetFit (SBERT fine-tuning)
        
        see <https://github.com/huggingface/setfit/blob/main/notebooks/zero-shot-classification.ipynb> > Note: some datasets on the Hugging Face Hub don't have a ClassLabel feature for the label column. In these cases, you should compute the candidate labels manually by first computing the id2label mapping as follows:
        
        2022-12-13 About
        
        Lewis Tunstall sur Twitter : "The SetFit library for few-shot learning with Sentence Transformers now supports *multi-label text classification*..."
        
        Tags:
        
        Multi-label Text classification
        SetFit (SBERT fine-tuning)
        Tweet
        
        Multilabel support [github issue](https://github.com/huggingface/setfit/issues/65)
        
        2022-10-14 About
        
        huggingface/setfit: Efficient few-shot learning with Sentence Transformers
        
        Tags:
        
        GitHub project
        SetFit (SBERT fine-tuning)
        
        2022-10-12 About
        
        [2209.11055] Efficient Few-Shot Learning Without Prompts
        
        Tags:
        
        Arxiv Doc
        Few-shot learning
        Moshe Wasserblat
        Nils Reimers
        SetFit (SBERT fine-tuning)
        
        [tweet](https://twitter.com/_akhaliq/status/1573109469646561280?s=20&t=RTpK9dh90az0zT1Xg2ohpQ): > So if I have 4 classes and say 2 labels per class, I would first fine tune an ST on these 4 pairs and then vectorize the 8 total examples for fine-tuning the classifier
        
        2022-09-23 About
        
        Sentence Transformer Fine-Tuning (SetFit): Outperforming GPT-3 on few-shot Text-Classification while being 1600 times smaller | by Moshe Wasserblat (2021-12)
        
        Tags:
        
        Few-shot text classification
        Moshe Wasserblat
        sBERT fine-tuning
        SetFit (SBERT fine-tuning)
        
        Finetuning d'un SBERT sur une tâche de classification (in fine, produit un SBERT) > **Few-shot text classification based on fine-tuning a Sentence Transformer with task-specific data** that can easily be implemented with the sentence-transformers library > Surprisingly, we did not find any work that performed an end-to-end ST fine-tuning for text classification in a Siamese manner. [COLAB](https://colab.research.google.com/github/MosheWasserb/SetFit/blob/main/SetFit_SST_2.ipynb) [Nils Reimers sur Twitter](doc:2022/03/nils_reimers_sur_twitter_gre)
        
        2022-03-31 About
        
        Nils Reimers sur Twitter : "Great post on SetFit"
        
        Tags:
        
        Nils Reimers
        SetFit (SBERT fine-tuning)
        Tweet
        
        About [Sentence Transformer Fine-Tuning (SetFit): Outperforming GPT-3 on few-shot Text-Classification while being 1600 times smaller | by Moshe Wasserblat](doc:2022/03/sentence_transformer_fine_tunin) > - Outperforms GPT-3 in few-shot text-classification (50 labeled examples, secret test set) > - 1600 times smaller > - Can be run on your CPU > - No limitation on the number of training examples > - Just few lines of code needed
        
        2022-03-31 About
        
        LlamaIndex 🦙 sur X : "Fine-tuning Embedding Models for RAG with LoRA'
        
        Tags:
        
        LlamaIndex
        LoRA
        sBERT fine-tuning
        
        2024-04-23 About
        
        [2008.11228] A simple method for domain adaptation of sentence embeddings
        
        Tags:
        
        Arxiv Doc
        Domain adaptation (NLP)
        sBERT fine-tuning
        Sentence-BERT: Domain adaptation
        Universal Sentence Encoder
        
        2022-04-01 About
        
        [2004.05119] Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer
        
        Tags:
        
        ACL 2020
        Arxiv Doc
        Low-Resource NLP
        NLP@Amazon
        NLP@Microsoft
        sBERT fine-tuning
        
        > Fine-tuning (FT) pre-trained sentence embedding models on small datasets has been shown to have limitations. In this paper we show that concatenating the embeddings from the pre-trained model with those from a simple sentence embedding model trained only on the target data, can improve over the performance of FT for few-sample tasks
        
        2022-03-31 About
        
        Sentence Transformer Fine-Tuning (SetFit): Outperforming GPT-3 on few-shot Text-Classification while being 1600 times smaller | by Moshe Wasserblat (2021-12)
        
        Tags:
        
        Few-shot text classification
        Moshe Wasserblat
        sBERT fine-tuning
        SetFit (SBERT fine-tuning)
        
        Finetuning d'un SBERT sur une tâche de classification (in fine, produit un SBERT) > **Few-shot text classification based on fine-tuning a Sentence Transformer with task-specific data** that can easily be implemented with the sentence-transformers library > Surprisingly, we did not find any work that performed an end-to-end ST fine-tuning for text classification in a Siamese manner. [COLAB](https://colab.research.google.com/github/MosheWasserb/SetFit/blob/main/SetFit_SST_2.ipynb) [Nils Reimers sur Twitter](doc:2022/03/nils_reimers_sur_twitter_gre)
        
        2022-03-31 About
        
        Sentence Embedding Fine-tuning for the French Language | by La Javaness R&D | Feb, 2022 | Medium
        
        Tags:
        
        CamemBERT
        NLP: French
        NLP sample code
        sBERT fine-tuning
        
        2022-03-31 About
        
        [2203.14655] Few-Shot Learning with Siamese Networks and Label Tuning
        
        Tags:
        
        Arxiv Doc
        Few-shot text classification
        sBERT fine-tuning
        Sentence-BERT
        Siamese networks
        
        > the problem of building text classifiers with little or no training data. > > In recent years, an approach based on neural textual entailment models has been found to give strong results on a diverse range of tasks. (cf. #[NLI](tag:nli), using the input text as the premise and the text representing the label as the hypothesis) > In this work, we show that **with proper pre-training, Siamese Networks that embed texts and labels** offer a competitive alternative. > > We introduce **label tuning: fine-tuning the label embeddings only**. While giving lower performance than model fine-tuning (which updates all params of the model), this approach has the architectural advantage that a single encoder can be shared by many different tasks (we only fine-tune the label embeddings) > The drop in quality can be compensated by using a variant of **[Knowledge distillation](tag:knowledge_distillation)** [Github](https://tinyurl.com/label-tuning), [Tweet](doc:2022/03/thomas_muller_sur_twitter_pa)
        
        2022-03-30 About
  - Sentence Transformers
    - BERT + Sentence Embeddings
      - Sentence-BERT
        sBERT fine-tuning
        Sentence-BERT: Domain adaptation
        SentenceBERT and Words
        
        Few-Shot Text Classification (Cloudera 2020)
        
        Tags:
        
        Few-shot text classification
        SentenceBERT and Words
        
        > Sentence-BERT has been optimized… well, for sentences! It’s reasonable to suspect that SBERT’s representations of single words or short phrases like “Business” or “Science & Technology” won’t be as semantically relevant as representations derived from a word-level method, like word2vec or GloVe
        
        2022-11-24 About
        
        skeskinen/bert.cpp: ggml implementation of BERT
        
        Tags:
        
        .cpp
        BERT
        Frugal LLMs
        Quantization
        Sentence-BERT
        Sentence Embeddings
        
        > ggml inference of BERT neural net architecture with pooling and normalization from SentenceTransformers (sbert.net). High quality sentence embeddings in pure C++ (with C API). > > The main goal of bert.cpp is to run the BERT model using **4-bit integer quantization on CPU**
        
        2023-05-09 About
        
        Multilingual Sentence Transformers | Pinecone
        
        Tags:
        
        Multilingual embeddings
        Pinecone
        Sentence-BERT
        
        Focus on **Multilingual Knowledge Distillation** > recent method introduced by Nils Reimers and Iryna Gurevych in 2020 > The teacher model is an already fine-tuned sentence transformer used for creating embeddings in a single language (most likely English). The student model is a transformer that has been pretrained on a multilingual corpus.
        
        2023-01-13 About
        
        Unsupervised Learning — Sentence-Transformers documentation
        
        Tags:
        
        Sentence-BERT
        TSDAE
        Unsupervised Sentence Embedding Learning
        
        > In our paper TSDAE we compare approaches for sentence embedding tasks, and in GPL we compare them for semantic search tasks (given a query, find relevant passages). While the unsupervised approach achieve acceptable performances for sentence embedding tasks, they perform poorly for semantic search tasks.
        
        2022-08-20 About
        
        Train and Fine-Tune Sentence Transformers Models
        
        Tags:
        
        Sentence-BERT
        
        2022-08-13 About
        
        sentence bert model in onnx format · Issue #46 · UKPLab/sentence-transformers
        
        Tags:
        
        ONNX
        Raphaël Sourty
        Sentence-BERT
        
        2022-06-13 About
        
        [2205.15952] Knowledge Graph -- Deep Learning: A Case Study in Question Answering in Aviation Safety Domain
        
        Tags:
        
        Arxiv Doc
        Knowledge Graph + Deep Learning
        Question Answering
        Sentence-BERT
        
        2022-06-11 About
        
        [2205.04260] EASE: Entity-Aware Contrastive Learning of Sentence Embedding
        
        Tags:
        
        Arxiv Doc
        Contrastive Learning
        Entities
        Entities and LM
        Entity type
        Ikuya Yamada
        Sentence-BERT
        Sentence Embeddings
        
        > we explore a type of supervision that has been under-explored in the literature: entity hyperlink annotations from Wikipedia. > > entities have been shown to be a strong indicator of text semantics > > a method for mining hard negatives based on the entity type Uses wikipedia2vec > the reliance on Wikipedia for training data may limit the application of the models to specific domains (e.g., general or encyclopedia domains). To apply EASE to other domains, one may need to annotate text from the domain either manually or automatically.
        
        2022-05-11 About
        
        [2203.14655] Few-Shot Learning with Siamese Networks and Label Tuning
        
        Tags:
        
        Arxiv Doc
        Few-shot text classification
        sBERT fine-tuning
        Sentence-BERT
        Siamese networks
        
        > the problem of building text classifiers with little or no training data. > > In recent years, an approach based on neural textual entailment models has been found to give strong results on a diverse range of tasks. (cf. #[NLI](tag:nli), using the input text as the premise and the text representing the label as the hypothesis) > In this work, we show that **with proper pre-training, Siamese Networks that embed texts and labels** offer a competitive alternative. > > We introduce **label tuning: fine-tuning the label embeddings only**. While giving lower performance than model fine-tuning (which updates all params of the model), this approach has the architectural advantage that a single encoder can be shared by many different tasks (we only fine-tune the label embeddings) > The drop in quality can be compensated by using a variant of **[Knowledge distillation](tag:knowledge_distillation)** [Github](https://tinyurl.com/label-tuning), [Tweet](doc:2022/03/thomas_muller_sur_twitter_pa)
        
        2022-03-30 About
        
        Document Matching for Job Descriptions | Semantic Scholar (2021)
        
        Tags:
        
        NLP@Stanford
        NLP + Human Resources
        Sentence-BERT
        
        > We train a document encoder to match online job descriptions to one of many standardized job roles from Singapore’s Skills Framework. The encoder generates semantically meaningful document encodings from textual descriptions of job roles, which are then compared using Cosine Similarity to determine matching. During training, we implement the methodology used by Sentence-BERT, fine tuning pre-trained BERT models using a siamese network architecture on labelled document pairs.
        
        2022-03-09 About
        
        Nils Reimers sur Twitter : "Creating intent classes for chatbots is challenging This tutorial shows how to use sentence-transformers to find potentially overlapping intent classes and how to improve your data annotation work." / Twitter
        
        Tags:
        
        Annotations (ML)
        Chatbots
        Nils Reimers
        Sentence-BERT
        Spectral clustering
        Tutorial
        Tweet
        
        2022-02-19 About
        
        Nils Reimers sur Twitter : "how to use the fast clustering algorithm from sentence-transformers..."
        
        Tags:
        
        Nils Reimers
        Sentence-BERT
        Short Text Clustering
        Tweet
        
        Clustering millions of sentences to optimize the ML-workflow
        
        2022-02-19 About
        
        sentence-transformers/fast_clustering.py at master · UKPLab/sentence-transformers
        
        Tags:
        
        Clustering
        Nils Reimers
        Sentence-BERT
        
        > This is a more complex example on performing clustering on large scale dataset. This examples find in a large set of sentences local communities, i.e., groups of sentences that are highly similar. You can freely configure the threshold what is considered as similar. A high threshold will only find extremely similar sentences, a lower threshold will find more sentence that are less similar. A second parameter is 'min_community_size': Only communities with at least a certain number of sentences will be returned. The method for finding the communities is extremely fast, for clustering 50k sentences it requires only 5 seconds (plus embedding comuptation). In this example, we download a large set of questions from Quora and then find similar questions in this set.
        
        2022-02-18 About
        
        gsarti/scibert-nli · Hugging Face
        
        Tags:
        
        Hugging Face
        NLP for scientific documents
        Sentence-BERT
        
        SciBERT fine-tuned on the SNLI and the MultiNLI datasets using the sentence-transformers library to produce universal sentence embeddings
        
        2022-01-29 About
        
        Semantic Search — Sentence-Transformers documentation
        
        Tags:
        
        Documentation
        Semantic Search
        Sentence-BERT
        
        **symmetric** semantic search vs **asymmetric** semantic search > - Suitable models for symmetric semantic search: Pre-Trained Sentence Embedding > - Suitable models for asymmetric semantic search: Pre-Trained MS MARCO Models
        
        2022-01-29 About
        
        Using pretrained SBERT model in cross-encoder · Issue #726 · UKPLab/sentence-transformers
        
        Tags:
        
        Cross-Encoders
        Sentence-BERT
        
        > so would it be a good idea to finetune a SBERT model on a cross-encoder task? > > The SBERT models are regular transformers model and hence can be used as base for cross encoders. Sometimes it could be helpful, otherwise it is better to use the original models. ([Nils Reimers](tag:nils_reimers))
        
        2021-12-17 About
        
        Advance BERT model via transferring knowledge from Cross-Encoders to Bi-Encoders | by Chien Vu | Towards Data Science
        
        Tags:
        
        BERT
        Cross-Encoders
        Data Augmentation
        Sentence-BERT
        Sentence Similarity
        
        Data Augmentation Method to improve SBERT Bi-Encoders for Pairwise Sentence Scoring Tasks (Semantic sentence tasks)
        
        2021-12-17 About
        
        Semantic search through a vectorized Wikipedia (SentenceBERT) with the Weaviate vector search engine
        
        Tags:
        
        Semantic Search
        Sentence-BERT
        Similarity queries / Vector search
        Weaviate
        Wikipedia
        
        2021-12-05 About
        
        Unsupervised_Extractive_Summarization - a Hugging Face Space by Hellisotherpeople
        
        Tags:
        
        Extractive Text Summarization
        GitHub project
        Sentence-BERT
        
        Unsupervised Extractive Text Summarization and Semantic Search [Github](https://github.com/Hellisotherpeople/CX_DB8)
        
        2021-12-03 About
        
        How to Fine-Tune Sentence-BERT for Question Answering | Capital One
        
        Tags:
        
        Question Answering
        Sentence-BERT
        Tutorial
        
        > tutorial on using the sentence-transformers library to fine-tune Sentence-BERT for question matching
        
        2021-11-21 About
        
        Multilingual Sentence Transformers | Pinecone
        
        Tags:
        
        Knowledge distillation
        Multilingual Language Models
        Pinecone
        Sentence-BERT
        Transformers
        
        How to make a text encoder multilingual using sentence transformers and multilingual knowledge distillation.
        
        2021-11-04 About
        
        Next-Gen Sentence Embeddings with Multiple Negatives Ranking Loss | Pinecone
        
        Tags:
        
        Multiple negatives ranking (MNR) loss
        Pinecone
        Sentence-BERT
        Sentence Embeddings
        
        > the world of sentence embeddings was ignited with the introduction of SBERT in 2019. Since then, many more sentence transformers have been introduced. These models quickly made the original SBERT obsolete. How did these newer sentence transformers manage to outperform SBERT so quickly? The answer is **multiple negatives ranking (MNR) loss**. > In short; **fine-tune your models with MNR loss, and do it with the [sentence-transformers](tag:sbert) library**. (mentionned in a [tweet](https://twitter.com/Nils_Reimers/status/1453001422400856086) by [Nils Reimers](tag:nils_reimers))
        
        2021-10-27 About
        
        Sentence Embeddings and Transformers | Pinecone
        
        Tags:
        
        Pinecone
        Sentence-BERT
        
        2021-10-23 About
        
        Sahajtomar/french_semantic · Hugging Face
        
        Tags:
        
        CamemBERT
        Discuté avec Raphaël
        huggingface / transformers
        NLP: French
        Sentence-BERT
        
        2021-10-14 About
        
        Semantic Search with S-BERT is all you need
        
        Tags:
        
        Howto
        Semantic Search
        Sentence-BERT
        
        > SentenceTransformers is designed in such way that fine-tuning your own sentence / text embeddings models is easy.
        
        2021-06-05 About
        
        Nils Reimers sur Twitter : "SBERT Release v1.1.0"
        
        Tags:
        
        Nils Reimers
        Sentence-BERT
        Tweet
        
        2021-04-22 About
        
        Nils Reimers sur Twitter : "New models for Neural Information Retrieval..."
        
        Tags:
        
        Information retrieval
        Nils Reimers
        Sentence-BERT
        Tweet
        
        2021-04-17 About
        
        SentenceTransformers Documentation
        
        Tags:
        
        Documentation
        Sentence-BERT
        
        2021-03-25 About
        
        Zero-Shot Learning in Modern NLP | Joe Davison Blog (2020-05)
        
        Tags:
        
        Hugging Face
        Natural Language Inference
        Sentence-BERT
        Text Classification
        Zero-Shot Learning
        Zero-shot Text Classifier
        
        > state-of-the-art NLP models for sequence classification without large annotated training sets. Simple idea: use a single model (eg. [Sentence-BERT](tag:sbert)) to embed both the text data and the class names into the same space. Pb: Sentence-BERT is designed to learn effective sentence-level, not single- or multi-word representations like our class names -> the label embeddings may not be as semantically salient as word-level embedding methods (i.e. word2vec). Solution 1: Learn a projection from sentence level embeddings of words to word2vec embeddings, use it for encoding when learning classifier. Can be adapted to few short learning Solution 2: "Classification as [#Natural Language Inference](tag:nli)". > A method which not only embeds sequences and labels into the same latent space where their distance can be measured, but that can actually tell us something about the compatibility of two distinct sequences out of the box.
        
        2021-02-23 About
        
        UKPLab/sentence-transformers: Sentence Embeddings with BERT & XLNet
        
        Tags:
        
        GitHub project
        huggingface / transformers
        Sentence-BERT
        XLNet
        
        [paper](doc:2019/08/_1908_10084_sentence_bert_sen)
        
        2020-07-14 About
        
        Richer Sentence Embeddings using Sentence-BERT — Part I
        
        Tags:
        
        Sentence-BERT
        
        Simplistic (and often used) methods for sentence embeddings with BERT are too simplistic to be good (avearaging the word vectors, or using the \[CLS\] special vector (start of sequence). [About this paper](/doc/2019/08/_1908_10084_sentence_bert_sen)
        
        2020-01-06 About
        
        [1908.10084] Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
        
        Tags:
        
        Arxiv Doc
        EMNLP 2019
        huggingface / transformers
        Nearest neighbor search
        Sentence-BERT
        Sentence Similarity
        Siamese networks
        
        > Sentence-BERT (SBERT), a modification of the pretrained BERT network that use siamese and triplet network structures to derive **semantically meaningful sentence embeddings** that can be compared using cosine-similarity. Important because - BERT ist unsuitable for semantic similarity search as well as for unsupervised tasks like clustering. - simple methods such as using the CLS token give low quality sentence embeddings However, the purpose of SBERT sentence embeddings are **not to be used for transfer learning for other tasks**. [Related blog post](/doc/2020/01/richer_sentence_embeddings_usin); [Github](https://github.com/UKPLab/sentence-transformers)
        
        2019-08-28 About
  - SIF embeddings
  - Universal Sentence Encoder
  - Unsupervised Sentence Embedding Learning
    - GPL (Generative Pseudo Labeling)
    - TSDAE

1 Document (Όλα τα έγγραφα)

CONCEPTUAL GROUNDING FOR TEXT REPRESENTATION LEARNING

Tags:

2019-12-03 About

Ιδιότητες

sl:creationDate : 2017-06-28
sl:creationTime : 2017-06-28T01:03:40Z
rdf:type : sl:Tag
skos:altLabel : Text Representation@en
skos:prefLabel : NLP: Text Representation