Semanlink - Information retrieval AND Internet

> **approach for training dense retrieval models that does not require any labeled training data**. Dense retrieval is a central challenge for open-domain tasks, such as Open QA, where state-of-the-art methods typically require large supervised datasets with custom hard-negative mining and denoising of positive examples.
>
> ART, in contrast, only requires access to unpaired inputs and outputs (e.g. questions and potential answer documents).
>
> It uses a new document-retrieval autoencoding scheme, where
> 1. an input question is used to retrieve a set of evidence documents, and
> 2. the documents are then used to compute the probability of reconstructing the original question.
>
> Training for retrieval based on question reconstruction enables effective unsupervised learning of both document and question encoders, which can be later incorporated into complete Open QA systems without any further finetuning.

[Tweet](doc:2022/07/devendra_singh_sachan_sur_twitt)

> Given an
input question, ART first retrieves a small set
of possible evidences documents. It then recon
structs
the original question by attending to these
documents
>
> The
key idea in ART is to consider the retrieved documents
as a noisy representation of the original
question and question reconstruction probability
as a way of denoising that provides soft-labels for
how likely each document is to have been the correct
result

Refers to [[IZACARD 2012.04584] Distilling Knowledge from Reader to Retriever for Question Answering](doc:2020/12/2012_04584_distilling_knowled)

2022-07-06 A propos

Understanding Semantic Search and Question Answering | deepset

Tags:

2022-06-03 A propos

BERTopic: The Future of Topic Modeling | Pinecone

Tags:

2022-05-12 A propos

[2203.08913] Memorizing Transformers

Tags:

2022-05-07 A propos

Devendra Singh Sachan sur Twitter : "...Unsupervised Passage Re-ranker (UPR), an approach to re-rank retrieved passages for information retrieval tasks."

Tags:

2022-04-18 A propos

NAVER LABS Europe : "@Nils_Reimers of @huggingface on 'Unsupervised domain adaptation for neural search'"

Tags:

2022-03-09 A propos

Semantic Keyword Clustering For 10,000+ Keywords [With Script]

Tags:

2022-02-18 A propos

How to Build a Semantic Search Engine With Transformers and Faiss | by Kostas Stathoulopoulos | Towards Data Science

Tags:

2022-01-29 A propos

[2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Tags:

2022-01-19 A propos

Domain Transfer with BERT | Pinecone

Tags:

2022-01-04 A propos

[2112.09118] Towards Unsupervised Dense Information Retrieval with Contrastive Learning

Tags:

2021-12-21 A propos

Making the Most of Data: Augmentation with BERT | Pinecone

Tags:

2021-12-18 A propos

Vespa Blog

Tags:

2021-12-17 A propos

Vespa - the big data serving engine

Tags:

Vespa

2021-12-17 A propos

[2112.07577] GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval

Tags:

An unsupervised domain adaptation technique for dense retrieval models

1. synthetic queries
are generated for each passage from the target corpus (using an existing pre-trained [T5](tag:text_to_text_transfer_transformer)
encoder-decoder)
2. the generated queries are used for mining negative
passages (retrieving the most similar
paragraphs using an existing dense retrieval
model == hard negatives!)
3. the query-passage pairs are labeled by a cross-encoder and used to train the domain-adapted
dense retriever (using method described in [Hofstätter et al.,
2020](doc:2021/12/2010_02666_improving_efficien))

[Nils Reimers sur Twitter](doc:2021/12/nils_reimers_sur_twitter_do_), [GitHub](https://github.com/UKPLab/gpl),  by the author of [TSDAE](doc:2021/09/2104_06979_tsdae_using_trans)

Claims to improve "Doc2Query" [Document Expansion by Query Prediction](doc:2022/01/1904_08375_document_expansion): ([src](https://twitter.com/KexinWang2049/status/1471435779415150598))

> - GPL: Uses doc2query to construct synthetic data and does knowledge distillation (i.e. training) on that data.
> - Doc2query: Generates queries to extend the documents and use BM25 on top of them w/o training.

2021-12-15 A propos

Semantic search through a vectorized Wikipedia (SentenceBERT) with the Weaviate vector search engine

Tags:

2021-12-05 A propos

Unsupervised Training for Sentence Transformers | Pinecone

Tags:

2021-11-24 A propos

Andrea Volpini sur Twitter : New blog post on #ContentHub for #Semantic #SEO. How to cluster content using Knowledge Graph Embeddings to detect: ✓ Clusters ✓ Sub-topics ✓ Author's topical authority

Tags:

2021-11-16 A propos

Multilingual Sentence Transformers | Pinecone

Tags:

2021-11-04 A propos

Next-Gen Sentence Embeddings with Multiple Negatives Ranking Loss | Pinecone

Tags:

2021-10-27 A propos

Isotropy in the Contextual Embedding Space: Clusters and Manifolds | OpenReview

Tags:

2021-10-26 A propos

Sentence Embeddings and Transformers | Pinecone

Tags:

2021-10-23 A propos

James Briggs sur Twitter : *free* course on vector similarity search and Faiss..."

Tags:

2021-10-16 A propos

Building a sentence embedding index with fastText and BM25 | by David Mezzetti | Towards Data Science

Tags:

2021-09-30 A propos

neuml/txtai: Build AI-powered semantic search applications

Tags:

txtai

2021-09-30 A propos

Agostina Calabrese sur Twitter : Structured Sentiment Analysis as Dependency Graph Parsing

Tags:

2021-08-02 A propos

A Moderate Proposal for Radically Better AI-powered Web Search

Tags:

2021-07-10 A propos

[2004.04906] Dense Passage Retrieval for Open-Domain Question Answering

Tags:

2021-06-03 A propos

Nils Reimers sur Twitter : "New models for Neural Information Retrieval..."

Tags:

2021-04-17 A propos

[2007.12603] IR-BERT: Leveraging BERT for Semantic Search in Background Linking for News Articles

Tags:

2021-04-12 A propos

[1901.04085] Passage Re-ranking with BERT

Tags:

2021-03-26 A propos

Rodrigo Nogueira sur Twitter : "Slides of our WSDM 2021 tutorial "Pretrained Transformers for Text Ranking: BERT and Beyond"

Tags:

2021-03-09 A propos

Hugging Face sur Twitter : "Transformers release of the Retrieval-Augmented Generation model in collaboration with @facebookai!"

Tags:

2021-02-23 A propos

Linking Entities with Knowledge Graphs | by Sigurd Berglann | Strise | Medium

Tags:

2021-02-05 A propos

[2002.08909] REALM: Retrieval-Augmented Language Model Pre-Training

Tags:

**Augment language model pre-training with a retriever module**, which
is trained using the masked language modeling objective.

> To capture knowledge in a more modular and interpretable way, we augment language model pre-training with a latent knowledge retriever, which allows the model to retrieve and attend over documents from a large corpus such as Wikipedia, used during pre-training, fine-tuning and inference. **For the first time, we show how to pre-train such a knowledge retriever in an unsupervised manner**, using masked language modeling as the learning signal and backpropagating through a retrieval step that considers millions of documents

Hum, #TODO: parallel to be drawn with techniques in [KG-augmented Language Models](tag:knowledge_graph_augmented_language_models) which focus "on the problem of capturing declarative knowledge in the learned parameters of a language model."

[Google AI Blog Post](doc:2020/08/google_ai_blog_realm_integrat)

[Summary](https://joeddav.github.io/blog/2020/03/03/REALM.html) for the [Hugging Face awesome-papers reading group](doc:2021/03/huggingface_awesome_papers_pap)

2020-12-12 A propos

[2012.04584] Distilling Knowledge from Reader to Retriever for Question Answering

Tags:

> a method to train an information retrieval module for downstream tasks, **without using pairs of queries and documents as annotations**.

Uses two models (standard pipeline for open-domain QA):

- the first one retrieves documents from a large source of knowledge (the retriever)
- the second one processes the support documents to solve the task (the reader).

> First the retriever selects support passages in a large knowledge
source. Then these passages are processed by the reader, along with the question, to generate an
answer

Inspired by knowledge distillation: the reader model is the teacher and the retriever is the student.

> More precisely, we use a sequence-to-sequence model as the reader, and use
the attention activations over the input documents as synthetic labels to train the retriever. 
> (**train the retriever by learning to approximate the attention score of the reader**)

Refers to:

- [REALM: Retrieval-Augmented Language Model Pre-Training](doc:2020/12/2002_08909_realm_retrieval_a)
- [Dehghani: Neural Ranking Models with Weak Supervision](doc:?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1704.08803)

2020-12-11 A propos

Exploration Engines - the koodos collective

Tags:

2020-12-10 A propos

Drew Tada sur Twitter : "Officially launching giantgra.ph A search engine for knowledge graphs..."

Tags:

2020-12-10 A propos

How to improve Elasticsearch search relevance with boolean queries | Elastic Blog

Tags:

ElasticSearch

2020-12-02 A propos

Recherche multilingue grâce à la détection de la langue dans Elasticsearch | Elastic Blog

Tags:

2020-11-26 A propos

Construire un bon analyzer français pour Elasticsearch

Tags:

ElasticSearch

2020-10-23 A propos

Building a Faster and Accurate Search Engine on Custom Dataset with Transformers 🤗 | by Shivanand Roy | Analytics Vidhya | Sep, 2020 | Medium

Tags:

2020-10-22 A propos

Elastic Transformers. Making BERT stretchy — Scalable… | by Mihail Dungarov | Sep, 2020 | Medium

Tags:

2020-09-08 A propos

Searching with Shingles | Elastic Blog

Tags:

2020-09-03 A propos

Google AI Blog: REALM: Integrating Retrieval into Language Representation Models

Tags:

2020-08-13 A propos

Tags:

2020-07-10 A propos

Ranked Entities in Search Results at Google

Tags:

2020-07-02 A propos

Tags:

2020-07-02 A propos

facebookresearch/faiss: A library for efficient similarity search and clustering of dense vectors.

Tags:

2020-06-09 A propos

[1911.03814] Scalable Zero-shot Entity Linking with Dense Entity Retrieval

Tags:

2020-05-02 A propos

Sebastian Riedel sur Twitter : Happy to introduce BLINK, the @facebookai open-source entity linker!...

Tags:

2020-05-02 A propos

[2004.06842] Layered Graph Embedding for Entity Recommendation using Wikipedia in the Yahoo! Knowledge Graph

Tags:

2020-04-17 A propos

[1911.02168] CoKE: Contextualized Knowledge Graph Embedding

Tags:

2020-03-22 A propos

Mapper Annotated Text Plugin | Elastic

Tags:

ElasticSearch: annotated text field

2020-03-14 A propos

MinHash token filter | Elasticsearch Reference

Tags:

2020-02-14 A propos

ElastiK Nearest Neighbors

Tags:

2020-02-13 A propos

Tech @ Cliqz

Tags:

Cliqz

2020-01-20 A propos

Building a Search Engine with BERT and TensorFlow - Towards Data Science

Tags:

2020-01-12 A propos

Building a real-time embeddings similarity matching system | Solutions | Google Cloud

Tags:

2020-01-11 A propos

Tags:

2020-01-10 A propos

Elasticsearch meets BERT: Building Search Engine with Elasticsearch and BERT

Tags:

2020-01-10 A propos

Joint Intent Classification and Slot Filling with Transformers (Jupyter Notebook Viewer)

Tags:

2020-01-09 A propos

[1902.10909] BERT for Joint Intent Classification and Slot Filling

Tags:

2020-01-09 A propos

Natural Language Understanding with Sequence to Sequence Models

Tags:

2020-01-09 A propos

[1911.00172] Generalization through Memorization: Nearest Neighbor Language Models

Tags:

2019-12-20 A propos

Détection d'intention: application industrielle d'un projet de recherche

Tags:

2019-12-14 A propos

Introducing the Annotated Text Plugin for Elasticsearch: Search for Things (not Strings) | Elastic Blog

Tags:

ElasticSearch: annotated text field

2019-12-14 A propos

mozilla/DeepSpeech: A TensorFlow implementation of Baidu's DeepSpeech architecture

Tags:

2019-12-10 A propos

Elasticsearch RSS feed indexer with Spacy entity extraction

Tags:

2019-11-24 A propos

Meetup Paris #40 : Beyond plain text: elasticsearch’s annotated text field type (en anglais) - YouTube

Tags:

2019-11-24 A propos

Building a Full-Text Search App Using Docker and Elasticsearch

Tags:

2019-11-10 A propos

[1910.03524] Beyond Vector Spaces: Compact Data Representation as Differentiable Weighted Graphs

Tags:

2019-10-09 A propos

[1908.10084] Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Tags:

2019-08-28 A propos

Supercharging Elasticsearch for extended Knowledge Graph use cases: Data science + business analytics training: Strata Data Conference

Tags:

2019-05-16 A propos

Introducing the Elastic Enterprise Search Beta: Search Everything, Anywhere | Elastic Blog

Tags:

ElasticSearch

2019-05-09 A propos

Article recommendation with Personalized PageRank and Full Text Search

Tags:

2019-02-28 A propos

[1703.03129] Learning to Remember Rare Events

Tags:

2018-10-23 A propos

Tutorial on Poincaré Embeddings (Jupyter Notebook )

Tags:

2018-05-20 A propos

Implementing Poincaré Embeddings | RARE Technologies

Tags:

2018-05-20 A propos

Open Semantic Search: Your own search engine for documents, images, tables, files, intranet & news

Tags:

2018-04-18 A propos

Tags:

2018-04-03 A propos

Semantic hashing using tags and topic modeling (2013)

Tags:

2018-03-22 A propos

ML-knn: A lazy learning approach to multi-label learning (2007)

Tags:

a lazy learning approach derived from the traditional [k-nearest neighbors algorithm](tag:k_nearest_neighbors_algorithm)

> for each unseen instance, its K nearest neighbors in the training set are firstly identified. After that, based on statistical information gained from the label sets of these neighboring instances, i.e. the number of neighboring instances belonging to each possible class, [Maximum a posteriori (MAP)](tag:maximum_a_posteriori_estimation) principle is utilized to determine the label set for the unseen instance.

Implemented in [scikit-multilearn](http://scikit.ml/api/skmultilearn.adapt.mlknn.html), in [java](https://github.com/lefman/mulan-extended/blob/master/mulan/src/mulan/classifier/lazy/MLkNN.java)

> the first lazy approach proposed specifically for multi-label classification. This is also a binary relevance approach which considers each label independently as a binary classification problem. Instead of a standard k-NN method, however, MLkNN uses the maximum a-posteriori (MAP) (Kelleher et al., 2015) approach combined with k-NN. [src](https://pdfs.semanticscholar.org/af9b/33da37d290c063cd826ab5923d96892a9767.pdf)

2018-03-18 A propos

[1710.04099] Wembedder: Wikidata entity embedding web service

Tags: