Semanlink - Neural networks ; Information retrieval: techniques AND Question Answering

Impression

Recherche de Mot-clé

Recherche de Doc

Préférences...

Mots-clés liés

Neural networks ; Information retrieval: techniques AND Question Answering

Descendants partagés

7 Documents

Yangqing Jia sur X : "The new ChatGPT search... Try searching with "what did the most popular post in hacker news say today?"

Tags:

2024-05-03 A propos

[2009.13013] SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval

Tags:

2023-04-06 A propos

[2205.11498] Domain Adaptation for Memory-Efficient Dense Retrieval

Tags:

2022-09-26 A propos

[2207.06300] Re2G: Retrieve, Rerank, Generate

Tags:

2022-07-14 A propos

Devendra Singh Sachan sur Twitter : "ART (Autoencoding-based Retriever Training), an unsupervised method to train a dense retriever that only uses questions and a collection of unpaired documents as the training data."

Tags:

2022-07-06 A propos

Zexuan Zhong sur Twitter : ...Does this really mean dense models are better? No Our #EMNLP2021 paper shows dense retrievers even fail to answer simple entity-centric questions

Tags:

2021-10-07 A propos

[2012.04584] Distilling Knowledge from Reader to Retriever for Question Answering

Tags:

> a method to train an information retrieval module for downstream tasks, **without using pairs of queries and documents as annotations**.

Uses two models (standard pipeline for open-domain QA):

- the first one retrieves documents from a large source of knowledge (the retriever)
- the second one processes the support documents to solve the task (the reader).

> First the retriever selects support passages in a large knowledge
source. Then these passages are processed by the reader, along with the question, to generate an
answer

Inspired by knowledge distillation: the reader model is the teacher and the retriever is the student.

> More precisely, we use a sequence-to-sequence model as the reader, and use
the attention activations over the input documents as synthetic labels to train the retriever. 
> (**train the retriever by learning to approximate the attention score of the reader**)

Refers to:

- [REALM: Retrieval-Augmented Language Model Pre-Training](doc:2020/12/2002_08909_realm_retrieval_a)
- [Dehghani: Neural Ranking Models with Weak Supervision](doc:?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1704.08803)

2020-12-11 A propos