Σχετικά με το έγγραφο αυτό
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Nicola De Cao
- sl:arxiv_num : 2010.00904
- sl:arxiv_published : 2020-10-02T10:13:31Z
- sl:arxiv_summary : Entities are at the center of how we represent and aggregate knowledge. For
instance, Encyclopedias such as Wikipedia are structured by entities (e.g., one
per article). The ability to retrieve such entities given a query is
fundamental for knowledge-intensive tasks such as entity linking and
open-domain question answering. One way to understand current approaches is as
classifiers among atomic labels, one for each entity. Their weight vectors are
dense entity representations produced by encoding entity information such as
descriptions. This approach leads to several shortcomings: i) context and
entity affinity is mainly captured through a vector dot product, potentially
missing fine-grained interactions between the two; ii) a large memory footprint
is needed to store dense representations when considering large entity sets;
iii) an appropriately hard set of negative data has to be subsampled at
training time. We propose GENRE, the first system that retrieves entities by
generating their unique names, left to right, token-by-token in an
autoregressive fashion, and conditioned on the context. This enables to
mitigate the aforementioned technical issues: i) the autoregressive formulation
allows us to directly capture relations between context and entity name,
effectively cross encoding both; ii) the memory footprint is greatly reduced
because the parameters of our encoder-decoder architecture scale with
vocabulary size, not entity count; iii) the exact softmax loss can be
efficiently computed without the need to subsample negative data. We show the
efficacy of the approach with more than 20 datasets on entity disambiguation,
end-to-end entity linking and document retrieval tasks, achieving new SOTA, or
very competitive results while using a tiny fraction of the memory of competing
systems. Finally, we demonstrate that new entities can be added by simply
specifying their unambiguous name.@en
- sl:arxiv_title : Autoregressive Entity Retrieval@en
- sl:arxiv_updated : 2020-10-02T10:13:31Z
- sl:bookmarkOf : https://arxiv.org/abs/2010.00904
- sl:creationDate : 2021-01-14
- sl:creationTime : 2021-01-14T10:04:01Z
Προτιμήσεις...