Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Emma J. Gerritse
- sl:arxiv_num : 2205.00820
- sl:arxiv_published : 2022-05-02T11:53:59Z
- sl:arxiv_summary : Pre-trained language models such as BERT have been a key ingredient to
achieve state-of-the-art results on a variety of tasks in natural language
processing and, more recently, also in information retrieval.Recent research
even claims that BERT is able to capture factual knowledge about entity
relations and properties, the information that is commonly obtained from
knowledge graphs. This paper investigates the following question: Do BERT-based
entity retrieval models benefit from additional entity information stored in
knowledge graphs? To address this research question, we map entity embeddings
into the same input space as a pre-trained BERT model and inject these entity
embeddings into the BERT model. This entity-enriched language model is then
employed on the entity retrieval task. We show that the entity-enriched BERT
model improves effectiveness on entity-oriented queries over a regular BERT
model, establishing a new state-of-the-art result for the entity retrieval
task, with substantial improvements for complex natural language queries and
queries requesting a list of entities with a certain property. Additionally, we
show that the entity information provided by our entity-enriched model
particularly helps queries related to less popular entities. Last, we observe
empirically that the entity-enriched BERT models enable fine-tuning on limited
training data, which otherwise would not be feasible due to the known
instabilities of BERT in few-sample fine-tuning, thereby contributing to
data-efficient training of BERT for entity search.@en
- sl:arxiv_title : Entity-aware Transformers for Entity Search@en
- sl:arxiv_updated : 2022-05-02T11:53:59Z
- sl:bookmarkOf : https://arxiv.org/abs/2205.00820
- sl:creationDate : 2022-07-12
- sl:creationTime : 2022-07-12T08:18:56Z
Documents with similar tags (experimental)
Préférences...