Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Robert L. Logan IV
- sl:arxiv_num : 1906.07241
- sl:arxiv_published : 2019-06-17T19:48:41Z
- sl:arxiv_summary : Modeling human language requires the ability to not only generate fluent text
but also encode factual knowledge. However, traditional language models are
only capable of remembering facts seen at training time, and often have
difficulty recalling them. To address this, we introduce the knowledge graph
language model (KGLM), a neural language model with mechanisms for selecting
and copying facts from a knowledge graph that are relevant to the context.
These mechanisms enable the model to render information it has never seen
before, as well as generate out-of-vocabulary tokens. We also introduce the
Linked WikiText-2 dataset, a corpus of annotated text aligned to the Wikidata
knowledge graph whose contents (roughly) match the popular WikiText-2
benchmark. In experiments, we demonstrate that the KGLM achieves significantly
better performance than a strong baseline language model. We additionally
compare different language model's ability to complete sentences requiring
factual knowledge, showing that the KGLM outperforms even very large language
models in generating facts.@en
- sl:arxiv_title : Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modeling@en
- sl:arxiv_updated : 2019-06-20T18:37:00Z
- sl:bookmarkOf : https://arxiv.org/abs/1906.07241
- sl:creationDate : 2020-05-11
- sl:creationTime : 2020-05-11T18:55:35Z
Documents with similar tags (experimental)
Préférences...