Au sujet de ce document
- sl:arxiv_author :
- Mohammed Samiul Saeef Department of Computer Science and Engineering, University of Texas at Arlington
- Farahnaz Akrami Department of Computer Science and Engineering, University of Texas at Arlington
- Qingheng Zhang State Key Laboratory for Novel Software Technology, Nanjing University
- Chengkai Li Department of Computer Science and Engineering, University of Texas at Arlington
- Wei Hu State Key Laboratory for Novel Software Technology, Nanjing University
- sl:arxiv_firstAuthor : Farahnaz Akrami Department of Computer Science and Engineering, University of Texas at Arlington
- sl:arxiv_num : 2003.08001
- sl:arxiv_published : 2020-03-18T01:18:09Z
- sl:arxiv_summary : In the active research area of employing embedding models for knowledge graph
completion, particularly for the task of link prediction, most prior studies
used two benchmark datasets FB15k and WN18 in evaluating such models. Most
triples in these and other datasets in such studies belong to reverse and
duplicate relations which exhibit high data redundancy due to semantic
duplication, correlation or data incompleteness. This is a case of excessive
data leakage---a model is trained using features that otherwise would not be
available when the model needs to be applied for real prediction. There are
also Cartesian product relations for which every triple formed by the Cartesian
product of applicable subjects and objects is a true fact. Link prediction on
the aforementioned relations is easy and can be achieved with even better
accuracy using straightforward rules instead of sophisticated embedding models.
A more fundamental defect of these models is that the link prediction scenario,
given such data, is non-existent in the real-world. This paper is the first
systematic study with the main objective of assessing the true effectiveness of
embedding models when the unrealistic triples are removed. Our experiment
results show these models are much less accurate than what we used to perceive.
Their poor accuracy renders link prediction a task without truly effective
automated solution. Hence, we call for re-investigation of possible effective
approaches.@en
- sl:arxiv_title : Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study@en
- sl:arxiv_updated : 2020-03-18T01:18:09Z
- sl:bookmarkOf : https://arxiv.org/abs/2003.08001
- sl:creationDate : 2020-05-15
- sl:creationTime : 2020-05-15T17:26:28Z
Documents with similar tags (experimental)
Préférences...