Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Mehdi Ali
- sl:arxiv_num : 2006.13365
- sl:arxiv_published : 2020-06-23T22:30:52Z
- sl:arxiv_summary : The heterogeneity in recently published knowledge graph embedding models'
implementations, training, and evaluation has made fair and thorough
comparisons difficult. In order to assess the reproducibility of previously
published results, we re-implemented and evaluated 19 interaction models in the
PyKEEN software package. Here, we outline which results could be reproduced
with their reported hyper-parameters, which could only be reproduced with
alternate hyper-parameters, and which could not be reproduced at all as well as
provide insight as to why this might be the case.
We then performed a large-scale benchmarking on four datasets with several
thousands of experiments and 21,246 GPU hours of computation time. We present
insights gained as to best practices, best configurations for each model, and
where improvements could be made over previously published best configurations.
Our results highlight that the combination of model architecture, training
approach, loss function, and the explicit modeling of inverse relations is
crucial for a model's performances, and not only determined by the model
architecture. We provide evidence that several architectures can obtain results
competitive to the state-of-the-art when configured carefully. We have made all
code, experimental configurations, results, and analyses that lead to our
interpretations available at https://github.com/pykeen/pykeen and
https://github.com/pykeen/benchmarking@en
- sl:arxiv_title : Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework@en
- sl:arxiv_updated : 2020-06-23T22:30:52Z
- sl:bookmarkOf : https://arxiv.org/abs/2006.13365
- sl:creationDate : 2020-06-26
- sl:creationTime : 2020-06-26T16:33:57Z
Documents with similar tags (experimental)
Préférences...