Semanlink - [2207.09980] ReFactorGNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective

[2207.09980] ReFactorGNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective

Tags:

Au sujet de ce document

sl:arxiv_author :
sl:arxiv_firstAuthor : Yihong Chen
sl:arxiv_num : 2207.09980
sl:arxiv_published : 2022-07-20T15:39:30Z
sl:arxiv_summary : Factorisation-based Models (FMs), such as DistMult, have enjoyed enduring success for Knowledge Graph Completion (KGC) tasks, often outperforming Graph Neural Networks (GNNs). However, unlike GNNs, FMs struggle to incorporate node features and to generalise to unseen nodes in inductive settings. Our work bridges the gap between FMs and GNNs by proposing ReFactorGNNs. This new architecture draws upon both modelling paradigms, which previously were largely thought of as disjoint. Concretely, using a message-passing formalism, we show how FMs can be cast as GNNs by reformulating the gradient descent procedure as message-passing operations, which forms the basis of our ReFactorGNNs. Across a multitude of well-established KGC benchmarks, our ReFactorGNNs achieve comparable transductive performance to FMs, and state-of-the-art inductive performance while using an order of magnitude fewer parameters.@en
sl:arxiv_title : ReFactorGNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective@en
sl:arxiv_updated : 2022-07-21T13:33:26Z
sl:bookmarkOf : https://arxiv.org/abs/2207.09980
sl:creationDate : 2022-07-23
sl:creationTime : 2022-07-23T12:57:37Z

Infos sur le fichier

Bookmark of: https://arxiv.org/abs/2207.09980

Documents with similar tags (experimental)

[2301.08210] Everything is Connected: Graph Neural Networks

Tags:

2023-01-21 A propos

[2011.02260] Graph Neural Networks in Recommender Systems: A Survey

Tags:

2020-11-11 A propos

[2010.05234] A Practical Guide to Graph Neural Networks

Tags:

2020-10-15 A propos

[2001.09522] TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network

Tags:

how to add a set of new concepts to an existing taxonomy.

[Tweet](https://twitter.com/mickeyjs6/status/1253772146142216194?s=20) [GitHub](https://github.com/mickeystroller/TaxoExpan)

> we study the taxonomy expansion task: given an
existing taxonomy and a set of new emerging concepts, we aim
to automatically expand the taxonomy to incorporate these new
concepts (without changing the existing relations in the given taxonomy).

> To the best of our knowledge, this is the first study on **how to
expand an existing directed acyclic graph (as we model a taxonomy
as a DAG) using self-supervised learning**.

Self-supervised framework, the existing taxonomy being used as training data: it learns a model to predict whether a query concept is the direct hyponym of an anchor concept.

> 2 techniques:
>
> 1. a **position-enhanced graph neural network that encodes the local structure of an anchor concept** in the existing taxonomy,
> 2. a noise-robust training objective that enables the learned model to be insensitive to the label noise in the self-supervision data.

Regarding 1: uses [GNN](/tag/graph_neural_networks.html) to model the "ego network" of concepts (potential “siblings”
and “grand parents” of the query concept).

> Regular
GNNs fail to distinguish nodes with different relative positions to
the query (i.e., some nodes are grand parents of the query while
the others are siblings of the query). To address this limitation, we
present a simple but effective enhancement to inject such position
information into GNNs using position embedding. We show that
such embedding can be easily integrated with existing GNN architectures
(e.g., [GCN](/tag/graph_convolutional_networks) and GAT) and significantly boosts the
prediction performance

Regarding point 2: uses InfoNCE loss, cf. [Contrastive Predictive Coding](/doc/?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1807.03748)

> Instead of predicting
whether each individual ⟨query concept, anchor concept⟩ pair
is positive or not, we first group all pairs sharing the same query
concept into a single training instance and learn a model to select
the positive pair among other negative ones from the group.

(Hum, ça me rappelle quelque chose)

> assume each concept (in existing taxonomy + set of new concepts) has an initial embedding
vector learned from some text associated with this concept.

To keep things tractable, only attempts to find a single parent node of each new concept.

2020-04-25 A propos

[2003.00330] Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective

Tags:

2020-03-15 A propos

[1901.00596] A Comprehensive Survey on Graph Neural Networks

Tags:

2019-07-15 A propos

[1806.01261] Relational inductive biases, deep learning, and graph networks

Tags:

2018-06-13 A propos