Semanlink - Machine learning: techniques AND Knowledge Graph Completion

BLP "BERT for Link Prediction". Central idea: **training an entity encoder with a
link prediction objective** (using the textual descriptions of entities when computing entity representations - hence not failing with entities unknown in training)

> a method for **learning representations
of entities**, that uses a **pre-trained Transformer** based
architecture as an entity encoder, and
**link prediction training on a knowledge graph
with textual entity descriptions**.

> using entity descriptions,
an entity encoder is trained for link prediction in
a knowledge graph. The encoder can then be used
without fine-tuning to obtain features for entity classification
and information retrieval

Cites [Xie et al](doc:2020/10/representation_learning_of_know) and [Kepler](doc:2020/11/1911_06136_kepler_a_unified_). They claim that their
objective targeted exclusively for link prediction (and not an objective that combines language modeling
and link prediction as Kepler)
performs better than Kepler's more complex one.

2020-11-03 About

[2009.00318] More is not Always Better: The Negative Impact of A-box Materialization on RDF2vec Knowledge Graph Embeddings

Tags:

2020-09-02 About

Peter Bloem sur Twitter : "One of the messages from Ruffinelli et al 2020..."

Tags:

2020-05-19 About

[2003.08001] Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study

Tags:

2020-05-15 About

[2001.09522] TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network

Tags:

how to add a set of new concepts to an existing taxonomy.

[Tweet](https://twitter.com/mickeyjs6/status/1253772146142216194?s=20) [GitHub](https://github.com/mickeystroller/TaxoExpan)

> we study the taxonomy expansion task: given an
existing taxonomy and a set of new emerging concepts, we aim
to automatically expand the taxonomy to incorporate these new
concepts (without changing the existing relations in the given taxonomy).

> To the best of our knowledge, this is the first study on **how to
expand an existing directed acyclic graph (as we model a taxonomy
as a DAG) using self-supervised learning**.

Self-supervised framework, the existing taxonomy being used as training data: it learns a model to predict whether a query concept is the direct hyponym of an anchor concept.

> 2 techniques:
>
> 1. a **position-enhanced graph neural network that encodes the local structure of an anchor concept** in the existing taxonomy,
> 2. a noise-robust training objective that enables the learned model to be insensitive to the label noise in the self-supervision data.

Regarding 1: uses [GNN](/tag/graph_neural_networks.html) to model the "ego network" of concepts (potential “siblings”
and “grand parents” of the query concept).

> Regular
GNNs fail to distinguish nodes with different relative positions to
the query (i.e., some nodes are grand parents of the query while
the others are siblings of the query). To address this limitation, we
present a simple but effective enhancement to inject such position
information into GNNs using position embedding. We show that
such embedding can be easily integrated with existing GNN architectures
(e.g., [GCN](/tag/graph_convolutional_networks) and GAT) and significantly boosts the
prediction performance

Regarding point 2: uses InfoNCE loss, cf. [Contrastive Predictive Coding](/doc/?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1807.03748)

> Instead of predicting
whether each individual ⟨query concept, anchor concept⟩ pair
is positive or not, we first group all pairs sharing the same query
concept into a single training instance and learn a model to select
the positive pair among other negative ones from the group.

(Hum, ça me rappelle quelque chose)

> assume each concept (in existing taxonomy + set of new concepts) has an initial embedding
vector learned from some text associated with this concept.

To keep things tractable, only attempts to find a single parent node of each new concept.

2020-04-25 About

Chengkai Li sur Twitter : "Link prediction methods on knowledge graphs don't work..."

Tags:

2020-03-29 About

[1909.03193] KG-BERT: BERT for Knowledge Graph Completion

Tags:

2020-03-22 About

[1911.02168] CoKE: Contextualized Knowledge Graph Embedding

Tags:

2020-03-22 About

[1902.10197] RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

Tags:

2020-03-03 About

davidsbatista/BREDS: "Bootstrapping Relationship Extractors with Distributional Semantics" (Batista et al., 2015) - code for EMNLP'15 paper

Tags:

2020-01-11 About

Reasoning With Neural Tensor Networks for Knowledge Base Completion (2013)

Tags:

**Predicting the likely truth of additional facts based on existing facts in the knowledge base.**

> we introduce an expressive neural
tensor network suitable for reasoning over relationships between two entities.

Most similar work: [Bordes et al.](http://127.0.0.1:8080/semanlink/doc/2019/08/learning_structured_embeddings_) (2011)

Contributions:

1. new neural tensor
network (**NTN**) suitable for reasoning over relationships between two entities. Generalizes several previous neural network models and provides a more
powerful way to model relational information than a standard neural network layer.
2. a new way to represent entities in knowledge bases, as the
average of  their constituting word vectorss, allowing the sharing of statistical strength between the words describing
each entity (e.g., Bank of China and China).
3. incorporation of word vectors which are trained on large unlabeled text

> We **learn to modify word representations
via grounding in world knowledge**. This essentially allows us to analyze word embeddings and
query them for specific relations. Furthermore, the resulting vectors could be used in other tasks
such as named entity recognition or relation classification in natural language

**Makes use of entity name**: NTN
first learns word vectors from an auxiliary news corpus, and
then initializes the representation of each entity by averaging
the vectors of words contained in its name. For example,
**the embedding of AlfredHitchcock is initialized by the
average word vectors of “alfred” and “hitchcock”**... This kind of methods model textual information
separately from KG facts, and hence fail to leverage
interactions between them.[src](doc:2019/05/knowledge_graph_embedding_a_su)

2019-08-03 About

A2N: Attending to Neighbors for Knowledge Graph Inference - ACL 2019

Tags:

2019-07-31 About

A Hybrid Graph Model for Distant Supervision Relation Extraction | Springer for Research & Development (ESWC 2019)

Tags:

2019-05-30 About

Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction - ACL Anthology

Tags:

2019-02-09 About

Romain Vial (Hyperlex) at Paris NLP meetup, slides

Tags:

2019-01-24 About

Weakly-supervised Relation Extraction by Pattern-enhanced Embedding Learning

Tags:

2018-05-10 About

Convolutional 2D Knowledge Graph Embeddings (2017)

Tags:

2018-02-26 About

Knowledge Graph Embedding by Translating on Hyperplanes (2014)

Tags:

2018-01-30 About

Knowledge base completion by learning pairwise-interaction differentiated embeddings | SpringerLink (2015)

Tags:

2018-01-27 About

Knowledge Graph and Text Jointly Embedding (2014)

Tags:

2018-01-05 About

Translating Embeddings for Modeling Multi-relational Data (2013)

Tags:

This work focuses on modeling multi-relational
data from KBs (Wordnet and Freebase in this paper), with the goal of providing an efficient
tool to complete them by automatically adding new facts, without requiring extra knowledge.

**Embedding entities and relationships of multirelational
data**: a method which **models relationships by interpreting them as translations** operating on the
low-dimensional embeddings of the entities. Motivation:
- hierarchical relationships are extremely common in KBs and translations are the natural transformations for representing them.
- cf. word embeddings and the “capital of” relationship between countries and cities, which are (coincidentally rather than willingly) represented by the model as translations in the embedding space. This suggests that there may exist embedding spaces in which 1-to-1 relationships between entities of different types may, as well, be represented by translations. The intention of our model is to enforce such a structure of the embedding space.

[Good blog post by PY Vandenbussche](http://pyvandenbussche.info/2017/translating-embeddings-transe/)

2018-01-05 About

Translating Embeddings (TransE) – Pierre-Yves Vandenbussche

Tags:

2017-09-09 About

[1506.01094] Traversing Knowledge Graphs in Vector Space

Tags:

2015-10-31 About