Semanlink - Knowledge Graph + Deep Learning AND Knowledge Graph Embeddings

[2010.03496] Inductive Entity Representations from Text via Link Prediction

Tags:

BLP "BERT for Link Prediction". Central idea: **training an entity encoder with a
link prediction objective** (using the textual descriptions of entities when computing entity representations - hence not failing with entities unknown in training)

> a method for **learning representations
of entities**, that uses a **pre-trained Transformer** based
architecture as an entity encoder, and
**link prediction training on a knowledge graph
with textual entity descriptions**.

> using entity descriptions,
an entity encoder is trained for link prediction in
a knowledge graph. The encoder can then be used
without fine-tuning to obtain features for entity classification
and information retrieval

Cites [Xie et al](doc:2020/10/representation_learning_of_know) and [Kepler](doc:2020/11/1911_06136_kepler_a_unified_). They claim that their
objective targeted exclusively for link prediction (and not an objective that combines language modeling
and link prediction as Kepler)
performs better than Kepler's more complex one.

2020-11-03 About

Bootstrapping Entity Alignment with Knowledge Graph Embedding | IJCAI

Tags:

2020-05-11 About

Deepak Nathani | Pay Attention, Relations are Important

Tags:

2020-04-30 About

[1906.01195] Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs

Tags:

2020-04-30 About

[1911.02168] CoKE: Contextualized Knowledge Graph Embedding

Tags:

2020-03-22 About

[1902.10197] RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

Tags:

2020-03-03 About

[1905.07129] ERNIE: Enhanced Language Representation with Informative Entities

Tags:

2019-08-05 About

Reasoning With Neural Tensor Networks for Knowledge Base Completion (2013)

Tags:

**Predicting the likely truth of additional facts based on existing facts in the knowledge base.**

> we introduce an expressive neural
tensor network suitable for reasoning over relationships between two entities.

Most similar work: [Bordes et al.](http://127.0.0.1:8080/semanlink/doc/2019/08/learning_structured_embeddings_) (2011)

Contributions:

1. new neural tensor
network (**NTN**) suitable for reasoning over relationships between two entities. Generalizes several previous neural network models and provides a more
powerful way to model relational information than a standard neural network layer.
2. a new way to represent entities in knowledge bases, as the
average of  their constituting word vectorss, allowing the sharing of statistical strength between the words describing
each entity (e.g., Bank of China and China).
3. incorporation of word vectors which are trained on large unlabeled text

> We **learn to modify word representations
via grounding in world knowledge**. This essentially allows us to analyze word embeddings and
query them for specific relations. Furthermore, the resulting vectors could be used in other tasks
such as named entity recognition or relation classification in natural language

**Makes use of entity name**: NTN
first learns word vectors from an auxiliary news corpus, and
then initializes the representation of each entity by averaging
the vectors of words contained in its name. For example,
**the embedding of AlfredHitchcock is initialized by the
average word vectors of “alfred” and “hitchcock”**... This kind of methods model textual information
separately from KG facts, and hence fail to leverage
interactions between them.[src](doc:2019/05/knowledge_graph_embedding_a_su)

2019-08-03 About

Knowledge Graph Embedding: A Survey of Approaches and Applications - IEEE (2017)

Tags:

- "facts only" based embeddings
    - Translational Distance Models, distance-based scoring functions
       - TransE and co: entities and relations as points in vector space
           - TransH: relation-specific hyperplanes.
       - Gaussian Embeddings: entities and relations as random variables
    - Semantic matching models, similarity-based scoring functions
        - RESCAL (entities as a vector, relation as a matrix) and familiy
            - DistMult (relation as diagonal matrix)
            - Holographic Embeddings (HolE)
            - etc.
        - Matching with Neural Networks
            - SME, etc
- incorporating other info: entity types, relation paths,
textual descriptions, as well as logical rules
    - entity types
        - semantically smooth embedding
(SSE):  entities of the same type to stay close
to each other in the embedding space
        - type-embodied knowledge representation
learning (TKRL), which can handle hierarchical
entity categories and multiple category labels
- Applications in downstream tasks

2019-05-27 About