Semanlink - Text, KG and embeddings

Parents :

Text, KG and embeddings

Amis :

Thèse IRIT-Renault NLP-KB

Descendants

Integrate LLM workflows with Knowledge Graph using Neo4j and APOC | by Tomaz Bratanic | Jun, 2023 | Towards Data Science

Tags:

2023-06-09 A propos

[1911.03681] E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT

Tags:

> way of **injecting factual knowledge about entities into the pretrained BERT model**.

(Feeding entity vectors
into BERT as if they
were wordpiece vectors without additional encoder
pretraining)

>
> **We align [Wikipedia2Vec](tag:wikipedia2vec) entity vectors (Yamada et al., 2016) with BERT's native wordpiece vector space and use the aligned entity vectors as if they were wordpiece vectors**. The resulting entity-enhanced version of BERT (called E-BERT) is similar in spirit to [ERNIE](tag:ernie) (Zhang et al., 2019) and [KnowBert](tag:knowbert) (Peters et al., 2019), but it **requires no expensive further pretraining of the BERT encoder**.
>
> Our vector space alignment strategy is inspired by
cross-lingual word vector alignment

Related work on Entity-enhanced BERT:

> ([ERNIE](doc:2019/08/_1905_07129_ernie_enhanced_la) and [Knowbert](doc:2020/05/1909_04164_knowledge_enhanced)) are based on the design principle
that BERT be adapted to entity vectors. They introduce
new encoder layers to feed pretrained entity
vectors into the Transformer, and they require additional
pretraining to integrate the new parameters.
In contrast, E-BERT’s design principle is that entity
vectors be adapted to BERT.
>
> Two other knowledge-enhanced MLMs are [KEPLER](doc:2020/11/1911_06136_kepler_a_unified_)
(Wang et al., 2019c) and K-Adapter (Wang
et al., 2020)... Their factual knowledge
does not stem from entity vectors – instead, they
are trained in a multi-task setting on relation classification
and knowledge base completion.

Not to be cofounded with [[2009.02835] E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commerce](doc:2020/12/2009_02835_e_bert_a_phrase_a)

2021-01-12 A propos

[1911.06136] KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation

Tags:

2020-11-03 A propos

[1802.05930] Learning beyond datasets: Knowledge Graph Augmented Neural Networks for Natural language Processing

Tags:

2020-10-02 A propos

Representing Text for Joint Embedding of Text and Knowledge Bases (EMNLP 2015)

Tags:

2020-09-14 A propos

[1905.07129] ERNIE: Enhanced Language Representation with Informative Entities

Tags:

2019-08-05 A propos

Propriétés

sl:creationDate : 2020-05-14
sl:creationTime : 2020-05-14T01:22:41Z
rdf:type : sl:Tag
skos:altLabel :
- Embeddings of Text + Knowledge Graphs@en
- Embeddings of text + KB
- Embeddings of Text + Knowledge Bases@en
skos:prefLabel : Text, KG and embeddings