Semanlink - Learning to Tag OOV Tokens by Integrating Contextual Representation and Background Knowledge (ACL Anthology 2020)

Learning to Tag OOV Tokens by Integrating Contextual Representation and Background Knowledge (ACL Anthology 2020)

Tags:

Aim to leverage both contextual representation of input text (deep LMs)  and  knowledge derived
from curated KBs ([Wordnet](tag:wordnet)) to improve [slot tagging](tag:slot_tagging) in the presence of [out-of-vocab](tag:oov) words ([few-shot scenario](tag:few_shot_learning))

Method:

1. retrieve potentially relevant KB entities and
encode them into distributed representations that
describe global graph-structured information
2. BERT encoder
layer to capture context-aware representations of
the sequence and attend to the KB embeddings
using multi-level graph attention
3. integrate
BERT embeddings and the KB embeddings
to predict the slot type

Contributions:

1. feasibility of applying lexical ontology
to facilitate recognizing OOV words. First to consider the large-scale background
knowledge for enhancing context-aware
slot tagging models.
2. a knowledge integration mechanism that uses multi-level graph
attention to model explicit lexical relations.
3.experiments on two benchmark datasets

> our method makes a notable difference in a
scenario where samples are linguistically diverse,
and large vocab exists.

(Better improvements when using RNN than BERT, because BERT already contains a lot of background knowledge)

Au sujet de ce document

sl:bookmarkOf : https://www.aclweb.org/anthology/2020.acl-main.58/
sl:creationDate : 2020-07-04
sl:creationTime : 2020-07-04T11:34:35Z

Infos sur le fichier

Bookmark of: https://www.aclweb.org/anthology/2020.acl-main.58/

Documents with similar tags (experimental)

Knowledge Graphs in Natural Language Processing @ ACL 2020 | by Michael Galkin

Tags:

2020-07-12 A propos