Semanlink - Contextualized word representations

Γονείς:

NLP techniques

Contextualized word representations

Σχετικές ετικέτες:

Απόγονοι

10 Έγγραφα (Όλα τα έγγραφα)

[1905.06316] What do you learn from context? Probing for sentence structure in contextualized word representations

Tags:

2020-08-02 About

[1909.04164] Knowledge Enhanced Contextual Word Representations

Tags:

2020-05-13 About

BERT, ELMo, & GPT-2: How Contextual are Contextualized Word Representations? | SAIL Blog

Tags:

2020-03-28 About

Lecture 14 – Contextual Vectors | Stanford CS224U: Natural Language Understanding | Spring 2019

Tags:

2020-01-05 About

Contextual String Embeddings for Sequence Labeling - ACL Anthology (2018)

Tags:

Contextualized word representations

2019-04-24 About

[1902.11269] Efficient Contextual Representation Learning Without Softmax Layer

Tags:

**how to accelerate contextual representation learning**.

> Contextual representation models are difficult to train due to the large parameter sizes and high computational complexity

> We find that the softmax layer (the output layer) causes significant inefficiency due to the large vocabulary size.
Therefore, we redesign the learning objectiv.
> Specifically, the proposed approach bypasses the softmax layer by performing language modeling with dimension reduction, and allows the models to leverage pre-trained word embeddings.
Our framework reduces the time spent on the output layer to a negligible level, eliminates almost all the trainable parameters of the softmax layer and performs language modeling without truncating the vocabulary.
When applied to ELMo, our method achieves a 4 times speedup and eliminates 80% trainable parameters while achieving competitive performance on downstream tasks.

**decouples learning contexts and words**

> Instead of using
a softmax layer to predict the distribution of the
missing word, we utilize and extend the SEMFIT
layer (Kumar and Tsvetkov, 2018) to **predict the
embedding of the missing word**.

2019-03-02 About

[1902.10618] Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition

Tags:

2019-02-28 About

Generalized Language Models

Tags:

2019-02-10 About

The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar

Tags:

2018-12-03 About

Learned in translation: contextualized word vectors (Salesforce Research)

Tags:

2017-09-18 About

Ιδιότητες

sl:creationDate : 2018-11-27
sl:creationTime : 2018-11-27T11:35:49Z
rdf:type : sl:Tag
skos:altLabel : Contextualized word embeddings@en
skos:prefLabel : Contextualized word representations@en