Semanlink - Distributional semantics AND Contextualized word representations

Impression

Recherche de Mot-clé

Recherche de Doc

Préférences...

Mots-clés liés

Amazon Alexa - Arxiv Doc - BERT - BERTopic - Contextualized word representations - ELMo - Embedding evaluation - GitHub project - Keras - NLP - salesforce - Topic Modeling - Transfer learning - Tutorial - Tweet - Word embeddings -

Distributional semantics AND Contextualized word representations

Descendants partagés

15 Documents

Maarten Grootendorst sur X : "BERTopic + LLMs + DataMapPlot"

Tags:

2024-01-06 A propos

Maarten Grootendorst sur Twitter : "The v0.14 release of BERTopic is here. Fine-tune your topic keywords and labels with models from @OpenAI, @huggingface, @CohereAI, @spacy_io, and @LangChainAI... An overview thread"

Tags:

2023-02-15 A propos

BERTopic: The Future of Topic Modeling | Pinecone

Tags:

2022-05-12 A propos

MaartenGr/BERTopic: Leveraging BERT and c-TF-IDF to create easily interpretable topics.

Tags:

2022-03-10 A propos

BERT Word Embeddings Tutorial · Chris McCormick

Tags:

2020-07-06 A propos

Lecture 14 – Contextual Vectors | Stanford CS224U: Natural Language Understanding | Spring 2019

Tags:

2020-01-05 A propos

[1902.11269] Efficient Contextual Representation Learning Without Softmax Layer

Tags:

**how to accelerate contextual representation learning**.

> Contextual representation models are difficult to train due to the large parameter sizes and high computational complexity

> We find that the softmax layer (the output layer) causes significant inefficiency due to the large vocabulary size.
Therefore, we redesign the learning objectiv.
> Specifically, the proposed approach bypasses the softmax layer by performing language modeling with dimension reduction, and allows the models to leverage pre-trained word embeddings.
Our framework reduces the time spent on the output layer to a negligible level, eliminates almost all the trainable parameters of the softmax layer and performs language modeling without truncating the vocabulary.
When applied to ELMo, our method achieves a 4 times speedup and eliminates 80% trainable parameters while achieving competitive performance on downstream tasks.

**decouples learning contexts and words**

> Instead of using
a softmax layer to predict the distribution of the
missing word, we utilize and extend the SEMFIT
layer (Kumar and Tsvetkov, 2018) to **predict the
embedding of the missing word**.

2019-03-02 A propos

[1902.10618] Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition

Tags:

2019-02-28 A propos

The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar

Tags:

2018-12-03 A propos

[1811.05370] Unsupervised Transfer Learning for Spoken Language Understanding in Intelligent Agents

Tags:

2018-11-20 A propos

iliaschalkidis/ELMo-keras: Re-implementation of ELMo on Keras

Tags:

2018-11-14 A propos

AI Can Recognize Images, But Text Has Been Tricky—Until Now | WIRED

Tags:

2018-09-08 A propos

[1806.06259] Evaluation of sentence embeddings in downstream and linguistic probing tasks

Tags:

2018-06-19 A propos

ELMo: Deep contextualized word representations (2018)

Tags:

ELMo

2018-02-16 A propos

Learned in translation: contextualized word vectors (Salesforce Research)

Tags:

2017-09-18 A propos