Semanlink - NLP@Princeton AND Arxiv Doc

Printer friendly

Search Tag:

Search Doc:

Preferences...

Tag Cloud

Arxiv Doc - Contrastive Learning - Dense Passage Retrieval - Discuté avec Raphaël - Embeddings - Emergence in LLMs - EMNLP 2018 - EMNLP 2021 - LM Fine-tuning - LMs: context length - NLP@Google - NLP@Princeton - Noise contrastive estimation - Open Domain Question Answering - Sanjeev Arora - SimCSE - Sparse coding - Word sense / Lexical ambiguity -

NLP@Princeton AND Arxiv Doc

Common descendants

10 Documents

[2307.15936] A Theory for Emergence of Complex Skills in Language Models

Tags:

2024-02-24 About

[2302.06600] Task-Specific Skill Localization in Fine-tuned Language Models

Tags:

2023-08-25 About

[2305.14788] Adapting Language Models to Compress Contexts

Tags:

2023-06-04 About

[2104.08821] SimCSE: Simple Contrastive Learning of Sentence Embeddings

Tags:

2022-10-17 About

[2012.12624] Learning Dense Representations of Phrases at Scale

Tags:

2022-05-11 About

[2202.14037] Understanding Contrastive Learning Requires Incorporating Inductive Biases

Tags:

2022-03-05 About

[2109.08133] Phrase Retrieval Learns Passage Retrieval, Too

Tags:

2021-09-30 About

[1703.02507] Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features

Tags:

2019-03-25 About

[1902.09229] A Theoretical Analysis of Contrastive Unsupervised Representation Learning

Tags:

2019-03-20 About

[1601.03764] Linear Algebraic Structure of Word Senses, with Applications to Polysemy

Tags:

> Here it is shown that multiple word senses reside
in linear superposition within the word
embedding and simple sparse coding can recover
vectors that approximately capture the
senses

> Each extracted word sense is accompanied by one of about  2000 “discourse atoms” that gives a succinct description of which other words co-occur with that word sense.

> The success of the approach is mathematically explained using a variant of
the random walk on discourses model

("random walk": a generative model for language). Under the assumptions of this model,  there
exists a linear relationship between the vector of a
word w and the vectors of the words in its contexts (It is not the average of the words in w's context, but in a given corpus the matrix of the linear relationship does not depend on w. It can be estimated, and so we can compute the embedding of a word from the contexts it belongs to)

[Related blog post](/doc/?uri=https%3A%2F%2Fwww.offconvex.org%2F2016%2F07%2F10%2Fembeddingspolysemy%2F)

2018-08-28 About