Wikipedia
Sanjeev Arora

best known for his work on probabilistically checkable proofs and, in particular, the PCP theorem.
[Off the convex path](http://www.offconvex.org/)

Related Tags:

17 Documents (Long List)

- Unsupervised Random Walk Sentence Embeddings: A Strong but Simple Baseline (2019)
*(About)*

2019-03-30 - [1902.09229] A Theoretical Analysis of Contrastive Unsupervised Representation Learning (2019)
*(About)*

[blog post](/doc/?uri=http%3A%2F%2Fwww.offconvex.org%2F2019%2F03%2F19%2FCURL%2F)

2019-03-20 - Contrastive Unsupervised Learning of Semantic Representations: A Theoretical Framework – Off the convex path
*(About)*

[paper](/doc/?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1902.09229). Why do objectives similar the one used by word2vec succeed in such diverse settings? ("Contrastive Unsupervised Representation Learning" (CURL)) > In contrastive learning the objective used at test time is very different from the training objective: generalization error is not the right way to think about this. -> a framework that formalizes the notion of semantic similarity that is implicitly used by these algorithms > **if the unsupervised loss happens to be small at the end of contrastive learning then the resulting representations perform well on downstream classification**

2019-03-20 - Word Embeddings: Explaining their properties – Off the convex path
*(About)*

second part for [this post](/doc/?uri=http%3A%2F%2Fwww.offconvex.org%2F2015%2F12%2F12%2Fword-embeddings-1%2F) >- What properties of natural languages cause these low-dimensional embeddings to exist? >- Why do low-dimensional embeddings work better at analogy solving than high dimensional embeddings?

2019-03-20 - Linear algebraic structure of word meanings – Off the convex path
*(About)*

[About this paper](/doc/?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1601.03764)

2018-09-20 - Simple and efficient semantic embeddings for rare words, n-grams, and language features – Off the convex path
*(About)*

> Distributional methods for capturing meaning, such as word embeddings, often require observing many examples of words in context. But most humans can infer a reasonable meaning from very few or even a single occurrence...

2018-09-18 - Off the convex path
*(About)*

2018-09-09 - A Latent Variable Model Approach to PMI-based Word Embeddings (2016)
*(About)*

[Related YouTube video](/doc/?uri=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DKR46z_V0BVw) Based on a generative model (random walk on words involving a latent discourse vector), a rigorous justification for models such as word2vec and GloVe, including the hyperparameter choices for the latter, and a mathematical explanation for why these word embeddings allow analogies to be solved using linear algebra.

2018-08-28 - [1601.03764] Linear Algebraic Structure of Word Senses, with Applications to Polysemy (2016 - revised 2018)
*(About)*

> Here it is shown that multiple word senses reside in linear superposition within the word embedding and simple sparse coding can recover vectors that approximately capture the senses > Each extracted word sense is accompanied by one of about 2000 “discourse atoms” that gives a succinct description of which other words co-occur with that word sense. > The success of the approach is mathematically explained using a variant of the random walk on discourses model ("random walk": a generative model for language). Under the assumptions of this model, there exists a linear relationship between the vector of a word w and the vectors of the words in its contexts (It is not the average of the words in w's context, but in a given corpus the matrix of the linear relationship does not depend on w. It can be estimated, and so we can compute the embedding of a word from the contexts it belongs to) [Related blog post](/doc/?uri=https%3A%2F%2Fwww.offconvex.org%2F2016%2F07%2F10%2Fembeddingspolysemy%2F)

2018-08-28 - Mathematics of Machine Learning: An introduction
*(About)*

2018-08-08 - Mathematics of Machine Learning and Deep Learning - Plenary talk at International Congress of Mathematicians 2018
*(About)*

[article](/doc/?uri=https%3A%2F%2Fwww.dropbox.com%2Fs%2Fy59petiffzq63gt%2Fmain.pdf%3Fdl%3D0)

2018-08-08 - Deep-learning-free Text and Sentence Embedding, Part 2 – Off the convex path
*(About)*

> Can we design a text embedding with the simplicity and transparency of SIF while also incorporating word order information? yes we can.

2018-06-25 - Deep-learning-free Text and Sentence Embedding, Part 1 – Off the convex path
*(About)*

> introduction to extremely simple ways of computing sentence embeddings, which on many standard tasks, beat many state-of-the-art deep learning methods. Related to [this paper](/doc/?uri=https%3A%2F%2Fopenreview.net%2Fforum%3Fid%3DSyK00v5xx) (BTW, contains a good intro to text embeddings)

2018-06-25 - Sanjeev Arora on "A theoretical approach to semantic representations" - YouTube (2016)
*(About)*

Why do low-dimensional word vectors exist? > a text corpus is imagined as being generated by a random walk in a latent variable space, and the word production is via a loglinear distribution. This model is shown to imply several empirically discovered past methods for word embedding like word2vec, GloVe, PMI etc [Related paper](/doc/?uri=http%3A%2F%2Fwww.aclweb.org%2Fanthology%2FQ16-1028)

2018-06-10 - A Theoretical Approach to Semantic Coding and Hashing | Simons Institute for the Theory of Computing (2016)
*(About)*

2018-05-26 - A Simple but Tough-to-Beat Baseline for Sentence Embeddings (2017)
*(About)*

> Use word embeddings computed using one of the popular methods on unlabeled corpus like Wikipedia, represent the sentence by a weighted average of the word vectors, and then modify them a bit using PCA/SVD [github project](https://github.com/PrincetonML/SIF) [blog post](/doc/?uri=http%3A%2F%2Fwww.offconvex.org%2F2018%2F06%2F17%2Ftextembeddings%2F) See also [youtube: Sanjeev Arora on "A theoretical approach to semantic representations"](https://www.youtube.com/watch?v=KR46z_V0BVw)

2018-05-10 - Semantic Word Embeddings – Off the convex path
*(About)*

([Part 2](/doc/?uri=http%3A%2F%2Fwww.offconvex.org%2F2016%2F02%2F14%2Fword-embeddings-2%2F))

2017-11-21

Properties

- sl:creationDate : 2018-05-26
- sl:creationTime : 2018-05-26T10:33:58Z
- sl:describedBy : https://en.wikipedia.org/wiki/Sanjeev_Arora
- rdf:type : sl:Tag
- skos:prefLabel : Sanjeev Arora