Semanlink - Technical girls and guys AND Distributional semantics

A Self-Training Approach for Short Text Clustering - (Hadifar 2019)

Tags:

2021-05-20 About

Unsupervised text classification with word embeddings - Max Halford

Tags:

2020-10-05 About

[1909.01259] Neural Attentive Bag-of-Entities Model for Text Classification

Tags:

A model that performs **text classification using entities in a knowledge base**.

> Entities provide unambiguous and relevant semantic signals that are beneficial for capturing semantics in texts. We combine **simple high-recall entity detection based on a dictionary** (word->list of entities), to detect entities in a document, with a novel neural **attention mechanism that enables the model to focus on a small number of unambiguous and relevant entities**.

2 steps:

1. Entity detection
2. Classification using the detected entities (+text) as inputs

Regarding entity linking, a local model which uses cosine
similarity between the embedding of the target
entity and the word-based representation of
the document to capture the relevance of an entity
given a document.

Embeddings from the KB: computed using [#Wikipedia2Vec](tag:wikipedia2vec) (similar words and entities
close to one another in a unified vector space)

Model using attention, with 2 features :

- cosine similarity between the
embedding of the entity and the word based
representation of the document
- the probability that the entity
name refers to the entity in KB.

Somewhat [related](doc:2020/01/investigating_entity_knowledge_)

### Conclusion:

>a neural
network model that performs text classification using
entities in Wikipedia. We combined simple
dictionary-based entity detection with a neural attention
mechanism to enable the model to focus
on a small number of unambiguous and relevant
entities in a document.

2020-09-02 About

[1812.06280] Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia

Tags:

2020-09-02 About

On word embeddings

Tags:

2020-06-05 About

Approximating the Softmax for Learning Word Embeddings

Tags:

2020-06-04 About

Wikify!: linking documents to encyclopedic knowledge (2007)

Tags:

2019-04-16 About

[1703.02507] Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features

Tags:

2019-03-25 About

Contrastive Unsupervised Learning of Semantic Representations: A Theoretical Framework – Off the convex path (2019-03)

Tags:

2019-03-20 About

Word Embeddings: Explaining their properties – Off the convex path (2016)

Tags:

2019-03-20 About

nlp-notebooks/Simple Sentence Similarity.ipynb at master · nlptown/nlp-notebooks

Tags:

2019-02-07 About

Querying machine learning distributional semantics with SPARQL - bobdc.blog

Tags:

2019-01-29 About

[1601.01343] Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation

Tags:

2019-01-27 About

Cross-Lingual Word Representations: Induction and Evaluation (Tutorial EMNLP 2017)

Tags:

2018-11-05 About

Unsupervised Random Walk Sentence Embeddings: A Strong but Simple Baseline (Ethayarajh 2018)

Tags:

2018-10-08 About

Linear algebraic structure of word meanings – Off the convex path

Tags:

2018-09-20 About

A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors (2018)

Tags:

A La Carte Embedding

2018-09-18 About

Simple and efficient semantic embeddings for rare words, n-grams, and language features – Off the convex path

Tags:

2018-09-18 About

A Latent Variable Model Approach to PMI-based Word Embeddings (2016)

Tags:

2018-08-28 About

[1601.03764] Linear Algebraic Structure of Word Senses, with Applications to Polysemy

Tags:

> Here it is shown that multiple word senses reside
in linear superposition within the word
embedding and simple sparse coding can recover
vectors that approximately capture the
senses

> Each extracted word sense is accompanied by one of about  2000 “discourse atoms” that gives a succinct description of which other words co-occur with that word sense.

> The success of the approach is mathematically explained using a variant of
the random walk on discourses model

("random walk": a generative model for language). Under the assumptions of this model,  there
exists a linear relationship between the vector of a
word w and the vectors of the words in its contexts (It is not the average of the words in w's context, but in a given corpus the matrix of the linear relationship does not depend on w. It can be estimated, and so we can compute the embedding of a word from the contexts it belongs to)

[Related blog post](/doc/?uri=https%3A%2F%2Fwww.offconvex.org%2F2016%2F07%2F10%2Fembeddingspolysemy%2F)

2018-08-28 About

Learning to Understand Phrases by Embedding the Dictionary (2016)

Tags:

2018-08-23 About

[1608.05426] A Strong Baseline for Learning Cross-Lingual Word Embeddings from Sentence Alignments

Tags:

2018-07-23 About

Deep-learning-free Text and Sentence Embedding, Part 2 – Off the convex path

Tags:

2018-06-25 About

Deep-learning-free Text and Sentence Embedding, Part 1 – Off the convex path

Tags:

2018-06-25 About

Sanjeev Arora on "A theoretical approach to semantic representations" - YouTube (2016)

Tags:

2018-06-10 About

A Tri-Partite Neural Document Language Model for Semantic Information Retrieval (2018 - ESWC conference)

Tags:

2018-06-08 About

A Theoretical Approach to Semantic Coding and Hashing | Simons Institute for the Theory of Computing (2016)

Tags:

2018-05-26 About

[1404.5367] Lexicon Infused Phrase Embeddings for Named Entity Resolution

Tags:

2018-05-22 About

[1706.04902] A Survey Of Cross-lingual Word Embedding Models

Tags:

2018-05-20 About

A Simple but Tough-to-Beat Baseline for Sentence Embeddings (2017)

Tags:

2018-05-10 About

[1411.4166] Retrofitting Word Vectors to Semantic Lexicons

Tags:

2018-02-25 About

Modèle Neuronal de Recherche d'Information Augmenté par une Ressource Sémantique (2017)

Tags:

2018-02-12 About

[1412.6623] Word Representations via Gaussian Embedding

Tags:

2018-01-28 About

Learning Concept-Driven Document Embeddings for Medical Information Search (2017)

Tags:

2018-01-03 About

Combining word and entity embeddings for entity linking (ESWC 2017)

Tags:

2017-12-30 About

[1712.09405] Advances in Pre-Training Distributed Word Representations

Tags:

2017-12-29 About

facebookresearch/MUSE: A library for Multilingual Unsupervised or Supervised word Embeddings

Tags:

2017-12-22 About

A Latent Variable Model Approach to PMI-basedWord Embeddings

Tags:

2017-12-07 About

Semantic Word Embeddings – Off the convex path

Tags:

2017-11-21 About

Improving Distributional Similarity with Lessons Learned from Word Embeddings (O Levy - 2015)

Tags:

2017-11-11 About

Semantics with Dense Vectors

Tags:

2017-11-11 About

Dependency-Based Word Embeddings | Omer Levy

Tags:

2017-11-08 About

Word embeddings in 2017: Trends and future directions

Tags:

2017-11-06 About

Using pre-trained word embeddings in a Keras model

Tags:

2017-10-23 About

[1710.04087] Word Translation Without Parallel Data

Tags:

2017-10-14 About

Distributed Word Representations for Information Retrieval

Tags:

2017-10-01 About

Learned in translation: contextualized word vectors (Salesforce Research)

Tags:

2017-09-18 About

[1607.01759] Bag of Tricks for Efficient Text Classification

Tags:

2017-09-10 About

[1703.00993] A Comparative Study of Word Embeddings for Reading Comprehension

Tags:

2017-08-28 About

[1507.07998] Document Embedding with Paragraph Vectors

Tags:

2017-08-20 About

An overview of word embeddings and their connection to distributional semantic models - AYLIEN (2016)

Tags:

2017-07-20 About

Embed, encode, attend, predict: The new deep learning formula for state-of-the-art NLP models | Blog | Explosion AI

Tags:

2017-07-20 About

Latent semantic indexing ("Introduction to Information Retrieval" Manning 2008)

Tags:

VSM : problem with synonymy and polysemy (eg. synonyms are accorded separate dimensions)

Could we use the co-occurrences of terms to capture the latent semantic associations of terms and alleviate these problems?

Concluding remarks:

- computational cost of the SVD is significant
    - biggest obstacle to the widespread adoption to LSI.
    - One approach to this obstacle: build the LSI representation on a randomly sampled subset of the documents, following which the remaining documents are ``folded in'' (cf Gensim tutorial "[Random Projection (used as an option to speed up LSI)](https://radimrehurek.com/gensim/models/rpmodel.html)")
- As we reduce k, recall tends to increase, as expected.
- **Most surprisingly**, a value of k in the low hundreds can actually increase precision. **This appears to suggest that for a suitable value of *k*, LSI addresses some of the challenges of synonymy**.
- LSI works best in applications where there is little overlap between queries and documents. (--??)

The experiments also documented some modes where LSI failed to match the effectiveness of more traditional indexes and score computations.

LSI shares two basic drawbacks of vector space retrieval:
    
- no good way of expressing negations
- no way of enforcing Boolean conditions.

LSI can be viewed as soft clustering by interpreting each dimension of the reduced space as a cluster and the value that a document has on that dimension as its fractional membership in that cluster.

2017-07-19 About

[1405.4053] Distributed Representations of Sentences and Documents

Tags:

2017-07-10 About

Distributed Semantics & Embeddings

Tags: