[1805.04032] From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018)(About) Survey focused on semantic representation of meaning (methods that try to directly model individual meanings of words).
Pb with word embeddings: the meaning conflation deficiency (representing a word with all its possible meanings as a single vector). Can be addressed by a method for modelling unambiguous lexical meaning.
two main branches of sense representation :
That Makes Sense: Joint Sense Retrofitting from Contextual and Ontological Information(About) post-processing method for generating low-dimensional sense embedding. Emploies the ontological and contextual information simultaneously.
(Poster at the Web Conf) [Github](https://github.com/y95847frank/Joint-Retrofitting)
Calcule des "sense embeddings", en partant de word embeddings pré-calculés (par ex avec word2vec), et de données de type lexicographiques (ex wordnet), en contraignant, pour un sens, la distance entre sense et word embedding.
> While recent word embedding models demonstrate their abilities to capture syntactic and semantic information, the demand for sense level embedding is getting higher. In this study, we propose a novel joint sense embedding learning model that retrofits the word representation into sense representation from contextual and ontological information. The experiments show the effectiveness and robustness of our model that outperforms previous approaches in four public available benchmark datasets.
> Given a trained word embedding and a lexical ontology that contains sense level relationships (e.g., synonym, hypernym, etc.), our model generates new sense vectors via constraining the distance between the sense vector and its word form vector, its sense neighbors and its contextual neighbors
[Influenced by](/doc/?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1411.4166) (which post-processes and modifies word vectors to incorporate knowledge from semantic lexicons, while this creates new sense vectors)
Towards a Seamless Integration of Word Senses into Downstream NLP Applications (2017)(About) By incorporating a novel disambiguation algorithm into a state-of-the-art classification model, we create a pipeline to integrate sense-level information into downstream NLP applications. We show that a simple disambiguation of the input text can lead to consistent performance improvement on multiple topic categorization and polarity detection datasets, particularly when the fine granularity of the underlying sense inventory is reduced and the document is sufficiently large.
Our results suggest that research in sense representation should put special emphasis on real-world evaluations on benchmarks for downstream applications, rather than on artificial tasks such as word similarity. In fact, research has previously shown that **word similarity might not constitute a reliable proxy to measure the performance of word embeddings in downstream applications**