A Tri-Partite Neural Document Language Model for Semantic Information Retrieval - 2018 ESWC-Conferences(About) from the abstract: Previous work in information retrieval have shown that using evidence, such as concepts and relations, from external knowledge sources could enhance the retrieval performance... This paper presents a new tri-partite neural document language framework that leverages explicit knowledge to jointly constrain word, concept, and document learning representations to tackle a number of issues including polysemy and granularity mismatch.
[1805.04032] From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018)(About) Survey focused on semantic representation of meaning (methods that try to directly model individual meanings of words).
Pb with word embeddings: the meaning conflation deficiency (representing a word with all its possible meanings as a single vector). Can be addressed by a method for modelling unambiguous lexical meaning.
two main branches of sense representation :
The Current Best of Universal Word Embeddings and Sentence Embeddings(About) Word embeddings SOTA: [ELMo](/tag/elmo)
Sentence embeddings: While unsupervised representation learning of sentences had been the
norm for quite some time, with simple baselines like averaging word embeddings, a few novel unsupervised and supervised
approaches, as well as multi-task learning schemes, have emerged in late
Deep Unordered Composition Rivals Syntactic Methods for Text Classification (2015)(About) > Many existing deep learning models for natural language processing tasks focus on learning the compositionality of their in- puts, which requires many expensive com- putations. We present a simple deep neural network that competes with and, in some cases, outperforms such models on sen- timent analysis and factoid question an- swering tasks while taking only a fraction of the training time. While our model is syntactically-ignorant, we show significant improvements over previous bag-of-words models by deepening our network and applying a novel variant of dropout
[1803.11175] Universal Sentence Encoder (2018)(About) models for encoding sentences into embedding vectors that specifically target transfer learning to other NLP tasks.
> With transfer learning via sentence embeddings, we observe surprisingly good performance with minimal amounts of supervised training data for a transfer task
Representations for Language: From Word Embeddings to Sentence Meanings - YouTube (2017)(About) What's special about human language? the only hope for explainable intelligence.
Symbols are not just an invention of logic / classical AI.
Meaning: a solution via distributional similarity based representations. One of the most successfull ideas of modern NLP.
> You shall know a word by the company it keeps (JR Firth 1957)
The BiLSTM hegemony
Neural Bag of words
> "Surprisingly effective for many tasks :-(" [cf "DAN", Deep Averaging Network, Iyyver et al.](/doc/?uri=http%3A%2F%2Fwww.cs.cornell.edu%2Fcourses%2Fcs5740%2F2016sp%2Fresources%2Fdans.pdf)
En Suède, un livret pour se préparer à la guerre(About) Mis en garde contre les « fake news », les habitants du royaume sont déjà prévenus : « Si la Suède est attaquée par un autre pays, nous ne nous rendrons jamais. Toutes les informations selon lesquelles la résistance doit cesser sont fausses. »