Weak Supervision: The New Programming Paradigm for Machine Learning(About) > a broad, high-level overview of recent weak supervision approaches, where noisier or higher-level supervision is used as a more expedient and flexible way to get supervision signal, in particular from **subject matter experts** (SMEs).
broad definition of weak supervision as being comprised of **one or more noisy conditional distributions over unlabeled data**.
Key practical motivation: what if a SME could spend an afternoon specifying a set of
heuristics or other resources, that–if handled properly–could effectively replace thousands of training
Contains a good comparison of the settings in active, semi-supervised, transfer learning (and links to surveys about them)
Active learning literature survey (2010)(About) The key idea behind active learning is that a machine learning algorithm can achieve greater accuracy with fewer labeled training instances if it is allowed to choose the data from which is learns
[1805.04032] From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018)(About) Survey focused on semantic representation of meaning (methods that try to directly model individual meanings of words).
Pb with word embeddings: the meaning conflation deficiency (representing a word with all its possible meanings as a single vector). Can be addressed by a method for modelling unambiguous lexical meaning.
two main branches of sense representation :
The Current Best of Universal Word Embeddings and Sentence Embeddings (2018)(About) Word embeddings SOTA: [ELMo](/tag/elmo)
Sentence embeddings: While unsupervised representation learning of sentences had been the
norm for quite some time, with simple baselines like averaging word embeddings, a few novel unsupervised and supervised
approaches, as well as multi-task learning schemes, have emerged in late