Practical guide to text classification | Google Developers(About) F. Chollet: "An important insight is that the ratio between number of training samples and mean number of words per sample can tell you whether you should be using a n-gram model or a sequence model -- and whether you should use pre-trained word embeddings or train your own from scratch."
Deep Unordered Composition Rivals Syntactic Methods for Text Classification (2015)(About) > Many existing deep learning models for natural language processing tasks focus on learning the compositionality of their inputs, which requires many expensive computations. We present a simple deep neural network that competes with and, in some cases, outperforms such models on sentiment analysis and factoid question answering tasks while taking only a fraction of the training time. While our model is syntactically-ignorant, we show significant improvements over previous bag-of-words models by deepening our network and applying a novel variant of dropout
Convolutional Neural Networks for Sentence Classification (2014)(About) experiments with convolutional neural networks (CNN) trained on top of pre-trained word vectors for sentence-level classification tasks.
[Github project](https://github.com/yoonkim/CNN_sentence) with code, updates to paper, and links to valuable resources, such as a [Denny Britz](/tag/denny_britz)'s [implementation in TensorFlow](https://github.com/dennybritz/cnn-text-classification-tf)
Bag of Tricks for Efficient Text Classification (arxiv) 2016(About) A simple and efficient baseline for text classification.
**Our word features can
be averaged** together to form good sentence representations.
Our experiments show that fastText is often on par with deep learning classifiers in terms of accuracy, and many orders of magnitude faster for training and evaluation. We can train fastText on more than one billion words in less than ten minutes using a standard multicore~CPU, and classify half a million sentences among~312K classes in less than a minute.
Text Classification With Word2Vec - DS lore(About) > Overall, we won’t be throwing away our SVMs any time soon in favor of word2vec but it has it’s place in text classification.
> 1. SVM’s are pretty great at text classification tasks
> 2. Models based on simple averaging of word-vectors can be surprisingly good too (given how much information is lost in taking the average)
> 3. but they only seem to have a clear advantage when there is ridiculously little labeled training data
> Update 2017: actually, the best way to utilise the pretrained embeddings would probably be this [using keras](https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras-model.html)
Sample code to benchmark a few text categorization models to test whehter word embeddings like word2vec can improve text classification accuracy.
Sample code (based on scikit-learn) includes an embedding vectorizer that is given embedding dataset and vectorizes texts by taking the mean of all the vectors corresponding to individual words.