Semanlink - Machine learning: problems AND NLP tasks / problems

a language
model with access to information available in a
KB, no assumptions
about the availability of additional components
(such as Named Entity Taggers) or annotations

> While classes of
named entities (e.g., person or location) occur frequently,
each individual name (e.g, Atherton or
Zhouzhuang) may be observed infrequently even
in a very large corpus of text. As a result language
models learn to represent accurately only the most
popular named entities

> knowing that Alice is a name
used to refer to a person should give ample information
about the context in which the word may
occur (e.g., Bob visited Alice).

> ---

> extends a traditional **RNN LM**

> we enhance a traditional LM with a
gating mechanism that controls whether a particular
word is modeled as a general word or as a reference
to an entity
>
> We train the model end-to-end
with only the traditional predictive language modeling
perplexity objective
>
> KALM is trained end-to-end using
a predictive objective on large corpus of text.

> To the best of our knowledge, KALM is the first
unsupervised neural NER approach.

> KALM extends a traditional, RNN-based neural
LM.

2022-08-31 A propos

Unsupervised Learning — Sentence-Transformers documentation

Tags:

2022-08-20 A propos

What Makes a Good Classification Example?

Tags:

2022-08-16 A propos

[1807.00745] Training a Neural Network in a Low-Resource Setting on Automatically Annotated Noisy Data

Tags:

2022-07-18 A propos

Dealing with Data Scarcity in Natural Language Processing | by Yves Peirsman | NLPTown | Medium 2019)

Tags:

2022-07-18 A propos

[2207.06300] Re2G: Retrieve, Rerank, Generate

Tags:

2022-07-14 A propos

[2205.00820] Entity-aware Transformers for Entity Search

Tags:

2022-07-12 A propos

[2205.15952] Knowledge Graph -- Deep Learning: A Case Study in Question Answering in Aviation Safety Domain

Tags:

2022-06-11 A propos

Domain transfer with GGPL: German Generative Pseudo Labeling 🥨 | by Matthias Richter | Jun, 2022 | ML6team

Tags:

2022-06-02 A propos

Nils Reimers sur Twitter : "GPL goes multi-lingual..."

Tags:

2022-06-01 A propos

DataBorg - Knowledge management simplified

Tags:

2022-05-14 A propos

[2012.12624] Learning Dense Representations of Phrases at Scale

Tags:

2022-05-11 A propos

[2008.09470] Top2Vec: Distributed Representations of Topics

Tags:

2022-04-28 A propos

Ramsri Goutham Golla sur Twitter : "Hi @Nils_Reimers For GPL you used "msmarco-distilbert-base-tas-b" model and ..."

Tags:

2022-04-27 A propos

[2110.08151] mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models

Tags:

[Ikuya Yamada sur Twitter : "Is entity representation effective to improve multilingual language models?..."](doc:2022/04/ikuya_yamada_sur_twitter_is_)

> Recent studies have shown that multilingual pretrained language models can be effectively improved with cross-lingual alignment information from Wikipedia entities. However, **existing methods only exploit entity information in pretraining and do not explicitly use entities in downstream tasks**. In this study, we explore the **effectiveness of leveraging entity representations for downstream cross-lingual tasks**.
>
> the key insight is that incorporating entity representations into the input allows us to extract more language-agnostic features.

[Github](https://github.com/studio-ousia/luke)

> Entity representations are known to enhance
language models in mono-lingual settings
(Zhang et al., 2019: [ERNIE](tag:ernie.html); Peters et al., 2019:  [[1909.04164] Knowledge Enhanced Contextual Word Representations](doc:2020/05/1909_04164_knowledge_enhanced); Wang et al.,
2021 [[1911.06136] KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](doc:2020/11/1911_06136_kepler_a_unified_); Xiong et al., 2020; Yamada et al., 2020: [[2010.01057] LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](doc:2020/11/2010_01057_luke_deep_context))
presumably by introducing real-world knowledge.
We show that using entity representations facilitates
cross-lingual transfer by providing languageindependent
features.
>
> Multilingual extension of LUKE. The model is trained with the multilingual
masked language modeling (MLM) task as well
as the masked entity prediction (MEP) task with
Wikipedia entity embeddings

> We investigate two ways of using the entity representations
in cross-lingual transfer tasks:
> 1. perform
entity linking for the input text, and append
the detected entity tokens to the input sequence.
The entity tokens are expected to provide language independent
features to the model
> 2. use the entity
[MASK] token from the MEP task as a languageindependent
feature extractor.

2022-04-17 A propos

I.A.B sur Twitter : "When we "know the meaning" of a word, what is it that we know? For example, what does knowing the words "dolphin" and "tiger" entail?..."

Tags:

2022-04-15 A propos

Ikuya Yamada sur Twitter : "Is entity representation effective to improve multilingual language models?..."

Tags:

2022-04-13 A propos

Tu Vu sur Twitter : "Enormous LMs like GPT-3 exhibit impressive few-shot performance, but w/ self-training a BERT base sized model can achieve much better results!

Tags:

2022-04-13 A propos

[2203.10581] Cluster & Tune: Boost Cold Start Performance in Text Classification

Tags:

[Leshem Choshen sur Twitter : "Labelled data is scarce, what can we do?..."](doc:2022/04/leshem_choshen_sur_twitter_l)

> **One-sentence Summary**: we suggest adding an unsupervised intermediate classification step, before finetunning and after pretraining BERT, and show it improves performance for data-constrained cases.

> for text classification cold start (when labeled
data is scarce), **add an intermediate unsupervised
classification task**, between the pretraining
and fine-tuning phases:
> perform clustering and
train the pre-trained model on predicting the
cluster labels.

> this additional
classification phase can significantly improve
performance, mainly for **topical classification**
tasks

> we use an efficient clustering technique,
that relies on simple Bag Of Words (BOW)
representations, to partition the unlabeled training
data into relatively homogeneous clusters of text
instances.
>
> Next, we treat these clusters as labeled
data for an intermediate text classification task, and
train the pre-trained model – with or without additional
MLM pretraining – with respect to this
multi-class problem, prior to the final fine-tuning
over the actual target-task labels

> The underlying
intuition is that inter-training the model
over a related text classification task would be more
beneficial compared to MLM inter-training, which
focuses on different textual entities, namely predicting
the identity of a single token.

2022-04-06 A propos

Leshem Choshen sur Twitter : "Labelled data is scarce, what can we do?..."

Tags:

2022-04-06 A propos

[2008.11228] A simple method for domain adaptation of sentence embeddings

Tags:

2022-04-01 A propos

[1910.06294] Training Compact Models for Low Resource Entity Tagging using Pre-trained Language Models

Tags:

2022-03-31 A propos

[2004.05119] Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer

Tags:

2022-03-31 A propos

Sentence Transformer Fine-Tuning (SetFit): Outperforming GPT-3 on few-shot Text-Classification while being 1600 times smaller | by Moshe Wasserblat (2021-12)

Tags:

2022-03-31 A propos

Nils Reimers sur Twitter : "Great post on SetFit"

Tags:

2022-03-31 A propos

Sentence Embedding Fine-tuning for the French Language | by La Javaness R&D | Feb, 2022 | Medium

Tags:

2022-03-31 A propos

Domain Adaptation — Sentence-Transformers documentation

Tags:

2022-03-31 A propos

[2203.14655] Few-Shot Learning with Siamese Networks and Label Tuning

Tags:

> the problem of building text classifiers with little or no training data.
>
> In recent years, an approach based on neural textual entailment models has been found to give strong results on a diverse range of tasks.

(cf. #[NLI](tag:nli), using the input text as the premise and the text representing the label as the hypothesis)

> In this work, we show that **with proper pre-training, Siamese Networks that embed texts and labels** offer a competitive alternative.
>
> We introduce **label tuning: fine-tuning the label embeddings only**. While giving lower performance than model fine-tuning (which updates all params of the model), this approach has the architectural advantage that a single encoder can be shared by many different tasks (we only fine-tune the label embeddings)
> The drop in quality can
be compensated by using a variant of **[Knowledge distillation](tag:knowledge_distillation)**

[Github](https://tinyurl.com/label-tuning), [Tweet](doc:2022/03/thomas_muller_sur_twitter_pa)

2022-03-30 A propos

[2006.00632] Neural Unsupervised Domain Adaptation in NLP---A Survey

Tags:

2022-03-30 A propos

Domain adaptation of word embeddings through the exploitation of in-domain corpora and knowledge bases (PhD Thesis 2021)

Tags:

Thèse de Hicham El Boukkouri, univ Paris-Saclay

[Github](https://github.com/helboukkouri/phd-code)

### Goal

Given a target specialized domain, improve the quality of general-domain
word representations using in-domain corpora and/or knowledge bases

### Contributions

#### a method for specializing general-domain embeddings in a [Low-Resource](tag:nlp_low_resource_scenarios) context.

> - train static representations on the task corpus, 
> - resume the
pre-training of general-domain contextual embeddings on the same task corpus,
> - finally, combine both static and contextual representations into
one final model

#### we tackle the issue of using a general-domain vocabulary in a specialized domain

#### Evaluation of  re-training vs training from scratch on specialized corpora using a specialized vocabulary

training from scratch better, but not that much: re-training from a general model
is still appropriate as it is less expensive and leads to comparable, although
slightly lower, performance

#### Regarding subword-based tokenization systems
> we argue that they are inconvenient in practice -> CharacterBERT, a variant
of BERT that uses ELMo’s character-based system instead of WordPieces. More convenient ti use, superior robustness to misspellings

#### Ways to specialize general-domain representations using knowledge bases

a strong baseline using a simple
method relying on graph embeddings and concatenation, using only is_a relation

> both static and contextual embeddings may effectively be
specialized using this simple approach

#### Knowledge Injection Modules (KIM) that
inject the knowledge representations directly within the BERT-like models' architecture

### Notes

> our experiments focused on a single
setting (i.e. the medical domain and the English language)

> meta-embeddings, an approach that consists in combining
different sets of representations for achieving improved performance

2022-03-23 A propos

GuideToTransformersDomainAdaptation.ipynb - Colaboratory

Tags:

2022-03-18 A propos

NLP | How to add a domain-specific vocabulary (new tokens) to a subword tokenizer already trained like BERT WordPiece | by Pierre Guillou | Medium

Tags:

2022-03-18 A propos

Retraining roberta-base using the RoBERTa MLM Procedure | Medium

Tags:

2022-03-18 A propos

Domain Adaptation with BERT-based Domain Classification and Data Selection - ACL Anthology (2019)

Tags:

2022-03-16 A propos

Studio Ousia sur Twitter : "Now using LUKE is easier than ever!" / Twitter

Tags:

2022-03-15 A propos

[2110.10778] Contrastive Document Representation Learning with Graph Attention Networks

Tags:

2022-03-10 A propos

ddangelov/Top2Vec: Top2Vec learns jointly embedded topic, document and word vectors.

Tags:

> Top2Vec is an algorithm for topic modeling and semantic search. It automatically detects topics present in text and generates jointly embedded topic, document and word vectors.
>
> "Update: Pre-trained Universal Sentence Encoders and BERT Sentence Transformer now available for embedding."

> **The assumption the algorithm makes is that many semantically similar documents are indicative of an underlying topic**. The first step is to create a joint embedding of document and word vectors. Once documents and words are embedded in a vector space the goal of the algorithm is to find dense clusters of documents, then identify which words attracted those documents together. Each dense area is a topic and the words that attracted the documents to the dense area are the topic words.

> Once you train the Top2Vec model you can:
> - ...
> - Get **hierarchical topics**.
> - Search topics by keywords.
> - Search documents by topic, by keywords.
> - Find similar words, similar documents.

Refered by [BERTopic](doc:2022/03/maartengr_bertopic_leveraging_)

2022-03-10 A propos

Unsupervised Training of Retrievers Using GenQ (The Art of Asking Questions with GenQ) | Pinecone

Tags:

2022-03-09 A propos

NAVER LABS Europe : "@Nils_Reimers of @huggingface on 'Unsupervised domain adaptation for neural search'"

Tags:

2022-03-09 A propos

[2109.06304] Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration

Tags:

2022-02-25 A propos

Nils Reimers sur Twitter : "Creating intent classes for chatbots is challenging This tutorial shows how to use sentence-transformers to find potentially overlapping intent classes and how to improve your data annotation work." / Twitter

Tags:

2022-02-19 A propos

Nils Reimers sur Twitter : "how to use the fast clustering algorithm from sentence-transformers..."

Tags:

2022-02-19 A propos

Semantic Keyword Clustering For 10,000+ Keywords [With Script]

Tags:

2022-02-18 A propos

NLP: POS (Part of speech) Tagging & Chunking | by Suneel Patel | Medium

Tags:

2022-02-16 A propos

Part-of-Speech(POS) Tag | Dependency Parsing | Constituency Parsing

Tags:

2022-02-06 A propos

gsarti/scibert-nli · Hugging Face

Tags:

2022-01-29 A propos

[2004.07180] SPECTER: Document-level Representation Learning using Citation-informed Transformers

Tags:

2022-01-29 A propos

Haystack Annotation Tool

Tags:

Haystack Annotation Tool

2022-01-27 A propos

[2108.13934] Robust Retrieval Augmented Generation for Zero-shot Slot Filling

Tags:

2022-01-19 A propos

Domain Transfer with BERT | Pinecone

Tags:

2022-01-04 A propos

[2112.09118] Towards Unsupervised Dense Information Retrieval with Contrastive Learning

Tags:

2021-12-21 A propos

Label-Specific Document Representation for Multi-Label Text Classification - ACL Anthology

Tags:

2021-12-20 A propos

Using pretrained SBERT model in cross-encoder · Issue #726 · UKPLab/sentence-transformers

Tags:

2021-12-17 A propos

Advance BERT model via transferring knowledge from Cross-Encoders to Bi-Encoders | by Chien Vu | Towards Data Science

Tags:

2021-12-17 A propos

[2112.07577] GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval

Tags:

An unsupervised domain adaptation technique for dense retrieval models

1. synthetic queries
are generated for each passage from the target corpus (using an existing pre-trained [T5](tag:text_to_text_transfer_transformer)
encoder-decoder)
2. the generated queries are used for mining negative
passages (retrieving the most similar
paragraphs using an existing dense retrieval
model == hard negatives!)
3. the query-passage pairs are labeled by a cross-encoder and used to train the domain-adapted
dense retriever (using method described in [Hofstätter et al.,
2020](doc:2021/12/2010_02666_improving_efficien))

[Nils Reimers sur Twitter](doc:2021/12/nils_reimers_sur_twitter_do_), [GitHub](https://github.com/UKPLab/gpl),  by the author of [TSDAE](doc:2021/09/2104_06979_tsdae_using_trans)

Claims to improve "Doc2Query" [Document Expansion by Query Prediction](doc:2022/01/1904_08375_document_expansion): ([src](https://twitter.com/KexinWang2049/status/1471435779415150598))

> - GPL: Uses doc2query to construct synthetic data and does knowledge distillation (i.e. training) on that data.
> - Doc2query: Generates queries to extend the documents and use BM25 on top of them w/o training.

2021-12-15 A propos

Nils Reimers sur Twitter : "Do dense retrieval models work out-of-the-box for your specific domain? Often the answer was No😢..."

Tags:

2021-12-15 A propos

Unsupervised_Extractive_Summarization - a Hugging Face Space by Hellisotherpeople

Tags:

2021-12-03 A propos

Unsupervised Training for Sentence Transformers | Pinecone

Tags:

2021-11-24 A propos

How to Fine-Tune Sentence-BERT for Question Answering | Capital One

Tags:

2021-11-21 A propos

[1911.02655] Towards Domain Adaptation from Limited Data for Question Answering Using Deep Neural Networks

Tags:

2021-11-19 A propos

[2108.13854] Contrastive Domain Adaptation for Question Answering using Limited Text Corpora

Tags:

2021-11-19 A propos

[1706.03610] Neural Domain Adaptation for Biomedical Question Answering

Tags:

2021-11-19 A propos

Multilingual Sentence Transformers | Pinecone

Tags:

2021-11-04 A propos

Nils Reimers sur Twitter : "Neural Search for Low Resource Scenarios..."

Tags:

2021-10-27 A propos

[2106.13474] Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains

Tags:

2021-10-21 A propos

[1908.11860] Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification

Tags:

2021-10-21 A propos

Application of Self-Organizing Maps in Text Clustering: A Review | IntechOpen (2012)

Tags:

2021-10-17 A propos

[1712.05972] Train Once, Test Anywhere: Zero-Shot Learning for Text Classification

Tags:

2021-10-16 A propos

[2010.07245] Text Classification Using Label Names Only: A Language Model Self-Training Approach

Tags:

2021-10-16 A propos

MasakhaNER: Named Entity Recognition for African Languages | MIT Press

Tags:

2021-10-14 A propos

[2109.04711] Pre-train or Annotate? Domain Adaptation with a Constrained Budget

Tags:

2021-10-14 A propos

[2109.08133] Phrase Retrieval Learns Passage Retrieval, Too

Tags:

2021-09-30 A propos

Building a sentence embedding index with fastText and BM25 | by David Mezzetti | Towards Data Science

Tags:

2021-09-30 A propos

[2010.12566] DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionaries

Tags:

2021-09-06 A propos

[2104.06979] TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning

Tags:

> The most
successful previous approaches like InferSent (Conneau
et al., 2017), Universial Sentence Encoder
(USE) (Cer et al., 2018) and SBERT (Reimers and
Gurevych, 2019) heavily relied on labeled data to
train sentence embedding models.
>
> TSDAE can
achieve up to 93.1% of the performance of indomain
supervised approaches. Further, we
show that TSDAE is **a strong domain adaptation
and pre-training method for sentence
embeddings**, significantly outperforming other
approaches like Masked Language Model.

> During training, TSDAE
encodes corrupted sentences into fixed-sized
vectors and requires the decoder to reconstruct the
original sentences from this sentence embedding.

- <https://www.sbert.net/examples/unsupervised_learning/TSDAE/README.html>
- [github](https://github.com/UKPLab/sentence-transformers/tree/master/examples/unsupervised_learning/TSDAE)
- [UKPLab/sentence-transformers: Sentence Embeddings with BERT & XLNet](doc:2020/07/ukplab_sentence_transformers_s)
- [twitter](https://twitter.com/KexinWang2049/status/1433361957579538432):

> **TSDAE can learn domain-specific sentence embeddings with unlabeled sentences**
>
> Most importantly, instead of STS (Semantic Textual Similarity), **we suggest evaluating unsupervised sentence embeddings on the domain-specific tasks&datasets, which is the real use case for them**. Actually, STS scores do not correlate with performance on specific tasks.

2021-09-01 A propos

(((ل()(ل() 'yoav))))👾 sur Twitter : "my two cents on why NLP as a field is focusing on the ML-ish / algorithmic / leaderboard-ish aspects (incl., now, LLMs) and not on the underlying language phenomena: it is just so much easier, on so many levels."

Tags:

2021-08-30 A propos

Agostina Calabrese sur Twitter : Structured Sentiment Analysis as Dependency Graph Parsing

Tags:

2021-08-02 A propos

[1911.02116] Unsupervised Cross-lingual Representation Learning at Scale

Tags:

2021-07-29 A propos

[2107.00676] A Primer on Pretrained Multilingual Language Models

Tags:

2021-07-13 A propos

[2103.11811] MasakhaNER: Named Entity Recognition for African Languages

Tags:

2021-07-06 A propos

Improving Quality of Search Results Clustering with Approximate Matrix Factorisations (2006)

Tags:

Lingo

2021-06-02 A propos

Simple Unsupervised Keyphrase Extraction using Sentence Embeddings - ACL Anthology (2018)

Tags:

2021-05-31 A propos

Carrot2 search results clustering engine (online)

Tags:

Carrot2

2021-05-27 A propos

Clustering of semantically enriched short texts (2018)

Tags:

2021-05-26 A propos

Term Based Semantic Clusters for Very Short Text Classification (2019)

Tags:

2021-05-26 A propos

Integrating Document Clustering and Multidocument Summarization

Tags:

2021-05-25 A propos

Adventures in Zero-Shot Text Classification

Tags:

2021-05-25 A propos

[2001.11631] Enhancement of Short Text Clustering by Iterative Classification

Tags:

2021-05-20 A propos

[2103.12953] Supporting Clustering with Contrastive Learning

Tags:

2021-05-20 A propos

A Self-Training Approach for Short Text Clustering - (Hadifar 2019)

Tags:

2021-05-20 A propos

Simultaneous Categorization of Text Documents And Identification of Cluster-dependent Keywords (2003)

Tags:

Clustering of text documents

2021-04-20 A propos

A Survey of Text Clustering Algorithms - C. C. Aggarwal (2012)

Tags:

2021-04-20 A propos

[2007.15779] Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

Tags:

2021-04-11 A propos

exBERT: Extending Pre-trained Models with Domain-specific Vocabulary Under Constrained Training Resources - ACL Anthology

Tags:

**Focus on the Embedding of Domain-specific Vocabulary.**

> exBERT
adds a new domain-specific vocabulary and the corresponding
embedding layer, as well as a small
extension module to the original unmodified model

> a pretraining
method allowing **low-cost embedding of
domain-specific vocabulary in the context of an
existing large pre-trained model such as BERT**

> exBERT... explicitly incorporates
the new domain’s vocabulary, while being able to
**reuse the original pre-trained model’s weights as is**
to reduce required computation and training data. Specifically, exBERT extends BERT by augmenting
its embeddings for the original vocabulary with
new embeddings for the domain-specific vocabulary
via **a learned small “extension” module**. **The
output of the original and extension modules are
combined via a trainable weighted sum operation**

In a way similar to concept developed in

> [[1902.00751] Parameter-Efficient Transfer Learning for NLP](doc:2021/04/1902_00751_parameter_efficien), but not in the fine-tuning paradigm.

[Github](https://github.com/cgmhaicenter/exBERT)

2021-04-11 A propos

New pipeline for zero-shot text classification - 🤗Transformers - Hugging Face Forums

Tags:

2021-03-15 A propos

Christopher Dengsø sur Twitter : "The moderation API now detects addresses in addition to other personal details."

Tags:

2021-03-09 A propos

asahi417/tner: Language model finetuning on NER

Tags:

2021-03-04 A propos

Named Entity Recognition without Labelled Data: A Weak Supervision Approach (2020)

Tags:

2021-02-24 A propos

Named Entity Recognition without Labelled Data: A Weak Supervision Approach (2020) (slides)

Tags:

2021-02-24 A propos

Distill our @huggingface zero-shot classifier with your specified class names

Tags:

2021-02-23 A propos

Zero-shot classifier distillation at master · huggingface/transformers

Tags:

2021-02-23 A propos

Zero-Shot Learning in Modern NLP | Joe Davison Blog (2020-05)

Tags:

2021-02-23 A propos

Why we switched from Spacy to Flair to anonymize French case law | by Michaël Benesty | Towards Data Science

Tags:

2021-02-10 A propos

kamalkraj/BERT-NER: Pytorch-Named-Entity-Recognition-with-BERT

Tags:

2021-02-07 A propos

Custom NLP Approaches to Data Anonymization | by Omri Mendels | Towards Data Science

Tags:

2021-02-01 A propos

X-BERT: eXtreme Multi-label Text Classification using Bidirectional Encoder Representations from Transformers

Tags:

2021-01-10 A propos

GitHub - explosion/sense2vec: Contextually-keyed word vectors

Tags:

2020-12-31 A propos

[2009.02835] E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commerce

Tags:

2020-12-14 A propos

[2004.10964] Don't Stop Pretraining: Adapt Language Models to Domains and Tasks

Tags:

2020-12-01 A propos

Domain-Specific BERT Models · Chris McCormick

Tags:

Domain-Specific BERT

2020-12-01 A propos

[2011.06993] FLERT: Document-Level Features for Named Entity Recognition

Tags:

2020-12-01 A propos

raphaelsty/ckb: Contextual knowledge bases

Tags:

2020-11-09 A propos

Sylvain Gugger sur Twitter : "Training a transformer model for text classification..."

Tags:

2020-10-19 A propos

[1911.11506] Word-Class Embeddings for Multiclass Text Classification

Tags:

> In supervised tasks such as multiclass
text classification (the focus of this article) it seems appealing to enhance word representations
with ad-hoc embeddings that encode task-specific information. We propose (supervised) word-class
embeddings (WCEs), and show that, when concatenated to (unsupervised) pre-trained word embeddings,
they substantially facilitate the training of deep-learning models in multiclass classification by
topic.
>
> A differentiating aspect of our method is that it keeps the modelling of word-class interactions separate from the
original word embedding. Word-class correlations are confined in a dedicated vector space, whose vectors enhance
(by concatenation) the unsupervised representations. The net effect is an embedding matrix that is better suited to
classification, and imposes no restriction to the network architecture using it.

[github](https://github.com/AlexMoreo/word-class-embeddings). Refers to [LEAM](doc:2020/02/joint_embedding_of_words_and_la) :

> [in LEAM] Once words and labels are embedded in a common vector space, word-label
compatibility is measured via cosine similarity. Our method instead models these compatibilities directly, without
generating intermediate embeddings for words or labels.

2020-10-11 A propos

[2004.03705] Deep Learning Based Text Classification: A Comprehensive Review

Tags:

2020-10-11 A propos

Top 6 Open Source Pretrained Models for Text Classification you should use

Tags:

Text Classification

2020-10-11 A propos

Towards Unsupervised Text Classification Leveraging Experts and Word Embeddings - (ACL 2019)

Tags:

2020-10-05 A propos

Unsupervised text classification with word embeddings - Max Halford

Tags:

2020-10-05 A propos

[2001.08053] Contextualized Embeddings in Named-Entity Recognition: An Empirical Study on Generalization

Tags:

2020-10-01 A propos

[1806.06478] Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment

Tags:

2020-09-06 A propos

[1909.01259] Neural Attentive Bag-of-Entities Model for Text Classification

Tags:

A model that performs **text classification using entities in a knowledge base**.

> Entities provide unambiguous and relevant semantic signals that are beneficial for capturing semantics in texts. We combine **simple high-recall entity detection based on a dictionary** (word->list of entities), to detect entities in a document, with a novel neural **attention mechanism that enables the model to focus on a small number of unambiguous and relevant entities**.

2 steps:

1. Entity detection
2. Classification using the detected entities (+text) as inputs

Regarding entity linking, a local model which uses cosine
similarity between the embedding of the target
entity and the word-based representation of
the document to capture the relevance of an entity
given a document.

Embeddings from the KB: computed using [#Wikipedia2Vec](tag:wikipedia2vec) (similar words and entities
close to one another in a unified vector space)

Model using attention, with 2 features :

- cosine similarity between the
embedding of the entity and the word based
representation of the document
- the probability that the entity
name refers to the entity in KB.

Somewhat [related](doc:2020/01/investigating_entity_knowledge_)

### Conclusion:

>a neural
network model that performs text classification using
entities in Wikipedia. We combined simple
dictionary-based entity detection with a neural attention
mechanism to enable the model to focus
on a small number of unambiguous and relevant
entities in a document.

2020-09-02 A propos

Initializing neural networks for hierarchical multi-label text classification (2017)

Tags:

2020-09-01 A propos

Hierarchical Multi-label Classification of Text with Capsule Networks (2019)

Tags:

2020-08-15 A propos

A Study of multilabel text classification and the effect of label hierarchy (2015)

Tags:

2020-08-15 A propos

[2003.11644] MAGNET: Multi-Label Text Classification using Attention-based Graph Neural Network

Tags:

2020-08-14 A propos

Hugging Face sur Twitter : "No labeled data? No problem. The 🤗 Transformers master branch now includes a built-in pipeline for zero-shot text classification...

Tags:

2020-08-12 A propos

Learning to Tag OOV Tokens by Integrating Contextual Representation and Background Knowledge (ACL Anthology 2020)

Tags:

Aim to leverage both contextual representation of input text (deep LMs)  and  knowledge derived
from curated KBs ([Wordnet](tag:wordnet)) to improve [slot tagging](tag:slot_tagging) in the presence of [out-of-vocab](tag:oov) words ([few-shot scenario](tag:few_shot_learning))

Method:

1. retrieve potentially relevant KB entities and
encode them into distributed representations that
describe global graph-structured information
2. BERT encoder
layer to capture context-aware representations of
the sequence and attend to the KB embeddings
using multi-level graph attention
3. integrate
BERT embeddings and the KB embeddings
to predict the slot type

Contributions:

1. feasibility of applying lexical ontology
to facilitate recognizing OOV words. First to consider the large-scale background
knowledge for enhancing context-aware
slot tagging models.
2. a knowledge integration mechanism that uses multi-level graph
attention to model explicit lexical relations.
3.experiments on two benchmark datasets

> our method makes a notable difference in a
scenario where samples are linguistically diverse,
and large vocab exists.

(Better improvements when using RNN than BERT, because BERT already contains a lot of background knowledge)

2020-07-04 A propos

dicksontsai/stanford-nlp-local-extension: Chrome extension for sending content to localhost server running Stanford NLP tools.

Tags:

2020-07-03 A propos

Representation Learning for Information Extraction from Form-like Documents – Google Research

Tags:

2020-06-15 A propos

[1909.04164] Knowledge Enhanced Contextual Word Representations

Tags:

2020-05-13 A propos

Disambiguating KBpedia Knowledge Graph Concepts

Tags:

2020-05-03 A propos

[2004.14958] A Call for More Rigor in Unsupervised Cross-lingual Learning

Tags:

2020-05-02 A propos

Unsupervised NER using BERT - Hands-on NLP model review - Quora

Tags:

2020-03-06 A propos

[2002.11402] Detecting Potential Topics In News Using BERT, CRF and Wikipedia

Tags:

2020-02-27 A propos

[1910.04126] Scalable Nearest Neighbor Search for Optimal Transport

Tags:

2020-02-20 A propos

FastHugs | ntentional

Tags:

2020-02-19 A propos

[1805.04174] Joint Embedding of Words and Labels for Text Classification (ACL Anthology 2018)

Tags:

2020-02-18 A propos

[1911.05507] Compressive Transformers for Long-Range Sequence Modelling

Tags:

2020-02-11 A propos

Semantic Text Matching for Long-Form Documents (2019)

Tags:

2020-01-23 A propos

Siamese Network for Image and Text similarity using Keras

Tags:

2020-01-22 A propos

pfliu-nlp/Named-Entity-Recognition-NER-Papers: An elaborate and exhaustive paper list for Named Entity Recognition (NER)

Tags:

2020-01-12 A propos

davidsbatista/BREDS: "Bootstrapping Relationship Extractors with Distributional Semantics" (Batista et al., 2015) - code for EMNLP'15 paper

Tags:

2020-01-11 A propos

Tags:

2020-01-10 A propos

[2001.01447] Improving Entity Linking by Modeling Latent Entity Type Information

Tags:

2020-01-09 A propos

Interpretable Named entity recognition with keras and LIME – Depends on the definition

Tags:

2020-01-09 A propos

Named Entity Recognition with Bert – Depends on the definition

Tags:

2020-01-09 A propos

Joint Intent Classification and Slot Filling with Transformers (Jupyter Notebook Viewer)

Tags:

2020-01-09 A propos

[1902.10909] BERT for Joint Intent Classification and Slot Filling

Tags:

2020-01-09 A propos

Natural Language Understanding with Sequence to Sequence Models

Tags:

2020-01-09 A propos

How to build deep neural network for custom NER with Keras

Tags:

2020-01-07 A propos

Lecture 14 – Contextual Vectors | Stanford CS224U: Natural Language Understanding | Spring 2019

Tags:

2020-01-05 A propos

Custom Named Entity Recognition Using spaCy - Towards Data Science

Tags:

2019-12-31 A propos

NER algo benchmark: spaCy, Flair, m-BERT and camemBERT on anonymizing French commercial legal cases

Tags:

2019-12-17 A propos

Named Entity Recognition with Pytorch Transformers – Pierre-Yves Vandenbussche

Tags:

2019-12-11 A propos - Copie locale

Meetup Paris #40 : Beyond plain text: elasticsearch’s annotated text field type (en anglais) - YouTube

Tags:

2019-11-24 A propos

Clustering of semantically enriched short texts (2019)

Tags:

2019-11-15 A propos

Efficient multi-lingual language model fine-tuning · fast.ai NLP

Tags:

2019-10-22 A propos

Feature-wise transformations. A simple and surprisingly effective family of conditioning mechanisms. (2018)

Tags:

2019-10-07 A propos

Baselines and Bigrams: Simple, Good Sentiment and Topic Classification. Sida Wang and Christopher D. Manning

Tags:

2019-09-03 A propos

[1908.08983] A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers

Tags:

2019-08-28 A propos

[1908.10084] Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Tags:

2019-08-28 A propos

Open-sourcing hyperparameter autotuning for fastText

Tags:

FastText

2019-08-27 A propos

4th Workshop on Representation Learning for NLP

Tags:

2019-08-12 A propos

Learning Text Similarity with Siamese Recurrent Networks (2016)

Tags:

2019-08-07 A propos

Classifying duplicate questions from Quora with Keras | R-bloggers

Tags:

2019-07-03 A propos

Tags:

2019-07-02 A propos

20% Accuracy Bump in Text Classification with ME-ULMFiT

Tags:

2019-06-23 A propos

[1905.10070] Label-aware Document Representation via Hybrid Attention for Extreme Multi-Label Text Classification

Tags:

2019-06-22 A propos

Visual and conceptual grounding for text representation learning

Tags:

2019-06-06 A propos

Knowledge-Based Short Text Categorization Using Entity and Category Embedding | Springer for Research & Development (2019)

Tags:

2019-05-30 A propos

A Technique for Building NLP Classifiers Efficiently with Transfer Learning and Weak Supervision

Tags:

2019-05-22 A propos

Combining Knowledge with Deep Convolutional Neural Networks for Short Text Classification (2017)

Tags:

2019-05-20 A propos

[1812.09449] A Survey on Deep Learning for Named Entity Recognition

Tags:

2019-04-24 A propos

[1904.08398] DocBERT: BERT for Document Classification

Tags:

2019-04-18 A propos

Phoneme Recognition (caveat emptor) – CMUSphinx Open Source Speech Recognition

Tags:

2019-04-16 A propos

[1806.04411] Named Entity Recognition with Extremely Limited Data

Tags:

2019-04-11 A propos

Evaluating Solutions for Named Entity Recognition | Novetta.com (2018)

Tags:

2019-04-11 A propos

A machine learning model to understand fancy abbreviations, trained on Tolkien

Tags:

2019-04-03 A propos

A SIMPLE ALGORITHM FOR IDENTIFYING ABBREVIATION DEFINITIONS IN BIOMEDICAL TEXT

Tags:

2019-04-03 A propos

nlp - Abbreviation detection - Stack Overflow

Tags:

Acronyms (NLP)

2019-04-03 A propos

How Search Engines Might Expand Abbreviations in Queries - SEO by the Sea ⚓

Tags:

Acronyms (NLP)

2019-04-03 A propos

Acronym Recognition - Recognizing acronyms in Swedish texts

Tags:

Acronyms (NLP)

2019-04-03 A propos

Sent2Vec: An unsupervised approach towards learning sentence embeddings | RARE Technologies

Tags:

2019-03-25 A propos

[1902.11269] Efficient Contextual Representation Learning Without Softmax Layer

Tags:

**how to accelerate contextual representation learning**.

> Contextual representation models are difficult to train due to the large parameter sizes and high computational complexity

> We find that the softmax layer (the output layer) causes significant inefficiency due to the large vocabulary size.
Therefore, we redesign the learning objectiv.
> Specifically, the proposed approach bypasses the softmax layer by performing language modeling with dimension reduction, and allows the models to leverage pre-trained word embeddings.
Our framework reduces the time spent on the output layer to a negligible level, eliminates almost all the trainable parameters of the softmax layer and performs language modeling without truncating the vocabulary.
When applied to ELMo, our method achieves a 4 times speedup and eliminates 80% trainable parameters while achieving competitive performance on downstream tasks.

**decouples learning contexts and words**

> Instead of using
a softmax layer to predict the distribution of the
missing word, we utilize and extend the SEMFIT
layer (Kumar and Tsvetkov, 2018) to **predict the
embedding of the missing word**.

2019-03-02 A propos

[1902.05196] Categorical Metadata Representation for Customized Text Classification

Tags:

2019-02-18 A propos

Keywords2vec

Tags:

2019-02-09 A propos

nlp-notebooks/Simple Sentence Similarity.ipynb at master · nlptown/nlp-notebooks

Tags:

2019-02-07 A propos

Enhancing Binary Classification by Modeling Uncertain Boundary in Three-Way Decisions - IEEE Journals & Magazine

Tags:

Three-way decisions

2019-02-02 A propos

[1711.09677] Binary classification models with "Uncertain" predictions

Tags:

2019-02-02 A propos

Cheatsheet · fastText

Tags:

2019-01-29 A propos

Using FastText models (not vectors) for robust embeddings | Kaggle

Tags:

2019-01-29 A propos

[1601.01343] Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation

Tags:

2019-01-27 A propos

[1812.04616] Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs

Tags:

2018-12-14 A propos

The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar

Tags:

2018-12-03 A propos

Tutorial on Text Classification (NLP) using ULMFiT and fastai Library in Python - Analytics Vidhya

Tags:

2018-12-01 A propos

[1811.05370] Unsupervised Transfer Learning for Spoken Language Understanding in Intelligent Agents

Tags:

2018-11-20 A propos

Associative Multichannel Autoencoder for Multimodal Word Representation (2018)

Tags:

2018-11-15 A propos

iliaschalkidis/ELMo-keras: Re-implementation of ELMo on Keras

Tags:

2018-11-14 A propos

Word Mover's Embedding: From Word2Vec to Document Embedding (2018)

Tags:

2018-11-10 A propos

Encoder-Decoder Recurrent Neural Network Models for Neural Machine Translation

Tags:

2018-11-07 A propos

Practical Text Classification With Python and Keras – Real Python

Tags:

2018-10-25 A propos

Effect of Non-linear Deep Architecture in Sequence Labeling

Tags:

2018-10-22 A propos

Deep Learning for Named Entity Recognition #1: Public Datasets and Annotation Methods

Tags:

Named Entity Recognition

2018-10-15 A propos

[1810.04805] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Tags:

2018-10-12 A propos

[1710.06632] Towards a Seamless Integration of Word Senses into Downstream NLP Applications

Tags:

2018-10-09 A propos

Unsupervised Text Summarization using Sentence Embeddings

Tags:

2018-10-09 A propos

[1704.05358] Representing Sentences as Low-Rank Subspaces

Tags:

2018-10-06 A propos

Linear algebraic structure of word meanings – Off the convex path

Tags:

2018-09-20 A propos

A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors (2018)

Tags:

A La Carte Embedding

2018-09-18 A propos

Simple and efficient semantic embeddings for rare words, n-grams, and language features – Off the convex path

Tags:

2018-09-18 A propos

Named Entity Recognition and Classification with Scikit-Learn

Tags:

2018-09-16 A propos

[Learning Note] StarSpace For Multi-label Text Classification

Tags:

2018-09-11 A propos

AI Can Recognize Images, But Text Has Been Tricky—Until Now | WIRED

Tags:

2018-09-08 A propos

[1809.00782] Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text

Tags:

2018-09-06 A propos

[1601.03764] Linear Algebraic Structure of Word Senses, with Applications to Polysemy

Tags:

> Here it is shown that multiple word senses reside
in linear superposition within the word
embedding and simple sparse coding can recover
vectors that approximately capture the
senses

> Each extracted word sense is accompanied by one of about  2000 “discourse atoms” that gives a succinct description of which other words co-occur with that word sense.

> The success of the approach is mathematically explained using a variant of
the random walk on discourses model

("random walk": a generative model for language). Under the assumptions of this model,  there
exists a linear relationship between the vector of a
word w and the vectors of the words in its contexts (It is not the average of the words in w's context, but in a given corpus the matrix of the linear relationship does not depend on w. It can be estimated, and so we can compute the embedding of a word from the contexts it belongs to)

[Related blog post](/doc/?uri=https%3A%2F%2Fwww.offconvex.org%2F2016%2F07%2F10%2Fembeddingspolysemy%2F)

2018-08-28 A propos

Representations for Language: From Word Embeddings to Sentence Meanings (2017) - Slides

Tags:

2018-08-28 A propos

Text feature extraction based on deep learning: a review (2017)

Tags:

2018-08-13 A propos

A Framework for Semi supervised Concept Extraction from MOOC content (2017)

Tags:

2018-08-12 A propos

Supplementary : Extreme Multi-label Learning with Label Features for Warm-start Tagging, Ranking & Recommendation

Tags:

2018-08-07 A propos

Practical guide to text classification | Google Developers

Tags:

2018-07-23 A propos

A Named Entity Recognition Shootout for German (2018)

Tags:

Named Entity Recognition

2018-07-12 A propos

[1806.04470] Design Challenges and Misconceptions in Neural Sequence Labeling

Tags:

2018-06-28 A propos

Training Classifiers with Natural Language Explanations

Tags:

2018-06-23 A propos

[1806.06259] Evaluation of sentence embeddings in downstream and linguistic probing tasks

Tags:

2018-06-19 A propos

A Tri-Partite Neural Document Language Model for Semantic Information Retrieval (2018 - ESWC conference)

Tags:

2018-06-08 A propos

A Word Embedding Approach to Predicting the Compositionality of Multiword Expressions (2015)

Tags:

2018-06-08 A propos

Deep Unordered Composition Rivals Syntactic Methods for Text Classification (2015)

Tags:

2018-05-29 A propos

Representations for Language: From Word Embeddings to Sentence Meanings (2017) - YouTube

Tags:

2018-05-29 A propos

Comparing Sentence Similarity Methods

Tags:

2018-05-25 A propos

A simple spell checker built from word vectors – Ed Rushton – Medium

Tags:

2018-05-25 A propos

[1404.5367] Lexicon Infused Phrase Embeddings for Named Entity Resolution

Tags:

2018-05-22 A propos

BiLSTM-CNN-CRF Implementation for Sequence Tagging

Tags:

2018-05-21 A propos

Named Entity Recognition and the Road to Deep Learning (2017)

Tags:

> the old
and the new-style NLP are not diametrically
opposed: just as it is possible (and useful!) to
incorporate neural-network features into a CRF,
CRFs have influenced some of the best deep
learning models for sequence labelling

This blog post go through the ways of doing NER, starting with CRF:

- When you develop a CRF,
a lot of time goes into finding feature functions (Does a word start with a capital? Is it uppercase? Is it a digit?...) 
- Pb: semantic similarity between words. 
- Gazetteers: lists with names of people, locations and organizations that are known in advance.
- feed Word Embeddings to a CRF: one way is to cluster a set
of word embeddings by distributional similarity, and
provide the CRF with the cluster IDs of a token and its
context words.
- Use word and character based embeddings
- LSTM not good enough -> biLSTM
- **biLSTM predict all labels independently of each other -> add a CRF layer** which outputs a matrix of transition scores between two states: dynamic programming can help find the optimal tag sequence for the sentence

2018-05-20 A propos

Named Entity Recognition and the Stanford NER Software (slides)

Tags:

2018-05-20 A propos

Named Entity Recognition: Milestone Papers, Models and Technologies (2017)

Tags:

2018-05-20 A propos

Stanford Named Entity Recognizer

Tags:

Stanford NER

2018-05-20 A propos

How to Train your Own Model with NLTK and Stanford NER Tagger? (for English, French, German…)

Tags:

2018-05-20 A propos

Tutorial on Poincaré Embeddings (Jupyter Notebook )

Tags:

2018-05-20 A propos

Implementing Poincaré Embeddings | RARE Technologies

Tags:

2018-05-20 A propos

NLP: Any libraries/dictionaries out there for fixing common spelling errors? - Part 2 & Alumni - Deep Learning Course Forums

Tags:

2018-05-18 A propos

[1607.07956] Joint Embedding of Hierarchical Categories and Entities for Concept Categorization and Dataless Classification (COLING 2016)

Tags:

2018-05-12 A propos

An Exploration of Embeddings for Generalized Phrases (2014)

Tags:

2018-05-12 A propos

Under the hood: Multilingual embeddings | Engineering Blog | Facebook Code

Tags:

2018-05-11 A propos

GitHub - Babylonpartners/fastText_multilingual: Multilingual word vectors

Tags:

2018-05-11 A propos

Weakly-supervised Relation Extraction by Pattern-enhanced Embedding Learning

Tags:

2018-05-10 A propos

Text Classification with TensorFlow Estimators

Tags:

2018-04-17 A propos

Part-of-Speech tagging tutorial with the Keras Deep Learning library - Cdiscount TechBlog

Tags:

2018-04-13 A propos

Sense2vec with spaCy and Gensim · Blog · Explosion AI

Tags:

2018-04-08 A propos

Exploring neural architectures for NER (CS224N 2018)

Tags:

2018-04-05 A propos

SPACY'S ENTITY RECOGNITION MODEL: incremental parsing with Bloom embeddings & residual CNNs - YouTube

Tags:

2018-03-06 A propos

Effective Word Representation for Named Entity Recognition (2017)

Tags:

2018-03-06 A propos

Deep learning with word embeddings improves biomedical named entity recognition | Bioinformatics | Oxford Academic (2017)

Tags:

2018-03-05 A propos

[1508.01991] Bidirectional LSTM-CRF Models for Sequence Tagging

Tags:

2018-03-05 A propos

Sequence Tagging with Tensorflow

Tags:

2018-03-05 A propos

[1603.01360] Neural Architectures for Named Entity Recognition

Tags:

2018-03-05 A propos

Examination of machine learning methods for multi-label classification of intellectual property documents (2017)

Tags:

2018-03-05 A propos

A survey of named entity recognition and classification (2006)

Tags:

Named Entity Recognition

2018-03-05 A propos

Extreme Multi-label Learning with Label Features for Warm-start Tagging, Ranking & Recommendation (2018)

Tags:

2018-03-04 A propos

Intro to text classification with Keras: automatically tagging Stack Overflow posts | Google Cloud Big Data and Machine Learning Blog

Tags:

2018-03-04 A propos

A Benchmark of Text Classification in PyTorch

Tags:

2018-02-28 A propos

ELMo: Deep contextualized word representations (2018)

Tags:

ELMo

2018-02-16 A propos

[1710.04099] Wembedder: Wikidata entity embedding web service

Tags:

2018-02-13 A propos

Evaluating the Impact of Word Embeddings on Similarity Scoring in Practical Information Retrieval (2017)

Tags:

> Transferring the success of word embeddings to Information Retrieval (IR) task is currently an active research topic. While embedding-based retrieval models could tackle the vocabulary mismatch problem by making use of the embedding’s inherent similarity between distinct words, most of them struggle to compete with the prevalent strong baselines such as TF-IDF and BM25.

Considering a practical ad-hoc IR task composed of two steps, matching and scoring, compares the performance of several techniques that leverage word embeddings in the retrieval models to compute the similarity between the query and the documents (namely word centroid similarity, paragraph vectors, Word Mover’s distance, as well as a novel inverse document frequency (IDF) re-weighted word centroid similarity).

> We confirm that word embeddings can be successfully employed in a practical information retrieval setting. The proposed cosine similarity of IDF re-weighted, aggregated word vectors is competitive to the TF-IDF baseline.

2018-01-28 A propos

RESEARCH TRACK: Web Content Analysis, Semantics and Knowledge

Tags:

[CFP](https://www2018.thewebconf.org/call-for-papers/research-tracks-cfp/web-content-analysis/)

> In previous years, ‘content analysis’ and ‘semantic and knowledge’ were in separate track. This year, we combined these tracks to emphasize the close relationship between these topics; **the use of content to curate knowledge and the use of knowledge to guide content analysis and intelligent usage**.

Some of the accepted papers:
### [Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN](https://doi.org/10.1145/3178876.3186005)

[Hierarchical Text Classification](/tag/nlp_hierarchical_text_classification): Text classification to a hierarchical taxonomy of topics, using graph representation of text, and CNN over this graph

Renvoie à ce qui a été vu dans le tutorial "Graph-based Text Representations"

from the abstract:

> a graph-CNN based deep learning model to first convert texts to graph-of-words, and then use graph convolution operations to convolve the word graph. Graph-of-words representation of texts has the advantage of capturing non-consecutive and long-distance semantics. CNN models have the advantage of learning different level of semantics. To further leverage the hierarchy of labels, we regularize the deep architecture with the dependency among labels

Conversion of text to graph: potentially given a single document

### [Weakly-supervised Relation Extraction by Pattern-enhanced Embedding Learning](https://doi.org/10.1145/3178876.3186024 )

Extraction de relations de corpus de textes de façon semi-supervisée, dans un contexte où on a peu de données labellisées décrivant les relations.

Par exemple, des données labellisées indique que le texte "Beijing, capital of China" correspond à la relation entre entités : ("Beijing", "Capital Of", "China), et on voudrait pouvoir extraire les entités et relations pertinentes à partir de texte tel que "Paris, France's capital,..."

Le papier décrit une méthode qui combine deux modules, l'un basé sur l'extraction automatique de patterns (par ex "[Head], Capital Of [Tail]") et l'autre sur la "sémantique distributionnelle" (du type "word embeddings"). Ces deux modules collaborent, le premier permettant de créer des instances de relations augmentant la base de connaissance sur lequel entrainer le second, et le second aidant le premier à déterminer des patterns informatifs ("co-entrainement")

### [Scalable Instance Reconstruction in Knowledge Bases via Relatedness Affiliated Embedding](https://doi.org/10.1145/3178876.3186017)

Knowledge base completion problem: usually, it is formulated as a link prediction problem, but not here. A novel knowledge embedding model ("Joint Modelling and Learning of Relatedness and Embedding")

### [Improving Word Embedding Compositionality using Lexicographic Definitions](https://doi.org/10.1145/3178876.3186007)

comment obtenir les meilleures représentations de texte à partir de représentations de mots (word embeddings) ? L'auteur utilise des ressources lexicographiques (wordnet) pour ses tests : l'embedding obtenu pour la définition d'un mot est-il proche de celui du mot ?

Le papier s'appuie sur une [thèse du même auteur](https://esc.fnwi.uva.nl/thesis/centraal/files/f1554608041.pdf), claire et bien écrite.

### [CESI: Canonicalizing Open Knowledge Bases using Embeddings and Side Information](https://doi.org/10.1145/3178876.3186030)

Amélioration de l'extraction de triplets (nom phrase, property, nom phrase) à partir de texte en calculant des embeddings pour les "nom phrases" (~entités)

### [Short-Text Topic Modeling via Non-negative Matrix Factorization Enriched with Local Word-Context Correlations](https://doi.org/10.1145/3178876.3186009)

Topic modeling for short texts, leveraging the word-context semantic correlations in the training

### [Towards Annotating Relational Data on the Web with Language Models](https://doi.org/10.1145/3178876.3186029)

### A paper by [David Blei](/tag/david_blei): (Dynamic Embeddings for Language Evolution)

2018-01-27 A propos

[1801.06146] Universal Language Model Fine-tuning for Text Classification

Tags:

2018-01-19 A propos

Combining word and entity embeddings for entity linking (ESWC 2017)

Tags:

2017-12-30 A propos

[1712.09405] Advances in Pre-Training Distributed Word Representations

Tags:

2017-12-29 A propos

gensim/WMD_tutorial.ipynb

Tags:

2017-12-23 A propos

facebookresearch/MUSE: A library for Multilingual Unsupervised or Supervised word Embeddings

Tags:

2017-12-22 A propos

LDA2vec: Word Embeddings in Topic Models – Towards Data Science

Tags:

LDA2vec

2017-12-11 A propos

(fastText) Euclidean distance instead of cosine-similarity?

Tags:

2017-12-07 A propos

machine learning - Text categorization: combining different kind of features - Data Science Stack Exchange

Tags:

2017-12-06 A propos

[1506.08422] Topic2Vec: Learning Distributed Representations of Topics

Tags:

2017-12-03 A propos

How can we get the vector of a paragraph? · Issue #26 · facebookresearch/fastText

Tags:

2017-11-25 A propos

How to use pre-trained word vectors from Facebook’s fastText

Tags:

2017-11-25 A propos

DS Toolbox - Topic Models - DS lore

Tags:

2017-11-21 A propos

Gaussian LDA for Topic Models with Word Embeddings (2015)

Tags:

2017-11-21 A propos

Introducing our Hybrid lda2vec Algorithm (2017) | Stitch Fix Technology – Multithreaded

Tags:

2017-11-21 A propos

Calculates Word Mover's Distance Insanely Fast

Tags:

2017-11-12 A propos

Word Mover’s Distance in Python

Tags:

Word Mover’s Distance

2017-11-12 A propos

Monolingual and Cross-Lingual Information Retrieval Models Based on (Bilingual) Word Embeddings (2015)

Tags:

2017-11-12 A propos

A Comparison of Vector-based Representations for Semantic Composition (Blacoe and Lapata - 2012)

Tags:

2017-11-12 A propos

Modifications for the Cluster Content Discovery and the Cluster Label Induction Phases of the Lingo Algorithm (2014)

Tags:

Lingo

2017-11-11 A propos

[1602.06797] Semi-supervised Clustering for Short Text via Deep Representation Learning

Tags:

2017-11-07 A propos

Semantic Enriched Short Text Clustering | SpringerLink

Tags:

Short Text Clustering

2017-11-07 A propos

Convolutional Neural Networks for Sentence Classification (2014)

Tags:

2017-11-07 A propos

Implementing a CNN for Text Classification in TensorFlow – WildML

Tags:

2017-11-06 A propos

Recurrent Convolutional Neural Networks for Text Classification (S Lai - ‎2015)

Tags:

2017-11-06 A propos

[1701.00185] Self-Taught Convolutional Neural Networks for Short Text Clustering

Tags:

2017-11-04 A propos

Spell Checker using Word2vec | Kaggle

Tags:

2017-11-03 A propos

Using Gensim Word2Vec Embeddings in Keras | Ben Bolte's Blog

Tags:

2017-10-23 A propos

Using pre-trained word embeddings in a Keras model

Tags:

2017-10-23 A propos

Towards a Seamless Integration of Word Senses into Downstream NLP Applications (2017)

Tags:

2017-10-21 A propos

[1710.04087] Word Translation Without Parallel Data

Tags:

2017-10-14 A propos

Efficient unsupervised keywords extraction using graphs

Tags:

2017-10-04 A propos

Named Entity Recognition using Word Embedding as a Feature (2016)

Tags:

2017-10-01 A propos

[1607.01759] Bag of Tricks for Efficient Text Classification

Tags:

2017-09-10 A propos

[1703.00993] A Comparative Study of Word Embeddings for Reading Comprehension

Tags:

2017-08-28 A propos

Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification - ScienceDirect

Tags:

2017-07-21 A propos

What is a simple but detailed explanation of Textrank? - Quora

Tags:

TextRank

2017-07-12 A propos

How does Textrank work? (slides)

Tags:

TextRank

2017-07-12 A propos

Dive Into NLTK, Part V: Using Stanford Text Analysis Tools in Python – Text Mining Online

Tags:

2017-07-11 A propos

NLTK: Installing Third Party Software · nltk Wiki

Tags:

2017-07-11 A propos

Lingo: Search Results Clustering Algorithm Based on Singular Value Decomposition (slides)

Tags:

2017-07-11 A propos

Lingo: Search Results Clustering Algorithm Based on Singular Value Decomposition (2004) (paper)

Tags:

2017-07-11 A propos

Source code for nltk.tag.stanford — NLTK documentation

Tags:

2017-07-11 A propos

RDRPOSTagger: A Rule-based Part-of-Speech and Morphological Tagging Toolkit

Tags:

2017-07-11 A propos

TreeTagger - a part-of-speech tagger for many languages

Tags:

2017-07-11 A propos

nltk.tag.stanford module — NLTK documentation

Tags:

2017-07-11 A propos

Stanford Log-linear Part-Of-Speech Tagger

Tags:

Stanford POS Tagger

2017-07-11 A propos

What are all possible pos tags of NLTK? - Stack Overflow

Tags:

2017-07-11 A propos

Penn Treebank P.O.S. Tags

Tags:

Part Of Speech Tagging

2017-07-11 A propos

Hierarchical clustering in Python and beyond

Tags:

2017-07-11 A propos

How can I tag and chunk French text using NLTK and Python? - Stack Overflow

Tags:

2017-06-29 A propos

Document Clustering with Python

Tags:

2017-06-28 A propos

gensim/FastText_Tutorial.ipynb

Tags:

2017-06-28 A propos

facebookresearch/fastText: Library for fast text representation and classification.

Tags:

2017-06-28 A propos

A Ranking Approach to Keyphrase Extraction - Microsoft Research (2009)

Tags:

2017-06-27 A propos

Document Representation and Dimension Reduction for Text Clustering

Tags:

2017-06-24 A propos

TextRank: Bringing Order into Texts (2004)

Tags:

2017-06-14 A propos

Automated Keyword Extraction – TF-IDF, RAKE, and TextRank (Less Than Dot - Blog)

Tags:

2017-06-13 A propos

An Intuitive Understanding of Word Embeddings: From Count Vectors to Word2Vec

Tags:

2017-06-09 A propos

word2vec-api

Tags:

2017-06-09 A propos

[1607.00570] Representation learning for very short texts using weighted word embedding aggregation

Tags:

2017-06-09 A propos

[1512.00765] Learning Semantic Similarity for Very Short Texts

Tags:

2017-06-09 A propos

From Word Embeddings To Document Distances (Kusner 2015)

Tags:

Word Mover’s Distance

2017-06-09 A propos

Topic Modeling for Short Texts with Auxiliary Word Embeddings

Tags:

2017-06-08 A propos

[1609.08496] Topic Modeling over Short Texts by Incorporating Word Embeddings

Tags:

2017-06-07 A propos

Lingo3G or Carrot2? | Carrot Search

Tags:

Carrot2

2017-06-04 A propos

Text Summarization with Gensim

Tags:

2017-06-02 A propos

gensim: models.word2vec – Deep learning with word2vec

Tags:

2017-06-01 A propos

Word2vec in gensim Tutorial | RaRe Technologies

Tags:

2017-06-01 A propos

An Efficient Way to Extract the Main Topics from a Sentence | The Tokenizer

Tags:

2017-05-24 A propos

Extract Subject Matter of Documents Using NLP – Alexander Crosson – Medium

Tags:

2017-05-24 A propos

Carrot2 manual

Tags:

Carrot2

2017-05-23 A propos

Carrot2: Text Clustering Algorithms and Applications

Tags:

2017-05-23 A propos

Result Clustering - Apache Solr Reference Guide - Apache Software Foundation

Tags:

2017-05-23 A propos

datquocnguyen/LFTM: Improving Topic Models with Latent Feature Word Representations (GitHub)

Tags:

2017-05-22 A propos

Lingo3G: real-time text clustering engine | Carrot Search

Tags:

2017-05-22 A propos

What are the best open source tools for unsupervised clustering of text documents? - Quora

Tags:

Clustering of text documents

2017-05-22 A propos

Improving Topic Models with Latent Feature Word Representations (slides)

Tags:

Topic Models + Word embedding

2017-05-20 A propos

Improving Topic Models with Latent Feature Word Representations | Nguyen | Transactions of the Association for Computational Linguistics

Tags:

2017-05-20 A propos

Using Word2Vec for topic modeling - Stack Overflow

Tags:

2017-05-19 A propos

Text Classification With Word2Vec - DS lore (2016)

Tags:

2017-05-18 A propos

Tags:

2017-05-18 A propos

Quick review on Text Clustering and Text Similarity Approaches

Tags:

Author: Maali Mnasri (PhD @ CEA)

First transform text units to vectors? not always (eg. sentence similarity task using lexical word alignment). But vectors are efficient to process, and benefit from existing clustering algorithms such as k-means.

Sentence level or document level? Sentence clustering to summarise large documents.

Thematic clustering vs Semantic clustering: depends on the similarity measure.

Text similarity measures:

- Cosine similarity of tf-idf (suitable to produce thematic clusters)
- Knowledge-based Measures (wordNet) (quantify semantic relatedness of words),
- Word embedings

Examples, sample code:

- using wordnet with NLTK, and the formula to compute sentence similarities from word similarities.
- computing similarities between docs using gensim/word2vec

Which clustering algorithm?

- when we have an approximation of the clusters number, and when the similarity measure is not expensive in terms of computation time, clustering algo are suitable and fast. Sample code of k-means clustering using tf-idf vectors with scikit-learn
- Hierarchical clustering algorithms
    - don't need to give the number of clusters
    - but time consuming (calculate a similarity matrix for the sentences) 
- for voluminous data, use an incremental clustering algorithm: sentences are processed one at a time ; each new sentence is compared to each of the already formed clusters.

2017-05-18 A propos

Create a Domain Text Classifier Using Cognonto | Frederick Giasson

Tags:

2016-10-25 A propos

Highly discriminative statistical features for email classification

Tags:

2015-10-20 A propos

Challenges of the email domain for text classification

Tags:

2015-10-20 A propos

Dexter, an Open Source Framework for Entity Linking

Tags:

2014-10-20 A propos

Extraction de la semantique

Tags:

2014-06-18 A propos

Hierarchical classification: Combining Bayes with SVM

Tags:

2014-05-18 A propos

Efficient Multi-label Classification with Many Labels (2013)

Tags:

2014-04-25 A propos

How to do text classification with label probabilities? - Stack Overflow

Tags:

2014-04-25 A propos

Sapping Attention: When you have a MALLET, everything looks like a nail

Tags:

2014-04-25 A propos

Provable Algorithms for Machine Learning Problems by Rong Ge.

Tags:

2014-04-23 A propos

nltk - hierarchical classification + topic model training data for internet articles and social media - Stack Overflow

Tags:

2014-04-23 A propos

Machine Learning Tutorial: The Max Entropy Text Classifier | DatumBox

Tags:

2014-04-08 A propos

Inductive learning algorithms and representations for text categorization

Tags:

Text Classification

2014-04-08 A propos

Text categorization - Scholarpedia

Tags:

Text Classification

2014-04-08 A propos

Text classification using string kernels

Tags:

Text Classification

2014-04-08 A propos

Learning Multilabel classification of news articles (2013)

Tags:

Multi-label Text classification

2014-04-08 A propos

java - Method(s) to output confidence score from Stanford Classifier? - Stack Overflow

Tags:

2014-04-08 A propos

[java-nlp-user] Stanford NER: confidence scores

Tags:

Stanford classifier

2014-04-08 A propos

10 Tips to Improve your Text Classification Algorithm Accuracy and Performance | Thinknook

Tags:

2014-04-07 A propos

Bing - Knowledge Widget (Beta)

Tags:

2014-04-04 A propos

Data Categorization using OpenNLP

Tags:

2014-03-27 A propos

Welcome to LibShortText documentation! — LibShortText 1.1 documentation

Tags:

LibShortText

2014-03-26 A propos

H.-F. Yu, C.-H. Ho, Y.-C. Juan, and C.-J. Lin. LibShortText: A Library for Short-text Classification and Analysis

Tags:

LibShortText

2014-03-26 A propos

LibShortText: A Library for Short-text Classification and Analysis

Tags:

LibShortText

2014-03-25 A propos

The Stanford classifier

Tags:

Stanford classifier

2014-03-16 A propos

python - Classifying Documents into Categories - Stack Overflow

Tags:

2014-03-15 A propos

nlp - Feature Selection and Reduction for Text Classification - Stack Overflow

Tags:

2014-03-15 A propos

Context-based Entity Linking | Blog | graus.nu

Tags:

Named Entity Recognition

2013-07-18 A propos

NERD: Named Entity Recognition and Disambiguation

Tags:

2013-07-10 A propos

Wikilinks - Information Extraction and Synthesis Laboratory

Tags:

Wikilinks Corpus

2013-03-12 A propos

Learning from Big Data: 40 Million Entities in Context

Tags:

Wikilinks Corpus

2013-03-12 A propos

Scientists See Advances in Deep Learning, a Part of Artificial Intelligence - NYTimes.com

Tags:

2012-11-30 A propos

Duped by Dupes | Wavii Blog

Tags:

2012-08-25 A propos

LINDEN: Linking Named Entities with Knowledge Base via Semantic Knowledge

Tags:

2012-04-19 A propos

A Spectrometry of Linked Data

Tags:

Entity mining is still a troublesome open problem. In past years many approaches allowed to automate the generation of equivalence links between references using schema matching or various heuristics based on the recognition of similar property values. In contrast, few of them considered the analysis of the network of equivalence links (“equivalence network”) as an indication of the likelihood and strength of the equivalence.

Could a URI reference (URIRef) be thought as exactly “attached” to its referent? Could it make sense to talk about entity “identifiers” or would it be better to talk about more ambiguous “references”, i.e., placeholders for any model that satisfies the formal semantics of the Semantic Web (Hayes)? Booth observes that the aforementioned question, which in the past has been often regarded as fundamental in the debate about identity on the Web, is relatively unimportant. As long as an entity, identified by whatsoever URIRef, is associated to at least one description containing machine understandable information, this information can be automatically processed and used by applications.

2012-04-16 A propos