Semanlink - NLP techniques AND Information extraction

Tags:

2024-04-12 About

Tags:

2024-04-12 About

L. Dietz @deeds@mastodon.social @dietz@idf.social sur X : "How to evaluate RAG / RACAS (Retrieval-Augmented-Clustering-Augmented-Summarization) systems for Article Generation..."

Tags:

2023-10-20 About

peft/examples/token_classification/peft_lora_token_cls.ipynb at main · huggingface/peft

Tags:

2023-08-27 About

[2304.02711] Structured prompt interrogation and recursive extraction of semantics (SPIRES): A method for populating knowledge bases using zero-shot learning

Tags:

2023-04-07 About

[1904.04458] Knowledge-Augmented Language Model and its Application to Unsupervised Named-Entity Recognition

Tags:

Knowledge Augmented
Language Model (KALM)

a language
model with access to information available in a
KB, no assumptions
about the availability of additional components
(such as Named Entity Taggers) or annotations

> While classes of
named entities (e.g., person or location) occur frequently,
each individual name (e.g, Atherton or
Zhouzhuang) may be observed infrequently even
in a very large corpus of text. As a result language
models learn to represent accurately only the most
popular named entities

> knowing that Alice is a name
used to refer to a person should give ample information
about the context in which the word may
occur (e.g., Bob visited Alice).

> ---

> extends a traditional **RNN LM**

> we enhance a traditional LM with a
gating mechanism that controls whether a particular
word is modeled as a general word or as a reference
to an entity
>
> We train the model end-to-end
with only the traditional predictive language modeling
perplexity objective
>
> KALM is trained end-to-end using
a predictive objective on large corpus of text.

> To the best of our knowledge, KALM is the first
unsupervised neural NER approach.

> KALM extends a traditional, RNN-based neural
LM.

2022-08-31 About

[2208.09982] GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarization

Tags:

2022-08-24 About

[2205.00820] Entity-aware Transformers for Entity Search

Tags:

2022-07-12 About

[1909.00426] Global Entity Disambiguation with BERT

Tags:

2022-04-18 About

[1910.06294] Training Compact Models for Low Resource Entity Tagging using Pre-trained Language Models

Tags:

2022-03-31 About

Building Transformer-Based Entity Linking System | by izuna385 | Medium (2021)

Tags:

2022-03-23 About

Studio Ousia sur Twitter : "Now using LUKE is easier than ever!" / Twitter

Tags:

2022-03-15 About

[2108.13934] Robust Retrieval Augmented Generation for Zero-shot Slot Filling

Tags:

2022-01-19 About

Unsupervised_Extractive_Summarization - a Hugging Face Space by Hellisotherpeople

Tags:

2021-12-03 About

kamalkraj/BERT-NER: Pytorch-Named-Entity-Recognition-with-BERT

Tags:

2021-02-07 About

Aran Komatsuzaki sur Twitter : "Big Bird: Transformers for Longer Sequences..."

Tags:

2020-07-29 About

Google AI Blog: Extracting Structured Data from Templatic Documents (2020)

Tags:

[About this paper](doc:2020/06/representation_learning_for_inf)

Templatic documents (eg. invoices): such documents do not contain “natural
language” but
instead resemble forms, with data often presented in tables

> an approach that **uses knowledge of target field types to identify
candidate fields**. These are then scored using **a neural network that
learns a dense representation of each candidate using the words in its
neighborhood**. Experiments on two corpora (invoices and receipts) show
that we’re able to generalize well to unseen layouts.
>
> An understanding of the **two-dimensional layout of text**
on the page is key to understanding such documents. On the other hand,
treating this purely as an image segmentation problem makes it difficult
to take advantage of the semantics of the text.
>
> Our approach to this problem allows developers to train and deploy an
extraction system for a given domain (like invoices) using **two inputs — a
target schema (i.e., a list of fields to extract and their corresponding
types) and a small collection of documents labeled with the ground truth
for use as a training set**

- The input document is first run through an [OCR service](doc:2020/06/detecter_le_texte_dans_les_fich).
- a candidate generator identifies spans of text in the OCR output that might correspond to
an instance of a given field (uses pre-existing
libraries associated with each field type)
- Each candidate is then scored using a neural
network (that is trained as a binary classifier)

2020-06-15 About

[1911.03814] Scalable Zero-shot Entity Linking with Dense Entity Retrieval

Tags:

2020-05-02 About

Sebastian Riedel sur Twitter : Happy to introduce BLINK, the @facebookai open-source entity linker!...

Tags:

2020-05-02 About

Martynas Jusevicius sur Twitter : "Is there a solution for entity recognition that would use a local #KnowledgeGraph to look for matches? Ideally any SPARQL datasource..."

Tags:

2020-03-13 About

AmbiverseNLU: A Natural Language Understanding suite by Max Planck Institute for Informatics

Tags:

2020-03-13 About

Unsupervised NER using BERT - Hands-on NLP model review - Quora

Tags:

2020-03-06 About

[2002.11402] Detecting Potential Topics In News Using BERT, CRF and Wikipedia

Tags:

2020-02-27 About

davidsbatista/BREDS: "Bootstrapping Relationship Extractors with Distributional Semantics" (Batista et al., 2015) - code for EMNLP'15 paper

Tags:

2020-01-11 About

[2003.05473] Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking (CoNNL 2019)

Tags:

Training BERT-base-uncased on English Wikipedia and then fine-tuned and evaluating it
on an entity linking (EL) benchmark (EL implemented as a token classification over the entity vocabulary)

> BERT+Entity is a straightforward extension on top
of BERT, i.e. we initialize BERT with the publicly
available weights from the BERT-base-uncased
model and add an output classification layer on
top of the architecture. Given a contextualized token,
the classifier computes the probability of an
entity link for each entry in the entity vocabulary.

Can BERT’s architecture learn all entity
linking steps jointly? To answer:

> an extreme
simplification of the **entity linking setup that
works surprisingly well**: simply cast it as **a
per token classification over the entire entity
vocabulary** (over 700K classes in our case).

> the model
is the first that performs entity linking without any
pipeline or any heuristics, compared to all prior
approaches. We found that with our approach we
can learn additional entity knowledge in BERT that
helps in entity linking. **However, we also found
that almost none of the downstream tasks really
required entity knowledge**.

### Related work

- > [Durrett and Klein (2014)](/doc/2020/01/a_joint_model_for_entity_analys) were the first to propose
jointly modelling Mention detection, Candidate generation and Entity disambiguation in a graphical
model and could show that each of those steps are
interdependent and benefit from a joint objective

This paper uses neural techniques instead of CRF.

- > [Yamada](/showprop.do?pptyuri=http%3A%2F%2Fwww.semanlink.net%2F2001%2F00%2Fsemanlink-schema%23arxiv_author&pptyval=Ikuya%2BYamada) (2016, 2017) was the first to
investigate neural text representations and entity
linking, but their approach is limited to ED.

cf. [#Wikipedia2Vec](tag:wikipedia2vec). Compare with [newer work by Yamada](doc:2020/09/1909_01259_neural_attentive_b)

2020-01-09 About

Interpretable Named entity recognition with keras and LIME – Depends on the definition

Tags:

2020-01-09 About

Named Entity Recognition with Bert – Depends on the definition

Tags:

2020-01-09 About

Microsoft Concept Graph and Concept Tagging Release

Tags:

Microsoft Concept Graph

2019-05-28 About

Microsoft Concept Graph: Mining Semantic Concepts for Short Text Understanding | MIT Press Journals (2019)

Tags:

2019-05-28 About

[1601.01343] Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation

Tags:

2019-01-27 About

Romain Vial (Hyperlex) at Paris NLP meetup, slides

Tags:

2019-01-24 About

Unsupervised Text Summarization using Sentence Embeddings

Tags:

2018-10-09 About

[1404.5367] Lexicon Infused Phrase Embeddings for Named Entity Resolution

Tags:

2018-05-22 About

Named Entity Recognition and the Road to Deep Learning (2017)

Tags:

> the old
and the new-style NLP are not diametrically
opposed: just as it is possible (and useful!) to
incorporate neural-network features into a CRF,
CRFs have influenced some of the best deep
learning models for sequence labelling

This blog post go through the ways of doing NER, starting with CRF:

- When you develop a CRF,
a lot of time goes into finding feature functions (Does a word start with a capital? Is it uppercase? Is it a digit?...) 
- Pb: semantic similarity between words. 
- Gazetteers: lists with names of people, locations and organizations that are known in advance.
- feed Word Embeddings to a CRF: one way is to cluster a set
of word embeddings by distributional similarity, and
provide the CRF with the cluster IDs of a token and its
context words.
- Use word and character based embeddings
- LSTM not good enough -> biLSTM
- **biLSTM predict all labels independently of each other -> add a CRF layer** which outputs a matrix of transition scores between two states: dynamic programming can help find the optimal tag sequence for the sentence

2018-05-20 About

Deep learning with word embeddings improves biomedical named entity recognition | Bioinformatics | Oxford Academic (2017)

Tags:

2018-03-05 About

Combining word and entity embeddings for entity linking (ESWC 2017)

Tags: