Semanlink - Favoris ; Entreprise AND Information extraction

> We show that it is feasible to perform **entity
linking by training a dual encoder (two-tower)
model that encodes mentions and entities in
the same dense vector space**, where candidate
entities are retrieved by approximate nearest
neighbor search. Unlike prior work, **this setup
does not rely on an alias table followed by a
re-ranker, and is thus the first fully learned entity
retrieval model**.

Contributions:

> -  a dual encoder architecture for
learning entity and mention encodings suitable for
retrieval. A key feature of the architecture is that it
employs a modular **hierarchy of sub-encoders that
capture different aspects of mentions and entities**
> - a simple, fully unsupervised **hard negative
mining** strategy that produces massive gains
in retrieval performance, compared to using only
random negatives
> - high
quality candidate entities very efficiently using approximate nearest neighbor search
> - outperforms discrete retrieval
baselines like an alias table or BM25

> strong retrieval
performance across all 5.7 million Wikipedia entities in
around 3ms per mention

> since we are using a two-tower or dual
encoder architecture, **our model cannot use any kind of attention over
both mentions and entities at once**, nor feature-wise
comparisons as done by Francis-Landau et al. (2016).
This is a fairly severe constraint – for example, **we cannot
directly compare the mention span to the entity title**
– but it permits retrieval with nearest neighbor search
for the entire context against a single, all encompassing
representation for each entity

2021-05-01 About

asahi417/tner: Language model finetuning on NER

Tags:

2021-03-04 About

kamalkraj/BERT-NER: Pytorch-Named-Entity-Recognition-with-BERT

Tags:

2021-02-07 About

Linking Entities with Knowledge Graphs | by Sigurd Berglann | Strise | Medium

Tags:

2021-02-05 About

Custom NLP Approaches to Data Anonymization | by Omri Mendels | Towards Data Science

Tags:

2021-02-01 About

[1903.11279] Graph Convolution for Multimodal Information Extraction from Visually Rich Documents

Tags:

2020-06-16 About

Representation Learning for Information Extraction from Form-like Documents – Google Research

Tags:

2020-06-15 About

Google AI Blog: Extracting Structured Data from Templatic Documents (2020)

Tags:

[About this paper](doc:2020/06/representation_learning_for_inf)

Templatic documents (eg. invoices): such documents do not contain “natural
language” but
instead resemble forms, with data often presented in tables

> an approach that **uses knowledge of target field types to identify
candidate fields**. These are then scored using **a neural network that
learns a dense representation of each candidate using the words in its
neighborhood**. Experiments on two corpora (invoices and receipts) show
that we’re able to generalize well to unseen layouts.
>
> An understanding of the **two-dimensional layout of text**
on the page is key to understanding such documents. On the other hand,
treating this purely as an image segmentation problem makes it difficult
to take advantage of the semantics of the text.
>
> Our approach to this problem allows developers to train and deploy an
extraction system for a given domain (like invoices) using **two inputs — a
target schema (i.e., a list of fields to extract and their corresponding
types) and a small collection of documents labeled with the ground truth
for use as a training set**

- The input document is first run through an [OCR service](doc:2020/06/detecter_le_texte_dans_les_fich).
- a candidate generator identifies spans of text in the OCR output that might correspond to
an instance of a given field (uses pre-existing
libraries associated with each field type)
- Each candidate is then scored using a neural
network (that is trained as a binary classifier)

2020-06-15 About

[1911.03814] Scalable Zero-shot Entity Linking with Dense Entity Retrieval

Tags:

2020-05-02 About

Unsupervised NER using BERT - Hands-on NLP model review - Quora

Tags:

2020-03-06 About

[2002.11402] Detecting Potential Topics In News Using BERT, CRF and Wikipedia

Tags:

2020-02-27 About

[2003.05473] Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking (CoNNL 2019)

Tags:

Training BERT-base-uncased on English Wikipedia and then fine-tuned and evaluating it
on an entity linking (EL) benchmark (EL implemented as a token classification over the entity vocabulary)

> BERT+Entity is a straightforward extension on top
of BERT, i.e. we initialize BERT with the publicly
available weights from the BERT-base-uncased
model and add an output classification layer on
top of the architecture. Given a contextualized token,
the classifier computes the probability of an
entity link for each entry in the entity vocabulary.

Can BERT’s architecture learn all entity
linking steps jointly? To answer:

> an extreme
simplification of the **entity linking setup that
works surprisingly well**: simply cast it as **a
per token classification over the entire entity
vocabulary** (over 700K classes in our case).

> the model
is the first that performs entity linking without any
pipeline or any heuristics, compared to all prior
approaches. We found that with our approach we
can learn additional entity knowledge in BERT that
helps in entity linking. **However, we also found
that almost none of the downstream tasks really
required entity knowledge**.

### Related work

- > [Durrett and Klein (2014)](/doc/2020/01/a_joint_model_for_entity_analys) were the first to propose
jointly modelling Mention detection, Candidate generation and Entity disambiguation in a graphical
model and could show that each of those steps are
interdependent and benefit from a joint objective

This paper uses neural techniques instead of CRF.

- > [Yamada](/showprop.do?pptyuri=http%3A%2F%2Fwww.semanlink.net%2F2001%2F00%2Fsemanlink-schema%23arxiv_author&pptyval=Ikuya%2BYamada) (2016, 2017) was the first to
investigate neural text representations and entity
linking, but their approach is limited to ED.

cf. [#Wikipedia2Vec](tag:wikipedia2vec). Compare with [newer work by Yamada](doc:2020/09/1909_01259_neural_attentive_b)

2020-01-09 About

[2001.01447] Improving Entity Linking by Modeling Latent Entity Type Information

Tags:

2020-01-09 About

Named Entity Recognition with Bert – Depends on the definition

Tags: