Semanlink - Favoris ; AI girls and guys ; Artificial Intelligence AND Entities

NER using DSPy

Tags:

2024-04-12 A propos

Entity Embedding Completion for Wide-Coverage Entity Disambiguation

Tags:

2022-12-11 A propos

[2205.04260] EASE: Entity-Aware Contrastive Learning of Sentence Embedding

Tags:

2022-05-11 A propos

[1909.00426] Global Entity Disambiguation with BERT

Tags:

2022-04-18 A propos

[2110.08151] mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models

Tags:

[Ikuya Yamada sur Twitter : "Is entity representation effective to improve multilingual language models?..."](doc:2022/04/ikuya_yamada_sur_twitter_is_)

> Recent studies have shown that multilingual pretrained language models can be effectively improved with cross-lingual alignment information from Wikipedia entities. However, **existing methods only exploit entity information in pretraining and do not explicitly use entities in downstream tasks**. In this study, we explore the **effectiveness of leveraging entity representations for downstream cross-lingual tasks**.
>
> the key insight is that incorporating entity representations into the input allows us to extract more language-agnostic features.

[Github](https://github.com/studio-ousia/luke)

> Entity representations are known to enhance
language models in mono-lingual settings
(Zhang et al., 2019: [ERNIE](tag:ernie.html); Peters et al., 2019:  [[1909.04164] Knowledge Enhanced Contextual Word Representations](doc:2020/05/1909_04164_knowledge_enhanced); Wang et al.,
2021 [[1911.06136] KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](doc:2020/11/1911_06136_kepler_a_unified_); Xiong et al., 2020; Yamada et al., 2020: [[2010.01057] LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](doc:2020/11/2010_01057_luke_deep_context))
presumably by introducing real-world knowledge.
We show that using entity representations facilitates
cross-lingual transfer by providing languageindependent
features.
>
> Multilingual extension of LUKE. The model is trained with the multilingual
masked language modeling (MLM) task as well
as the masked entity prediction (MEP) task with
Wikipedia entity embeddings

> We investigate two ways of using the entity representations
in cross-lingual transfer tasks:
> 1. perform
entity linking for the input text, and append
the detected entity tokens to the input sequence.
The entity tokens are expected to provide language independent
features to the model
> 2. use the entity
[MASK] token from the MEP task as a languageindependent
feature extractor.

2022-04-17 A propos

Ikuya Yamada sur Twitter : "Is entity representation effective to improve multilingual language models?..."

Tags:

2022-04-13 A propos

[1910.06294] Training Compact Models for Low Resource Entity Tagging using Pre-trained Language Models

Tags:

2022-03-31 A propos

(((ل()(ل() 'yoav))))👾 sur Twitter : "my two cents on why NLP as a field is focusing on the ML-ish / algorithmic / leaderboard-ish aspects (incl., now, LLMs) and not on the underlying language phenomena: it is just so much easier, on so many levels."

Tags:

2021-08-30 A propos

[2101.00345] Modeling Fine-Grained Entity Types with Box Embeddings

Tags:

2021-06-22 A propos

[2010.00904] Autoregressive Entity Retrieval

Tags:

2021-01-14 A propos

[2010.01057] LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention

Tags:

2020-11-26 A propos

[2001.08053] Contextualized Embeddings in Named-Entity Recognition: An Empirical Study on Generalization

Tags:

2020-10-01 A propos

[1909.01259] Neural Attentive Bag-of-Entities Model for Text Classification

Tags:

A model that performs **text classification using entities in a knowledge base**.

> Entities provide unambiguous and relevant semantic signals that are beneficial for capturing semantics in texts. We combine **simple high-recall entity detection based on a dictionary** (word->list of entities), to detect entities in a document, with a novel neural **attention mechanism that enables the model to focus on a small number of unambiguous and relevant entities**.

2 steps:

1. Entity detection
2. Classification using the detected entities (+text) as inputs

Regarding entity linking, a local model which uses cosine
similarity between the embedding of the target
entity and the word-based representation of
the document to capture the relevance of an entity
given a document.

Embeddings from the KB: computed using [#Wikipedia2Vec](tag:wikipedia2vec) (similar words and entities
close to one another in a unified vector space)

Model using attention, with 2 features :

- cosine similarity between the
embedding of the entity and the word based
representation of the document
- the probability that the entity
name refers to the entity in KB.

Somewhat [related](doc:2020/01/investigating_entity_knowledge_)

### Conclusion:

>a neural
network model that performs text classification using
entities in Wikipedia. We combined simple
dictionary-based entity detection with a neural attention
mechanism to enable the model to focus
on a small number of unambiguous and relevant
entities in a document.

2020-09-02 A propos

[1812.06280] Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia

Tags:

2020-09-02 A propos

Trust, but verify! Better entity linking through automatic verification (2017)

Tags:

2020-01-10 A propos

HITS at TAC KBP 2015:Entity Discovery and Linking, and Event Nugget Detection

Tags:

2020-01-10 A propos

Extraction de relation via la validation de relation

Tags:

2019-12-03 A propos

Reasoning With Neural Tensor Networks for Knowledge Base Completion (2013)

Tags:

**Predicting the likely truth of additional facts based on existing facts in the knowledge base.**

> we introduce an expressive neural
tensor network suitable for reasoning over relationships between two entities.

Most similar work: [Bordes et al.](http://127.0.0.1:8080/semanlink/doc/2019/08/learning_structured_embeddings_) (2011)

Contributions:

1. new neural tensor
network (**NTN**) suitable for reasoning over relationships between two entities. Generalizes several previous neural network models and provides a more
powerful way to model relational information than a standard neural network layer.
2. a new way to represent entities in knowledge bases, as the
average of  their constituting word vectorss, allowing the sharing of statistical strength between the words describing
each entity (e.g., Bank of China and China).
3. incorporation of word vectors which are trained on large unlabeled text

> We **learn to modify word representations
via grounding in world knowledge**. This essentially allows us to analyze word embeddings and
query them for specific relations. Furthermore, the resulting vectors could be used in other tasks
such as named entity recognition or relation classification in natural language

**Makes use of entity name**: NTN
first learns word vectors from an auxiliary news corpus, and
then initializes the representation of each entity by averaging
the vectors of words contained in its name. For example,
**the embedding of AlfredHitchcock is initialized by the
average word vectors of “alfred” and “hitchcock”**... This kind of methods model textual information
separately from KG facts, and hence fail to leverage
interactions between them.[src](doc:2019/05/knowledge_graph_embedding_a_su)

2019-08-03 A propos

Entity Linking | NLP-progress

Tags:

2019-04-25 A propos

[1601.01343] Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation

Tags:

2019-01-27 A propos

Effect of Non-linear Deep Architecture in Sequence Labeling

Tags:

2018-10-22 A propos

[Learning Note] StarSpace For Multi-label Text Classification

Tags:

2018-09-11 A propos

A Tri-Partite Neural Document Language Model for Semantic Information Retrieval (2018 - ESWC conference)

Tags:

2018-06-08 A propos

[1404.5367] Lexicon Infused Phrase Embeddings for Named Entity Resolution

Tags:

2018-05-22 A propos

Named Entity Recognition and the Road to Deep Learning (2017)

Tags:

> the old
and the new-style NLP are not diametrically
opposed: just as it is possible (and useful!) to
incorporate neural-network features into a CRF,
CRFs have influenced some of the best deep
learning models for sequence labelling

This blog post go through the ways of doing NER, starting with CRF:

- When you develop a CRF,
a lot of time goes into finding feature functions (Does a word start with a capital? Is it uppercase? Is it a digit?...) 
- Pb: semantic similarity between words. 
- Gazetteers: lists with names of people, locations and organizations that are known in advance.
- feed Word Embeddings to a CRF: one way is to cluster a set
of word embeddings by distributional similarity, and
provide the CRF with the cluster IDs of a token and its
context words.
- Use word and character based embeddings
- LSTM not good enough -> biLSTM
- **biLSTM predict all labels independently of each other -> add a CRF layer** which outputs a matrix of transition scores between two states: dynamic programming can help find the optimal tag sequence for the sentence

2018-05-20 A propos

[1709.03856] StarSpace: Embed All The Things!

Tags:

2018-05-13 A propos

SPACY'S ENTITY RECOGNITION MODEL: incremental parsing with Bloom embeddings & residual CNNs - YouTube

Tags:

2018-03-06 A propos

Sequence Tagging with Tensorflow

Tags:

2018-03-05 A propos

[1603.01360] Neural Architectures for Named Entity Recognition

Tags:

2018-03-05 A propos

Modèle Neuronal de Recherche d'Information Augmenté par une Ressource Sémantique (2017)

Tags:

2018-02-12 A propos

Translating Embeddings for Modeling Multi-relational Data (2013)

Tags:

This work focuses on modeling multi-relational
data from KBs (Wordnet and Freebase in this paper), with the goal of providing an efficient
tool to complete them by automatically adding new facts, without requiring extra knowledge.

**Embedding entities and relationships of multirelational
data**: a method which **models relationships by interpreting them as translations** operating on the
low-dimensional embeddings of the entities. Motivation:
- hierarchical relationships are extremely common in KBs and translations are the natural transformations for representing them.
- cf. word embeddings and the “capital of” relationship between countries and cities, which are (coincidentally rather than willingly) represented by the model as translations in the embedding space. This suggests that there may exist embedding spaces in which 1-to-1 relationships between entities of different types may, as well, be represented by translations. The intention of our model is to enforce such a structure of the embedding space.

[Good blog post by PY Vandenbussche](http://pyvandenbussche.info/2017/translating-embeddings-transe/)

2018-01-05 A propos

Learning Concept-Driven Document Embeddings for Medical Information Search (2017)

Tags:

2018-01-03 A propos

Combining word and entity embeddings for entity linking (ESWC 2017)

Tags:

2017-12-30 A propos