Semanlink - Artificial Intelligence AND Knowledge

> we propose to
leverage the factual knowledge from KGs to enhance LLMs,
while still benefiting from circumventing the burdensome
training expenses by using pre-trained LLMs

> Graph Neural Prompting
(GNP), a plug-and-play method to assist pre-trained
LLMs in learning beneficial knowledge from KGs
>
> GNP
encodes the pertinent grounded knowledge and complex
structural information to derive Graph Neural Prompt, an
embedding vector that can be sent into LLMs to provide
guidance and instructions

> - GNP first utilizes
a GNN to capture and encode the
intricate graph knowledge into **entity/node embeddings**. 
> - Then,
a cross-modality pooling module is present to determine
the **most relevant node embeddings in relation to the text
input**, and consolidate these node embeddings into **a holistic
graph-level embedding**.
> - After that, GNP encompasses a
**domain projector** to bridge the inherent disparities between
the graph and text domains.
> - Finally, a **self-supervised link
prediction objective** is introduced to enhance the model
comprehension of relationships between entities and capture
graph knowledge in a self-supervised manner.

2023-09-28 A propos

Omar Khattab sur X : "This isn't the main point of this great new paper by @sophiaalthammer et al. But it's incredible how ColBERT at 1000 training queries is better than DPR trained at *50,000* queries!"

Tags:

2023-09-14 A propos

Inside DSPy: The New Language Model Programming Framework You Need… – Towards AI

Tags:

DSPy (Demonstrate-Search-Predict)

2023-09-06 A propos

Omar Khattab sur X : "Announcing 𝗗𝗦𝗣y..."

Tags:

2023-08-24 A propos

Knowledge Graph RAG Query Engine - LlamaIndex

Tags:

2023-08-17 A propos

Jerry Liu on X: “Graph RAG” in @llama_index: a new method of augmenting LLMs with context from a graph database

Tags:

2023-08-17 A propos

GitHub - neo4j/NaLLM: Repository for the NaLLM project

Tags:

2023-08-14 A propos

Knowledge Graphs & LLMs: Fine-Tuning Vs. Retrieval-Augmented Generation | by Tomaz Bratanic | Neo4j Developer Blog (June 6 2023)

Tags:

2023-08-13 A propos

[2308.00081] Towards Semantically Enriched Embeddings for Knowledge Graph Completion

Tags:

2023-08-02 A propos

ML Blog - Improve ChatGPT with Knowledge Graphs

Tags:

2023-07-04 A propos

Personal Knowledge Graphs - strategic structures

Tags:

Personal Knowledge Graph

2023-07-04 A propos

[2212.14024] Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP

Tags:

2023-06-23 A propos

[2306.08302] Unifying Large Language Models and Knowledge Graphs: A Roadmap

Tags:

2023-06-18 A propos

Jerry Liu sur Twitter : "The DSP project carries a lot of insights for improving RAG..."

Tags:

2023-06-18 A propos

Retrieval-augmented LMs, and prompting

Tags:

2023-06-16 A propos

Integrate LLM workflows with Knowledge Graph using Neo4j and APOC | by Tomaz Bratanic | Jun, 2023 | Towards Data Science

Tags:

2023-06-09 A propos

Jerry Liu sur Twitter : "LLMs + knowledge graphs backed by a graph db (@NebulaGraph). New stack for retrieval-augmented generation. Now possible with @llama_index..."

Tags:

2023-06-09 A propos

Daniel Daza sur Twitter : "BioBLP, a method for learning embeddings on multimodal knowledge graphs...."

Tags:

2023-06-07 A propos

LangChain Retrieval Webinar - YouTube

Tags:

2023-05-27 A propos

[2305.15294] Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy

Tags:

2023-05-26 A propos

Text Translation cognitive skill - Azure Cognitive Search | Microsoft Learn

Tags:

2023-05-09 A propos

The Embedding Archives: Millions of Wikipedia Article Embeddings in Many Languages

Tags:

2023-04-21 A propos

Jerry Liu sur Twitter : "To evaluate retrieval-augmented LLM’s, we can compare the query/response/source context to see if they match. We just added this to @gpt_index..."

Tags:

2023-04-11 A propos

[2304.02711] Structured prompt interrogation and recursive extraction of semantics (SPIRES): A method for populating knowledge bases using zero-shot learning

Tags:

2023-04-07 A propos

Meta-Learning for Keyphrase Extraction | by Sinequa | Medium (2021)

Tags:

2023-04-07 A propos

Classifying long textual documents (up to 25 000 tokens) using BERT | by Sinequa | (2020)

Tags:

2023-04-07 A propos

Sinequa Brings the Power of ChatGPT to the Enterprise With Neural Search

Tags:

2023-04-07 A propos

[2304.01982] Rethinking the Role of Token Retrieval in Multi-Vector Retrieval

Tags:

2023-04-05 A propos

harley turan sur Twitter : "a force-directed knowledge graph interface for @OpenAI’s gpt-4..."

Tags:

2023-03-31 A propos

The Knowledge Graph Conference 2023

Tags:

The Knowledge Graph Conference

2023-02-24 A propos

stanfordnlp/dspy: 𝗗𝗦𝗣: Demonstrate-Search-Predict. A framework for composing retrieval and language models for knowledge-intensive NLP.

Tags:

2023-02-18 A propos

[2302.05019] A Comprehensive Survey on Automatic Knowledge Graph Construction

Tags:

2023-02-15 A propos

deepset sur Twitter : " Generative models have taken the world of NLP by storm. But LLMs do not know about your personal data. This makes personal assistants, enterprise knowledge management and many other applications challenging. Retrieval augmented pipelines are the answer"

Tags:

2023-02-02 A propos

fly51fly sur Twitter : "REPLUG: Retrieval-Augmented Black-Box Language Models..."

Tags:

2023-01-31 A propos

Omar Khattab sur Twitter : "New features in 𝗖𝗼𝗹𝗕𝗘𝗥𝗧 ‣ Index Updater. Simple API to add/remove documents..."

Tags:

2023-01-30 A propos

François Chollet sur Twitter : "I'm pretty optimistic that the LLM reliability / factualness issue can be fixed. The key is to use LLMs as a dialog interface and not as a store of knowledge..."

Tags:

2023-01-24 A propos

[2002.01808] K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters

Tags:

2023-01-12 A propos

Uri Alon sur Twitter : "K-nearest neighbors language models (kNN-LMs; @ukhndlwl et al., ICLR'2020) improve the perplexity of standard LMs... Why?"

Tags:

2023-01-11 A propos

Inductive Link Prediction in Knowledge Graphs | by Michael Galkin | Towards Data Science

Tags:

2023-01-03 A propos

Graph ML in 2023: The State of Affairs | by Michael Galkin | Towards Data Science

Tags:

2023-01-03 A propos

IBM/zshot: Zero and Few shot named entity & relationships recognition

Tags:

2022-12-23 A propos

Akari Asai sur Twitter : "Can we solely rely on LLMs’ memories (eg replace search w ChatGPT)? Probably not... Our analysis shows how retrieval is complementary to LLMs’ parametric knowledge..."

Tags:

2022-12-21 A propos

Marcel Fröhlich sur Twitter : what enterprise knowledge graphs are about..."

Tags:

2022-11-15 A propos

[2210.13952] KnowGL: Knowledge Generation and Linking from Text

Tags:

2022-11-13 A propos

KGF22: Knowledge Graphs and The Not So Quiet Cognitive Revolution - Ontotext

Tags:

Knowledge Graphs

2022-11-06 A propos

[2210.09338] Deep Bidirectional Language-Knowledge Graph Pretraining

Tags:

2022-10-23 A propos

Yi Tay sur Twitter : "Don't retrieve, recite!..."

Tags:

2022-10-06 A propos

David Chapman sur Twitter : "AI labs should compete to build the smallest possible language models, which “know” as little as possible..."

Tags:

2022-10-01 A propos

Zshot: Zero and Few shot named entity & relationships recognition

Tags:

2022-10-01 A propos

Perspectives on knowledge acquisition & mobilization with neural net - Hugo Larochelle - CoLLAs 2022 - YouTube

Tags:

2022-09-17 A propos

Michael Galkin sur Twitter : "GraphGPS is now accepted at #NeurIPS2022..."

Tags:

2022-09-15 A propos

[1904.04458] Knowledge-Augmented Language Model and its Application to Unsupervised Named-Entity Recognition

Tags:

Knowledge Augmented
Language Model (KALM)

a language
model with access to information available in a
KB, no assumptions
about the availability of additional components
(such as Named Entity Taggers) or annotations

> While classes of
named entities (e.g., person or location) occur frequently,
each individual name (e.g, Atherton or
Zhouzhuang) may be observed infrequently even
in a very large corpus of text. As a result language
models learn to represent accurately only the most
popular named entities

> knowing that Alice is a name
used to refer to a person should give ample information
about the context in which the word may
occur (e.g., Bob visited Alice).

> ---

> extends a traditional **RNN LM**

> we enhance a traditional LM with a
gating mechanism that controls whether a particular
word is modeled as a general word or as a reference
to an entity
>
> We train the model end-to-end
with only the traditional predictive language modeling
perplexity objective
>
> KALM is trained end-to-end using
a predictive objective on large corpus of text.

> To the best of our knowledge, KALM is the first
unsupervised neural NER approach.

> KALM extends a traditional, RNN-based neural
LM.

2022-08-31 A propos

[2006.10713] Zero-Shot Learning with Common Sense Knowledge Graphs

Tags:

2022-08-29 A propos

Materials for ACL-2022 tutorial: Knowledge-Augmented Methods for NLP

Tags:

2022-08-26 A propos

[2208.09982] GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarization

Tags:

2022-08-24 A propos

GitHub - raphaelsty/kgsearch: Query and visualize knowledge graphs

Tags:

2022-08-13 A propos

[2208.03299] Few-shot Learning with Retrieval Augmented Language Model

Tags:

2022-08-08 A propos

PyKEEN, Python package for reproducible, facile knowledge graph embeddings.

Tags:

2022-08-06 A propos

[2208.00635] DictBERT: Dictionary Description Knowledge Enhanced Language Model Pre-training via Contrastive Learning

Tags:

2022-08-02 A propos

[2207.09980] ReFactorGNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective

Tags:

2022-07-23 A propos

StATIK: Structure and Text for Inductive Knowledge Graph Completion - ACL Anthology (2022)

Tags:

2022-07-17 A propos

[2207.06300] Re2G: Retrieve, Rerank, Generate

Tags:

2022-07-14 A propos

[2205.00820] Entity-aware Transformers for Entity Search

Tags:

2022-07-12 A propos

Recent Advances in Retrieval-Augmented Text Generation

Tags:

2022-07-11 A propos

cs224n Lecture 13: Integrating Knowledge in Language Models

Tags:

2022-07-08 A propos

[2206.06520] Memory-Based Model Editing at Scale

Tags:

2022-07-07 A propos

[2205.08012] CascadER: Cross-Modal Cascading for Knowledge Graph Link Prediction

Tags:

2022-07-07 A propos

Tara Safavi sur Twitter : "CascadER, a new knowledge graph (KG) link prediction method leveraging structured relations + unstructured text..."

Tags:

2022-07-07 A propos

[2008.12813] HittER: Hierarchical Transformers for Knowledge Graph Embeddings

Tags:

2022-06-30 A propos

[2205.15952] Knowledge Graph -- Deep Learning: A Case Study in Question Answering in Aviation Safety Domain

Tags:

2022-06-11 A propos

KGC Workshop

Tags:

2022-05-30 A propos

International Workshop on Knowledge Graph Generation from Text (Text2KG) 2022

Tags:

2022-05-30 A propos

A review of some techniques for inclusion of domain‐knowledge into deep neural networks

Tags:

2022-05-19 A propos

[2205.08184] SKILL: Structured Knowledge Infusion for Large Language Models

Tags:

2022-05-18 A propos

DataBorg - Knowledge management simplified

Tags:

2022-05-14 A propos

BERT-INT: A BERT-based Interaction Model For Knowledge Graph Alignment

Tags:

Entity alignment

2022-05-11 A propos

[2205.04260] EASE: Entity-Aware Contrastive Learning of Sentence Embedding

Tags:

2022-05-11 A propos

[2204.11428] Personal Research Knowledge Graphs

Tags:

2022-04-30 A propos

[2110.08151] mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models

Tags:

[Ikuya Yamada sur Twitter : "Is entity representation effective to improve multilingual language models?..."](doc:2022/04/ikuya_yamada_sur_twitter_is_)

> Recent studies have shown that multilingual pretrained language models can be effectively improved with cross-lingual alignment information from Wikipedia entities. However, **existing methods only exploit entity information in pretraining and do not explicitly use entities in downstream tasks**. In this study, we explore the **effectiveness of leveraging entity representations for downstream cross-lingual tasks**.
>
> the key insight is that incorporating entity representations into the input allows us to extract more language-agnostic features.

[Github](https://github.com/studio-ousia/luke)

> Entity representations are known to enhance
language models in mono-lingual settings
(Zhang et al., 2019: [ERNIE](tag:ernie.html); Peters et al., 2019:  [[1909.04164] Knowledge Enhanced Contextual Word Representations](doc:2020/05/1909_04164_knowledge_enhanced); Wang et al.,
2021 [[1911.06136] KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](doc:2020/11/1911_06136_kepler_a_unified_); Xiong et al., 2020; Yamada et al., 2020: [[2010.01057] LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](doc:2020/11/2010_01057_luke_deep_context))
presumably by introducing real-world knowledge.
We show that using entity representations facilitates
cross-lingual transfer by providing languageindependent
features.
>
> Multilingual extension of LUKE. The model is trained with the multilingual
masked language modeling (MLM) task as well
as the masked entity prediction (MEP) task with
Wikipedia entity embeddings

> We investigate two ways of using the entity representations
in cross-lingual transfer tasks:
> 1. perform
entity linking for the input text, and append
the detected entity tokens to the input sequence.
The entity tokens are expected to provide language independent
features to the model
> 2. use the entity
[MASK] token from the MEP task as a languageindependent
feature extractor.

2022-04-17 A propos

Ikuya Yamada sur Twitter : "Is entity representation effective to improve multilingual language models?..."

Tags:

2022-04-13 A propos

EASE: Entity-Aware Contrastive Learning of Sentence Embedding | Papers With Code

Tags:

2022-04-08 A propos

[2203.13088] Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction

Tags:

2022-03-30 A propos

Jason Weston sur Twitter : "SeeKeR: An open source search-augmented language model"

Tags:

2022-03-25 A propos

[2101.12294] Combining pre-trained language models and structured knowledge

Tags:

2022-03-25 A propos

Domain adaptation of word embeddings through the exploitation of in-domain corpora and knowledge bases (PhD Thesis 2021)

Tags:

Thèse de Hicham El Boukkouri, univ Paris-Saclay

[Github](https://github.com/helboukkouri/phd-code)

### Goal

Given a target specialized domain, improve the quality of general-domain
word representations using in-domain corpora and/or knowledge bases

### Contributions

#### a method for specializing general-domain embeddings in a [Low-Resource](tag:nlp_low_resource_scenarios) context.

> - train static representations on the task corpus, 
> - resume the
pre-training of general-domain contextual embeddings on the same task corpus,
> - finally, combine both static and contextual representations into
one final model

#### we tackle the issue of using a general-domain vocabulary in a specialized domain

#### Evaluation of  re-training vs training from scratch on specialized corpora using a specialized vocabulary

training from scratch better, but not that much: re-training from a general model
is still appropriate as it is less expensive and leads to comparable, although
slightly lower, performance

#### Regarding subword-based tokenization systems
> we argue that they are inconvenient in practice -> CharacterBERT, a variant
of BERT that uses ELMo’s character-based system instead of WordPieces. More convenient ti use, superior robustness to misspellings

#### Ways to specialize general-domain representations using knowledge bases

a strong baseline using a simple
method relying on graph embeddings and concatenation, using only is_a relation

> both static and contextual embeddings may effectively be
specialized using this simple approach

#### Knowledge Injection Modules (KIM) that
inject the knowledge representations directly within the BERT-like models' architecture

### Notes

> our experiments focused on a single
setting (i.e. the medical domain and the English language)

> meta-embeddings, an approach that consists in combining
different sets of representations for achieving improved performance

2022-03-23 A propos

Building Transformer-Based Entity Linking System | by izuna385 | Medium (2021)

Tags:

2022-03-23 A propos

Studio Ousia sur Twitter : "Now using LUKE is easier than ever!" / Twitter

Tags:

2022-03-15 A propos

SPIKE for Knowledge Base Construction

Tags:

2022-02-22 A propos

Yosi Shamay sur Twitter : "a new platform for rapid ad-hoc knowledgebase construction using extractive search...."

Tags:

2022-02-22 A propos

REALM: Retrieval-Augmented Language Model Pre-Training (Paper Explained) - YouTube

Tags:

2022-01-23 A propos

[2108.13934] Robust Retrieval Augmented Generation for Zero-shot Slot Filling

Tags:

2022-01-19 A propos

Integrate ORQA and REALM for Open Domain Question Answering · Issue #312 · deepset-ai/haystack

Tags:

2022-01-12 A propos

[2004.12832] ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

Tags:

2022-01-12 A propos

[2007.00814] Relevance-guided Supervision for OpenQA with ColBERT

Tags:

2022-01-07 A propos

Improving Language Models by Retrieving from Trillions of Tokens | DeepMind

Tags:

2021-12-09 A propos

facebookresearch/DrQA: Reading Wikipedia to Answer Open-Domain Questions

Tags:

2021-12-08 A propos

Semantic search through a vectorized Wikipedia (SentenceBERT) with the Weaviate vector search engine

Tags:

2021-12-05 A propos

[2112.01488] ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction

Tags:

2021-12-05 A propos

5 Reasons Knowledge Graph will never bloom | by Dongsheng Wang | Medium

Tags:

2021-12-02 A propos

Andrea Volpini sur Twitter : New blog post on #ContentHub for #Semantic #SEO. How to cluster content using Knowledge Graph Embeddings to detect: ✓ Clusters ✓ Sub-topics ✓ Author's topical authority

Tags:

2021-11-16 A propos

[2110.06176] Mention Memory: incorporating textual knowledge into Transformers through entity mention attention

Tags:

2021-10-13 A propos

Building Scalable, Explainable, and Adaptive NLP Models with Retrieval | SAIL Blog

Tags:

2021-10-07 A propos

Enterprise Knowledge Graph Foundation

Tags:

2021-10-02 A propos

Haystack (deepset)

Tags:

2021-09-20 A propos

Link Prediction with Graph Neural Networks and Knowledge Extraction

Tags:

2021-09-03 A propos

Knowledge Graphs in Natural Language Processing @ ACL 2021 | by Michael Galkin | Aug, 2021

Tags:

2021-08-06 A propos

A Moderate Proposal for Radically Better AI-powered Web Search

Tags:

2021-07-10 A propos

[2106.04612] Neural Extractive Search

Tags:

how to extend a
search paradigm we call “**extractive search**” with
neural similarity techniques.

> some information needs require extracting
and aggregating sub-sentence information
(words, phrases, or entities) from multiple documents
(e.g. a list of all the risk factors for a specific
disease and their number of mentions, or a comprehensive
table of startups and CEOs).

> extractive search combines
document selection with information extraction. **The query is extended with capture slots**:
these are **search terms that act as variables, whose
values should be extracted**.
> The user
is then presented with the matched documents, each
annotated with the corresponding captured spans,
as well as aggregate information over the captured
spans

Conclusion :

> We presented a system for neural extractive search.
While we found our system to be useful for scientific
search, it also has clear limitations and areas
for improvement, both in terms of accuracy (only
72.2% of the returned results are relevant, both the
alignment and similarity models generalize well to
some relations but not to others), and in terms of
scale

[Video of demo](https://www.youtube.com/watch?v=TtqWi2GgB5A&t=1832s)

2021-06-23 A propos

librAIry

Tags:

2021-06-22 A propos

[2001.03765] Learning Cross-Context Entity Representations from Text

Tags:

2021-06-22 A propos

[2101.00345] Modeling Fine-Grained Entity Types with Box Embeddings

Tags:

2021-06-22 A propos

[2102.07043] Reasoning Over Virtual Knowledge Bases With Open Predicate Relations

Tags:

> a method for constructing **a virtual KB (VKB) trained entirely from text**

Open Predicate Query Language (OPQL): constructing a virtual knowledge base (VKB) that supports KB reasoning & open-domain QA, tackling the incompleteness of knowledge bases by constructing a virtual KB only from text

> OPQL constructs
a VKB by **encoding and indexing a set of
relation mentions** in a way that naturally enables
reasoning and can be trained without any structured
supervision.

> can be used
as an **external memory integrated into a language
model**

cf. this earlier paper [[2002.10640] Differentiable Reasoning over a Virtual Knowledge Base](doc:2020/07/2002_10640_differentiable_rea). But does not require an initial structured KB for distant
supervision.

> The key idea in constructing the OPQL VKB is to use a
dual-encoder pre-training process, similar to 
[[1906.03158] Matching the Blanks: Distributional Similarity for Relation Learning](doc:2021/05/1906_03158_matching_the_blank)

Related work section refers to [[1909.04164] Knowledge Enhanced Contextual Word Representations](doc:2020/05/1909_04164_knowledge_enhanced). Also refers to [[2007.00849] Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge](doc:2020/07/2007_00849_facts_as_experts_) (some authors in common)

2021-06-20 A propos

[1410.5859] Towards a Model Theory for Distributed Representations

Tags:

2021-06-10 A propos

Matching and mining in knowledge graphs of the Web of data Applications in pharmacogenomics

Tags:

2021-06-10 A propos

scikit-learn Pipelines meet Knowledge Graphs - The Python kgextension Package ESWC 2021

Tags:

2021-06-10 A propos

An Introduction to Knowledge Graphs | SAIL Blog

Tags:

2021-05-25 A propos

Enterprise Knowledge Graph Solutions | eccenca

Tags:

2021-05-19 A propos

Sinequa : Enterprise Search Platform

Tags:

2021-05-19 A propos

[2009.12030] AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding

Tags:

2021-05-17 A propos

[1911.09419] Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction

Tags:

2021-05-17 A propos

[2012.02558] Pre-trained language models as knowledge bases for Automotive Complaint Analysis

Tags:

2021-04-11 A propos

Pulling Turtle RDF triples from the Google Knowledge Graph

Tags:

2021-03-30 A propos

[2103.12876] Complex Factoid Question Answering with a Free-Text Knowledge Graph

Tags:

2021-03-30 A propos

BabelNet | Le plus grand dictionnaire encyclopédique et réseau sémantique

Tags:

2021-03-09 A propos

Hugging Face sur Twitter : "Transformers release of the Retrieval-Augmented Generation model in collaboration with @facebookai!"

Tags:

2021-02-23 A propos

Pre-trained Models | GraphVite

Tags:

2021-02-16 A propos

raphaelsty/kdmlm: Combine knowledge bases with language models.

Tags:

2021-02-16 A propos

Retrieval Augmented Generation with Huggingface Transformers and Ray | Distributed Computing with Ray

Tags:

2021-02-11 A propos

Linking Entities with Knowledge Graphs | by Sigurd Berglann | Strise | Medium

Tags:

2021-02-05 A propos

DerwenAI/kglab: an abstraction layer in Python for building knowledge graphs

Tags:

2021-01-30 A propos

KGC 2021 Call for Presentations | The Knowledge Graph Conference

Tags:

The Knowledge Graph Conference

2021-01-30 A propos

The complexity to construct knowledge graphs and how low code tools can help or hurt you | Medium

Tags:

2021-01-30 A propos

Exploring The MET Art Collections with Hume #2 | GraphAware

Tags:

2021-01-19 A propos

Goldfire Cognitive Search | IHS Markit

Tags:

2021-01-18 A propos

Language Models are Open Knowledge Graphs... but are hard to mine - Towards Data Science

Tags:

2021-01-17 A propos

[1911.03681] E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT

Tags:

> way of **injecting factual knowledge about entities into the pretrained BERT model**.

(Feeding entity vectors
into BERT as if they
were wordpiece vectors without additional encoder
pretraining)

>
> **We align [Wikipedia2Vec](tag:wikipedia2vec) entity vectors (Yamada et al., 2016) with BERT's native wordpiece vector space and use the aligned entity vectors as if they were wordpiece vectors**. The resulting entity-enhanced version of BERT (called E-BERT) is similar in spirit to [ERNIE](tag:ernie) (Zhang et al., 2019) and [KnowBert](tag:knowbert) (Peters et al., 2019), but it **requires no expensive further pretraining of the BERT encoder**.
>
> Our vector space alignment strategy is inspired by
cross-lingual word vector alignment

Related work on Entity-enhanced BERT:

> ([ERNIE](doc:2019/08/_1905_07129_ernie_enhanced_la) and [Knowbert](doc:2020/05/1909_04164_knowledge_enhanced)) are based on the design principle
that BERT be adapted to entity vectors. They introduce
new encoder layers to feed pretrained entity
vectors into the Transformer, and they require additional
pretraining to integrate the new parameters.
In contrast, E-BERT’s design principle is that entity
vectors be adapted to BERT.
>
> Two other knowledge-enhanced MLMs are [KEPLER](doc:2020/11/1911_06136_kepler_a_unified_)
(Wang et al., 2019c) and K-Adapter (Wang
et al., 2020)... Their factual knowledge
does not stem from entity vectors – instead, they
are trained in a multi-task setting on relation classification
and knowledge base completion.

Not to be cofounded with [[2009.02835] E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commerce](doc:2020/12/2009_02835_e_bert_a_phrase_a)

2021-01-12 A propos

Pablo Castro sur Twitter : "Knowledge mining using the knowledge store feature of #AzureSearch"

Tags:

2020-12-19 A propos

[2009.02835] E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commerce

Tags:

2020-12-14 A propos

[2002.08909] REALM: Retrieval-Augmented Language Model Pre-Training

Tags:

**Augment language model pre-training with a retriever module**, which
is trained using the masked language modeling objective.

> To capture knowledge in a more modular and interpretable way, we augment language model pre-training with a latent knowledge retriever, which allows the model to retrieve and attend over documents from a large corpus such as Wikipedia, used during pre-training, fine-tuning and inference. **For the first time, we show how to pre-train such a knowledge retriever in an unsupervised manner**, using masked language modeling as the learning signal and backpropagating through a retrieval step that considers millions of documents

Hum, #TODO: parallel to be drawn with techniques in [KG-augmented Language Models](tag:knowledge_graph_augmented_language_models) which focus "on the problem of capturing declarative knowledge in the learned parameters of a language model."

[Google AI Blog Post](doc:2020/08/google_ai_blog_realm_integrat)

[Summary](https://joeddav.github.io/blog/2020/03/03/REALM.html) for the [Hugging Face awesome-papers reading group](doc:2021/03/huggingface_awesome_papers_pap)

2020-12-12 A propos

[2012.04584] Distilling Knowledge from Reader to Retriever for Question Answering

Tags:

> a method to train an information retrieval module for downstream tasks, **without using pairs of queries and documents as annotations**.

Uses two models (standard pipeline for open-domain QA):

- the first one retrieves documents from a large source of knowledge (the retriever)
- the second one processes the support documents to solve the task (the reader).

> First the retriever selects support passages in a large knowledge
source. Then these passages are processed by the reader, along with the question, to generate an
answer

Inspired by knowledge distillation: the reader model is the teacher and the retriever is the student.

> More precisely, we use a sequence-to-sequence model as the reader, and use
the attention activations over the input documents as synthetic labels to train the retriever. 
> (**train the retriever by learning to approximate the attention score of the reader**)

Refers to:

- [REALM: Retrieval-Augmented Language Model Pre-Training](doc:2020/12/2002_08909_realm_retrieval_a)
- [Dehghani: Neural Ranking Models with Weak Supervision](doc:?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1704.08803)

2020-12-11 A propos

Supporting content decision makers with machine learning | Dec, 2020 | Netflix TechBlog

Tags:

2020-12-11 A propos

Exploration Engines - the koodos collective

Tags:

2020-12-10 A propos

Drew Tada sur Twitter : "Officially launching giantgra.ph A search engine for knowledge graphs..."

Tags:

2020-12-10 A propos

Knowledge Base Embedding By Cooperative Knowledge Distillation - ACL Anthology

Tags:

KD-MKB paper

2020-12-05 A propos

FP Servant sur Twitter : "constructing a personal knowledge graph as a support for learning (and a metaphor of the learning experience)..."

Tags:

2020-12-03 A propos

[2010.01057] LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention

Tags:

2020-11-26 A propos

Knowledge Graphs in NLP @ EMNLP 2020 | by Michael Galkin | Nov, 2020 | Medium

Tags:

2020-11-24 A propos

Knowledge Graph, AI Web Data Extraction and Crawling | Diffbot

Tags:

2020-11-14 A propos

raphaelsty/ckb: Contextual knowledge bases

Tags:

2020-11-09 A propos

Text-Enhanced Representation Learning for Knowledge Graph (IJCAI 2016)

Tags:

2020-11-03 A propos

[1911.06136] KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation

Tags:

2020-11-03 A propos

[2010.03496] Inductive Entity Representations from Text via Link Prediction

Tags:

BLP "BERT for Link Prediction". Central idea: **training an entity encoder with a
link prediction objective** (using the textual descriptions of entities when computing entity representations - hence not failing with entities unknown in training)

> a method for **learning representations
of entities**, that uses a **pre-trained Transformer** based
architecture as an entity encoder, and
**link prediction training on a knowledge graph
with textual entity descriptions**.

> using entity descriptions,
an entity encoder is trained for link prediction in
a knowledge graph. The encoder can then be used
without fine-tuning to obtain features for entity classification
and information retrieval

Cites [Xie et al](doc:2020/10/representation_learning_of_know) and [Kepler](doc:2020/11/1911_06136_kepler_a_unified_). They claim that their
objective targeted exclusively for link prediction (and not an objective that combines language modeling
and link prediction as Kepler)
performs better than Kepler's more complex one.

2020-11-03 A propos

Knowledge Graphs: An Information Retrieval Perspective

Tags:

2020-10-30 A propos

[2010.11967] Language Models are Open Knowledge Graphs

Tags:

2020-10-26 A propos

TAGME: on-the-fly annotation of short text fragments!

Tags:

2020-10-11 A propos

[2010.00402] From Trees to Continuous Embeddings and Back: Hyperbolic Hierarchical Clustering

Tags:

2020-10-03 A propos

[1802.05930] Learning beyond datasets: Knowledge Graph Augmented Neural Networks for Natural language Processing

Tags:

2020-10-02 A propos

SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions (AAAI 2017)

Tags:

2020-10-02 A propos

Representation learning of knowledge graphs with entity descriptions (AAAI 2016)

Tags:

"Description-Embodied Knowledge Representation Learning" (DKRL)

> In most knowledge graphs there are usually concise descriptions for entities, which cannot be well utilized by existing methods... Experimental results on real-world datasets show that, our method outperforms other baselines on the knowledge graph completion and entity classification tasks, especially under the zero-shot setting, which indicates that **our method is capable of building representations for novel entities according to their descriptions**. [Source code on github](https://github.com/xrb92/DKRL).

For fact triples: TransE. Meanwhile, given an entity we will also learn to maximize
the likelihood of predicting its description (using either CBOW or CNN encoder) (head + relation = tail, also in "text space")

Two types of representations for entities: structure-based
representations and description-based representations. They are learned simultaneously into the same
vector space but not forced to be unified **so that novel
entities with only descriptions can be represented**.

2020-10-02 A propos

[2009.07938] Type-augmented Relation Prediction in Knowledge Graphs

Tags:

2020-09-19 A propos

Representing Text for Joint Embedding of Text and Knowledge Bases (EMNLP 2015)

Tags:

2020-09-14 A propos

[1806.06478] Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment

Tags:

2020-09-06 A propos

[1803.07828] Expeditious Generation of Knowledge Graph Embeddings

Tags:

2020-09-02 A propos

[2009.00318] More is not Always Better: The Negative Impact of A-box Materialization on RDF2vec Knowledge Graph Embeddings

Tags:

2020-09-02 A propos

[1909.01259] Neural Attentive Bag-of-Entities Model for Text Classification

Tags:

A model that performs **text classification using entities in a knowledge base**.

> Entities provide unambiguous and relevant semantic signals that are beneficial for capturing semantics in texts. We combine **simple high-recall entity detection based on a dictionary** (word->list of entities), to detect entities in a document, with a novel neural **attention mechanism that enables the model to focus on a small number of unambiguous and relevant entities**.

2 steps:

1. Entity detection
2. Classification using the detected entities (+text) as inputs

Regarding entity linking, a local model which uses cosine
similarity between the embedding of the target
entity and the word-based representation of
the document to capture the relevance of an entity
given a document.

Embeddings from the KB: computed using [#Wikipedia2Vec](tag:wikipedia2vec) (similar words and entities
close to one another in a unified vector space)

Model using attention, with 2 features :

- cosine similarity between the
embedding of the entity and the word based
representation of the document
- the probability that the entity
name refers to the entity in KB.

Somewhat [related](doc:2020/01/investigating_entity_knowledge_)

### Conclusion:

>a neural
network model that performs text classification using
entities in Wikipedia. We combined simple
dictionary-based entity detection with a neural attention
mechanism to enable the model to focus
on a small number of unambiguous and relevant
entities in a document.

2020-09-02 A propos

[1812.06280] Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia

Tags:

2020-09-02 A propos

RDF2vec.org

Tags:

RDF2VEC

2020-09-02 A propos

Triple Classification Using Regions and Fine-Grained Entity Typing (AAAI 2019)

Tags:

Triple Classification

2020-08-30 A propos

[2008.08995] Constructing a Knowledge Graph from Unstructured Documents without External Alignment

Tags:

2020-08-21 A propos

Google AI Blog: REALM: Integrating Retrieval into Language Representation Models

Tags:

2020-08-13 A propos

[1911.03903] A Re-evaluation of Knowledge Graph Completion Methods

Tags:

2020-07-28 A propos

raphaelsty/mkb: Knowledge Base Embedding By Cooperative Knowledge Distillation

Tags:

2020-07-24 A propos

Knowledge Graphs in Natural Language Processing @ ACL 2020 | by Michael Galkin

Tags:

2020-07-12 A propos

[2004.07202] Entities as Experts: Sparse Memory Access with Entity Supervision

Tags:

2020-07-11 A propos

[2002.10640] Differentiable Reasoning over a Virtual Knowledge Base

Tags:

2020-07-11 A propos

[2007.00849] Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge

Tags:

> a neural language model that includes **an explicit interface between symbolically interpretable factual information and subsymbolic neural knowledge.**... **The model can be updated without re-training by manipulating its symbolic representations**. In particular this model allows us to add new facts and overwrite existing ones.

> a **neural language model which learns to access information
in a symbolic knowledge graph.**

> This
model builds on the recently-proposed [Entities as
Experts](doc:2020/07/2004_07202_entities_as_expert) (EaE) language model (Févry et al., 2020),
which extends the same transformer (Vaswani
et al., 2017) architecture of BERT (Devlin et al., 2019) with an additional external memory for entities.
>
> After training EaE, the embedding associated
with an entity will (ideally) capture information
about the textual context in which that
entity appears, and by inference, the entity’s semantic
properties
>
> we include an additional
memory called a fact memory, which encodes
triples from a symbolic KB.
>
> This combination results in a
neural language model which learns to access information
in a the symbolic knowledge graph.

TODO:

- read again IBM's [Span Selection Pre-training for Question Answering](doc:2019/09/_1909_04120_span_selection_pre) ("an effort to avoid encoding general knowledge in the transformer network itself")
- compare with [[1907.05242] Large Memory Layers with Product Keys](doc:2019/07/_1907_05242_large_memory_layer)
- how does it relate with [[2002.08909] REALM: Retrieval-Augmented Language Model Pre-Training](doc:2020/12/2002_08909_realm_retrieval_a)?

2020-07-09 A propos

awslabs/dgl-ke: package for learning large-scale knowledge graph embeddings.

Tags:

2020-07-07 A propos

End-to-End Learning with Text & Knowledge Bases (Bhuwan Dhingra PhD Thesis)

Tags:

2020-07-06 A propos

Learning to Tag OOV Tokens by Integrating Contextual Representation and Background Knowledge (ACL Anthology 2020)

Tags:

Aim to leverage both contextual representation of input text (deep LMs)  and  knowledge derived
from curated KBs ([Wordnet](tag:wordnet)) to improve [slot tagging](tag:slot_tagging) in the presence of [out-of-vocab](tag:oov) words ([few-shot scenario](tag:few_shot_learning))

Method:

1. retrieve potentially relevant KB entities and
encode them into distributed representations that
describe global graph-structured information
2. BERT encoder
layer to capture context-aware representations of
the sequence and attend to the KB embeddings
using multi-level graph attention
3. integrate
BERT embeddings and the KB embeddings
to predict the slot type

Contributions:

1. feasibility of applying lexical ontology
to facilitate recognizing OOV words. First to consider the large-scale background
knowledge for enhancing context-aware
slot tagging models.
2. a knowledge integration mechanism that uses multi-level graph
attention to model explicit lexical relations.
3.experiments on two benchmark datasets

> our method makes a notable difference in a
scenario where samples are linguistically diverse,
and large vocab exists.

(Better improvements when using RNN than BERT, because BERT already contains a lot of background knowledge)

2020-07-04 A propos

[2006.13365] Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework

Tags:

2020-06-26 A propos

facebookresearch/faiss: A library for efficient similarity search and clustering of dense vectors.

Tags:

2020-06-09 A propos

Label unstructured data using Enterprise Knowledge Graphs 2

Tags:

2020-05-22 A propos

Peter Bloem sur Twitter : "One of the messages from Ruffinelli et al 2020..."

Tags:

2020-05-19 A propos

[2003.08001] Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study

Tags:

2020-05-15 A propos

[1909.04164] Knowledge Enhanced Contextual Word Representations

Tags:

2020-05-13 A propos

[1907.04829] BAM! Born-Again Multi-Task Networks for Natural Language Understanding

Tags:

2020-05-12 A propos

[1912.08422] Distilling Structured Knowledge into Embeddings for Explainable and Accurate Recommendation

Tags:

2020-05-12 A propos

[1807.08447] LinkNBed: Multi-Graph Representation Learning with Entity Linkage

Tags:

2020-05-11 A propos

Bootstrapping Entity Alignment with Knowledge Graph Embedding | IJCAI

Tags:

2020-05-11 A propos

[1706.00384] Deep Mutual Learning

Tags:

2020-05-11 A propos

[1906.07241] Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modeling

Tags:

2020-05-11 A propos

At Columbia University virtual conference, masters ply the strange and beautiful art of knowledge graphs | ZDNet

Tags:

Knowledge Graphs

2020-05-10 A propos

A knowledge graph embedding library for reproducible research

Tags:

2020-05-04 A propos

You CAN Teach an Old Dog New Tricks! On Training Knowledge Graph Embeddings (ICLR 2020)

Tags:

2020-05-04 A propos

[1910.12507] A Survey on Knowledge Graph Embeddings with Literals: Which model links better Literal-ly?

Tags:

2020-05-04 A propos

[2004.14843] Knowledge Graph Embeddings and Explainable AI

Tags:

2020-05-04 A propos

Disambiguating KBpedia Knowledge Graph Concepts

Tags:

2020-05-03 A propos

Cognonto - Data. Structure. Meaning.

Tags:

KBPedia

2020-05-03 A propos

[1911.03814] Scalable Zero-shot Entity Linking with Dense Entity Retrieval

Tags:

2020-05-02 A propos

Sebastian Riedel sur Twitter : Happy to introduce BLINK, the @facebookai open-source entity linker!...

Tags:

2020-05-02 A propos

Deepak Nathani | Pay Attention, Relations are Important

Tags:

2020-04-30 A propos

[1906.01195] Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs

Tags:

2020-04-30 A propos

Iterative Entity Alignment with Improved Neural Attribute Embedding

Tags:

2020-04-29 A propos

A Comprehensive Survey of Knowledge Graph Embeddings with Literals: Techniques and Applications

Tags:

2020-04-29 A propos

CEUR-WS.org/Vol-2377 - Workshop on Deep Learning for Knowledge Graphs 2019

Tags:

2020-04-29 A propos

Knowledge Graphs @ ICLR 2020 - Michael Galkin - Medium

Tags:

2020-04-28 A propos

[2001.09522] TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network

Tags:

how to add a set of new concepts to an existing taxonomy.

[Tweet](https://twitter.com/mickeyjs6/status/1253772146142216194?s=20) [GitHub](https://github.com/mickeystroller/TaxoExpan)

> we study the taxonomy expansion task: given an
existing taxonomy and a set of new emerging concepts, we aim
to automatically expand the taxonomy to incorporate these new
concepts (without changing the existing relations in the given taxonomy).

> To the best of our knowledge, this is the first study on **how to
expand an existing directed acyclic graph (as we model a taxonomy
as a DAG) using self-supervised learning**.

Self-supervised framework, the existing taxonomy being used as training data: it learns a model to predict whether a query concept is the direct hyponym of an anchor concept.

> 2 techniques:
>
> 1. a **position-enhanced graph neural network that encodes the local structure of an anchor concept** in the existing taxonomy,
> 2. a noise-robust training objective that enables the learned model to be insensitive to the label noise in the self-supervision data.

Regarding 1: uses [GNN](/tag/graph_neural_networks.html) to model the "ego network" of concepts (potential “siblings”
and “grand parents” of the query concept).

> Regular
GNNs fail to distinguish nodes with different relative positions to
the query (i.e., some nodes are grand parents of the query while
the others are siblings of the query). To address this limitation, we
present a simple but effective enhancement to inject such position
information into GNNs using position embedding. We show that
such embedding can be easily integrated with existing GNN architectures
(e.g., [GCN](/tag/graph_convolutional_networks) and GAT) and significantly boosts the
prediction performance

Regarding point 2: uses InfoNCE loss, cf. [Contrastive Predictive Coding](/doc/?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1807.03748)

> Instead of predicting
whether each individual ⟨query concept, anchor concept⟩ pair
is positive or not, we first group all pairs sharing the same query
concept into a single training instance and learn a model to select
the positive pair among other negative ones from the group.

(Hum, ça me rappelle quelque chose)

> assume each concept (in existing taxonomy + set of new concepts) has an initial embedding
vector learned from some text associated with this concept.

To keep things tractable, only attempts to find a single parent node of each new concept.

2020-04-25 A propos

Retail Graph — Walmart’s Product Knowledge Graph

Tags:

2020-04-09 A propos

Chengkai Li sur Twitter : "Link prediction methods on knowledge graphs don't work..."

Tags:

2020-03-29 A propos

[1909.03193] KG-BERT: BERT for Knowledge Graph Completion

Tags:

2020-03-22 A propos

[1911.02168] CoKE: Contextualized Knowledge Graph Embedding

Tags:

2020-03-22 A propos

Combining knowledge graphs, quickly and accurately (2020)

Tags:

2020-03-19 A propos

Martynas Jusevicius sur Twitter : "Is there a solution for entity recognition that would use a local #KnowledgeGraph to look for matches? Ideally any SPARQL datasource..."

Tags:

2020-03-13 A propos

AmbiverseNLU: A Natural Language Understanding suite by Max Planck Institute for Informatics

Tags:

2020-03-13 A propos

[1909.07606] K-BERT: Enabling Language Representation with Knowledge Graph

Tags:

2020-03-08 A propos

[2003.02320] Knowledge Graphs

Tags:

2020-03-07 A propos

LinkedDataHub - AtomGraph's open-source Knowledge Graph management system

Tags:

Knowledge Graphs

2020-03-05 A propos

[1902.10197] RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

Tags:

2020-03-03 A propos

[2002.11402] Detecting Potential Topics In News Using BERT, CRF and Wikipedia

Tags:

2020-02-27 A propos

Adam Roberts sur Twitter : "New preprint: How Much Knowledge Can You Pack into the Parameters of a Language Model?..."

Tags:

2020-02-11 A propos

davidsbatista/BREDS: "Bootstrapping Relationship Extractors with Distributional Semantics" (Batista et al., 2015) - code for EMNLP'15 paper

Tags:

2020-01-11 A propos

[2001.01447] Improving Entity Linking by Modeling Latent Entity Type Information

Tags:

2020-01-09 A propos

Advancing Natural Language Processing (NLP) for Enterprise Domains

Tags:

2020-01-07 A propos

[1911.00172] Generalization through Memorization: Nearest Neighbor Language Models

Tags:

2019-12-20 A propos

CONCEPTUAL GROUNDING FOR TEXT REPRESENTATION LEARNING

Tags:

2019-12-03 A propos

Extraction de relation via la validation de relation

Tags:

2019-12-03 A propos

Barack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling (ACL 2019)

Tags:

2019-11-17 A propos

[1910.09760] Question Answering over Knowledge Graphs via Structural Query Patterns

Tags:

2019-11-06 A propos

Knowledge Graphs and Knowledge modelling took center stage at ISWC 2019 | LinkedIn

Tags:

2019-11-02 A propos

A Brief History of Knowledge Graph's Main Ideas: A tutorial, Claudio Gutierrez and Juan F. Sequeda

Tags:

2019-10-27 A propos

Improving long-form question answering by compressing search results

Tags:

2019-10-24 A propos

[1910.03524] Beyond Vector Spaces: Compact Data Representation as Differentiable Weighted Graphs

Tags:

2019-10-09 A propos

[1909.04120] Span Selection Pre-training for Question Answering

Tags:

> a **new pre-training task inspired by reading
comprehension** and an **effort to avoid encoding general knowledge in the transformer network itself**

Current transformer architectures store general knowledge -> large models, long pre-training time. Better to offload the requirement of general knowledge to a sparsely activated network.

"Span selection" as an additional auxiliary task: the query is a sentence drawn from a corpus
with a term replaced with a special token: [BLANK]. The term replaced by the blank is the answer term. The passage is
relevant as determined by a BM25 search, and answer-bearing (containing the answer
term). Unlike BERT’s cloze task, where the answer must be drawn from the model itself, the answer is found in a passage
using language understanding.

> **We hope to progress to a model of general purpose language modeling that uses an indexed long
term memory to retrieve world knowledge, rather than holding it in the densely activated transformer encoder layers.**

2019-09-18 A propos

Knowledge Graphs and Machine Learning - Towards Data Science

Tags:

2019-09-12 A propos

[1909.01066] Language Models as Knowledge Bases?

Tags:

2019-09-05 A propos

Knowledge Graphs and Natural Language Processing. The Year of the Graph Newsletter, July/August 2019 | Linked Data Orchestration

Tags:

Knowledge Graphs and NLP

2019-08-29 A propos

[1904.02342] Text Generation from Knowledge Graphs with Graph Transformers

Tags:

2019-08-23 A propos

[1905.07854] KGAT: Knowledge Graph Attention Network for Recommendation

Tags:

2019-08-23 A propos

Neural Knowledge Acquisition via Mutual Attention between Knowledge Graph and Text (2018)

Tags:

2019-08-23 A propos

The knowledge graph as the default data model for learning on heterogeneous knowledge (2017)

Tags:

2019-08-22 A propos

Knowledge Graph Reasoning Papers

Tags:

2019-08-09 A propos

[1905.07129] ERNIE: Enhanced Language Representation with Informative Entities

Tags:

2019-08-05 A propos

Knowledge graphs in Natural Language Processing @ ACL 2019 - Michael Galkin

Tags:

2019-08-05 A propos

Learning Structured Embeddings of Knowledge Bases (2011)

Tags:

2019-08-03 A propos

Reasoning With Neural Tensor Networks for Knowledge Base Completion (2013)

Tags:

**Predicting the likely truth of additional facts based on existing facts in the knowledge base.**

> we introduce an expressive neural
tensor network suitable for reasoning over relationships between two entities.

Most similar work: [Bordes et al.](http://127.0.0.1:8080/semanlink/doc/2019/08/learning_structured_embeddings_) (2011)

Contributions:

1. new neural tensor
network (**NTN**) suitable for reasoning over relationships between two entities. Generalizes several previous neural network models and provides a more
powerful way to model relational information than a standard neural network layer.
2. a new way to represent entities in knowledge bases, as the
average of  their constituting word vectorss, allowing the sharing of statistical strength between the words describing
each entity (e.g., Bank of China and China).
3. incorporation of word vectors which are trained on large unlabeled text

> We **learn to modify word representations
via grounding in world knowledge**. This essentially allows us to analyze word embeddings and
query them for specific relations. Furthermore, the resulting vectors could be used in other tasks
such as named entity recognition or relation classification in natural language

**Makes use of entity name**: NTN
first learns word vectors from an auxiliary news corpus, and
then initializes the representation of each entity by averaging
the vectors of words contained in its name. For example,
**the embedding of AlfredHitchcock is initialized by the
average word vectors of “alfred” and “hitchcock”**... This kind of methods model textual information
separately from KG facts, and hence fail to leverage
interactions between them.[src](doc:2019/05/knowledge_graph_embedding_a_su)

2019-08-03 A propos

A2N: Attending to Neighbors for Knowledge Graph Inference - ACL 2019

Tags:

2019-07-31 A propos

Knowledge Graphs: Technical Review

Tags:

2019-07-15 A propos

Nandana Mihindukulasooriya sur Twitter : "I wonder if there is a service for disambiguation and entity linking for @wikidata where I can pass a text and get Wikidata entities mentioned in it!

Tags:

2019-07-13 A propos

A Common Sense View of Knowledge Graphs | AI3:::Adaptive Information

Tags:

2019-07-03 A propos

[1810.10531] A mathematical theory of semantic development in deep neural networks

Tags:

2019-06-29 A propos

[1812.00417] Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale

Tags:

2019-06-28 A propos

[1905.10070] Label-aware Document Representation via Hybrid Attention for Extreme Multi-Label Text Classification

Tags:

2019-06-22 A propos

Visual and conceptual grounding for text representation learning

Tags:

2019-06-06 A propos

Trip Report: AKBC 2019 (1st Conference on Automated Knowledge Base Construction)

Tags:

2019-06-03 A propos

A Hybrid Graph Model for Distant Supervision Relation Extraction | Springer for Research & Development (ESWC 2019)

Tags:

2019-05-30 A propos

Virtual Knowledge Graphs: An Overview of Systems and Use Cases | MIT Press Journals (2019)

Tags:

2019-05-28 A propos

Microsoft Concept Graph and Concept Tagging Release

Tags:

Microsoft Concept Graph

2019-05-28 A propos

Microsoft Concept Graph: Mining Semantic Concepts for Short Text Understanding | MIT Press Journals (2019)

Tags:

2019-05-28 A propos

Knowledge Graph Embedding: A Survey of Approaches and Applications - IEEE (2017)

Tags:

- "facts only" based embeddings
    - Translational Distance Models, distance-based scoring functions
       - TransE and co: entities and relations as points in vector space
           - TransH: relation-specific hyperplanes.
       - Gaussian Embeddings: entities and relations as random variables
    - Semantic matching models, similarity-based scoring functions
        - RESCAL (entities as a vector, relation as a matrix) and familiy
            - DistMult (relation as diagonal matrix)
            - Holographic Embeddings (HolE)
            - etc.
        - Matching with Neural Networks
            - SME, etc
- incorporating other info: entity types, relation paths,
textual descriptions, as well as logical rules
    - entity types
        - semantically smooth embedding
(SSE):  entities of the same type to stay close
to each other in the embedding space
        - type-embodied knowledge representation
learning (TKRL), which can handle hierarchical
entity categories and multiple category labels
- Applications in downstream tasks

2019-05-27 A propos

Knowledge Graph Technology and Applications 2019 - Simia

Tags:

2019-05-23 A propos

Supercharging Elasticsearch for extended Knowledge Graph use cases: Data science + business analytics training: Strata Data Conference

Tags:

2019-05-16 A propos

Knowledge Graph Conference 2019, Day 1 - Simia

Tags:

2019-05-09 A propos

Learning to Map Wikidata Entities To Predefined Topics

Tags:

2019-05-03 A propos

open-knowledge-embeddings.pdf - (2019)

Tags:

2019-04-29 A propos

Federated KBs at AKBC2019

Tags:

2019-04-29 A propos

[1810.09164] Named Entity Disambiguation using Deep Learning on Graphs

Tags:

2019-04-26 A propos

Dandelion API - Semantic Text Analytics as a service

Tags:

2019-04-24 A propos

thunlp/OpenKE: An Open-Source Package for Knowledge Embedding (KE)

Tags:

2019-04-23 A propos

Wikify!: linking documents to encyclopedic knowledge (2007)

Tags:

2019-04-16 A propos

2019 Knowledge Graph Conference - Knowledge Graphs for AI in the Enterprise

Tags:

Knowledge Graph Conference 2019

2019-04-11 A propos

Accenture/AmpliGraph: Python library for Representation Learning on Knowledge Graphs

Tags:

2019-03-25 A propos

[1903.05872] Interactive Concept Mining on Personal Data -- Bootstrapping Semantic Services

Tags:

2019-03-17 A propos

Weak Supervision: A New Programming Paradigm for Machine Learning | SAIL Blog

Tags:

2019-03-12 A propos

Integrating Domain-Knowledge into Deep Learning (2019)

Tags:

2019-03-03 A propos

[1901.11504] Multi-Task Deep Neural Networks for Natural Language Understanding

Tags:

2019-02-17 A propos

Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction - ACL Anthology

Tags:

2019-02-09 A propos

BPEmb: Subword Embeddings

Tags:

2019-01-31 A propos

[1601.01343] Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation

Tags:

2019-01-27 A propos

Romain Vial (Hyperlex) at Paris NLP meetup, slides

Tags:

2019-01-24 A propos

Wikipedia2Vec

Tags:

2018-12-23 A propos

Deep Learning with Knowledge Graphs – Octavian – Medium

Tags:

Knowledge Graph + Deep Learning

2018-12-09 A propos

Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction

Tags:

2018-11-04 A propos

Ontotext | Semantic Technology Developer

Tags:

2018-10-13 A propos

[1809.01797] Describing a Knowledge Base

Tags:

2018-09-07 A propos

[1809.00782] Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text

Tags:

2018-09-06 A propos

HyperE: Hyperbolic Embeddings for Entities

Tags:

2018-07-27 A propos

Extending NLP - Stardog

Tags:

Stardog

2018-06-14 A propos

A Tri-Partite Neural Document Language Model for Semantic Information Retrieval (2018 - ESWC conference)

Tags:

2018-06-08 A propos

D2KLab/entity2rec: entity2rec generates item recommendation from knowledge graphs

Tags:

2018-06-04 A propos

Stardog: The Enterprise Knowledge Graph Platform

Tags:

Enterprise Knowledge Graph

2018-05-31 A propos

Tutorial on Poincaré Embeddings (Jupyter Notebook )

Tags:

2018-05-20 A propos

Implementing Poincaré Embeddings | RARE Technologies

Tags:

2018-05-20 A propos

Weakly-supervised Relation Extraction by Pattern-enhanced Embedding Learning

Tags:

2018-05-10 A propos

[1804.04526] EventKG: A Multilingual Event-Centric Temporal Knowledge Graph

Tags:

2018-04-15 A propos

[1804.01486] Clinical Concept Embeddings Learned from Massive Sources of Multimodal Medical Data

Tags:

2018-04-14 A propos

ML-knn: A lazy learning approach to multi-label learning (2007)

Tags:

a lazy learning approach derived from the traditional [k-nearest neighbors algorithm](tag:k_nearest_neighbors_algorithm)

> for each unseen instance, its K nearest neighbors in the training set are firstly identified. After that, based on statistical information gained from the label sets of these neighboring instances, i.e. the number of neighboring instances belonging to each possible class, [Maximum a posteriori (MAP)](tag:maximum_a_posteriori_estimation) principle is utilized to determine the label set for the unseen instance.

Implemented in [scikit-multilearn](http://scikit.ml/api/skmultilearn.adapt.mlknn.html), in [java](https://github.com/lefman/mulan-extended/blob/master/mulan/src/mulan/classifier/lazy/MLkNN.java)

> the first lazy approach proposed specifically for multi-label classification. This is also a binary relevance approach which considers each label independently as a binary classification problem. Instead of a standard k-NN method, however, MLkNN uses the maximum a-posteriori (MAP) (Kelleher et al., 2015) approach combined with k-NN. [src](https://pdfs.semanticscholar.org/af9b/33da37d290c063cd826ab5923d96892a9767.pdf)

2018-03-18 A propos

Convolutional 2D Knowledge Graph Embeddings (2017)

Tags:

2018-02-26 A propos

[1710.04099] Wembedder: Wikidata entity embedding web service

Tags:

2018-02-13 A propos

Modèle Neuronal de Recherche d'Information Augmenté par une Ressource Sémantique (2017)

Tags:

2018-02-12 A propos

Workshop on Deep Learning for Knowledge Graphs and Semantic Technologies

Tags:

2018-02-01 A propos

Knowledge Graph Embedding by Translating on Hyperplanes (2014)

Tags:

2018-01-30 A propos

WORKSHOP: BigNet @ WWW 2018 Workshop on Learning Representations for Big Networks

Tags:

2018-01-27 A propos

Knowledge base completion by learning pairwise-interaction differentiated embeddings | SpringerLink (2015)

Tags:

2018-01-27 A propos

LEARNING GRAPH EMBEDDINGS FOR NODE LABELING AND INFORMATION DIFFUSION IN SOCIAL NETWORKS (2017)

Tags:

2018-01-23 A propos

Knowledge Graph and Text Jointly Embedding (2014)

Tags:

2018-01-05 A propos

Translating Embeddings for Modeling Multi-relational Data (2013)

Tags:

This work focuses on modeling multi-relational
data from KBs (Wordnet and Freebase in this paper), with the goal of providing an efficient
tool to complete them by automatically adding new facts, without requiring extra knowledge.

**Embedding entities and relationships of multirelational
data**: a method which **models relationships by interpreting them as translations** operating on the
low-dimensional embeddings of the entities. Motivation:
- hierarchical relationships are extremely common in KBs and translations are the natural transformations for representing them.
- cf. word embeddings and the “capital of” relationship between countries and cities, which are (coincidentally rather than willingly) represented by the model as translations in the embedding space. This suggests that there may exist embedding spaces in which 1-to-1 relationships between entities of different types may, as well, be represented by translations. The intention of our model is to enforce such a structure of the embedding space.

[Good blog post by PY Vandenbussche](http://pyvandenbussche.info/2017/translating-embeddings-transe/)

2018-01-05 A propos

Combining semantics and deep learning for intelligent information services

Tags:

2018-01-03 A propos

Global RDF Vector Space Embeddings

Tags:

RDF embeddings

2018-01-03 A propos

RDF2Vec: RDF Graph Embeddings for Data Mining - (2016)

Tags:

RDF2VEC

2018-01-03 A propos

RDF2Vec: RDF Graph Embeddings for Data Mining

Tags:

RDF2VEC

2018-01-03 A propos

Awesome Knowledge Graph Embedding Approaches

Tags:

Knowledge Graph Embeddings

2018-01-03 A propos

Learning Concept-Driven Document Embeddings for Medical Information Search (2017)

Tags:

2018-01-03 A propos

Combining word and entity embeddings for entity linking (ESWC 2017)

Tags:

2017-12-30 A propos

[1705.08039] Poincaré Embeddings for Learning Hierarchical Representations

Tags:

2017-12-16 A propos

wikidata-taxonomy

Tags:

2017-10-29 A propos

Towards a Seamless Integration of Word Senses into Downstream NLP Applications (2017)

Tags:

2017-10-21 A propos

Andrew Moore on "TOKeN: The Open Knowledge Network" - YouTube

Tags:

Open Knowledge Network

2017-10-21 A propos

Open Knowledge Network

Tags:

2017-10-21 A propos

Knowledge Maps: Structure Versus Meaning - DATAVERSITY

Tags:

2017-10-19 A propos

Enriching Word Embeddings Using Knowledge Graph for Semantic Tagging in Conversational Dialog Systems - Microsoft Research (2015)

Tags:

> new simple, yet effective approaches to
learn domain specific word embeddings.

## Intro

> Adapting word embeddings, such as jointly capturing
syntactic and semantic information, can further enrich semantic
word representations for several tasks, e.g., sentiment
analysis (Tang et al. 2014), named entity recognition
(Lebret, Legrand, and Collobert 2013), entity-relation extraction
(Weston et al. 2013), etc. (Yu and Dredze 2014)
has introduced a lightly supervised word embedding learning
extending word2vec. They incorporate prior information to the objective
function as a regularization term considering synonymy relations
between words from Wordnet (Fellbaum 1999).

> In this work, we go one step further and investigate if
enriching the word2vec word embeddings trained on unstructured/
unlabeled text with domain specific semantic relations
obtained from knowledge sources (e.g., knowledge
graphs, search query logs, etc.) can help to discover relation
aware word embeddings. Unlike earlier work, **we encode the
information about the relations between phrases, thereby,
entities and relation mentions are all embedded into a low dimensional
vector space**.

## Related work (Learning Word Embeddings with Priors)

- word2vec
- Relational Constrained Model (RTM) (Yu and Dredze 2014)
While CBOW learns lexical word embeddings from provided text, the RTM learns embeddings of words based on their similarity to other words provided by a knowledge resource (eg. wordnet)
- Joint model  (Yu and Dredze 2014)
combines CBOW and RTM through linear combination

2017-10-02 A propos

ConceptNet

Tags:

2017-09-18 A propos

Translating Embeddings (TransE) – Pierre-Yves Vandenbussche

Tags:

2017-09-09 A propos

Burger King’s new ad forces Google Home to advertise the Whopper - The Verge

Tags:

2017-04-13 A propos

What IBM, the Semantic Web Company, and Siemens are doing with semantic technologies | ZDNet

Tags:

2016-09-17 A propos

One Genius’ Lonely Crusade to Teach a Computer Common Sense | WIRED

Tags:

2016-03-27 A propos

IBM Watson APIs hold key to broader cognitive computing use

Tags:

2015-12-30 A propos

Google Knowledge Graph Search API | Knowledge Graph Search API | Google Developers

Tags:

Google Knowledge Graph

2015-12-19 A propos

[1506.01094] Traversing Knowledge Graphs in Vector Space

Tags:

2015-10-31 A propos

Wikinizer: Introducing MindGraph | LinkedUp: Linking Web Data for Education - An EU project about the potential of open data in education

Tags:

2014-12-22 A propos

Machine Learning with Knowledge Graphs - VideoLectures.NET

Tags:

2014-10-06 A propos

Cognitive Computing Makes It Possible to Build Truly Amazing Apps | ProgrammableWeb

Tags:

Cognitive computing

2014-09-09 A propos

Big Structure: At The Nexus of Knowledge Bases, the Semantic Web and Artificial Intelligence | AI3:::Adaptive Information

Tags:

Neither the semantic Web nor linked data has developed the practices, tooling or experience to actually interoperate data across the Web... Where it is compliant and from authoritative information sources, linked data can be a gold standard in data publishing. But, linked data is neither necessary nor essential, and may even be a diversion if it sucks the air from the room for what is more broadly useful.<br/>
Interoperability requires reference structures, what we are calling Big Structure.<br/>
Most analysts see firms that are actively pursuing data integration innovations as forward-thinking and more competitive.<br/>
The semantic Web, in our view, is properly understood as a sub-domain of artificial intelligence. Semantic technologies mesh smoothly with natural language tasks and objectives.  But, as we noted in a recent review article, artificial intelligence is itself undergoing a renaissance. These advances are coming about because of the use of knowledge-based AI (KBAI), which combines knowledge bases with machine learning and other AI approaches. Natural language and spoken interfaces combined with background knowledge and a few machine-language utilities are what underlie Apple’s Siri, for example.<br/>

2014-07-24 A propos

Google I/O 2014 Recap: Android, Knowledge Graph and more | Alexandre Passant

Tags:

2014-07-02 A propos

How to Build Your Own Knowledge Graph (Video – Part 1) - Semanticweb.com

Tags:

2014-06-27 A propos

Knowledge-base Extractor (github)

Tags:

2014-05-29 A propos

How To Tell Search Engines What "Entities" Are On Your Web Pages

Tags: