Semanlink - [2304.02711] Structured prompt interrogation and recursive extraction of semantics (SPIRES): A method for populating knowledge bases using zero-shot learning

[2304.02711] Structured prompt interrogation and recursive extraction of semantics (SPIRES): A method for populating knowledge bases using zero-shot learning

Tags:

About This Document

sl:arxiv_author :
sl:arxiv_firstAuthor : J. Harry Caufield
sl:arxiv_num : 2304.02711
sl:arxiv_published : 2023-04-05T19:07:04Z
sl:arxiv_summary : Creating knowledge bases and ontologies is a time consuming task that relies on a manual curation. AI/NLP approaches can assist expert curators in populating these knowledge bases, but current approaches rely on extensive training data, and are not able to populate arbitrary complex nested knowledge schemas. Here we present Structured Prompt Interrogation and Recursive Extraction of Semantics (SPIRES), a Knowledge Extraction approach that relies on the ability of Large Language Models (LLMs) to perform zero-shot learning (ZSL) and general-purpose query answering from flexible prompts and return information conforming to a specified schema. Given a detailed, user-defined knowledge schema and an input text, SPIRES recursively performs prompt interrogation against GPT-3+ to obtain a set of responses matching the provided schema. SPIRES uses existing ontologies and vocabularies to provide identifiers for all matched elements. We present examples of use of SPIRES in different domains, including extraction of food recipes, multi-species cellular signaling pathways, disease treatments, multi-step drug mechanisms, and chemical to disease causation graphs. Current SPIRES accuracy is comparable to the mid-range of existing Relation Extraction (RE) methods, but has the advantage of easy customization, flexibility, and, crucially, the ability to perform new tasks in the absence of any training data. This method supports a general strategy of leveraging the language interpreting capabilities of LLMs to assemble knowledge bases, assisting manual knowledge curation and acquisition while supporting validation with publicly-available databases and ontologies external to the LLM. SPIRES is available as part of the open source OntoGPT package: https://github.com/ monarch-initiative/ontogpt.@en
sl:arxiv_title : Structured prompt interrogation and recursive extraction of semantics (SPIRES): A method for populating knowledge bases using zero-shot learning@en
sl:arxiv_updated : 2023-04-05T19:07:04Z
sl:bookmarkOf : https://arxiv.org/abs/2304.02711
sl:creationDate : 2023-04-07
sl:creationTime : 2023-04-07T14:49:22Z

File info

Bookmark of: https://arxiv.org/abs/2304.02711

Documents with similar tags (experimental)

[2308.00081] Towards Semantically Enriched Embeddings for Knowledge Graph Completion

Tags:

2023-08-02 About

[2210.13952] KnowGL: Knowledge Generation and Linking from Text

Tags:

2022-11-13 About

[2006.10713] Zero-Shot Learning with Common Sense Knowledge Graphs

Tags:

2022-08-29 About

[2108.13934] Robust Retrieval Augmented Generation for Zero-shot Slot Filling

Tags:

2022-01-19 About

[1712.05972] Train Once, Test Anywhere: Zero-Shot Learning for Text Classification

Tags:

2021-10-16 About

[2009.07938] Type-augmented Relation Prediction in Knowledge Graphs

Tags:

2020-09-19 About

[2009.00318] More is not Always Better: The Negative Impact of A-box Materialization on RDF2vec Knowledge Graph Embeddings

Tags:

2020-09-02 About

[1911.03903] A Re-evaluation of Knowledge Graph Completion Methods

Tags:

2020-07-28 About

[2006.15020] Pre-training via Paraphrasing

Tags:

2020-06-30 About

[2003.08001] Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study

Tags:

2020-05-15 About

[2001.09522] TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network

Tags:

how to add a set of new concepts to an existing taxonomy.

[Tweet](https://twitter.com/mickeyjs6/status/1253772146142216194?s=20) [GitHub](https://github.com/mickeystroller/TaxoExpan)

> we study the taxonomy expansion task: given an
existing taxonomy and a set of new emerging concepts, we aim
to automatically expand the taxonomy to incorporate these new
concepts (without changing the existing relations in the given taxonomy).

> To the best of our knowledge, this is the first study on **how to
expand an existing directed acyclic graph (as we model a taxonomy
as a DAG) using self-supervised learning**.

Self-supervised framework, the existing taxonomy being used as training data: it learns a model to predict whether a query concept is the direct hyponym of an anchor concept.

> 2 techniques:
>
> 1. a **position-enhanced graph neural network that encodes the local structure of an anchor concept** in the existing taxonomy,
> 2. a noise-robust training objective that enables the learned model to be insensitive to the label noise in the self-supervision data.

Regarding 1: uses [GNN](/tag/graph_neural_networks.html) to model the "ego network" of concepts (potential “siblings”
and “grand parents” of the query concept).

> Regular
GNNs fail to distinguish nodes with different relative positions to
the query (i.e., some nodes are grand parents of the query while
the others are siblings of the query). To address this limitation, we
present a simple but effective enhancement to inject such position
information into GNNs using position embedding. We show that
such embedding can be easily integrated with existing GNN architectures
(e.g., [GCN](/tag/graph_convolutional_networks) and GAT) and significantly boosts the
prediction performance

Regarding point 2: uses InfoNCE loss, cf. [Contrastive Predictive Coding](/doc/?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1807.03748)

> Instead of predicting
whether each individual ⟨query concept, anchor concept⟩ pair
is positive or not, we first group all pairs sharing the same query
concept into a single training instance and learn a model to select
the positive pair among other negative ones from the group.

(Hum, ça me rappelle quelque chose)

> assume each concept (in existing taxonomy + set of new concepts) has an initial embedding
vector learned from some text associated with this concept.

To keep things tractable, only attempts to find a single parent node of each new concept.

2020-04-25 About

[1909.03193] KG-BERT: BERT for Knowledge Graph Completion

Tags:

2020-03-22 About

[1902.10197] RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

Tags:

2020-03-03 About

[1703.07464] No Fuss Distance Metric Learning using Proxies

Tags:

> We address the problem of distance metric learning (DML), defined as learning a distance consistent with a notion of semantic similarity...
> Traditionnaly, supervision is expressed in the form of sets of points that follow
an ordinal relationship – an anchor point x is similar to
a set of positive points Y , and dissimilar to a set of negative
points Z, and a loss defined over these distances is minimized.
> Triplet-Based methods are challenging to optimize (a main issue is the need for finding informative triplets).
>
> We propose to **optimize the triplet loss on a different space of triplets, consisting of an anchor data point and similar and dissimilar proxy points which are learned as well**. These proxies approximate the original data points, so that a triplet loss over the proxies is a tight upper bound of the original loss.

Mentioned in this [blog post](/doc/2020/01/training_a_speaker_embedding_fr):

> "**Proxy based triplet learning**": instead of generating triplets, we learn an embedding for each class and use the learnt embedding as a proxy for triplets as part of the training. In other words, we can train end to end without the computationally expensive step of resampling triplets after each network update.

Near the conclusion:

> Our formulation of Proxy-NCA loss produces a loss very
similar to the standard cross-entropy loss used in classification.
However, we arrive at our formulation from a different
direction: we are not interested in the actual classifier and
indeed discard the proxies once the model has been trained.
Instead, the proxies are auxiliary variables, enabling more
effective optimization of the embedding model parameters.
**As such, our formulation not only enables us to surpass the
state of the art in zero-shot learning, but also offers an explanation
to the effectiveness of the standard trick of training
a classifier, and using its penultimate layer’s output as the
embedding.**

2020-02-09 About

[1506.01094] Traversing Knowledge Graphs in Vector Space

Tags:

2015-10-31 About