Semanlink - [1410.5859] Towards a Model Theory for Distributed Representations

[1410.5859] Towards a Model Theory for Distributed Representations

Tags:

Au sujet de ce document

sl:arxiv_author : Ramanathan Guha
sl:arxiv_firstAuthor : Ramanathan Guha
sl:arxiv_num : 1410.5859
sl:arxiv_published : 2014-10-21T21:15:45Z
sl:arxiv_summary : Distributed representations (such as those based on embeddings) and discrete representations (such as those based on logic) have complementary strengths. We explore one possible approach to combining these two kinds of representations. We present a model theory/semantics for first order logic based on vectors of reals. We describe the model theory, discuss some interesting properties of such a system and present a simple approach to query answering.@en
sl:arxiv_title : Towards a Model Theory for Distributed Representations@en
sl:arxiv_updated : 2015-02-05T03:06:09Z
sl:bookmarkOf : https://arxiv.org/abs/1410.5859
sl:creationDate : 2021-06-10
sl:creationTime : 2021-06-10T16:30:07Z

Infos sur le fichier

Bookmark of: https://arxiv.org/abs/1410.5859

Documents with similar tags (experimental)

[2308.00081] Towards Semantically Enriched Embeddings for Knowledge Graph Completion

Tags:

2023-08-02 A propos

[2009.12030] AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding

Tags:

2021-05-17 A propos

[1806.06478] Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment

Tags:

2020-09-06 A propos

[1803.07828] Expeditious Generation of Knowledge Graph Embeddings

Tags:

2020-09-02 A propos

[2009.00318] More is not Always Better: The Negative Impact of A-box Materialization on RDF2vec Knowledge Graph Embeddings

Tags:

2020-09-02 A propos

[2007.00849] Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge

Tags:

> a neural language model that includes **an explicit interface between symbolically interpretable factual information and subsymbolic neural knowledge.**... **The model can be updated without re-training by manipulating its symbolic representations**. In particular this model allows us to add new facts and overwrite existing ones.

> a **neural language model which learns to access information
in a symbolic knowledge graph.**

> This
model builds on the recently-proposed [Entities as
Experts](doc:2020/07/2004_07202_entities_as_expert) (EaE) language model (Févry et al., 2020),
which extends the same transformer (Vaswani
et al., 2017) architecture of BERT (Devlin et al., 2019) with an additional external memory for entities.
>
> After training EaE, the embedding associated
with an entity will (ideally) capture information
about the textual context in which that
entity appears, and by inference, the entity’s semantic
properties
>
> we include an additional
memory called a fact memory, which encodes
triples from a symbolic KB.
>
> This combination results in a
neural language model which learns to access information
in a the symbolic knowledge graph.

TODO:

- read again IBM's [Span Selection Pre-training for Question Answering](doc:2019/09/_1909_04120_span_selection_pre) ("an effort to avoid encoding general knowledge in the transformer network itself")
- compare with [[1907.05242] Large Memory Layers with Product Keys](doc:2019/07/_1907_05242_large_memory_layer)
- how does it relate with [[2002.08909] REALM: Retrieval-Augmented Language Model Pre-Training](doc:2020/12/2002_08909_realm_retrieval_a)?

2020-07-09 A propos

[2006.13365] Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework

Tags:

2020-06-26 A propos

[1910.12507] A Survey on Knowledge Graph Embeddings with Literals: Which model links better Literal-ly?

Tags:

2020-05-04 A propos

[2004.14843] Knowledge Graph Embeddings and Explainable AI

Tags:

2020-05-04 A propos

[1906.01195] Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs

Tags:

2020-04-30 A propos

[1911.02168] CoKE: Contextualized Knowledge Graph Embedding

Tags:

2020-03-22 A propos

[1902.10197] RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

Tags:

2020-03-03 A propos

[1807.00082] Amanuensis: The Programmer's Apprentice

Tags:

**The use of natural language to facilitate communication
between the expert programmer and apprentice AI system.**

> an overview of the material covered in a course taught at Stanford in the spring quarter of 2018. The course draws upon **insight from cognitive and systems neuroscience to implement hybrid connectionist and symbolic reasoning systems** that leverage and extend the state of the art in machine learning **by integrating human and machine intelligence**. As a concrete example we focus on digital assistants that learn from continuous dialog with an expert software engineer while providing initial value as powerful analytical, computational and mathematical savants.

> [#Dehaene](/tag/stanislas_dehaene)'s work extends the [#Global Workspace Theory](/tag/global_workspace_theory) of Bernard Baars. Dehaene’s version of the theory combined with Yoshua Bengio’s concept of a [#consciousness prior](/tag/consciousness_prior.html) and deep reinforcement learning suggest a model for constructing and maintaining the cognitive states that arise and persist during complex problem solving.

2019-11-12 A propos

[1907.03950] Learning by Abstraction: The Neural State Machine

Tags:

2019-07-10 A propos

[1905.12149] SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver

Tags:

2019-05-31 A propos

[1806.01261] Relational inductive biases, deep learning, and graph networks

Tags:

2018-06-13 A propos

[1801.00631] Deep Learning: A Critical Appraisal

Tags:

2018-01-03 A propos

[1506.01094] Traversing Knowledge Graphs in Vector Space

Tags:

2015-10-31 A propos