Semanlink - NN / Symbolic AI hybridation AND Machine learning

Stéphane d'Ascoli sur X : "Think Transfomers are terrible at logical reasoning? Think again. Transformers trained with Boolean inputs and symbolic outputs..."

Tags:

2023-09-26 About

Gary Marcus 🇺🇦 sur Twitter : "Hybrid neurosymbolic system beats end-to-end deep learning in autonomous driving study from Caltech." / Twitter

Tags:

2022-04-06 About

Deep Learning for AI | July 2021 | Communications of the ACM

Tags:

2021-08-02 About

[1410.5859] Towards a Model Theory for Distributed Representations

Tags:

2021-06-10 About

[2007.00849] Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge

Tags:

> a neural language model that includes **an explicit interface between symbolically interpretable factual information and subsymbolic neural knowledge.**... **The model can be updated without re-training by manipulating its symbolic representations**. In particular this model allows us to add new facts and overwrite existing ones.

> a **neural language model which learns to access information
in a symbolic knowledge graph.**

> This
model builds on the recently-proposed [Entities as
Experts](doc:2020/07/2004_07202_entities_as_expert) (EaE) language model (Févry et al., 2020),
which extends the same transformer (Vaswani
et al., 2017) architecture of BERT (Devlin et al., 2019) with an additional external memory for entities.
>
> After training EaE, the embedding associated
with an entity will (ideally) capture information
about the textual context in which that
entity appears, and by inference, the entity’s semantic
properties
>
> we include an additional
memory called a fact memory, which encodes
triples from a symbolic KB.
>
> This combination results in a
neural language model which learns to access information
in a the symbolic knowledge graph.

TODO:

- read again IBM's [Span Selection Pre-training for Question Answering](doc:2019/09/_1909_04120_span_selection_pre) ("an effort to avoid encoding general knowledge in the transformer network itself")
- compare with [[1907.05242] Large Memory Layers with Product Keys](doc:2019/07/_1907_05242_large_memory_layer)
- how does it relate with [[2002.08909] REALM: Retrieval-Augmented Language Model Pre-Training](doc:2020/12/2002_08909_realm_retrieval_a)?

2020-07-09 About

End-to-End Learning with Text & Knowledge Bases (Bhuwan Dhingra PhD Thesis)

Tags:

2020-07-06 About

[2003.00330] Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective

Tags:

2020-03-15 About

Max Little sur Twitter : "Causal bootstrapping - a simple way of doing causal inference using arbitrary machine learning algo..."

Tags:

2020-03-08 About

[1807.00082] Amanuensis: The Programmer's Apprentice

Tags:

**The use of natural language to facilitate communication
between the expert programmer and apprentice AI system.**

> an overview of the material covered in a course taught at Stanford in the spring quarter of 2018. The course draws upon **insight from cognitive and systems neuroscience to implement hybrid connectionist and symbolic reasoning systems** that leverage and extend the state of the art in machine learning **by integrating human and machine intelligence**. As a concrete example we focus on digital assistants that learn from continuous dialog with an expert software engineer while providing initial value as powerful analytical, computational and mathematical savants.

> [#Dehaene](/tag/stanislas_dehaene)'s work extends the [#Global Workspace Theory](/tag/global_workspace_theory) of Bernard Baars. Dehaene’s version of the theory combined with Yoshua Bengio’s concept of a [#consciousness prior](/tag/consciousness_prior.html) and deep reinforcement learning suggest a model for constructing and maintaining the cognitive states that arise and persist during complex problem solving.

2019-11-12 About

[1907.03950] Learning by Abstraction: The Neural State Machine

Tags: