Semanlink - Favoris ; Machine learning: techniques AND NN / Symbolic AI hybridation

Deep Learning for AI | July 2021 | Communications of the ACM

Tags:

2021-08-02 About

[1410.5859] Towards a Model Theory for Distributed Representations

Tags:

2021-06-10 About

[2007.00849] Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge

Tags:

> a neural language model that includes **an explicit interface between symbolically interpretable factual information and subsymbolic neural knowledge.**... **The model can be updated without re-training by manipulating its symbolic representations**. In particular this model allows us to add new facts and overwrite existing ones.

> a **neural language model which learns to access information
in a symbolic knowledge graph.**

> This
model builds on the recently-proposed [Entities as
Experts](doc:2020/07/2004_07202_entities_as_expert) (EaE) language model (Févry et al., 2020),
which extends the same transformer (Vaswani
et al., 2017) architecture of BERT (Devlin et al., 2019) with an additional external memory for entities.
>
> After training EaE, the embedding associated
with an entity will (ideally) capture information
about the textual context in which that
entity appears, and by inference, the entity’s semantic
properties
>
> we include an additional
memory called a fact memory, which encodes
triples from a symbolic KB.
>
> This combination results in a
neural language model which learns to access information
in a the symbolic knowledge graph.

TODO:

- read again IBM's [Span Selection Pre-training for Question Answering](doc:2019/09/_1909_04120_span_selection_pre) ("an effort to avoid encoding general knowledge in the transformer network itself")
- compare with [[1907.05242] Large Memory Layers with Product Keys](doc:2019/07/_1907_05242_large_memory_layer)
- how does it relate with [[2002.08909] REALM: Retrieval-Augmented Language Model Pre-Training](doc:2020/12/2002_08909_realm_retrieval_a)?

2020-07-09 About

[2003.00330] Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective

Tags:

2020-03-15 About

From Deep Learning of Disentangled Representations to Higher-level Cognition - YouTube

Tags:

2018-09-28 About

[1806.01261] Relational inductive biases, deep learning, and graph networks

Tags:

2018-06-13 About

[1801.00631] Deep Learning: A Critical Appraisal

Tags:

2018-01-03 About

[1709.08568] The Consciousness Prior

Tags:

2017-09-29 About