Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Andrew M. Saxe
- sl:arxiv_num : 1810.10531
- sl:arxiv_published : 2018-10-23T22:20:27Z
- sl:arxiv_summary : An extensive body of empirical research has revealed remarkable regularities
in the acquisition, organization, deployment, and neural representation of
human semantic knowledge, thereby raising a fundamental conceptual question:
what are the theoretical principles governing the ability of neural networks to
acquire, organize, and deploy abstract knowledge by integrating across many
individual experiences? We address this question by mathematically analyzing
the nonlinear dynamics of learning in deep linear networks. We find exact
solutions to this learning dynamics that yield a conceptual explanation for the
prevalence of many disparate phenomena in semantic cognition, including the
hierarchical differentiation of concepts through rapid developmental
transitions, the ubiquity of semantic illusions between such transitions, the
emergence of item typicality and category coherence as factors controlling the
speed of semantic processing, changing patterns of inductive projection over
development, and the conservation of semantic similarity in neural
representations across species. Thus, surprisingly, our simple neural model
qualitatively recapitulates many diverse regularities underlying semantic
development, while providing analytic insight into how the statistical
structure of an environment can interact with nonlinear deep learning dynamics
to give rise to these regularities.@en
- sl:arxiv_title : A mathematical theory of semantic development in deep neural networks@en
- sl:arxiv_updated : 2018-10-23T22:20:27Z
- sl:bookmarkOf : https://arxiv.org/abs/1810.10531
- sl:creationDate : 2019-06-29
- sl:creationTime : 2019-06-29T15:22:55Z
Documents with similar tags (experimental)
Préférences...