Σχετικά με το έγγραφο αυτό
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Heinrich van Deventer
- sl:arxiv_num : 2208.05388
- sl:arxiv_published : 2022-08-10T15:10:35Z
- sl:arxiv_summary : Artificial neural networks (ANNs), despite their universal function
approximation capability and practical success, are subject to catastrophic
forgetting. Catastrophic forgetting refers to the abrupt unlearning of a
previous task when a new task is learned. It is an emergent phenomenon that
hinders continual learning. Existing universal function approximation theorems
for ANNs guarantee function approximation ability, but do not predict
catastrophic forgetting. This paper presents a novel universal approximation
theorem for multi-variable functions using only single-variable functions and
exponential functions. Furthermore, we present ATLAS: a novel ANN architecture
based on the new theorem. It is shown that ATLAS is a universal function
approximator capable of some memory retention, and continual learning. The
memory of ATLAS is imperfect, with some off-target effects during continual
learning, but it is well-behaved and predictable. An efficient implementation
of ATLAS is provided. Experiments are conducted to evaluate both the function
approximation and memory retention capabilities of ATLAS.@en
- sl:arxiv_title : ATLAS: Universal Function Approximator for Memory Retention@en
- sl:arxiv_updated : 2022-08-10T15:10:35Z
- sl:bookmarkOf : https://arxiv.org/abs/2208.05388
- sl:creationDate : 2022-08-28
- sl:creationTime : 2022-08-28T10:22:54Z
Προτιμήσεις...