Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Wan-Duo Kurt Ma
- sl:arxiv_num : 1908.01580
- sl:arxiv_published : 2019-08-05T12:23:24Z
- sl:arxiv_summary : We introduce the HSIC (Hilbert-Schmidt independence criterion) bottleneck for
training deep neural networks. The HSIC bottleneck is an alternative to the
conventional cross-entropy loss and backpropagation that has a number of
distinct advantages. It mitigates exploding and vanishing gradients, resulting
in the ability to learn very deep networks without skip connections. There is
no requirement for symmetric feedback or update locking. We find that the HSIC
bottleneck provides performance on MNIST/FashionMNIST/CIFAR10 classification
comparable to backpropagation with a cross-entropy target, even when the system
is not encouraged to make the output resemble the classification labels.
Appending a single layer trained with SGD (without backpropagation) to reformat
the information further improves performance.@en
- sl:arxiv_title : The HSIC Bottleneck: Deep Learning without Back-Propagation@en
- sl:arxiv_updated : 2019-12-05T09:24:24Z
- sl:bookmarkOf : https://arxiv.org/abs/1908.01580
- sl:creationDate : 2019-08-15
- sl:creationTime : 2019-08-15T17:13:21Z
Documents with similar tags (experimental)
Préférences...