Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Léonard Blier
- sl:arxiv_num : 1802.07044
- sl:arxiv_published : 2018-02-20T10:15:26Z
- sl:arxiv_summary : Solomonoff's general theory of inference and the Minimum Description Length
principle formalize Occam's razor, and hold that a good model of data is a
model that is good at losslessly compressing the data, including the cost of
describing the model itself. Deep neural networks might seem to go against this
principle given the large number of parameters to be encoded.
We demonstrate experimentally the ability of deep neural networks to compress
the training data even when accounting for parameter encoding. The compression
viewpoint originally motivated the use of variational methods in neural
networks. Unexpectedly, we found that these variational methods provide
surprisingly poor compression bounds, despite being explicitly built to
minimize such bounds. This might explain the relatively poor practical
performance of variational methods in deep learning. On the other hand, simple
incremental encoding methods yield excellent compression values on deep
networks, vindicating Solomonoff's approach.@en
- sl:arxiv_title : The Description Length of Deep Learning Models@en
- sl:arxiv_updated : 2018-11-01T11:23:09Z
- sl:bookmarkOf : https://arxiv.org/abs/1802.07044
- sl:creationDate : 2019-10-11
- sl:creationTime : 2019-10-11T01:59:35Z
Documents with similar tags (experimental)
Préférences...