DL: why does it work?
http://www.semanlink.net/tag/dl_why_does_it_work
Documents tagged with DL: why does it work?[1802.07044] The Description Length of Deep Learning Models
http://www.semanlink.net/doc/2019/10/_1802_07044_the_description_le
> Solomonoff’s general theory of inference (Solomonoff, 1964) and the [Minimum Description Length Principle](tag:minimum_description_length_principle) (Grünwald, 2007; Rissanen, 2007) formalize [Occam's razor](tag:occam_s_razor), and hold that **a good model of data is a model that is good at losslessly
compressing the data, including the cost of describing the model itself**. Deep neural
networks might seem to go against this principle given the large number of
parameters to be encoded.
We demonstrate experimentally the ability of deep neural networks to compress
the training data even when accounting for parameter encoding.
2019-10-11T01:59:35ZNew Theory Cracks Open the Black Box of Deep Learning | Quanta Magazine
https://www.quantamagazine.org/new-theory-cracks-open-the-black-box-of-deep-learning-20170921/
A new idea called the “information bottleneck” is helping to explain the puzzling success of today’s artificial-intelligence algorithms — and might also explain how human brains learn.
2017-12-30T11:07:53ZThe Extraordinary Link Between Deep Neural Networks and the Nature of the Universe
https://www.technologyreview.com/s/602344/the-extraordinary-link-between-deep-neural-networks-and-the-nature-of-the-universe/
Nobody understands why deep neural networks are so good at solving complex problems. Now physicists say the secret is buried in the laws of physics.
2016-09-11T00:38:49Z