Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Esteban Real
- sl:arxiv_num : 2003.03384
- sl:arxiv_published : 2020-03-06T19:00:04Z
- sl:arxiv_summary : Machine learning research has advanced in multiple aspects, including model
structures and learning methods. The effort to automate such research, known as
AutoML, has also made significant progress. However, this progress has largely
focused on the architecture of neural networks, where it has relied on
sophisticated expert-designed layers as building blocks---or similarly
restrictive search spaces. Our goal is to show that AutoML can go further: it
is possible today to automatically discover complete machine learning
algorithms just using basic mathematical operations as building blocks. We
demonstrate this by introducing a novel framework that significantly reduces
human bias through a generic search space. Despite the vastness of this space,
evolutionary search can still discover two-layer neural networks trained by
backpropagation. These simple neural networks can then be surpassed by evolving
directly on tasks of interest, e.g. CIFAR-10 variants, where modern techniques
emerge in the top algorithms, such as bilinear interactions, normalized
gradients, and weight averaging. Moreover, evolution adapts algorithms to
different task types: e.g., dropout-like techniques appear when little data is
available. We believe these preliminary successes in discovering machine
learning algorithms from scratch indicate a promising new direction for the
field.@en
- sl:arxiv_title : AutoML-Zero: Evolving Machine Learning Algorithms From Scratch@en
- sl:arxiv_updated : 2020-03-06T19:00:04Z
- sl:bookmarkOf : https://arxiv.org/abs/2003.03384
- sl:creationDate : 2020-03-17
- sl:creationTime : 2020-03-17T21:57:40Z
Documents with similar tags (experimental)
Préférences...