Semanlink - Emergence (AI)

Parents :

Emergence

Emergence (AI)

Amis :

Descendants

Emergence in LLMs

3 Documents (Liste longue)

Physics of AI - YouTube

Tags:

- Intelligence has emerged: why? how?
- Let's study this with *controlled experiments* and *toy models*
- Clean and clear insights that peer slightly behind the magic curtain

emergence; rise of scale. Phase transition: fct(compute time for training: nb of params, dataset size)

(12:25) Transformers: the jump when compared to traditional NN: instead on oparating on single input X, operates on a set of inputs. (cf. training to say whether an image contains 2 elements of the same kind). **The aptitude to compare elements in the input sequence enables analogies** - the essence of reasoning.(Classical NN treats one high dimensional vector, tranformers hanlde sets of vectors. Attention layer replaces learned filters W of classical NN by the other elements in the sequence. Attention module compares part of input against other part of input. "Relative machines rather than absolute". Sets are a very powerful level of abstraction)

emergence
> The truth is that nobody has a clue what is going on"
>
> "**Something unknown is doing we don't know what**" (A. Eddington)

being inspired by physics' methodology: controled experiments and toy mathematical models

2023-03-28 A propos

Matthew Honnibal sur Twitter : "Some of the things ChatGPT can do are emergent behaviours... Other things it can do have been specifically taught to it. Is there some speculative list somewhere about what tasks were supervised?" / Twitter

Tags:

2023-01-14 A propos

Unveiling Transformers with LEGO - YouTube

Tags:

2022-06-30 A propos

Propriétés

sl:creationDate : 2023-01-14
sl:creationTime : 2023-01-14T16:42:16Z
rdf:type : sl:Tag
skos:prefLabel : Emergence (AI)