Semanlink - Emergence (AI)

Γονείς:

Emergence

Emergence (AI)

Σχετικές ετικέτες:

Απόγονοι

Emergence in LLMs

3 Έγγραφα (Όλα τα έγγραφα)

Physics of AI - YouTube

Tags:

- Intelligence has emerged: why? how?
- Let's study this with *controlled experiments* and *toy models*
- Clean and clear insights that peer slightly behind the magic curtain

emergence; rise of scale. Phase transition: fct(compute time for training: nb of params, dataset size)

(12:25) Transformers: the jump when compared to traditional NN: instead on oparating on single input X, operates on a set of inputs. (cf. training to say whether an image contains 2 elements of the same kind). **The aptitude to compare elements in the input sequence enables analogies** - the essence of reasoning.(Classical NN treats one high dimensional vector, tranformers hanlde sets of vectors. Attention layer replaces learned filters W of classical NN by the other elements in the sequence. Attention module compares part of input against other part of input. "Relative machines rather than absolute". Sets are a very powerful level of abstraction)

emergence
> The truth is that nobody has a clue what is going on"
>
> "**Something unknown is doing we don't know what**" (A. Eddington)

being inspired by physics' methodology: controled experiments and toy mathematical models

2023-03-28 About

Matthew Honnibal sur Twitter : "Some of the things ChatGPT can do are emergent behaviours... Other things it can do have been specifically taught to it. Is there some speculative list somewhere about what tasks were supervised?" / Twitter

Tags:

2023-01-14 About

Unveiling Transformers with LEGO - YouTube

Tags:

2022-06-30 About

Ιδιότητες

sl:creationDate : 2023-01-14
sl:creationTime : 2023-01-14T16:42:16Z
rdf:type : sl:Tag
skos:prefLabel : Emergence (AI)