Semanlink

Full interview: "Godfather of artificial intelligence" talks impact and potential of AI - YouTube

Tags:

```
2:45 – why mainstream AI in the 1980s shunned neural networks
3:21 – Hinton believed in neural networks because the brain worked this way 
4:31 – two different paths to intelligence
6:15 – lack of data and compute that impeded deep learning’s progress
8:57 – the start of deep learning in 2006
10:04 – two big deep learning developments: speech recognition at Google and object recognition at University of Toronto
10:49 – how object recognition works in layman’s terms
15:47 – breakthroughs in object recognition influencing the AI community
18:28 – why Hinton likes the company Cohere
19:20 – biological brains vs digital brains
21:39 – ChatGPT as an “idiot savant” with a lack of understanding of truth
24:56 – how society should handle this new AI
30:52 – self-healing minefield proposed by the government
31:43 – how to create an effective autonomous solder and the alignment problem
33:12 – are large language models “just autocomplete”? Are humans “just autocomplete”?
33:52 – translating “The trophy would not fit in the suitcase because it was too big” into French. English-French translation and the insights it provides into LLMs’ understanding of the world
35:50 – computers coming up with new ideas for itself
37:00 – AI displacing jobs
38:15 – how big of a revolution is AI?
40:37 – is AI sentient?
```

2023-03-27 About

Jim Fan sur Twitter : "...NVIDIA AI Foundations "

Tags:

2023-03-24 About

Andrej Karpathy sur Twitter : "Base LLMs (non-finetuned) make very strong few-shot classifiers. Describe task in English, give few examples, read off the label probabilities on test example. No gradient-based optimization necessary. It brings a cannon to a knife fight but is fast, convenient, strong baseline." / Twitter

Tags:

2023-03-19 About

Alpaca: A Strong Open-Source Instruction-Following Model

Tags:

2023-03-14 About

Sergey Levine sur Twitter : "What if we train a language model on images & robot data? That's the idea behind PaLM-E: a huge LLM (562B params) that is trained on language and "multimodal sentences" that include images and language"

Tags:

2023-03-08 About

Jerry Liu sur Twitter : "A key goal of @gpt_index is to enable end users to ask an LLM *any* questions over their own data..."

Tags:

2023-02-27 About

LLM Powered Assistants for Complex Interfaces - Nick Arner

Tags:

2023-02-26 About

abacaj/awesome-transformers: A curated list of awesome transformer models.

Tags:

2023-02-25 About

Guillaume Lample sur Twitter : "Today we release LLaMA, 4 foundation models ranging from 7B to 65B parameters..."

Tags:

2023-02-25 About

Maria Khalusova @maria@recsys.social sur Twitter : "Did you know that you can tweak the text output generated by a LLM without changing any of the trainable parameters?..."

Tags:

2023-02-23 About

[2302.08091] Do We Still Need Clinical Language Models?

Tags:

2023-02-17 About

[2203.14465] STaR: Bootstrapping Reasoning With Reasoning

Tags:

2023-02-07 About

Yann LeCun sur Twitter : "On the highway towards Human-Level AI, Large Language Model is an off-ramp."

Tags:

2023-02-05 About

deepset sur Twitter : " Generative models have taken the world of NLP by storm. But LLMs do not know about your personal data. This makes personal assistants, enterprise knowledge management and many other applications challenging. Retrieval augmented pipelines are the answer"

Tags:

2023-02-02 About

hwchase17/langchain: ⚡ Building applications with LLMs through composability ⚡

Tags:

LLM

2023-02-02 About

The Flan Collection: Advancing open source methods for instruction tuning – Google AI Blog

Tags:

2023-02-02 About

An empirical analysis of compute-optimal large language model training

Tags:

2023-01-26 About

Andrej Karpathy sur Twitter : "Great post (5mo ago) "chinchilla's wild implications" giving context to LLM goldrush shifting from model size to dataset size..."

Tags:

2023-01-05 About

Some remarks on Large Language Models

Tags:

> There turned out to be a phase shift somewhere between 60B parameters and 175B parameters, that made language models super impressive.

> **The performance of current days language models are not obtained by language modeling**
>
>    - [Traditional] LMs are not [grounded](tag:grounded_language_learning)
> 
> **3 conceptual steps between GPT-3 and chatGPT: Instructions, code, RLHF.** The last one is, I think, the least interesting despite getting the most attention
>
> Instruction tuning: For example, the human annotators would write something like "please summarize this text", followed by some text they got, followed by a summary they produced of this text. -> Some symbols ("summarize", "translate", "formal") are used in a consistent way together with the concept/task they denote. And they always appear in the beginning of the text. -> the act of producing a summary grounded to the human concept of "summary"
>
> code: programming language code data, and specifically data that contains both natural language instructions or descriptions (in the form of code comments) and the corresponding programming language code. This produced another very direct form of grounding. the human language describes concepts (or intents), which are then realized in the form of the corresponding programs.
>
> "[RL with Human Feedback](tag:reinforcement_learning_from_human_feedback)". This is a fancy way of saying that the model now observes two humans in a conversation, one playing the role of a user, and another playing the role of "the AI", demonstrating how the AI should respond in different situations. This clearly helps the model learn how dialogs work, and how to keep track of information across dialog states (something that is very hard to learn from just "found" data).

2023-01-03 About