Semanlink - Peter J. Liu sur Twitter : "RLHF-alternative without RL"

Εκτύπωση

Βρες μου:

Search Doc:

Προτιμήσεις...

Peter J. Liu sur Twitter : "RLHF-alternative without RL"

Tags:

Σχετικά με το έγγραφο αυτό

sl:bookmarkOf : https://twitter.com/peterjliu/status/1659023597447565312?s=20
sl:creationDate : 2023-05-18
sl:creationTime : 2023-05-18T09:53:46Z

Πληροφορία αρχείου

Bookmark of: https://twitter.com/peterjliu/status/1659023597447565312?s=20

Documents with similar tags (experimental)

LlamaIndex 🦙 sur X : "RAFT - Retrieval Augmented Fine Tuning..."

Tags:

2024-03-28 About

Archit Sharma sur Twitter : "Direct Preference Optimization (DPO) allows you to fine-tune LMs directly from preferences via a simple classification loss, no RL required"

Tags:

2023-05-31 About

Eric sur Twitter : "...Introducing Direct Preference Optimization (DPO), a simple classification loss provably equivalent to RLHF"

Tags:

2023-05-31 About

Hyung Won Chung sur Twitter : "RLHF as an instance of using a learned objective function"

Tags:

2023-05-18 About

Niels Rogge sur Twitter : "Made some new demo notebooks! - fine-tune @MetaAI's SAM and @GoogleAI's Pix2Struct on custom data"

Tags:

2023-05-09 About

Aran Komatsuzaki sur Twitter : "JaxPruner: A concise library for sparsity research An open-source JAX-based pruning and sparse training library for machine learning research repo"

Tags:

2023-04-28 About

Aran Komatsuzaki sur Twitter : "Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning"

Tags:

2023-04-27 About

Reinforcement Learning for Language Models

Tags:

> I was puzzled for a while as to why we need RL for LM training, rather than just using supervised instruct tuning. I now have a convincing argument, which is also reflected in a recent talk by @johnschulman2.

1st convincing argument:

> supervised learning allows only positive feedback (we show the model a series of questions and their correct answers) while **RL allows also for negative feedback** (the model is allowed to generate an answer an get a feedback saying "this is not correct")...if you as a learner are allowed to form your own hypotheses and ask the teacher if they are correct (as in the RL setting), even an adversarial teacher can no longer trick you into latching on to a wrong hypothesis.

2nd convincing argument is about knowledge-seeking queries

> we want to encourage the model to answer based on its internal knowledge, but we don't know what this internal knowledge contains. In supervised training, we present the model with a question and its correct answer, and train the model to replicate the provided answer... But if we are succeed in training the model to generalize in [the cases it doesn't know], then we essentially teaches the model to make stuff up! it actively encourages the model to "lie".

2023-04-23 About

Shayne Longpre sur Twitter : "A 🧵 on @OpenAI LLM "Alignment" (e.g. #ChatGPT)..."

Tags:

2023-02-27 About

(((ل()(ل() 'yoav))))👾 sur Twitter : "there is this genre of papers that show you can train/tune only some subset of a network's weights, freezing the rest, and things still work as well as (or better than) full training/tuning..."

Tags:

2023-02-15 About

Shital Shah sur Twitter : "Surprising and important paper: TLDR; All the gains we get by first pre-training on large dataset and then fine tuning on small dataset could be obtained by just small dataset but with pre-training objective!!"

Tags:

2023-02-04 About

Tanishq Mathew Abraham sur Twitter : "Are you wondering how large language models like ChatGPT and InstructGPT actually work? One of the secret ingredients is RLHF... Let's dive into how RLHF works in 8 tweets!" / Twitter

Tags:

2022-12-28 About

Stephanie Chan sur Twitter : "Transformer inductive biases..."

Tags:

2022-10-14 About

Prithviraj (Raj) Ammanabrolu sur Twitter : "The secret to aligning LMs to human preferences is reinforcement learning. ..."

Tags:

2022-10-06 About

Yi Tay sur Twitter : "Don't retrieve, recite!..."

Tags:

2022-10-06 About

Tu Vu sur Twitter : "Enormous LMs like GPT-3 exhibit impressive few-shot performance, but w/ self-training a BERT base sized model can achieve much better results!

Tags:

2022-04-13 About

Ananya Kumar sur Twitter : "How should you fine-tune a large pretrained model (CLIP, SimCLR) robustly?..."

Tags:

2022-03-03 About