Semanlink - Machine Learning for Humans, Part 5: Reinforcement Learning

Machine Learning for Humans, Part 5: Reinforcement Learning

Tags:

Reinforcement learning

About This Document

sl:bookmarkOf : https://medium.com/machine-learning-for-humans/reinforcement-learning-6eacf258b265
sl:creationDate : 2019-09-23
sl:creationTime : 2019-09-23T23:36:26Z

File info

Bookmark of: https://medium.com/machine-learning-for-humans/reinforcement-learning-6eacf258b265

Documents with similar tags (experimental)

Peter J. Liu sur Twitter : "RLHF-alternative without RL"

Tags:

2023-05-18 About

Aran Komatsuzaki sur Twitter : "Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning"

Tags:

2023-04-27 About

Reinforcement Learning for Language Models

Tags:

> I was puzzled for a while as to why we need RL for LM training, rather than just using supervised instruct tuning. I now have a convincing argument, which is also reflected in a recent talk by @johnschulman2.

1st convincing argument:

> supervised learning allows only positive feedback (we show the model a series of questions and their correct answers) while **RL allows also for negative feedback** (the model is allowed to generate an answer an get a feedback saying "this is not correct")...if you as a learner are allowed to form your own hypotheses and ask the teacher if they are correct (as in the RL setting), even an adversarial teacher can no longer trick you into latching on to a wrong hypothesis.

2nd convincing argument is about knowledge-seeking queries

> we want to encourage the model to answer based on its internal knowledge, but we don't know what this internal knowledge contains. In supervised training, we present the model with a question and its correct answer, and train the model to replicate the provided answer... But if we are succeed in training the model to generalize in [the cases it doesn't know], then we essentially teaches the model to make stuff up! it actively encourages the model to "lie".

2023-04-23 About

Prithviraj (Raj) Ammanabrolu sur Twitter : "The secret to aligning LMs to human preferences is reinforcement learning. ..."

Tags:

2022-10-06 About

Key Papers in Deep RL — OpenAI - Spinning Up documentation

Tags:

Reinforcement learning

2018-11-09 About

Time-Contrastive Networks: Self-Supervised Learning from Video (2017)

Tags:

Self-supervised approach for learning representations and robotic behaviors entirely from unlabeled videos recorded from multiple viewpoints, and study how this representation can be used in two robotic imitation settings: imitating object interactions from videos of humans, and imitating human poses.

> We train our representations using a metric learning loss, where multiple simultaneous viewpoints of the same observation are attracted in the embedding space, while being repelled from temporal neighbors which are often visually similar but functionally different. In other words, the model simultaneously learns to recognize what is common between different-looking images, and what is different between similar-looking images.
> This signal causes our model to discover attributes that do not change across viewpoint, but do change across time, while ignoring nuisance variables such as occlusions, motion blur, lighting and background. We demonstrate that this representation can be used by a robot to directly mimic human poses without an explicit correspondence, and that it can be used as a reward function within a reinforcement learning algorithm.

2018-10-27 About

Reinforcement Learning from scratch – Insight Data

Tags: