Semanlink - Time-Contrastive Networks: Self-Supervised Learning from Video (2017)

Time-Contrastive Networks: Self-Supervised Learning from Video (2017)

Tags:

Self-supervised approach for learning representations and robotic behaviors entirely from unlabeled videos recorded from multiple viewpoints, and study how this representation can be used in two robotic imitation settings: imitating object interactions from videos of humans, and imitating human poses.

> We train our representations using a metric learning loss, where multiple simultaneous viewpoints of the same observation are attracted in the embedding space, while being repelled from temporal neighbors which are often visually similar but functionally different. In other words, the model simultaneously learns to recognize what is common between different-looking images, and what is different between similar-looking images.
> This signal causes our model to discover attributes that do not change across viewpoint, but do change across time, while ignoring nuisance variables such as occlusions, motion blur, lighting and background. We demonstrate that this representation can be used by a robot to directly mimic human poses without an explicit correspondence, and that it can be used as a reward function within a reinforcement learning algorithm.

About This Document

sl:creationDate : 2018-10-27
sl:creationTime : 2018-10-27T14:59:43Z

File info

Bookmark of: https://sermanet.github.io/imitate/