by Dr Wei Wu (Microsoft Xiaolce - chatbot with 200 millions users in China) and Dr Rui Yan (Peking Univ)
- Chit-chat (casual, non goal oriented) open-domain. Must be relevant to the context and diverse (informative) to be engaging.
- why creating a chat? to prove an AI can speak like a human, commercial reasons, link to services.
Task oriented vs non task oriented: this tutorial is about the second one.
Retrieval based vs generation based.
Basic knowledge of DL for chatbots:
- word embeddings
- sentence embeddings (CNN, RNN)
- dialogue modeling: seq-to-seq with attention
Response selection for retrieval based chatbots:
- single turn response selection (slides 37-57)
- framework 1: matching with seq embeddings
- framework 2: matching with message-response interaction (46)
- extension of 1: KnowledgeMatching with External Knowledge (53)
- extension of 2: RepresentationsMatching with Multiple Levels of Representations (54)
- insights from comparison between 1 and 2 (57)
- multi turn response selection (62)
- context is now: mess + history
- again, 2 frameworks
Emerging directions (79):
- matching with better representations
- Self-Attention (82)
- fusing multiple types of repr. But how to fuse matters (83)
- pre-training
Learning a matching model for response selection (84)
Generation based models for chatbots:
- single turn generarion (89)
- Basic generation model
- seq2seq
- Attention
- Bi-directional modeling
- multi turn generation
- Contexts are important
- Context sensitive models
- Hierarchical context modeling
- Latent variable modeling
- Hierarchical memory networks
Diversity in conversations (99)
Content introducing (106)
Additional elements (113)
- Topics in cnversation
- Emotions
Persona in chat:
- Persona
- ...
- Knowledge
- Common sense
RL and Adversarial learning in conversations (125)
Evaluation (132)
Future trends:
- Reasoning in dialogues
- X-grounded dialogues