Semanlink - Tutorial AND Chatbots

Nils Reimers sur Twitter : "Creating intent classes for chatbots is challenging This tutorial shows how to use sentence-transformers to find potentially overlapping intent classes and how to improve your data annotation work." / Twitter

Tags:

2022-02-19 About

Joint Intent Classification and Slot Filling with Transformers (Jupyter Notebook Viewer)

Tags:

2020-01-09 About

Deep Chit-Chat: deep learning for chatbots (EMNLP 2018 Tutorial)

Tags:

by Dr Wei Wu (Microsoft Xiaolce - chatbot with 200 millions users in China) and Dr Rui Yan (Peking Univ)

- Chit-chat (casual, non goal oriented) open-domain. Must be relevant to the context and diverse (informative) to be engaging.
- why creating a chat? to prove an AI can speak like a human, commercial reasons, link to services.

Task oriented vs non task oriented: this tutorial is about the second one.

Retrieval based vs generation based.

Basic knowledge of DL for chatbots:

- word embeddings
- sentence embeddings (CNN, RNN)
- dialogue modeling: seq-to-seq with attention

Response selection for retrieval based chatbots:

- single turn response selection (slides 37-57)
    - framework 1: matching with seq embeddings
    - framework 2: matching with message-response interaction (46)
    - extension of 1: KnowledgeMatching with External Knowledge (53)
    - extension of 2: RepresentationsMatching with Multiple Levels of Representations (54)
    - insights from comparison between 1 and 2 (57)
- multi turn response selection (62)
    - context is now: mess + history
    - again, 2 frameworks

Emerging directions (79):

- matching with better representations
    - Self-Attention (82)
    - fusing multiple types of repr. But how to fuse matters (83)
    - pre-training

Learning a matching model for response selection (84)

Generation based models for chatbots:

- single turn generarion (89)
    - Basic generation model
        - seq2seq
        - Attention
        - Bi-directional modeling
- multi turn generation
    - Contexts are important
    - Context sensitive models
    - Hierarchical context modeling
    - Latent variable modeling
    - Hierarchical memory networks

Diversity in conversations (99)

Content introducing (106)

Additional elements (113)

- Topics in cnversation
- Emotions

Persona in chat:

- Persona
- ...
- Knowledge
- Common sense

RL and Adversarial learning in conversations (125)

Evaluation (132)

Future trends:

- Reasoning in dialogues
- X-grounded dialogues

2018-11-06 About