Semanlink - The Natural Language Decathlon: Multitask Learning as Question Answering (slides)

Tags:

About This Document

sl:bookmarkOf : https://web.stanford.edu/class/cs224n/slides/cs224n-2019-lecture17-multitask.pdf
sl:creationDate : 2021-01-26
sl:creationTime : 2021-01-26T14:43:51Z
sl:relatedDoc : https://github.com/salesforce/decaNLP

File info

Bookmark of: https://web.stanford.edu/class/cs224n/slides/cs224n-2019-lecture17-multitask.pdf

Linked From

The Natural Language Decathlon: Multitask Learning as Question Answering (2018) Salesforce research

Tags:

2018-06-21 About

Documents with similar tags (experimental)

Joint Models in NLP - Slides - Tutorial (EMNLP 2018) - Yue Zhang

Tags:

**Joint models: solve 2 tasks at once.**

Related tasks: POS tagging, NER, chuncking. Pipeline tasks

Motivations:

- reduce error propagation
- information exchange between tasks

Challenges:

- Joint learning
- Search

History: statistical models. 2 kinds:

- Graph-Based Methods
    - Traditional solution:
        - Score each candidate, select the highest-scored output
        - Search-space typically exponential
- Transition-Based Methods
    - Transition-Based systems: Automata
        - State: partial result during decoding, Action: operations that can be applied for state transition
        - Output constructed incrementally

- Deep learning based model
    - Neural transition based models
    - Neural graph-based models
        - Cross task
            - Seminal work: Collobert, Ronan, et al. "Natural language processing (almost) from scratch."
            - Not all tasks are mutually beneficial
            - Ramachandran, et al.  “Unsupervised pretraining for sequence to sequence learning.”
            - Peters, Matthew E., et al. "Deep contextualized word representations." (ELMo)
            - "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding."
            - ULMFIT
            - Correlation between multi-task learning and pretraining
        - Cross lingual
        - Cross domain
        - Cross standard

2018-11-06 About

Slides motivating true multitask learning in AI and NLP

Tags:

2018-07-25 About

The Natural Language Decathlon: Multitask Learning as Question Answering (2018) Salesforce research

Tags:

2018-06-21 About