Semanlink - Multi-task learning AND NLP

- Graph-Based Methods
    - Traditional solution:
        - Score each candidate, select the highest-scored output
        - Search-space typically exponential
- Transition-Based Methods
    - Transition-Based systems: Automata
        - State: partial result during decoding, Action: operations that can be applied for state transition
        - Output constructed incrementally

- Deep learning based model
    - Neural transition based models
    - Neural graph-based models
        - Cross task
            - Seminal work: Collobert, Ronan, et al. "Natural language processing (almost) from scratch."
            - Not all tasks are mutually beneficial
            - Ramachandran, et al.  “Unsupervised pretraining for sequence to sequence learning.”
            - Peters, Matthew E., et al. "Deep contextualized word representations." (ELMo)
            - "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding."
            - ULMFIT
            - Correlation between multi-task learning and pretraining
        - Cross lingual
        - Cross domain
        - Cross standard

2018-11-06 About

An Overview of Multi-Task Learning for Deep Learning

Tags:

2018-10-02 About

Slides motivating true multitask learning in AI and NLP

Tags:

2018-07-25 About

The Natural Language Decathlon: Multitask Learning as Question Answering (2018) Salesforce research

Tags:

2018-06-21 About

[1103.0398] Natural Language Processing (almost) from Scratch

Tags:

2018-01-17 About