Semanlink - EMNLP ; Multi-task learning AND Multi-task learning

Pre-training + Massive Multi-tasking, Benchmarking in NLP, EMNLP primer, 🤗 NLP Course, ACL 2021 recap, | Revue

Tags:

2021-11-07 About

Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction - ACL Anthology

Tags:

2019-02-09 About

Joint Models in NLP - Slides - Tutorial (EMNLP 2018) - Yue Zhang

Tags:

**Joint models: solve 2 tasks at once.**

Related tasks: POS tagging, NER, chuncking. Pipeline tasks

Motivations:

- reduce error propagation
- information exchange between tasks

Challenges:

- Joint learning
- Search

History: statistical models. 2 kinds:

- Graph-Based Methods
    - Traditional solution:
        - Score each candidate, select the highest-scored output
        - Search-space typically exponential
- Transition-Based Methods
    - Transition-Based systems: Automata
        - State: partial result during decoding, Action: operations that can be applied for state transition
        - Output constructed incrementally

- Deep learning based model
    - Neural transition based models
    - Neural graph-based models
        - Cross task
            - Seminal work: Collobert, Ronan, et al. "Natural language processing (almost) from scratch."
            - Not all tasks are mutually beneficial
            - Ramachandran, et al.  “Unsupervised pretraining for sequence to sequence learning.”
            - Peters, Matthew E., et al. "Deep contextualized word representations." (ELMo)
            - "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding."
            - ULMFIT
            - Correlation between multi-task learning and pretraining
        - Cross lingual
        - Cross domain
        - Cross standard

2018-11-06 About