Joint Models in NLP - Slides - Tutorial (EMNLP 2018) - Yue Zhang
**Joint models: solve 2 tasks at once.** Related tasks: POS tagging, NER, chuncking. Pipeline tasks Motivations: - reduce error propagation - information exchange between tasks Challenges: - Joint learning - Search History: statistical models. 2 kinds: - Graph-Based Methods - Traditional solution: - Score each candidate, select the highest-scored output - Search-space typically exponential - Transition-Based Methods - Transition-Based systems: Automata - State: partial result during decoding, Action: operations that can be applied for state transition - Output constructed incrementally - Deep learning based model - Neural transition based models - Neural graph-based models - Cross task - Seminal work: Collobert, Ronan, et al. "Natural language processing (almost) from scratch." - Not all tasks are mutually beneficial - Ramachandran, et al. “Unsupervised pretraining for sequence to sequence learning.” - Peters, Matthew E., et al. "Deep contextualized word representations." (ELMo) - "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." - ULMFIT - Correlation between multi-task learning and pretraining - Cross lingual - Cross domain - Cross standard
About This Document
File info