Joint models
solve 2 tasks at once
Related Tags:
2 Documents (Long List
  • Joint Models in NLP - Slides - Tutorial (EMNLP 2018) - Yue Zhang (About)
    **Joint models: solve 2 tasks at once.** Related tasks: POS tagging, NER, chuncking. Pipeline tasks Motivations: - reduce error propagation - information exchange between tasks Challenges: - Joint learning - Search History: statistical models. 2 kinds: - Graph-Based Methods - Traditional solution: - Score each candidate, select the highest-scored output - Search-space typically exponential - Transition-Based Methods - Transition-Based systems: Automata - State: partial result during decoding, Action: operations that can be applied for state transition - Output constructed incrementally - Deep learning based model - Neural transition based models - Neural graph-based models - Cross task - Seminal work: Collobert, Ronan, et al. "Natural language processing (almost) from scratch." - Not all tasks are mutually beneficial - Ramachandran, et al. “Unsupervised pretraining for sequence to sequence learning.” - Peters, Matthew E., et al. "Deep contextualized word representations." (ELMo) - "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." - ULMFIT - Correlation between multi-task learning and pretraining - Cross lingual - Cross domain - Cross standard
  • Natural Language Processing (almost) from Scratch - Collobert and Weston (2011) (About)
    seminal work Abstract: > a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements