Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Peter Izsak
- sl:arxiv_num : 1910.06294
- sl:arxiv_published : 2019-10-14T17:22:37Z
- sl:arxiv_summary : Training models on low-resource named entity recognition tasks has been shown
to be a challenge, especially in industrial applications where deploying
updated models is a continuous effort and crucial for business operations. In
such cases there is often an abundance of unlabeled data, while labeled data is
scarce or unavailable. Pre-trained language models trained to extract
contextual features from text were shown to improve many natural language
processing (NLP) tasks, including scarcely labeled tasks, by leveraging
transfer learning. However, such models impose a heavy memory and computational
burden, making it a challenge to train and deploy such models for inference
use. In this work-in-progress we combined the effectiveness of transfer
learning provided by pre-trained masked language models with a semi-supervised
approach to train a fast and compact model using labeled and unlabeled
examples. Preliminary evaluations show that the compact models can achieve
competitive accuracy with 36x compression rate when compared with a
state-of-the-art pre-trained language model, and run significantly faster in
inference, allowing deployment of such models in production environments or on
edge devices.@en
- sl:arxiv_title : Training Compact Models for Low Resource Entity Tagging using Pre-trained Language Models@en
- sl:arxiv_updated : 2019-10-17T08:07:19Z
- sl:bookmarkOf : https://arxiv.org/abs/1910.06294
- sl:creationDate : 2022-03-31
- sl:creationTime : 2022-03-31T21:06:23Z
Documents with similar tags (experimental)
Préférences...