Semanlink - Favoris ; Machine learning AND Low-Resource NLP

Printer friendly

Search Tag:

Search Doc:

Preferences...

Favoris ; Machine learning AND Low-Resource NLP

Common descendants

22 Documents

[2309.06131] Annotating Data for Fine-Tuning a Neural Ranker? Current Active Learning Strategies are not Better than Random Selection

Tags:

2023-09-14 About

Do large language models work on Tagalog?

Tags:

2023-08-07 About

Comparing Africa-centric Models to OpenAI's GPT3.5 - Lelapa

Tags:

2023-02-10 About

Towards a Tagalog NLP pipeline

Tags:

2023-02-04 About

Colin Leong sur Twitter : "This book is about the only "dataset" I ever found for Hani. My first ever foray into the field, I found an electronic copy and munged it into a Hani/English parallel corpus, and trained a JoeyNMT model with the help of @MasakhaneNLP and @KreutzerJulia in particular." / Twitter

Tags:

2023-01-05 About

How to Train an mT5 Model for Translation With Simple Transformers | by Thilina Rajapakse | Towards Data Science

Tags:

2022-09-25 About

[2203.09435] Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation

Tags:

2022-09-08 About

[2209.00099] Efficient Methods for Natural Language Processing: A Survey

Tags:

2022-09-04 About

[1807.00745] Training a Neural Network in a Low-Resource Setting on Automatically Annotated Noisy Data

Tags:

2022-07-18 About

Dealing with Data Scarcity in Natural Language Processing | by Yves Peirsman | NLPTown | Medium 2019)

Tags:

2022-07-18 About

[1910.06294] Training Compact Models for Low Resource Entity Tagging using Pre-trained Language Models

Tags:

2022-03-31 About

[2004.05119] Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer

Tags:

2022-03-31 About

Domain adaptation of word embeddings through the exploitation of in-domain corpora and knowledge bases (PhD Thesis 2021)

Tags:

Thèse de Hicham El Boukkouri, univ Paris-Saclay

[Github](https://github.com/helboukkouri/phd-code)

### Goal

Given a target specialized domain, improve the quality of general-domain
word representations using in-domain corpora and/or knowledge bases

### Contributions

#### a method for specializing general-domain embeddings in a [Low-Resource](tag:nlp_low_resource_scenarios) context.

> - train static representations on the task corpus, 
> - resume the
pre-training of general-domain contextual embeddings on the same task corpus,
> - finally, combine both static and contextual representations into
one final model

#### we tackle the issue of using a general-domain vocabulary in a specialized domain

#### Evaluation of  re-training vs training from scratch on specialized corpora using a specialized vocabulary

training from scratch better, but not that much: re-training from a general model
is still appropriate as it is less expensive and leads to comparable, although
slightly lower, performance

#### Regarding subword-based tokenization systems
> we argue that they are inconvenient in practice -> CharacterBERT, a variant
of BERT that uses ELMo’s character-based system instead of WordPieces. More convenient ti use, superior robustness to misspellings

#### Ways to specialize general-domain representations using knowledge bases

a strong baseline using a simple
method relying on graph embeddings and concatenation, using only is_a relation

> both static and contextual embeddings may effectively be
specialized using this simple approach

#### Knowledge Injection Modules (KIM) that
inject the knowledge representations directly within the BERT-like models' architecture

### Notes

> our experiments focused on a single
setting (i.e. the medical domain and the English language)

> meta-embeddings, an approach that consists in combining
different sets of representations for achieving improved performance

2022-03-23 About

[2112.01488] ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction

Tags:

2021-12-05 About

[1911.02655] Towards Domain Adaptation from Limited Data for Question Answering Using Deep Neural Networks

Tags:

2021-11-19 About

[2108.13854] Contrastive Domain Adaptation for Question Answering using Limited Text Corpora

Tags:

2021-11-19 About

[1706.03610] Neural Domain Adaptation for Biomedical Question Answering

Tags:

2021-11-19 About

Nils Reimers sur Twitter : "Neural Search for Low Resource Scenarios..."

Tags:

2021-10-27 About

VaLaR NMT: Vastly Lacking Resources Neural Machine Translation (2019)

Tags:

2021-10-14 About

[2109.04513] Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach

Tags:

2021-09-23 About

Koren Lazar sur Twitter : "...Modern pre-trained language models are applicable even in extreme low-resource settings as the case of the ancient Akkadian language."

Tags:

2021-09-23 About

[2107.00676] A Primer on Pretrained Multilingual Language Models

Tags:

2021-07-13 About