Semanlink - Favoris ; Transformers AND Fine-tuning

Impression

Recherche de Mot-clé

Recherche de Doc

Préférences...

Mots-clés liés

Alpaca - Arxiv Doc - BERT - BERT fine-tuning - ChatGPT - Domain adaptation - Domain adaptation (NLP) - Embedding Model Fine-tuning - Few-shot learning - Few-shot text classification - Fine-tuning - GitHub project - GPL (Generative Pseudo Labeling) - Instruction tuning - LLaMA - LlamaIndex - LLM+SQL - LM Fine-tuning - Moshe Wasserblat - Neural Search - Nils Reimers - NLP: French - NLP@Google - NLP@Microsoft - NLP@Stanford - OpenAI GPT - Parameter-efficient fine-tuning (PEFT) - Prompting/In-context learning - RL from Human Feedback - sBERT fine-tuning - Sentence-BERT - Sentence-BERT: Domain adaptation - Sentence embeddings: Domain adaptation - SetFit (SBERT fine-tuning) - Synthetic QA data - Text to SQL - Transfer learning - Transfer learning in NLP - Transformers - Tweet - Unsupervised Domain Adaptation (NLP) - Unsupervised Sentence Embedding Learning -

Favoris ; Transformers AND Fine-tuning

Descendants partagés

46 Documents

LlamaIndex 🦙 sur X : "Fine-tuning Embedding Models for RAG with LoRA'

Tags:

2024-04-23 A propos

Jeremy Howard sur X : "FSDP/QDoRA with Llama3 : I believe that this combination is likely to create better task-specific models than anything else at any cost."

Tags:

2024-04-23 A propos

Fine-tuning transformers: : Vocabulary transfer: Artificial Intelligence: Vol 317, No C

Tags:

2024-02-29 A propos

UKP Lab sur X : "a lightweight solution for few-shot domain-specific sentence classification: AdaSent!..."

Tags:

2023-12-09 A propos

openai-cookbook/examples/Customizing_embeddings.ipynb

Tags:

2023-09-17 A propos

GPT-3.5 Turbo fine-tuning and API updates

Tags:

2023-08-22 A propos

Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Models to Unique Applications | Anyscale

Tags:

2023-08-18 A propos

Easily Finetune Llama 2 for Your Text-to-SQL Applications | by Jerry Liu | Aug, 2023

Tags:

2023-08-18 A propos

clem 🤗 sur Twitter : "Llama 2 by @Meta is already integrated with @huggingface transformers, TGI, inference endpoints, PEFT and much more..."

Tags:

2023-07-19 A propos

[2306.07536] TART: A plug-and-play Transformer module for task-agnostic reasoning

Tags:

2023-06-15 A propos

How to use Alpaca-LoRA to fine-tune a model like ChatGPT

Tags:

2023-05-26 A propos

Domain Adaptation with Generative Pseudo-Labeling (GPL) | Pinecone

Tags:

2023-04-09 A propos

Daniel Vila Suero sur Twitter : "Data quality is key for LLMs, but we're building Open Source LLMs with data of "unknown" quality... Introducing Alpaca GarbageCollector..."

Tags:

2023-04-05 A propos

Alpaca Finetuning of Llama on a 24G Consumer GPU

Tags:

2023-03-27 A propos

tloen/alpaca-lora: Instruct-tune LLaMA on consumer hardware

Tags:

2023-03-22 A propos

LLM Zoo at Home: LLaMA & Alpaca | bergis universe of software, hardware and ideas

Tags:

2023-03-20 A propos

Shayne Longpre sur Twitter : "A 🧵 on @OpenAI LLM "Alignment" (e.g. #ChatGPT)..."

Tags:

2023-02-27 A propos

Prompting, Instruction Finetuning, and RLHF (CS224N)

Tags:

2023-02-16 A propos

[2302.04870] Offsite-Tuning: Transfer Learning without Full Model

Tags:

2023-02-11 A propos

karpathy/nanoGPT: The simplest, fastest repository for training/finetuning medium-sized GPTs.

Tags:

2023-02-02 A propos

Shayne Longpre sur Twitter : "What’s the best completely public competitor to #ChatGPT? Flan-T5 beats all public models we tested..."

Tags:

2023-02-01 A propos

ValueError "invalid literal for int() with base 10" in trainer.evaluate (dataset created from pandas) · Issue #228 · huggingface/setfit

Tags:

2022-12-13 A propos

Lewis Tunstall sur Twitter : "The SetFit library for few-shot learning with Sentence Transformers now supports *multi-label text classification*..."

Tags:

2022-10-14 A propos

huggingface/setfit: Efficient few-shot learning with Sentence Transformers

Tags:

2022-10-12 A propos

[2205.11498] Domain Adaptation for Memory-Efficient Dense Retrieval

Tags:

2022-09-26 A propos

[2209.11055] Efficient Few-Shot Learning Without Prompts

Tags:

2022-09-23 A propos

[2106.10199] BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

Tags:

2022-09-01 A propos

On Stability of Few-Sample Transformer Fine-Tuning | Kaggle

Tags:

2022-08-29 A propos

Unsupervised Learning — Sentence-Transformers documentation

Tags:

2022-08-20 A propos

adapter-hub/adapter-transformers: Huggingface Transformers + Adapters

Tags:

2022-07-22 A propos

Domain transfer with GGPL: German Generative Pseudo Labeling 🥨 | by Matthias Richter | Jun, 2022 | ML6team

Tags:

2022-06-02 A propos

Nils Reimers sur Twitter : "GPL goes multi-lingual..."

Tags:

2022-06-01 A propos

Ramsri Goutham Golla sur Twitter : "Hi @Nils_Reimers For GPL you used "msmarco-distilbert-base-tas-b" model and ..."

Tags:

2022-04-27 A propos

[2008.11228] A simple method for domain adaptation of sentence embeddings

Tags:

2022-04-01 A propos

[2004.05119] Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer

Tags:

2022-03-31 A propos

Sentence Transformer Fine-Tuning (SetFit): Outperforming GPT-3 on few-shot Text-Classification while being 1600 times smaller | by Moshe Wasserblat (2021-12)

Tags:

2022-03-31 A propos

Nils Reimers sur Twitter : "Great post on SetFit"

Tags:

2022-03-31 A propos

Sentence Embedding Fine-tuning for the French Language | by La Javaness R&D | Feb, 2022 | Medium

Tags:

2022-03-31 A propos

Domain Adaptation — Sentence-Transformers documentation

Tags:

2022-03-31 A propos

[2203.14655] Few-Shot Learning with Siamese Networks and Label Tuning

Tags:

> the problem of building text classifiers with little or no training data.
>
> In recent years, an approach based on neural textual entailment models has been found to give strong results on a diverse range of tasks.

(cf. #[NLI](tag:nli), using the input text as the premise and the text representing the label as the hypothesis)

> In this work, we show that **with proper pre-training, Siamese Networks that embed texts and labels** offer a competitive alternative.
>
> We introduce **label tuning: fine-tuning the label embeddings only**. While giving lower performance than model fine-tuning (which updates all params of the model), this approach has the architectural advantage that a single encoder can be shared by many different tasks (we only fine-tune the label embeddings)
> The drop in quality can
be compensated by using a variant of **[Knowledge distillation](tag:knowledge_distillation)**

[Github](https://tinyurl.com/label-tuning), [Tweet](doc:2022/03/thomas_muller_sur_twitter_pa)

2022-03-30 A propos

[2006.05987] Revisiting Few-sample BERT Fine-tuning

Tags:

2022-03-21 A propos

NAVER LABS Europe : "@Nils_Reimers of @huggingface on 'Unsupervised domain adaptation for neural search'"

Tags:

2022-03-09 A propos

[2112.07577] GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval

Tags:

An unsupervised domain adaptation technique for dense retrieval models

1. synthetic queries
are generated for each passage from the target corpus (using an existing pre-trained [T5](tag:text_to_text_transfer_transformer)
encoder-decoder)
2. the generated queries are used for mining negative
passages (retrieving the most similar
paragraphs using an existing dense retrieval
model == hard negatives!)
3. the query-passage pairs are labeled by a cross-encoder and used to train the domain-adapted
dense retriever (using method described in [Hofstätter et al.,
2020](doc:2021/12/2010_02666_improving_efficien))

[Nils Reimers sur Twitter](doc:2021/12/nils_reimers_sur_twitter_do_), [GitHub](https://github.com/UKPLab/gpl),  by the author of [TSDAE](doc:2021/09/2104_06979_tsdae_using_trans)

Claims to improve "Doc2Query" [Document Expansion by Query Prediction](doc:2022/01/1904_08375_document_expansion): ([src](https://twitter.com/KexinWang2049/status/1471435779415150598))

> - GPL: Uses doc2query to construct synthetic data and does knowledge distillation (i.e. training) on that data.
> - Doc2query: Generates queries to extend the documents and use BM25 on top of them w/o training.

2021-12-15 A propos

Unsupervised Training for Sentence Transformers | Pinecone

Tags:

2021-11-24 A propos

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines (2021)

Tags:

2021-10-30 A propos

[1902.00751] Parameter-Efficient Transfer Learning for NLP

Tags:

2021-04-11 A propos