Semanlink - Language Model: Adaptation AND NLP: pretraining

Impression

Recherche de Mot-clé

Recherche de Doc

Préférences...

Mots-clés liés

ADAM (ML) - AI on consumer hardware - Allen Institute for AI (AI2) - Andrej Karpathy - Arxiv Doc - BERT - BERT fine-tuning - Cannibalizing LLMs - Catastrophic forgetting - Classification - Continual Learning - Domain adaptation (NLP) - EMNLP 2023 - fast.ai - Fine-tuning - Good - Google - Hugging Face - Jeremy Howard - Kaggle - Knowledge distillation - LLM - LLM Fine-tuning vs prompting - LM Fine-tuning - LMs: context length - LoRA - Matthew Honnibal - Neo4j - NLP: pretraining - NLP@Google - NLP@Meta - NLP@Microsoft - NLP@Stanford - OpenAI - Open Source - Open-source LLM - Parameter-efficient fine-tuning (PEFT) - Pre-Trained Language Models - Prompting/In-context learning - Prompts - Quantization - Sentence embeddings: Domain adaptation - Slides - Text to SQL - Transfer learning in NLP - Tutorial - Tweet - Vanishing gradient - Yoav Goldberg -

Language Model: Adaptation AND NLP: pretraining

Descendants partagés

27 Documents

LlamaIndex 🦙 sur X : "Fine-tuning Embedding Models for RAG with LoRA'

Tags:

2024-04-23 A propos

Jeremy Howard sur X : "FSDP/QDoRA with Llama3 : I believe that this combination is likely to create better task-specific models than anything else at any cost."

Tags:

2024-04-23 A propos

UKP Lab sur X : "a lightweight solution for few-shot domain-specific sentence classification: AdaSent!..."

Tags:

2023-12-09 A propos

How many Labelled Examples do you need for a BERT-sized Model to Beat GPT-4 on Predictive Tasks? - Speaker Deck

Tags:

2023-10-27 A propos

fast.ai - Can LLMs learn from a single example?

Tags:

2023-10-21 A propos

Leandro von Werra sur X : "Companies are currently deciding whether to invest in fine-tuning their own open LLMs (e.g. Llama 2) or to use closed LLMs via an API (e.g. Claude or GPT4)..."

Tags:

2023-10-01 A propos

[2309.12307] LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models

Tags:

2023-09-26 A propos

P-tuning for sequence classification

Tags:

2023-08-28 A propos

Why You (Probably) Don't Need to Fine-tune an LLM - Tidepool by Aquarium

Tags:

LLM Fine-tuning vs prompting

2023-08-19 A propos

Rachel Woods sur X : "There's a resurgence of interest in fine tuning LLMs I've yet to see a successful public use case where fine tuning > prompting. But here's where I see fine tuning *mattering*..."

Tags:

2023-08-19 A propos

Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Models to Unique Applications | Anyscale

Tags:

2023-08-18 A propos

Easily Finetune Llama 2 for Your Text-to-SQL Applications | by Jerry Liu | Aug, 2023

Tags:

2023-08-18 A propos

Knowledge Graphs & LLMs: Fine-Tuning Vs. Retrieval-Augmented Generation | by Tomaz Bratanic | Neo4j Developer Blog (June 6 2023)

Tags:

2023-08-13 A propos

clem 🤗 sur Twitter : "Llama 2 by @Meta is already integrated with @huggingface transformers, TGI, inference endpoints, PEFT and much more..."

Tags:

2023-07-19 A propos

How to use Alpaca-LoRA to fine-tune a model like ChatGPT

Tags:

2023-05-26 A propos

Google "We Have No Moat, And Neither Does OpenAI"

Tags:

> low-cost public involvement was enabled
by a vastly cheaper mechanism for fine tuning called low
rank adaptation ()[LoRA](tag:lora)

> **Part of what makes LoRA so effective is that ... it’s stackable.**
>
> By contrast, training giant models from scratch not only
throws away the pretraining, but also any iterative
improvements that have been made on top.

> LoRA updates are very cheap to produce (~$100) for the
most popular model sizes.

> Many of these projects are saving time by training on
small, highly curated datasets...
> These
datasets are built using synthetic methods (e.g. filtering
the best responses from an existing model) and
scavenging from other projects

> Directly Competing With Open Source
Is a Losing Proposition

> Paradoxically, the one clear winner in all of this is Meta.
Because the leaked model was theirs ([LLaMA](tag:llama)), they have
effectively garnered an entire planet's worth of free labor.
Since most open source innovation is happening on top of
their architecture, there is nothing stopping them from
directly incorporating it into their products.

2023-05-04 A propos

Andrej Karpathy sur Twitter : "There's a chance that LoRA finetunes work so well that it dramatically alters the finetuning vs. retrieval + few-shot prompting power dynamic..."

Tags:

2023-04-21 A propos

[2205.05638] Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

Tags:

2022-12-15 A propos

[2106.10199] BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

Tags:

2022-09-01 A propos

On Stability of Few-Sample Transformer Fine-Tuning | Kaggle

Tags:

2022-08-29 A propos

[2006.05987] Revisiting Few-sample BERT Fine-tuning

Tags:

2022-03-21 A propos

NLP | How to add a domain-specific vocabulary (new tokens) to a subword tokenizer already trained like BERT WordPiece | by Pierre Guillou | Medium

Tags:

2022-03-18 A propos

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines (2021)

Tags:

2021-10-30 A propos

[2106.13474] Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains

Tags:

2021-10-21 A propos

[1902.00751] Parameter-Efficient Transfer Learning for NLP

Tags:

2021-04-11 A propos

How many data points is a prompt worth?

Tags:

2021-04-08 A propos

[2004.10964] Don't Stop Pretraining: Adapt Language Models to Domains and Tasks

Tags:

2020-12-01 A propos