Semanlink - Fine-tuning AND OpenAI

Impression

Recherche de Mot-clé

Recherche de Doc

Préférences...

Mots-clés liés

AI on consumer hardware - Alpaca - Andrej Karpathy - Arxiv Doc - Cannibalizing LLMs - ChatGPT - cs224n - Dataset quality - Fine-tuning - GitHub project - Good - Google - Instruction tuning - Jeremy Howard - LLaMA - LLM - LM Fine-tuning - NLP@Meta - OpenAI - OpenAI GPT - Open Source - Open-source LLM - Parameter-efficient fine-tuning (PEFT) - Quantization - RL from Human Feedback - Slides - Text to SQL - Transfer learning - Tutorial - Tweet -

Fine-tuning AND OpenAI

Descendants partagés

19 Documents

Jeremy Howard sur X : "FSDP/QDoRA with Llama3 : I believe that this combination is likely to create better task-specific models than anything else at any cost."

Tags:

2024-04-23 A propos

openai-cookbook/examples/Customizing_embeddings.ipynb

Tags:

2023-09-17 A propos

GPT-3.5 Turbo fine-tuning and API updates

Tags:

2023-08-22 A propos

Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Models to Unique Applications | Anyscale

Tags:

2023-08-18 A propos

Easily Finetune Llama 2 for Your Text-to-SQL Applications | by Jerry Liu | Aug, 2023

Tags:

2023-08-18 A propos

clem 🤗 sur Twitter : "Llama 2 by @Meta is already integrated with @huggingface transformers, TGI, inference endpoints, PEFT and much more..."

Tags:

2023-07-19 A propos

How to use Alpaca-LoRA to fine-tune a model like ChatGPT

Tags:

2023-05-26 A propos

Hyung Won Chung sur Twitter : "RLHF as an instance of using a learned objective function"

Tags:

2023-05-18 A propos

Google "We Have No Moat, And Neither Does OpenAI"

Tags:

> low-cost public involvement was enabled
by a vastly cheaper mechanism for fine tuning called low
rank adaptation ()[LoRA](tag:lora)

> **Part of what makes LoRA so effective is that ... it’s stackable.**
>
> By contrast, training giant models from scratch not only
throws away the pretraining, but also any iterative
improvements that have been made on top.

> LoRA updates are very cheap to produce (~$100) for the
most popular model sizes.

> Many of these projects are saving time by training on
small, highly curated datasets...
> These
datasets are built using synthetic methods (e.g. filtering
the best responses from an existing model) and
scavenging from other projects

> Directly Competing With Open Source
Is a Losing Proposition

> Paradoxically, the one clear winner in all of this is Meta.
Because the leaked model was theirs ([LLaMA](tag:llama)), they have
effectively garnered an entire planet's worth of free labor.
Since most open source innovation is happening on top of
their architecture, there is nothing stopping them from
directly incorporating it into their products.

2023-05-04 A propos

Daniel Vila Suero sur Twitter : "Data quality is key for LLMs, but we're building Open Source LLMs with data of "unknown" quality... Introducing Alpaca GarbageCollector..."

Tags:

2023-04-05 A propos

Fine-tuning - OpenAI API

Tags:

2023-03-30 A propos

Alpaca Finetuning of Llama on a 24G Consumer GPU

Tags:

2023-03-27 A propos

tloen/alpaca-lora: Instruct-tune LLaMA on consumer hardware

Tags:

2023-03-22 A propos

LLM Zoo at Home: LLaMA & Alpaca | bergis universe of software, hardware and ideas

Tags:

2023-03-20 A propos

Shayne Longpre sur Twitter : "A 🧵 on @OpenAI LLM "Alignment" (e.g. #ChatGPT)..."

Tags:

2023-02-27 A propos

Prompting, Instruction Finetuning, and RLHF (CS224N)

Tags:

2023-02-16 A propos

[2302.04870] Offsite-Tuning: Transfer Learning without Full Model

Tags:

2023-02-11 A propos

karpathy/nanoGPT: The simplest, fastest repository for training/finetuning medium-sized GPTs.

Tags:

2023-02-02 A propos

Shayne Longpre sur Twitter : "What’s the best completely public competitor to #ChatGPT? Flan-T5 beats all public models we tested..."

Tags:

2023-02-01 A propos