Semanlink - Self-Attention AND NLP@Meta

Impression

Recherche de Mot-clé

Recherche de Doc

Préférences...

Mots-clés liés

AI on consumer hardware - Alpaca - Andrej Karpathy - Apple M1/M2 - Arxiv Doc - Attention mechanism - BERT - CamemBERT - ChatGPT - Cross-lingual NLP - Dan Jurafsky - Dataset quality - Discuté avec Raphaël - Entity Retrieval - Fine-tuning - Foundation Models - France is AI 2018 - Generalization - GitHub - GitHub project - Guillaume Lample - Hugging Face - Information retrieval - Jeremy Howard - Job matching - Language Model - Language Models + Knowledge - LLaMA - Llama 2 - LLM - LM Fine-tuning - Long documents - Ludovic Denoyer - Mémoire - Memory in deep learning - NLP: French - NLP@Facebook - NLP@Meta - NLP@Stanford - NLP sample code - Parameter-efficient fine-tuning (PEFT) - Paris NLP meetup - Pre-Trained Language Models - Publicly available datasets - Siamese networks - Text to SQL - Transformers - Tutorial - Tweet -

Self-Attention AND NLP@Meta

Descendants partagés

35 Documents

Jeremy Howard sur X : "FSDP/QDoRA with Llama3 : I believe that this combination is likely to create better task-specific models than anything else at any cost."

Tags:

2024-04-23 A propos

Philipp Schmid sur X : "Can we combine multiple fine-tuned LLMs into a single MoE?..."

Tags:

2024-03-30 A propos

Getting started with Llama 2 - AI at Meta

Tags:

2023-10-28 A propos

Philipp Schmid sur X : “YaRN” allows you to scale LLMs like llama 2 to over 100k context!...

Tags:

2023-09-01 A propos

Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Models to Unique Applications | Anyscale

Tags:

2023-08-18 A propos

Easily Finetune Llama 2 for Your Text-to-SQL Applications | by Jerry Liu | Aug, 2023

Tags:

2023-08-18 A propos

Andrej Karpathy sur Twitter : "How is LLaMa.cpp possible?"

Tags:

2023-08-16 A propos

Simon Willison sur Twitter : "Run Llama 2 on your own Mac using LLM and Homebrew..."

Tags:

2023-08-02 A propos

Andrej Karpathy sur Twitter : "My fun weekend hack: llama2.c Lets you train a baby Llama 2 model in PyTorch, then inference it with one 500-line file with no dependencies, in pure C."

Tags:

2023-07-24 A propos

Llama 2 is here - get it on Hugging Face

Tags:

2023-07-19 A propos

clem 🤗 sur Twitter : "Llama 2 by @Meta is already integrated with @huggingface transformers, TGI, inference endpoints, PEFT and much more..."

Tags:

2023-07-19 A propos

Llama 2 - Meta AI

Tags:

Llama 2

2023-07-19 A propos

[2305.07185] MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers

Tags:

2023-07-01 A propos

ggerganov/llama.cpp: Port of Facebook's LLaMA model in C/C++

Tags:

2023-06-04 A propos

How to use Alpaca-LoRA to fine-tune a model like ChatGPT

Tags:

2023-05-26 A propos

Daniel Vila Suero sur Twitter : "Data quality is key for LLMs, but we're building Open Source LLMs with data of "unknown" quality... Introducing Alpaca GarbageCollector..."

Tags:

2023-04-05 A propos

Jeremy Howard sur Twitter : "There's a lot of folks under the misunderstanding that it's now possible to run a 30B param LLM in <6GB, based on this GitHub discussion. This is not the case..." (thread)

Tags:

2023-04-03 A propos

Using ChatGPT Plugins with LLaMA

Tags:

2023-03-27 A propos

Alpaca Finetuning of Llama on a 24G Consumer GPU

Tags:

2023-03-27 A propos

tloen/alpaca-lora: Instruct-tune LLaMA on consumer hardware

Tags:

2023-03-22 A propos

LLM Zoo at Home: LLaMA & Alpaca | bergis universe of software, hardware and ideas

Tags:

2023-03-20 A propos

Guillaume Lample sur Twitter : "Today we release LLaMA, 4 foundation models ranging from 7B to 65B parameters..."

Tags:

2023-02-25 A propos

Akari Asai sur Twitter : "Can we solely rely on LLMs’ memories (eg replace search w ChatGPT)? Probably not... Our analysis shows how retrieval is complementary to LLMs’ parametric knowledge..."

Tags:

2022-12-21 A propos

Sentence Embedding Fine-tuning for the French Language | by La Javaness R&D | Feb, 2022 | Medium

Tags:

2022-03-31 A propos

Sahajtomar/french_semantic · Hugging Face

Tags:

2021-10-14 A propos

[1901.04085] Passage Re-ranking with BERT

Tags:

2021-03-26 A propos

Hugging Face sur Twitter : "Transformers release of the Retrieval-Augmented Generation model in collaboration with @facebookai!"

Tags:

2021-02-23 A propos

Guillaume Lample sur Twitter : "Last year, we showed that you can outperform a 24-layer transformer in language modeling with just...

Tags:

2020-10-10 A propos

[1911.03814] Scalable Zero-shot Entity Linking with Dense Entity Retrieval

Tags:

2020-05-02 A propos

Paris NLP Season 4 Meetup #3 – Paris NLP (2020)

Tags:

2020-01-23 A propos

[1911.00172] Generalization through Memorization: Nearest Neighbor Language Models

Tags:

2019-12-20 A propos

CamemBERT

Tags:

CamemBERT

2019-11-10 A propos

[1911.01464] Emerging Cross-lingual Structure in Pretrained Language Models

Tags:

2019-11-06 A propos

[1907.05242] Large Memory Layers with Product Keys

Tags:

> **a structured memory which can be easily integrated into a neural network.** The memory is very large by design and therefore significantly increases the capacity of the architecture, by up to a billion parameters with a negligible computational overhead. Its design and access pattern is based on **product keys**, which enable fast and exact nearest neighbor search. The ability to increase the number of parameters while keeping the same computational budget lets the overall system strike a better trade-off between prediction accuracy and computation efficiency both at training and test time.

> a key-value memory layer that can increase model capacity for a negligible computational cost. A 12-layer transformer with a memory outperforms a 24-layer transformer, and is 2x faster!

[Implementation](/doc/2019/08/product_key_memory_pkm_minima)

TODO: compare with [[2007.00849] Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge](doc:2020/07/2007_00849_facts_as_experts_)

2019-07-13 A propos

[1503.08895] End-To-End Memory Networks

Tags:

2018-10-23 A propos