Semanlink - NLP@Microsoft AND Pre-Trained Language Models

Impression

Recherche de Mot-clé

Recherche de Doc

Préférences...

NLP@Microsoft AND Pre-Trained Language Models

Descendants partagés

Semantic Kernel

22 Documents

LlamaIndex 🦙 sur X : "Fine-tuning Embedding Models for RAG with LoRA'

Tags:

2024-04-23 A propos

Jeremy Howard sur X : "FSDP/QDoRA with Llama3 : I believe that this combination is likely to create better task-specific models than anything else at any cost."

Tags:

2024-04-23 A propos

GraphRAG: Unlocking LLM discovery on narrative private data - Microsoft Research

Tags:

2024-03-07 A propos

GitHub - microsoft/LLMLingua: To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

Tags:

2024-01-18 A propos

microsoft/chat-copilot

Tags:

Semantic Kernel

2023-11-16 A propos

microsoft/semantic-kernel: Integrate cutting-edge LLM technology quickly and easily into your apps

Tags:

Semantic Kernel

2023-11-16 A propos

microsoft/semantic-kernel: Integrate cutting-edge LLM technology quickly and easily into your apps

Tags:

2023-10-19 A propos

[2309.12307] LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models

Tags:

2023-09-26 A propos

Sebastien Bubeck sur X : "How far does one billion parameters take you? ... Releasing phi-1.5, a 1.3B parameter LLM exhibiting emergent behaviors surprisingly close to much larger LLMs"

Tags:

2023-09-12 A propos

[2002.06275] TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval

Tags:

2023-08-27 A propos

[2307.08621] Retentive Network: A Successor to Transformer for Large Language Models

Tags:

2023-07-20 A propos

How to use Alpaca-LoRA to fine-tune a model like ChatGPT

Tags:

2023-05-26 A propos

State of GPT (Andrej Karpathy at Microsoft Build)

Tags:

2023-05-24 A propos

Google "We Have No Moat, And Neither Does OpenAI"

Tags:

> low-cost public involvement was enabled
by a vastly cheaper mechanism for fine tuning called low
rank adaptation ()[LoRA](tag:lora)

> **Part of what makes LoRA so effective is that ... it’s stackable.**
>
> By contrast, training giant models from scratch not only
throws away the pretraining, but also any iterative
improvements that have been made on top.

> LoRA updates are very cheap to produce (~$100) for the
most popular model sizes.

> Many of these projects are saving time by training on
small, highly curated datasets...
> These
datasets are built using synthetic methods (e.g. filtering
the best responses from an existing model) and
scavenging from other projects

> Directly Competing With Open Source
Is a Losing Proposition

> Paradoxically, the one clear winner in all of this is Meta.
Because the leaked model was theirs ([LLaMA](tag:llama)), they have
effectively garnered an entire planet's worth of free labor.
Since most open source innovation is happening on top of
their architecture, there is nothing stopping them from
directly incorporating it into their products.

2023-05-04 A propos

BioGPT

Tags:

2023-02-02 A propos

[2004.05119] Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer

Tags:

2022-03-31 A propos

[2106.13474] Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains

Tags:

2021-10-21 A propos

[2007.15779] Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

Tags:

2021-04-11 A propos

[2002.02925] BERT-of-Theseus: Compressing BERT by Progressive Module Replacing

Tags:

2020-02-10 A propos

Canwen Xu sur Twitter : "WTF? We brutally dismember BERT and replace all his organs?"

Tags:

2020-02-10 A propos

Robust Language Representation Learning via Multi-task Knowledge Distillation - Microsoft Research

Tags:

2019-05-19 A propos

[1901.11504] Multi-Task Deep Neural Networks for Natural Language Understanding

Tags:

2019-02-17 A propos