Semanlink - Microsoft AND Transformers

Printer friendly

Search Tag:

Search Doc:

Preferences...

Microsoft AND Transformers

Common descendants

Windows Copilot

21 Documents

LlamaIndex 🦙 sur X : "Fine-tuning Embedding Models for RAG with LoRA'

Tags:

2024-04-23 About

Jeremy Howard sur X : "FSDP/QDoRA with Llama3 : I believe that this combination is likely to create better task-specific models than anything else at any cost."

Tags:

2024-04-23 About

[2002.06275] TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval

Tags:

2023-08-27 About

[2307.08621] Retentive Network: A Successor to Transformer for Large Language Models

Tags:

2023-07-20 About

Introducing Azure OpenAI Service On Your Data in Public Preview - Microsoft Community Hub

Tags:

2023-06-21 About

How to use Alpaca-LoRA to fine-tune a model like ChatGPT

Tags:

2023-05-26 About

State of GPT (Andrej Karpathy at Microsoft Build)

Tags:

2023-05-24 About

Bringing the power of AI to Windows 11

Tags:

Windows Copilot

2023-05-24 About

Azure-Samples/azure-search-openai-demo: A sample app for the Retrieval-Augmented Generation pattern running in Azure

Tags:

2023-04-14 About

Physics of AI - YouTube

Tags:

- Intelligence has emerged: why? how?
- Let's study this with *controlled experiments* and *toy models*
- Clean and clear insights that peer slightly behind the magic curtain

emergence; rise of scale. Phase transition: fct(compute time for training: nb of params, dataset size)

(12:25) Transformers: the jump when compared to traditional NN: instead on oparating on single input X, operates on a set of inputs. (cf. training to say whether an image contains 2 elements of the same kind). **The aptitude to compare elements in the input sequence enables analogies** - the essence of reasoning.(Classical NN treats one high dimensional vector, tranformers hanlde sets of vectors. Attention layer replaces learned filters W of classical NN by the other elements in the sequence. Attention module compares part of input against other part of input. "Relative machines rather than absolute". Sets are a very powerful level of abstraction)

emergence
> The truth is that nobody has a clue what is going on"
>
> "**Something unknown is doing we don't know what**" (A. Eddington)

being inspired by physics' methodology: controled experiments and toy mathematical models

2023-03-28 About

Microsoft launches Teams Premium with features powered by OpenAI - The Verge

Tags:

2023-02-07 About

BioGPT

Tags:

2023-02-02 About

[2008.12813] HittER: Hierarchical Transformers for Knowledge Graph Embeddings

Tags:

2022-06-30 About

Unveiling Transformers with LEGO - YouTube

Tags:

2022-06-30 About

Lenka Zdeborova sur Twitter : "Amazing lecture of @SebastienBubeck giving intuition on transformers and how to approach them theoretically..."

Tags:

2022-06-30 About

[2004.05119] Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer

Tags:

2022-03-31 About

[2007.15779] Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

Tags:

2021-04-11 About

[2002.02925] BERT-of-Theseus: Compressing BERT by Progressive Module Replacing

Tags:

2020-02-10 About

Canwen Xu sur Twitter : "WTF? We brutally dismember BERT and replace all his organs?"

Tags:

2020-02-10 About

Robust Language Representation Learning via Multi-task Knowledge Distillation - Microsoft Research

Tags:

2019-05-19 About

[1901.11504] Multi-Task Deep Neural Networks for Natural Language Understanding

Tags:

2019-02-17 About