Semanlink - Education AND Language Model

Printer friendly

Search Tag:

Search Doc:

Preferences...

Tag Cloud

Education AND Language Model

Common descendants

67 Documents

[2404.03592] ReFT: Representation Finetuning for Language Models

Tags:

2024-04-08 About

ColBERT gist:c1182551fa609736d47df4af82f7c5ab

Tags:

2024-03-08 About

ColBERT Inference in the Browser

Tags:

2024-03-08 About

Announcing Vespa Long-Context ColBERT | Vespa Blog

Tags:

2024-03-03 About

Omar Khattab sur X : "ColBERT in 81 languages by generalizing from English training! ..."

Tags:

2024-02-28 About

[2307.15936] A Theory for Emergence of Complex Skills in Language Models

Tags:

2024-02-24 About

Sanjeev Arora sur X : "It's better to use just 5% of the instruction-tuning data (suitably selected) instead of the full dataset."

Tags:

2024-02-15 About

New Theory Suggests Chatbots Can Understand Text | Quanta Magazine

Tags:

2024-02-11 About

Omar Khattab sur X : "A thread on late interaction, how it works efficiently, and why/where it's been shown to improve quality"

Tags:

2024-02-05 About

Craig Macdonald sur X : "a thread of our main ColBERT research findings"

Tags:

2024-02-01 About

long document ranking · Issue #11 · stanford-futuredata/ColBERT

Tags:

2024-01-31 About

Exploring ColBERT with RAGatouille | Simon Willison’s TILs

Tags:

2024-01-29 About

Jo Kristian Bergum sur X : (on "why using ColBERT?")

Tags:

2024-01-28 About

bclavie/RAGatouille

Tags:

2024-01-26 About

Omar Khattab sur X : "A🧵on beating the hardware lottery for retrieval: the internals of the late interaction stack. ColBERT..."

Tags:

2023-12-29 About

Zain Hasan sur X : "why do large language models pay more attention to and reason better over the beginning and end of what you tell them in prompts?"

Tags:

2023-11-19 About

raphaelsty/neural-cherche: Neural Search

Tags:

2023-11-17 About

fast.ai - Can LLMs learn from a single example?

Tags:

2023-10-21 About

Omar Khattab sur X : "This isn't the main point of this great new paper by @sophiaalthammer et al. But it's incredible how ColBERT at 1000 training queries is better than DPR trained at *50,000* queries!"

Tags:

2023-09-14 About

[2302.06600] Task-Specific Skill Localization in Fine-tuned Language Models

Tags:

2023-08-25 About

Sanjeev Arora sur Twitter : "new `skills' induced by LLM fine-tuning can be localized in tiny fraction of the model."

Tags:

2023-07-07 About

Sanjeev Arora sur Twitter : "Fine-tuning language models using just forward pass!...r

Tags:

2023-06-09 About

[2305.14788] Adapting Language Models to Compress Contexts

Tags:

2023-06-04 About

LangChain Retrieval Webinar - YouTube

Tags:

2023-05-27 About

How to use Alpaca-LoRA to fine-tune a model like ChatGPT

Tags:

2023-05-26 About

Intelligence artificielle : « C’est dans un rapport perverti aux connaissances que réside la menace de ChatGPT »

Tags:

2023-05-11 About

Andrej Karpathy sur Twitter : "Excellent TED talk from Sal Khan: - many inspiring examples of GPTs finetuned into socratic tutors, assisting without giving away answers...

Tags:

2023-05-02 About

[2304.09848] Evaluating Verifiability in Generative Search Engines

Tags:

2023-04-23 About

Daniel Vila Suero sur Twitter : "Data quality is key for LLMs, but we're building Open Source LLMs with data of "unknown" quality... Introducing Alpaca GarbageCollector..."

Tags:

2023-04-05 About

[2304.01982] Rethinking the Role of Token Retrieval in Multi-Vector Retrieval

Tags:

2023-04-05 About

Alpaca Finetuning of Llama on a 24G Consumer GPU

Tags:

2023-03-27 About

tloen/alpaca-lora: Instruct-tune LLaMA on consumer hardware

Tags:

2023-03-22 About

LLM Zoo at Home: LLaMA & Alpaca | bergis universe of software, hardware and ideas

Tags:

2023-03-20 About

stanfordnlp/dspy: 𝗗𝗦𝗣: Demonstrate-Search-Predict. A framework for composing retrieval and language models for knowledge-intensive NLP.

Tags:

2023-02-18 About

Prompting, Instruction Finetuning, and RLHF (CS224N)

Tags:

2023-02-16 About

[2203.14465] STaR: Bootstrapping Reasoning With Reasoning

Tags:

2023-02-07 About

Omar Khattab sur Twitter : "New features in 𝗖𝗼𝗹𝗕𝗘𝗥𝗧 ‣ Index Updater. Simple API to add/remove documents..."

Tags:

2023-01-30 About

Andrej Karpathy sur Twitter : "Let's build GPT: from scratch, in code, spelled out."

Tags:

2023-01-28 About

Stanford studied 30 large language models so you don’t have to

Tags:

2022-12-20 About

Talking to Models: Stanford U & Microsoft Method Enables Developers to Correct Model Bugs via Natural Language Patches | Synced

Tags:

2022-11-27 About

Meet CoAuthor, an Experiment in Human-AI Collaborative Writing

Tags:

2022-11-13 About

[2210.09338] Deep Bidirectional Language-Knowledge Graph Pretraining

Tags:

2022-10-23 About

Sanjeev Arora sur Twitter : "A priori, fine-tuning a huge LM on a few datapoints could lead to catastrophic overfitting. So why doesn’t it? Our theory + experiments..."

Tags:

2022-10-14 About

[2208.01066] What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

Tags:

2022-09-17 About

[2208.03299] Few-shot Learning with Retrieval Augmented Language Model

Tags:

2022-08-08 About

cs224n Lecture 13: Integrating Knowledge in Language Models

Tags:

2022-07-08 About

[2203.13088] Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction

Tags:

2022-03-30 About

[2101.12294] Combining pre-trained language models and structured knowledge

Tags:

2022-03-25 About

[2006.05987] Revisiting Few-sample BERT Fine-tuning

Tags:

2022-03-21 About

Document Matching for Job Descriptions | Semantic Scholar (2021)

Tags:

2022-03-09 About

Integrate ORQA and REALM for Open Domain Question Answering · Issue #312 · deepset-ai/haystack

Tags:

2022-01-12 About

[2004.12832] ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

Tags:

2022-01-12 About

[2007.00814] Relevance-guided Supervision for OpenQA with ColBERT

Tags:

2022-01-07 About

[2112.01488] ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction

Tags:

2021-12-05 About

Seth Stafford sur Twitter : "Here’s a nice paper (ICLR spotlight) on how to apply masking in LM training..."

Tags:

2021-10-16 About

Building Scalable, Explainable, and Adaptive NLP Models with Retrieval | SAIL Blog

Tags:

2021-10-07 About

A Moderate Proposal for Radically Better AI-powered Web Search

Tags:

2021-07-10 About

exBERT: Extending Pre-trained Models with Domain-specific Vocabulary Under Constrained Training Resources - ACL Anthology

Tags:

**Focus on the Embedding of Domain-specific Vocabulary.**

> exBERT
adds a new domain-specific vocabulary and the corresponding
embedding layer, as well as a small
extension module to the original unmodified model

> a pretraining
method allowing **low-cost embedding of
domain-specific vocabulary in the context of an
existing large pre-trained model such as BERT**

> exBERT... explicitly incorporates
the new domain’s vocabulary, while being able to
**reuse the original pre-trained model’s weights as is**
to reduce required computation and training data. Specifically, exBERT extends BERT by augmenting
its embeddings for the original vocabulary with
new embeddings for the domain-specific vocabulary
via **a learned small “extension” module**. **The
output of the original and extension modules are
combined via a trainable weighted sum operation**

In a way similar to concept developed in

> [[1902.00751] Parameter-Efficient Transfer Learning for NLP](doc:2021/04/1902_00751_parameter_efficien), but not in the fine-tuning paradigm.

[Github](https://github.com/cgmhaicenter/exBERT)

2021-04-11 About

[2012.04584] Distilling Knowledge from Reader to Retriever for Question Answering

Tags:

> a method to train an information retrieval module for downstream tasks, **without using pairs of queries and documents as annotations**.

Uses two models (standard pipeline for open-domain QA):

- the first one retrieves documents from a large source of knowledge (the retriever)
- the second one processes the support documents to solve the task (the reader).

> First the retriever selects support passages in a large knowledge
source. Then these passages are processed by the reader, along with the question, to generate an
answer

Inspired by knowledge distillation: the reader model is the teacher and the retriever is the student.

> More precisely, we use a sequence-to-sequence model as the reader, and use
the attention activations over the input documents as synthetic labels to train the retriever. 
> (**train the retriever by learning to approximate the attention score of the reader**)

Refers to:

- [REALM: Retrieval-Augmented Language Model Pre-Training](doc:2020/12/2002_08909_realm_retrieval_a)
- [Dehghani: Neural Ranking Models with Weak Supervision](doc:?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1704.08803)

2020-12-11 About

BERT, ELMo, & GPT-2: How Contextual are Contextualized Word Representations? | SAIL Blog

Tags:

2020-03-28 About

Lecture 14 – Contextual Vectors | Stanford CS224U: Natural Language Understanding | Spring 2019

Tags:

2020-01-05 About

[1911.00172] Generalization through Memorization: Nearest Neighbor Language Models

Tags:

2019-12-20 About

Efficient multi-lingual language model fine-tuning · fast.ai NLP

Tags:

2019-10-22 About

Introducing FastBert — A simple Deep Learning library for BERT Models

Tags:

2019-05-23 About

Tutorial on Text Classification (NLP) using ULMFiT and fastai Library in Python - Analytics Vidhya

Tags:

2018-12-01 About

Language models - Jordan Boyd-Graber - University of Maryland

Tags:

2012-03-24 About

SRILM - The SRI Language Modeling Toolkit

Tags:

2012-03-24 About