Semanlink - Favoris ; Semantic technology AND LMs: context length

Printer friendly

Search Tag:

Search Doc:

Preferences...

Favoris ; Semantic technology AND LMs: context length

Common descendants

LMs: context length

18 Documents

[2404.11018] Many-Shot In-Context Learning

Tags:

2024-04-21 About

GitHub - microsoft/LLMLingua: To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

Tags:

2024-01-18 About

Albert Gu sur X : "Quadratic attention has been indispensable for information-dense modalities such as language... until now..."

Tags:

2023-12-05 About

Zain Hasan sur X : "why do large language models pay more attention to and reason better over the beginning and end of what you tell them in prompts?"

Tags:

2023-11-19 About

[2310.03025] Retrieval meets Long Context Large Language Models

Tags:

2023-10-07 About

[2309.12307] LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models

Tags:

2023-09-26 About

Philipp Schmid sur X : “YaRN” allows you to scale LLMs like llama 2 to over 100k context!...

Tags:

2023-09-01 About

NLP News #66: ✨ Flashier Attention, 🤐 Gzip classifiers

Tags:

2023-08-14 About

LlamaIndex 🦙 (GPT Index) sur Twitter : "Pretty much everyone building LLM apps over data has to figure out how to... cram arbitrary data into limited context windows?"

Tags:

2023-07-09 About

[2307.02486] LongNet: Scaling Transformers to 1,000,000,000 Tokens

Tags:

2023-07-06 About

[2305.07185] MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers

Tags:

2023-07-01 About

[2306.07174] Augmenting Language Models with Long-Term Memory

Tags:

2023-06-13 About

[2305.14788] Adapting Language Models to Compress Contexts

Tags:

2023-06-04 About

State of GPT (Andrej Karpathy at Microsoft Build)

Tags:

2023-05-24 About

Aran Komatsuzaki sur Twitter : "Unleashing Infinite-Length Input Capacity for Large-scale Language Models with Self-Controlled Memory System Enables LLMs to outperform ChatGPT in scenarios involving ultra-long document summarization or long-term conversations."

Tags:

2023-04-27 About

Aran Komatsuzaki sur Twitter : "Scaling Transformer to 1M tokens and beyond with Recurrent Memory Transformer..."

Tags:

2023-04-25 About

Does ChatGPT remember what happened earlier in the conversation? | OpenAI Help Center

Tags:

[aistackexchange](https://ai.stackexchange.com/questions/38150/how-does-chatgpt-retain-the-context-of-previous-questions)

> [me]I heard people using the phrase "context length" about you. Is it the same thing as the number of tokens you just spoke of?
>
> [ChatGPT]"Context length" and "number of tokens" are related concepts, but they refer to slightly different things. The number of tokens refers to the total number of individual units of meaning (e.g. words or subwords) in a given input text.
>
> Context length, on the other hand, refers to the number of tokens used as a context for generating a response. The context is the information the model uses to generate a response. So, context length is the number of tokens that the model will take into account to generate a response.
>
> In other words, the context length is a subset of the number of tokens, it's the number of tokens used to generate the response. **It's usually a sliding window of tokens that goes back and forth on the input text**. The larger the context length, the more information the model has to work with, which can lead to more accurate and relevant responses but also requires more computational resources and can make the model slower.
>
> [me][surprised]

2023-01-27 About

[2203.08913] Memorizing Transformers

Tags:

2022-05-07 About