Semanlink - LLM AND LMs: context length

Impression

Recherche de Mot-clé

Recherche de Doc

Préférences...

Mots-clés liés

Andrej Karpathy - Arxiv Doc - DeepMind - Discuté avec Raphaël - Hugo Larochelle - k-nearest neighbors - Language Model - LLM - LLMs: extended context window - LMs: context length - Long documents - Memory in deep learning - NLP@Google - NLP@Meta - NLP@Microsoft - OpenAI GPT - Transformers - Transformers: Quadratic complexity - Tweet -

LLM AND LMs: context length

Descendants partagés

13 Documents

[2404.11018] Many-Shot In-Context Learning

Tags:

2024-04-21 A propos

Albert Gu sur X : "Quadratic attention has been indispensable for information-dense modalities such as language... until now..."

Tags:

2023-12-05 A propos

[2310.03025] Retrieval meets Long Context Large Language Models

Tags:

2023-10-07 A propos

[2309.12307] LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models

Tags:

2023-09-26 A propos

Philipp Schmid sur X : “YaRN” allows you to scale LLMs like llama 2 to over 100k context!...

Tags:

2023-09-01 A propos

LlamaIndex 🦙 (GPT Index) sur Twitter : "Pretty much everyone building LLM apps over data has to figure out how to... cram arbitrary data into limited context windows?"

Tags:

2023-07-09 A propos

[2307.02486] LongNet: Scaling Transformers to 1,000,000,000 Tokens

Tags:

2023-07-06 A propos

[2305.07185] MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers

Tags:

2023-07-01 A propos

State of GPT (Andrej Karpathy at Microsoft Build)

Tags:

2023-05-24 A propos

Aran Komatsuzaki sur Twitter : "Unleashing Infinite-Length Input Capacity for Large-scale Language Models with Self-Controlled Memory System Enables LLMs to outperform ChatGPT in scenarios involving ultra-long document summarization or long-term conversations."

Tags:

2023-04-27 A propos

Aran Komatsuzaki sur Twitter : "Scaling Transformer to 1M tokens and beyond with Recurrent Memory Transformer..."

Tags:

2023-04-25 A propos

Does ChatGPT remember what happened earlier in the conversation? | OpenAI Help Center

Tags:

[aistackexchange](https://ai.stackexchange.com/questions/38150/how-does-chatgpt-retain-the-context-of-previous-questions)

> [me]I heard people using the phrase "context length" about you. Is it the same thing as the number of tokens you just spoke of?
>
> [ChatGPT]"Context length" and "number of tokens" are related concepts, but they refer to slightly different things. The number of tokens refers to the total number of individual units of meaning (e.g. words or subwords) in a given input text.
>
> Context length, on the other hand, refers to the number of tokens used as a context for generating a response. The context is the information the model uses to generate a response. So, context length is the number of tokens that the model will take into account to generate a response.
>
> In other words, the context length is a subset of the number of tokens, it's the number of tokens used to generate the response. **It's usually a sliding window of tokens that goes back and forth on the input text**. The larger the context length, the more information the model has to work with, which can lead to more accurate and relevant responses but also requires more computational resources and can make the model slower.
>
> [me][surprised]

2023-01-27 A propos

[2203.08913] Memorizing Transformers

Tags:

2022-05-07 A propos