]> 2024-03-04T08:15:12Z 2024-03-04 Mexico 1968 Official Film | The Olympics in Mexico 2024-03-31T14:18:04Z 2024-03-31 Aran Komatsuzaki sur X : "performance of LLMs with user prompt at the top vs. bottom of the user input (e.g. this image)" 2024-03-23T11:44:52Z Matteo Pagliardini sur X : "#DenseFormer, a simple modification that performs—after each transformer block—a weighted average of past representations." 2024-03-23 Understanding Hierarchical Navigable Small Worlds (HNSW) 2024-03-08 2024-03-08T12:33:18Z Ollama: good name, anyway 2024-03-30T01:39:42Z 2024-03-30 Leonie sur X : "Ollama allows you to run open source LLMs LOCALLY..." raphaelsrty sur X : "My personal Knowledge Base made to the front page of HackerNews today..." 2024-03-23 2024-03-23T11:53:35Z 2024-03-07T14:12:15Z 2024-03-07 GraphRAG: Unlocking LLM discovery on narrative private data - Microsoft Research > GraphRAG uses **LLM generated knowledge graphs** to provide substantial improvements in question-and-answer performance when conducting document analysis of complex information. > power of **prompt augmentation** when performing **discovery** on private datasets (data that the LLM is not trained on and has never seen before, such as an enterprise’s proprietary research, business documents..) > GraphRAG uses the LLM to **create a knowledge graph based on the private dataset**. This graph is then used alongside graph machine learning to perform **prompt augmentation** at query time. > the GraphRAG approach [can] **discover entities in the query**. This allows the LLM to ground itself in the graph and results in superior answer that contains provenance through links to the original supporting text GraphRAG can answer queries such as "**what are the top five themes in the data?**" 2024-03-09 2024-03-09T11:55:54Z Akshay 🚀 sur X : "Let's build a "Chat with your code" RAG application, step-by-step" > Ludwig Boltzmann, who spent much of his life studying statistical mechanics, died in 1906, by his own hand. Paul Ehrenfest, carrying on the work, died similarly in 1933. Now it is our turn to study statistical mechanics. Massimo sur X : "Dr. David L. Goodstein’s «States of Matter», a book published in January 1985, has a very remarkable opening passage." 2024-03-17T08:22:33Z 2024-03-17 DSPy Cheatsheet | DSPy 2024-03-30 2024-03-30T01:35:56Z 2024-03-30T17:27:40Z Philipp Schmid sur X : "Can we combine multiple fine-tuned LLMs into a single MoE?..." 2024-03-30 > 1. Select pre-trained LLM as the seed model > 2. Fine-tune individual LLMs on dedicated task, domain or language > 3. Combine feedforward parameters of LLMs (2) in MoE layers and average the remaining parameters > 4. Fine-Tune combined MoE to learn token-level routing (assuming experts are frozen), allowing the model to activate the appropriate experts for different inputs selectively. Shruti Mishra sur X : "NVIDIA just released FREE online courses in AI..." 2024-03-23 2024-03-23T14:47:52Z 2024-03-18T22:57:09Z 2024-03-18 Der Verlorene (L'Homme perdu) 1951 De et avec Peter Lorre 2024-03-09 2024-03-09T11:37:47Z Krista Opsahl-Ong sur X : "Got a pipeline with **multiple prompts**, like a DSPy program? ... Introducing MIPRO, a Multi-prompt Instruction Proposal Optimizer...." 2024-03-15T23:56:26Z 2024-03-15 Agriculture : une réforme express de la PAC qui « détricote les acquis environnementaux » > The study demonstrates that RAG significantly improves LLM performance, **even on questions within their training domain**. > RAG could enable smaller, less costly, or private models to deliver high-quality results in tasks requiring simple factual reasoning 2024-03-13T22:49:36Z 2024-03-13 RAG makes LLMs better and equal | Pinecone How to Build a RAG System With LlamaIndex, OpenAI, and MongoDB Vector Database | MongoDB 2024-03-03T10:21:00Z 2024-03-03 2024-03-19 [tweet](https://x.com/Nils_Reimers/status/1769809006762037368?s=20) > Instead of reducing the number of dimensions, a better method is to train the model specifically to use fewer bytes per dimension. By using 1 byte per dimension, we reduce the memory 4x (954 GB → 238 GB) while keeping 99.99% of the original search quality. We can go even further... 2024-03-19T23:31:14Z Cohere int8 & binary Embeddings - Scale Your Vector Database to Large Datasets 2024-03-13 2024-03-13T23:27:05Z Command-R: RAG at Production Scale 2024-01-22T18:09:52Z 2024-03-17T07:58:15Z Thomas Demeester Chris Develder 2024-03-17 Karel D'Oosterlinck François Remy Karel D'Oosterlinck In-Context Learning for Extreme Multi-Label Classification Christopher Potts Omar Khattab Multi-label classification problems with thousands of classes are hard to solve with in-context learning alone, as language models (LMs) might lack prior knowledge about the precise classes or how to assign them, and it is generally infeasible to demonstrate every class in a prompt. We propose a general program, $\texttt{Infer--Retrieve--Rank}$, that defines multi-step interactions between LMs and retrievers to efficiently tackle such problems. We implement this program using the $\texttt{DSPy}$ programming model, which specifies in-context systems in a declarative manner, and use $\texttt{DSPy}$ optimizers to tune it towards specific datasets by bootstrapping only tens of few-shot examples. Our primary extreme classification program, optimized separately for each task, attains state-of-the-art results across three benchmarks (HOUSE, TECH, TECHWOLF). We apply the same program to a benchmark with vastly different characteristics and attain competitive performance as well (BioDEX). Unlike prior work, our proposed solution requires no finetuning, is easily applicable to new tasks, alleviates prompt engineering, and requires only tens of labeled examples. Our code is public at https://github.com/KarelDO/xmc.dspy. [2401.12178] In-Context Learning for Extreme Multi-Label Classification 2024-01-22T18:09:52Z 2401.12178 2024-03-17T08:07:54Z 2024-03-17 Saurabh Dashora sur X : "12 database types and when to use them" LlamaIndex 🦙 sur X : "An emerging technique to better represent your data for RAG/LLM applications is to only chunk the data, but also hierarchically cluster and index it..." 2024-03-30 2024-03-30T01:29:05Z Answer.AI - You can now train a 70b language model at home 2024-03-09 2024-03-09T10:06:03Z learning human actions on computer applications 2024-03-07T16:02:46Z rabbit - research 2024-03-07 2024-03-16 2024-03-16T10:28:38Z Benjamin Clavié sur X : "Introducing rerankers: a lightweight library to provide a unified way to use various reranking methods" 2024-03-16 2024-03-16T10:08:07Z Jo Kristian Bergum sur X : "...helping people understand the shortcomings of text embedding models for their data. The most powerful has been demonstrating how the embedding models' tokenizers work." 2024-03-05 2024-03-05T22:38:53Z Risques liés aux « nouveaux OGM » : l’Anses recommande une évaluation au cas par cas, dans un avis resté confidentiel 2024-03-12T08:10:20Z 2024-03-12 On a testé Le Chat, l’étonnant ChatGPT à la française de Mistral AI « Il n’existe aucun scénario de transition qui n’implique des changements profonds dans notre relation aux animaux » Pour fuir une battue, une laie sur le point de mettre bas se jette à la mer, nage 10 km jusqu'à l'île de Groix où elle accouche de 3 marcassins. La presse en parle. Ils sont abattus par des chasseurs. 2024-03-31 2024-03-31T13:44:33Z 2024-03-07 tools to easily embed and cluster texts as well as label clusters semantically huggingface/text-clustering: Easily embed, cluster and semantically label text datasets 2024-03-07T13:04:38Z 2024-03-07T16:10:39Z 2024-03-07 Topics | IBM Research Aux Pays-Bas, l’éventuelle délocalisation du géant ASML agite les autorités politiques **Le groupe de haute technologie dénonce une politique de plus en plus restrictive en matière d’immigration** et un environnement pas assez « probusiness ». 2024-03-12T22:46:41Z 2024-03-12 2024-03-08 ColBERT gist:c1182551fa609736d47df4af82f7c5ab > a quick gist that does synthetic data gen, fine-tuning, eval. Just add your own documents, or try it on a PG essay. @JoshPurtell 2024-03-08T23:31:23Z 2024-03-10T11:25:15Z RAG CLI - LlamaIndex CLI tool to ingest local files into a local vector database that is then used for a Chat Q&A repl within your terminal. 2024-03-10 Raptor Retriever LlamaPack 2024-03-03T22:17:10Z 2024-03-03 Demo of ColBERT query-passage scoring interpretability - try with the following: "what are the mentioned EICPS?" and passage "There is a security risk related to EICPS 67" - MaxSim Score: 20.71 - Estimated Relevance: 64.71% - highlights: There related - then "what are the mentioned animals?" and "There is a security risk related to lions" - MaxSim Score: 9.18 - Estimated Relevance: 28.68% - highlights: related lions ``` Effects of climate change on marine ecosystems MaxSim Score: 27.90 Estimated Relevance: 87.17% Effects of global warming on marine ecosystems MaxSim Score: 24.62 Estimated Relevance: 76.94% Effects of global warming on life in the oceans MaxSim Score: 19.64 Estimated Relevance: 61.39% Effects of global warming on life on Mars MaxSim Score: 13.65 Estimated Relevance: 42.65% ``` 2024-03-08T18:07:53Z 2024-03-08 ColBERT Inference in the Browser 2024-03-28 2024-03-28T08:37:59Z Abhishek sur X : "ChatGPT can now create Mind Maps.." Enhancing RAG-based application accuracy by constructing and leveraging knowledge graphs 2024-03-16 2024-03-16T16:13:35Z Hrishi sur X : "WalkingRAG is finally out!..." 2024-03-09 2024-03-09T11:28:51Z Announcing Vespa Long-Context ColBERT | Vespa Blog 2024-03-03T09:01:52Z 2024-03-03 Comment Opendatasoft est devenue l’acteur incontournable de l’ouverture des données publiques 2024-03-05T14:04:14Z 2024-03-05 2024-03-03T14:23:09Z Naomi Oreskes, historienne des sciences : « Nous mettons en œuvre aux Etats-Unis des idées politiques qui ne fonctionnent pas. Nous payons le prix fort du libre marché » 2024-03-03 un ours polaire nage plusieurs centaines de kilomètres pour venir s'échouer en Islande, et y être abattu par la police. 2024-03-31T16:16:27Z 2024-03-31 Après avoir dérivé en Arctique, un ours polaire est abattu par la police islandaise (2008) 2024-03-16T14:05:21Z 2024-03-16 Santiago sur X : "In 1973, UC Berkeley was on the brink of getting sued for gender bias...." Ecologie : « Le gouvernement n’hésite pas à dissimuler ou à retenir des informations cruciales pour le débat démocratique » 2024-03-17 2024-03-17T09:00:15Z 2024-03-03 2024-03-03T10:14:19Z Jerry Liu sur X : "To better augment LLMs with context, it makes a lot of sense to organize context not just as a flat list of text chunks, but as a hierarchy of high-level to low-level details. RAPTOR..." > To better augment LLMs with context, it makes a lot of sense to organize context not just as a flat list of text chunks, but as a hierarchy of high-level to low-level details. > > RAPTOR is a super simple but neat idea towards this direction. Hierarchically cluster and summarize the text into a tree (the clustering is important, allows semantically related concepts to be grouped together and doesn't purely rely on spatial positioning!). During query-time dynamically retrieve the most relevant context to the question. > Tell me something I don't know... 2024-03-07T01:35:13Z 2024-03-07 Yrjänä Rankka 🌻 sur X : "@ParisHilton You are a disambiguation benchmark for semantic reasoning." / X (2018) > in high dimensional space, the concept of proximity, distance or nearest neighbor may not even be qualitatively meaningful. 2024-03-03 On the Surprising Behavior of Distance Metrics in High Dimensional Space (Aggarwal 2001) 2024-03-03T21:33:43Z Christopher Manning sur X : "Now that everyone is writing LLM programs, the idea of doing approximate bayesian inference by sampling along linguistic pipelines (rather than k-best, etc.) is more relevant again" 2024-03-19 2024-03-19T23:51:46Z "Solving the Problem of Cascading Errors: Approximate Bayesian Inference for Linguistic Annotation Pipelines" (2006) > The end-to-end performance of natural language processing systems for compound tasks, such as question answering and textual entailment, is often hampered by use of a greedy 1-best pipeline archi- tecture, which causes errors to propagate and compound at each stage. We present a novel architecture, which models these pipelines as Bayesian networks, with each low level task corresponding to a variable in the network, and then we perform approximate inference to find the best labeling. 2024-03-23T15:01:31Z 2024-03-23 Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval 2024-03-01 Intro to DSPy: Goodbye Prompting, Hello Programming! | by Leonie Monigatti | Feb, 2024 | Towards Data Science 2024-03-01T02:17:40Z 2024-03-28 2024-03-28T08:32:20Z LlamaIndex 🦙 sur X : "RAFT - Retrieval Augmented Fine Tuning..." LlamaIndex sur X : "Save Memory (and Money) in RAG pipeline with @Cohere 's Int8 and Binary Embeddings..." 2024-03-30 2024-03-30T17:48:43Z 2024-03-15 2024-03-15T23:35:08Z Frank van Harmelen sur X : "...GenAI is rapidly becoming the best motivation for symbolic AI in a long time!" 2024-03-30 2024-03-30T01:31:45Z Jo Kristian Bergum sur X : "Vespa is the only vector database that supports..." 2024-03-07 2024-03-07T15:38:33Z KGC23 Keynote: The Future of Knowledge Graphs in a World of LLMs — Denny Vrandečić, Wikimedia - YouTube 2024-03-11T10:09:22Z 2024-03-11 What you should know about RAG (from beginner to advanced) | by Jonathan Nguyen | Medium Nils Reimers sur X : "Embeddings can store only 1 aspect/topic per embedding well." > On Wikipedia, one paragraph typically focuses on one topic. So this gives you a good chunking for Wikipeda 2024-03-13 2024-03-13T23:20:09Z (thread by the person of [WalkingRAG](tag:walkingrag)) > The similarities between WalkingRAG and RAPTOR are that both attempt to capture relationships in the data into a higher structure using LLMs... This is a tree in RAPTOR's case, with WalkingRAG it's a graph. 2024-03-09 2024-03-09T11:30:15Z Hrishi sur X : "RAPTOR is... one of the very few [RAG architectures] that actively presumes and uses the structure in a document...."