supercoderhawk

🕊️The Paloma paper is truly impressive - a must-read for anyone caring about the language model evaluation. It addresses two crucial questions that had previously left me puzzled: ❓Can the validation loss on one corpus (e.g., C4) represent all domains? The answer is no🚫.…

RAG And Context Understanding A great diagram that showcases the challenges with RAG benchmarking and LLM context understanding RAG systems are complex because of the following 4 issues. Stuffing the context of the LLM rarely helps and typically confuses the LLM We need a…

Microsoft presents UFO A UI-Focused Agent for Windows OS Interaction paper page: huggingface.co/papers/2402.07… introduce UFO, an innovative UI-Focused agent to fulfill user requests tailored to applications on Windows OS, harnessing the capabilities of GPT-Vision. UFO employs a…

New paper: How can you tell when a model is hallucinating? Let it cheat! An expert doesn't need to cheat, so if your model learns to cheat, there must be something it doesn't know. Our general new approach for measuring uncertainty: arxiv.org/abs/2402.08733

An incredible skill that I have witnessed, especially at OpenAI, is the ability to make “yolo runs” work. The traditional advice in academic research is, “change one thing at a time.” This approach forces you to understand the effect of each component in your model, and…

so i guess this is a thing now universities running ads to resell students' data for training llms 💰💰💰

It’s year 2024, and n-gram LMs are making a comeback!! We develop infini-gram, an engine that efficiently processes n-gram queries with unbounded n and trillion-token corpora. It takes merely 20 milliseconds to count the frequency of an arbitrarily long n-gram in RedPajama (1.4T…

Large Language Model (LLM) agents promise to free us from mundane tasks, but how should they best interact with our world? Introducing CodeAct, an agent {framework, instruction-tuning dataset, model}, employs executable Python code to unify the actions of LLM agents. 🧵1/

Continual Learning for LLMs One of the biggest challenges of working with LLMs is keeping them updated. Continual learning aims to enhance the overall linguistic and reasoning capabilities of LLMs. This survey paper provides an overview of developments in continual learning.…

A Novel RAG Approach That Understands The Whole Document Context RAG has rapidly evolved to be the standard way to apply LLMs in production. However, most methods are still limited because most existing methods retrieve only short contiguous chunks from a retrieval corpus,…

Lots of compelling AI research ideas this week ranging from self-correcting RAG to sparsified LVLMs. A few papers I’ve been reading this week: - OLMo - SliceGPT - MoE-LLaVa - Corrective RAG - Rephrasing the Web - Redefining Retrieval in RAG - LLMs for Mathematical Reasoning…

We just opened sourced SQLCoder-70B! It outperforms all publicly accessible LLMs for Postgres text-to-SQL generation by a very wide margin. SQLCoder is finetuned on @AIatMeta's CodeLlama-70B model that was released yesterday on less than 20,000 hand-curated prompt completion…

(1/5)🚀 Our OpenMoE Paper is out! 📄 Including: 🔍ALL Checkpoints 📊 In-depth MoE routing analysis 🤯Learning from mistakes & solutions Three important findings: (1) Context-Independent Specialization; (2) Early Routing Learning; (3) Drop-towards-the-End. Paper Link:…

I'm currently looking into different metrics and frameworks around Retrieval-Augmented Generation (RAG) evaluation. This is a first brain dump. But the landscape is already quite broad. What RAG evaluation metrics and frameworks have you already tested? And which ones did you…

MuGI: Enhancing Information Retrieval through Multi-Text Generation Intergration with Large Language Models Proposes a framework that leverages LLM text generation to expand queries and substantially improves IR performance. 📝arxiv.org/abs/2401.06311 👨🏽‍💻github.com/lezhang7/Retri…

Improving Information Retrieval in LLMs One effective way to use open-source LLMs is for search tasks, which could power many other applications. This work explores the use of instruction tuning to improve a language model's proficiency in information retrieval (IR) tasks.…

Here’s a neat paper by Barnett et al. (@DeakinA2I2) that outlines 7 failure points in building a RAG pipeline over your data. 🚫 Missing content (did not index it) 🚫 Missing in top-k retrieved set 🚫 Missing in reranked set 🚫 Not extracted (in context but LLM couldn’t use) 🚫…

There was a lot of cool RAG research in the past year or two, and luckily for you, all of these efforts are tracked under one place! “Retrieval-Augmented Generation for Large Language Models: A Survey” by Gao et al. does an admirable job categorizing all RAG research into three…

Although there are abundant work studying long-context LLMs, most of them talks about architecture / positional encoding, almost none of existing papers talk about data. In this work, we take a close look at data influence on context scaling yaofu.notion.site/Understanding-…

New RAG technique alert 🚨 We’ve come up with an advanced RAG technique in @llama_index that lets you ask structured questions over many documents ✨: 1. Model each document as a metadata dictionary - store more attributes beyond a simple text summary. (e.g. a row in SQL…