Thanks for your tutorial.

what's the diagram tool used? anyone knows?

Taylor Sharon Robinson Donald Lopez Barbara

Thanks for the amazing content. Is there any video where he discusses memory in rag. I try to give a summary of the previous query and response along with the new query to limit token usage. As the chat continues the LLM hallucinates. I believe there might be a better way to manage old user queries and a way to "forget" by analyzing whether it is a new topic or not . Please let me know 🙂

THis helped a lot .

#greatness

RAG-Fusion uses the RRF to calculate a fused score of retrieved documents that were previously ranked by their respective retrievers. What I don't understand is that here only one retriever is being used and it's responsible for retrieving a list of documents per query, so in such a scenario, query1 might lead to fetch doc1, doc2 doc3, and so on. Yes, it's quite possible that doc1 is ranked 1 for query1 and ranked 2 for query2. But even in such a case, it's only based on a single retriever's ranking. Then, isn't it redundant to apply RRF on top of it?

The accent is so hard to catch up

dang, there still isn't a way to connect Ollama instead of an openai key at time of writing. if someone has figured out before this has been made very available and you see this- please enlighten us! <3

Does anyone know why I get a RateLimitError when I try using my OpenAI API Key with my paid ChatGPT account? It says: "You exceeded your current quota, please check your plan and billing details" but I have never done anything with this API Key before. Does anyone know how to fix this? Alternatively, does anyone know how to perform RAG without this? Please help!

love from India, keep doing the great work, Lance <3

🎯 Key points for quick navigation:

00:00 📚 Introduction to RAG by Lance Martin, a LangChain engineer.
00:14 💡 Explanation of how RAG combines custom data with LLMs.
00:28 🔍 Motivation: Most data is private, but LLMs are trained on public data.
01:08 🗃️ Context windows in LLMs are growing, allowing more private data to be input.
01:48 ⚙️ Overview of RAG: Indexing, retrieval, and generation stages.
02:54 📊 RAG unites LLMs' processing with large-scale private data.
03:24 🧠 Breakdown of RAG components: Query translation, routing, construction, and more.
04:46 ⭐ Methods for document retrieval and reranking in RAG.
05:55 💾 Indexing external documents and converting them to numerical representations.
08:25 🧩 Splitting documents for embedding due to context window limits.
10:00 🖥️ Computing fixed-length vectors for documents using embeddings.
12:45 🔍 Using k-nearest neighbors to find similar documents.
15:59 📝 Generating answers based on retrieved documents in RAG.
17:07 📝 Prompt templates for generating answers in LLMs.
19:02 🔗 Combining prompts, LLMs, and retrievers into chains.
22:14 🚀 Introduction to advanced query translation in RAG.
23:07 ✔️ Importance of rewriting queries for effective retrieval.
24:05 🌐 Multi-query approach: Rewriting questions from different perspectives.
25:38 🚀 Indexed a blog post on agents in a vector store.
26:19 🔍 Split question into sub-questions and retrieve relevant documents.
28:08 🔧 Used LangSmith to trace intermediate and final steps.
30:42 🗂️ Built a consolidated list from multiple retrievals.
35:02 🧩 Discussed sub-question decomposition retrieval.
36:23 🔄 Combined answers to iterative sub-questions for final answer.
38:18 🔗 Connected question-answer pairs sequentially in prompts.
41:02 📚 Stepback prompting for generating more abstract questions.
43:02 🪜 Generated more generic questions to enhance context for retrieval.
44:45 🔄 Retrieval performed on both original and stepback questions.
48:50 🌐 HYDE involves converting questions into hypothetical documents for better alignment with document embeddings.
49:43 🔎 Generated hypothetical documents based on questions for more effective retrieval.
51:15 📝 Hypothetical Document: Demonstrated hypothetical document generation and retrieval process.
51:44 🌟 Performance: Using hypothetical document generation can improve retrieval performance.
52:13 🚦 Routing: Involves translating a query and routing it to appropriate data sources.
53:48 🔍 Semantic Routing: Embeds and compares questions to prompts for routing.
56:08 🔗 Routing Mechanism: Connects the intended data source to specific retriever chains.
58:11 🚀 Semantic Routing Example: Demonstrates choosing a prompt based on semantic similarity.
59:47 💬 Query Construction: Transforms natural language queries to structured queries for metadata filters.
01:00:15 🗓️ Example Query: Converts natural questions into structured queries with date filters and metadata.
01:04:26 📚 Query Optimization: Optimizes retrieval by translating natural language into data-querying domain-specific languages.
01:11:48 🗄️ Hierarchical Indexing: Raptor technique deals with questions needing detailed and broader information.
01:12:57 🧩 Hierarchical indexing helps in retrieving more relevant document chunks by clustering and summarizing documents recursively.
01:14:08 🤏 Summaries provide high-level semantic representations, while raw chunks offer detailed, document-specific insights.
01:15:04 🧪 Comprehensive studies indicate that hierarchical indexing enhances semantic search by offering better coverage across different question types.
01:17:19 📇 Process involved embedding, clustering, and recursive summarization to build a tree structure of document information.
01:20:09 🛠️ Code demonstration included creating a vector store, embedding documents, clustering, summarizing, and managing tokens.
01:22:22 🔍 CoBER method enhances semantic search by generating embeddings for every token and computing maximum similarities between questions and documents.
01:24:57 🧑‍💻 RoBERTA library facilitates playing with CoBER, which showcases good performance but requires evaluating production readiness due to possible latency issues.
01:26:40 🌐 CoBER demonstrated through LangChain retriever integration, offering an efficient and unique indexing approach.
01:28:10 🗺️ Langraph released for building more complex state machines and diverse logical flows in RAG applications.
01:33:05 🔍 Corrective RAG workflow improved retrieval by re-assessing document relevance and performing web searches for ambiguous results.
01:35:06 🧩 Functions for state modification in Langraph illustrated how each state (node) in the flow modifies the document retrieval process.
01:37:08 🔍 Logical filtering: Use a grading chain to mark documents as relevant or not and perform actions based on the results.
01:37:32 🚦 Conditional routing: Based on the 'search' value, route the workflow to either transform the query for a web search or proceed to generate a response.
01:39:13 📑 Document relevance check: Filter documents for relevance before transforming the query and performing a web search.
01:39:55 🔄 Query transformation: Adjust the query based on information retrieved from a web search to improve relevance.
01:40:52 📊 Detailed node inspection: Use tools like LangSmith to inspect each node's output to ensure the logical flow is correct.
01:42:26 🚀 Moving from chains to flows: Transitioning from simple chains to complex flows offers cleaner and more sophisticated workflows.
01:44:06 🔧 Flow engineering: Flow engineering with Lang graph is intuitive and allows for sophisticated logical reasoning workflows.
01:45:03 🧩 Integrating ideas: Combining query analysis and adaptive flow engineering improves your RAG pipeline's efficiency.
01:46:14 📚 Corrective workflows: Use unit tests to ensure smooth corrective workflows during model inference.
01:48:34 💡 Command R: Uses Command R model with structured output, enabling binary yes/no responses for easier logical flow control.
01:56:21 ⚙️ Binding functions to nodes: Bind each node in your graph to a specific function to handle different logical decisions and flows.
01:58:24 🔄 If tool calls are not in the response, a fallback mechanism is triggered to choose the next data source.
01:59:18 🔍 Different data sources (web search vs. Vector store) are used, and their outputs determine the subsequent nodes in the graph.
02:00:25 🧾 Conditional edges in the graph handle logic such as document relevance and hallucination checks.
02:01:05 📊 Functions are defined as nodes and edges in the graph, following a flow that matches a predefined diagram for logic.
02:03:18 🗂️ The flow diagram for the graph aligns with the logic drawn out earlier, ensuring consistent data routing and processing.
02:05:10 ⏱️ The implemented RAG system processes questions quickly, demonstrating efficient retrieval and generation handling.
02:07:15 ⚡ Command R model shows rapid performance and effective handling of relevance, hallucination, and answer usefulness checks within the RAG system.
02:08:55 🧠 Lang graph provides a reliable, less flexible solution compared to agents, suitable for defined flows and faster implementation.
02:10:51 🧩 Agents offer more flexibility for open-ended workflows at the cost of reliability, especially when working with smaller LLMs.
02:11:46 💻 Open-source models like Command R can be run locally, enabling fast inference and practical use for online applications.
02:12:46 🔧 Practical implementation of RAG systems combines Lang graph with Command R for a fast, reliable solution adaptable for various workflows.
02:17:09 📉 Tested GPT-4's ability to retrieve and reason over multiple facts within a large context window, showing degradation in performance as complexity and context length increase.
02:18:27 🧩 Observations included the difficulty of retrieving facts placed at the beginning of a large context window, potentially due to a recency bias.
02:19:10 🔄 Confirmed that adding reasoning tasks exacerbates retrieval difficulty, highlighting limits within LLMs without a retrieval augmentation system.
02:19:52 🚩 Be skeptical of single-needle retrievals as they often oversimplify the retrieval problem.
02:21:00 🎯 Focus on the retrieval of precise document chunks, but be cautious of over-engineering.
02:22:48 🏗️ Consider document-centric RAG over chunking to simplify retrieval and reduce complexity.
02:26:30 🧩 Clustering documents and summarizing clusters help to handle queries requiring multiple pieces of information.
02:28:07 🔍 Use long-context embedding models to embed full documents effectively.
02:31:33 🖥️ Using open-source models can make RAG systems more accessible and efficient, even on local machines.

Made with HARPA AI

Tbh, this is not from scratch if you are using an heavily abstracted framework(Langchain). Its misleading.

I don't understand how this precious course is available for free. So well explained that too with the code. I am so grateful for Lance and freeCodeCamp. Thank you guys, thank you so much.

amazing content on RAG

Thank you

Hello there!

This material is amazing. I would love to read the articles cited in the video. Could someone please list them for me? Thank you!

may be not the best, explanation and not understandable easily

Very, very good! Can you post the links in the comments? Not all are in the notebooks (e.g., 2 of 4 at the 1:21:38 mark).

Is it just me or are there straight up always errors based on changed libs?
Even tho when you simply try to execute their code.
It's really frustrating working with langchain at this point.

Where can we find this video about Chunking?

This felt like a semester condensed in a few hours. This dude reads a lot. I learned about so many of interesting things.