RAG

This article aims to explore the architectural design and specific code implementation of the RAG algorithm through the interpretation of papers and source code. This article mainly discusses GraphRAG, LightRAG and RAPTOR RAG, and also mentions Contextual Retrieval proposed by Anthropic and the evaluation method of the RAG algorithm. In the end, it is recommended that different methods be selected according to the size of the knowledge base document.

With the popularity of the Transformer architecture, many Embedding and Rerank models are now based on this architecture. Taking this opportunity, we will sort out the process and history of the research, and take stock of the architectures adopted by several well-known Rerank models and the companies that developed them. Finally, we will return to the topic and briefly discuss whether Rerank should be used in RAG scenarios.

This paper provides a detailed introduction to various vector similarity search methods, such as KD trees, IVF inverted indexes, HNSW and LSH. It provides a detailed introduction from data structures to algorithm implementations by analyzing the specific implementations in the source codes of Annoy, Faiss, PGVector and FALCONN.