MCP vs RAG: what's the difference?
RAG and MCP both get outside knowledge to a model, at different moments. RAG (retrieval-augmented generation) searches an index, usually of embedded text, and pastes the best passages into the prompt before generation. MCP is a protocol that lets the model call tools and read resources during its run, so it can fetch live data and take actions.
Part of Glossary: the terms behind MCP, coding agents and shared pages, defined plainly.
RAG, ahead of time: you split documents into chunks, turn each into an embedding, and store the vectors in an index.
RAG, at question time: your app embeds the question, pulls the nearest chunks, and puts them in the prompt. The model never chooses what gets fetched.
MCP, at startup: the client connects to one or more servers and lists their tools and resources, each with a description the model can read.
MCP, while answering: the model decides to call a tool or read a resource. The result lands in its context, and it can call again with what it learned.
Together: put the search step behind an MCP tool. The model then decides when to search and with what query, instead of always getting the top chunks.
Source: Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)
RAG versus MCP, layer by layer
The two are not at the same level, which is why the comparison confuses people. RAG is an architecture. The 2020 paper that named it paired a pretrained sequence-to-sequence model (its parametric memory) with a dense vector index of Wikipedia reached through a neural retriever (its non-parametric memory), so the model could cite and update knowledge it did not have to store in its weights. Today the phrase covers any system that retrieves text and prepends it to the prompt.
MCP is a protocol. It says how a client and a server exchange JSON-RPC messages, how the server advertises tools, resources and prompts, and how the model's host invokes them. It says nothing about embeddings, chunking or ranking. A model using MCP might never retrieve a document; it might only create a ticket.
So the real contrast is about timing and control. In RAG, your application decides what the model sees, before the model runs, using a similarity score. In MCP, the model decides what to fetch, while it runs, by reading tool descriptions and choosing. One is a pipeline; the other is a conversation.
Example: a documentation assistant
Suppose you want an assistant that answers questions about your product docs. The classic RAG build embeds every page, and on each question retrieves the five closest chunks and asks the model to answer from them. It is fast and the cost per question is flat. It also fails in a specific way: if the first retrieval misses, the model has no second chance, and it cannot check the live status page or open the ticket the user is asking about.
The MCP build gives the agent a server with a search_docs tool, a read_page tool and a get_ticket tool. The model searches, reads the page the hit came from, notices the question is really about a recent incident, and calls the ticket tool. Three round trips instead of one, more tokens, slower, and a far better answer on the hard questions. The search tool inside that server is, of course, your RAG retriever.
When to use which
Use plain RAG when the corpus is large and stable, questions are answerable from it, and you want predictable latency and cost. Use MCP when the answer depends on live state, when the task includes actions, or when the model needs to look in several places and decide as it goes. Use both when you have a corpus and an agent: wrap the retriever as a tool so the model can search on demand, and keep a few high-value pages as resources it can read directly.
Common mistakes
- Pasting everything into the prompt. RAG was invented because the model cannot hold the whole corpus. An agent with a search tool has the same limit; retrieving twenty chunks per call fills the context window and crowds out the task.
- Expecting a tool to be used. With RAG the retrieval always happens. With MCP the model may decide not to search. If a lookup is mandatory, do it in your pipeline rather than hoping.
- Comparing them on accuracy alone. The honest comparison includes latency, cost per answer and how much the model is trusted to steer.
FAQ
- Does MCP replace RAG?
- No. RAG is a technique for finding relevant text; MCP is a wire protocol for connecting a model to tools and data. A RAG pipeline is a natural thing to expose as an MCP tool, and plenty of MCP servers do exactly that.
- When is RAG better than an MCP tool?
- When the knowledge is a large, mostly static corpus and every answer needs some of it. Retrieval before generation is cheap, predictable and does not depend on the model deciding to look. Use MCP when the data changes, lives behind a service, or the task involves doing something, not just knowing.
- Can MCP resources do what RAG does?
- Partly. Resources let a server offer documents the client can read into context, which covers small, well-named sources. They do not do similarity search over thousands of chunks; for that you still want a retriever, exposed as a tool.
Stop sending HTMLover Slack.
Create an account in a minute, connect your coding agent, and make your first board.