The Agentic AI Stack — Every Layer, Mapped
Embeddings, vector stores, orchestration, memory, guardrails, observability. A full map of the agentic AI landscape and where each piece fits.
Every few weeks I catch myself re-explaining the agentic AI landscape to someone — a recruiter, a fellow engineer, or honestly, myself at 2am trying to remember if I need a vector store for this or not.
So I built the map once, properly, and I’m keeping it here.
This isn’t a “best tool” ranking. It’s a layer-by-layer breakdown of what each category of tool does, why it exists, and what trade-off you’re accepting when you pick one over another.
Why a Map, Not a List
Most “agentic AI tools” posts mix categories — they’ll put Pinecone and LangGraph in the same bullet list even though one stores vectors and the other orchestrates agent logic. That’s like comparing a database to a web framework.
The stack actually breaks into distinct layers. Here they are.
1. Embeddings
Turns text into vectors so meaning can be compared mathematically.
| Tool | Best For | Trade-off |
|---|---|---|
| Voyage AI | RAG paired with Claude | Smaller ecosystem than OpenAI’s |
| OpenAI Embeddings | Fast prototyping, huge community | Not domain-tuned |
| Cohere Embed | Multilingual + built-in reranking | Smaller community |
| Jina AI | Self-hosted, cost-sensitive setups | More ops work if self-hosted |
I use Voyage AI in the Site Search Agent project — it’s Anthropic’s recommended pairing, and the accuracy on product-catalog text has been solid.
2. Retrieval Techniques
The actual strategy for finding relevant context, independent of which vector store you use.
- RAG — retrieve, then inject into the prompt. The baseline.
- GraphRAG — builds a knowledge graph first, retrieves via graph traversal. Expensive to build, but captures relationships plain RAG misses.
- Agentic RAG — the agent itself decides when and what to retrieve, and can multi-hop across several retrieval rounds.
- HyDE — generates a hypothetical answer first, embeds that to search. Helps when the query’s phrasing doesn’t match the documents’ phrasing.
For the Site Search Agent, plain RAG was enough — the vocabulary mismatch problem (“footwear for jogging” → running shoes) is solved by the embedding step itself, not by a fancier retrieval strategy.
3. Vector Stores
Where the vectors actually live.
| Tool | Best For | Trade-off |
|---|---|---|
| Pinecone | Zero-ops production apps | Cost scales with usage, lock-in |
| Weaviate | Hybrid keyword + vector search | More config than Pinecone |
| Qdrant | Cost-conscious, on-prem | Smaller ecosystem |
| Milvus | Billion-scale vector counts | Heavy ops overhead |
| Chroma | Local dev, prototyping | Not built for production scale |
| pgvector | Teams already on Postgres | Less optimized at huge scale |
This is the one that actually matters most for my stack. I’m on Neon Postgres for everything already — adding pgvector means no new infrastructure, no new vendor, no new bill. For a portfolio-scale RAG project, that’s the right trade every time.
4. Agent Memory
Persisting context across sessions, not just within one conversation.
- Mem0 — auto-summarizing long-term memory layer for agents.
- Zep — fast retrieval of conversational history across sessions.
Neither of my projects need this yet — everything so far is single-session. But the Family Assistant platform will eventually need to remember a child’s preferences across visits, and that’s exactly this layer’s job.
5. Orchestration Frameworks
The layer that actually runs the agent loop.
| Tool | Best For | Trade-off |
|---|---|---|
| Claude Agent SDK | Native Claude agents, no extra abstraction | Smaller community than LangChain |
| LangGraph | Complex, conditional multi-step flows | Steeper learning curve |
| CrewAI | Role-based multi-agent collaboration | Less low-level control |
| AutoGen | Research, agent-to-agent dialogue | Harder to productionize |
| LlamaIndex Agents | Retrieval-heavy agents | Less general-purpose outside RAG |
I’ve deliberately avoided LangChain across all four projects — every
agentic loop I’ve built (while(stop_reason === "tool_use")) is
hand-rolled against the raw Anthropic API. It’s more code up front,
but I understand every line of it, and there’s nothing to debug
through an abstraction layer when something breaks.
6. Tool-Use Protocols
How the model actually calls your code.
- MCP (Model Context Protocol) — open, vendor-neutral standard. Write the tool server once, any MCP-compatible client can use it.
- OpenAI Function Calling — simple, but locked to OpenAI’s format.
- LangChain Tools — huge pre-built library, but abstraction overhead.
Project 4 (Restaurant Agent) was my first real MCP build —
restaurant-mcp-server exposes the tools, restaurant-agent
consumes them. The appeal is portability: those same tools could be
wired into Claude Desktop, Claude Code, or any other MCP client
without touching the server code.
7. Reasoning & Planning
Not a tool category — a set of prompting/looping patterns.
- ReAct — reason, then act, then observe, repeat. The default pattern underneath almost every agent framework.
- Chain-of-Thought — step-by-step reasoning before answering.
- Tree-of-Thought — explores multiple reasoning branches, expensive but useful for uncertain, multi-path problems.
- Reflexion — the agent critiques its own past attempt and retries.
- Plan-and-Execute — plan once, execute many steps. Cheaper than pure ReAct for long-running tasks, but the plan can go stale if the world state changes mid-execution.
8. Evaluation & Observability
How you know the agent is actually working, and keeps working.
| Tool | Best For | Trade-off |
|---|---|---|
| LangSmith | LangGraph/LangChain apps | Best value locked to that ecosystem |
| Langfuse | Framework-agnostic, self-hostable | Less polished UI than LangSmith |
| Braintrust | Rigorous eval/regression pipelines | Eval-focused, less general tracing |
| Helicone | Quick cost/latency monitoring | Less deep tracing |
Since none of my projects use LangChain, Langfuse or Helicone are the natural fit here — I just haven’t wired one in yet. It’s on the list.
9. Guardrails & Safety
- Guardrails AI — validates outputs against a schema.
- NeMo Guardrails — programmable conversation-flow control.
- Rebuff — purpose-built prompt injection detection.
- Lakera — commercial, broader LLM security coverage.
Relevant the moment an agent handles untrusted input — user uploads, scraped web content, anything you didn’t write yourself. The Autism Worksheet Studio’s content moderation pipeline is a version of this: pre- and post-generation checks before anything reaches a family.
10. Deployment & Low-Code Layers
- Claude Code / Claude Cowork — agentic dev and knowledge-work tools, no separate infra to stand up.
- Vercel AI SDK — easiest path from prototype to a deployed Next.js/React app.
- n8n, Lindy, Relevance AI — visual, no-code agent builders for business workflows.
- Bedrock Agents / Vertex AI Agent Builder — managed, but you inherit AWS or GCP lock-in.
Where My Stack Actually Sits
Laying all four projects against this map:
Embeddings → Voyage AI (Project 3)
Retrieval → RAG (Projects 2, 3)
Vector store → pgvector on Neon (Project 3)
Orchestration → hand-rolled loop (Projects 1–4)
Tool protocol → MCP (Project 4)
Reasoning pattern → ReAct (Projects 1–4)
Deployment → Cloudflare Workers (Project 4, this site)
No LangChain, no managed agent platform, no vector DB vendor beyond what Neon already gives me. It’s more typing up front than reaching for a framework — but every piece is something I can explain line-by-line, which matters more to me right now than saving a week of setup time.
Next up: wiring Langfuse into the Restaurant Agent so I can actually see what these agent loops are doing in production, not just trust that they’re working.