Problem
A small services business wants a support agent that doesn’t just answer questions but actually does things: opens a ticket, updates a customer record, sends a follow-up, books a slot. The moment an agent can write to a database, the interesting question stops being “is it smart?” and becomes “what happens when it’s wrong?” A wrong answer is embarrassing. A wrong write is a phone call.
Constraints
- Every action that mutates state has to be reviewable by a human before it runs. Not as a prompt instruction, as a property of the system.
- The approval has to survive across requests and workers. A customer proposes an action now and a manager approves it later, possibly on a different process.
- The agent has to be measurable. “It seems to work” isn’t shippable.
- The whole thing should run on plain Postgres so it deploys anywhere, including Supabase.
System
Six tools live on an MCP server: two read-only (search_knowledge_base, lookup_customer) and four mutating (create_ticket, update_customer, send_email, book_slot). Every tool’s arguments and returns are Pydantic models, so validation has one source of truth.
The LangGraph agent runs retrieve → plan → approval → execute → respond. The retrieve node does read-only RAG over the knowledge base automatically. The plan node (Gemini 2.5 Flash bound to all six tools) either answers or proposes a tool call. If the proposed tool is in the mutating set, the graph routes through an approval node that calls interrupt() and suspends the run. State lives in a Postgres checkpointer, so /chat returns pending_approval plus a thread ID, and /approve can resume it from any worker.
Langfuse tracing is wired but silent without keys, so dev and CI pay nothing for it.
Decisions
The gate is enforced by the graph, not the prompt. Mutating tools are tagged twice, with MCP’s native readOnlyHint and an explicit mutating flag, and a single MUTATING_TOOLS set drives routing. The model can’t talk its way past it, because the model never gets to choose whether the gate applies.
MCP for the tool layer, even though it added a process boundary. It made the tools testable through a real client session independent of any agent framework, and it means the same tools can serve a different orchestrator later.
A 44-case golden set with a CI threshold. The eval measures tool-selection accuracy, argument validity, retrieval recall, task completion, and the safety metric that matters most: false-action rate, which is proposing a write when the case says it shouldn’t. The runner exits non-zero under 90% accuracy or over 10% false actions, and CI runs it as a gate when the API key is present.
Self-initializing schema. The container creates and seeds its own tables on startup, because managed Postgres has no docker-initdb hook and “works on the compose file but not on the cloud DB” is a classic deployment surprise.
Outcome
Latest eval: 95.5% tool-selection accuracy, 100% argument validity, 100% retrieval recall@5, 84.6% task completion, 0% false-action rate. 71 offline tests plus two live end-to-end runs. The full chat → gate → approve → database-write flow is verified running in-container, with rate limiting and an optional API key for a public demo.
What I’d do next: the task-completion gap (84.6%) is mostly argument-shaping on multi-field updates, which is a prompt-and-schema problem rather than an architecture one.