Agent in production needs infrastructure + observability + guardrails + evals. The checklist that actually works.
For background see our glossary on agent observability and AI evals.
Step 1: Pick infrastructure (1 day)
For most agents in 2026:
Compute: Vercel, Railway, Fly.io for HTTP-triggered agents. AWS Lambda for cron/event-driven.
Model API: Anthropic Claude or OpenAI GPT-5. Both have OpenAI-compatible APIs.
State storage: Postgres for structured state. Redis for hot cache. Vector DB (pgvector or Pinecone) for retrieval.
Job queue: Inngest, Trigger.dev, or BullMQ for long-running tasks.
Step 2: Set up observability (1 day — non-negotiable)
Without observability, debugging production agents is impossible.
Tools:
- Helicone (free tier excellent)
- LangSmith (LangGraph-native)
- Braintrust (combines evals + observability)
Trace every:
- LLM call (prompt, response, cost, latency)
- Tool call (input, output, errors)
- State transition
- User interaction
See our LLM observability glossary entry.
Step 3: Build the eval suite (2-3 days)
Before launching, write 50-200 test cases covering:
- Happy path scenarios
- Known failure modes
- Edge cases discovered in dev
- Security probes (prompt injection attempts)
Use Promptfoo, Braintrust, or LangSmith. Run on every model swap or prompt change.
See AI evals.
Step 4: Layer guardrails (2-3 days)
Three layers, none optional:
Input filtering: Detect prompt injection, PII leakage, off-topic queries.
Output classification: Re-classify model outputs. Block content that violates policy.
Tool-call allowlists: Agent can only call tools the deployment explicitly authorized.
Tools: NeMo Guardrails, Llama Guard, Lakera, or custom.
Step 5: Cost optimization (1 day)
Three patterns that cut costs 50-90%:
1. Prompt caching. Anthropic and OpenAI discount cached input tokens 50-90%. Structure prompts with stable content at the top.
2. Model routing. Strong model for planning + verification. Cheaper model for routine tool calls.
3. Aggressive max-iteration caps. Prevents runaway costs from infinite loops.
Step 6: Pre-launch checklist (1 day)
Before shipping to real users:
- Evals pass at >90% on happy path
- Observability tracing every LLM/tool call
- Guardrails active on input + output + tool calls
- Max-iteration cap set
- Approval gates on irreversible actions
- Cost monitoring with budget alerts
- Red-team eval run at least once
- Incident response runbook documented
- On-call rotation set if critical
Step 7: First 30 days post-launch
Watch for:
Day 1-7: Cost spikes, latency outliers, eval regressions.
Day 8-14: User feedback on quality. Categorize: prompt issue, model issue, infrastructure issue.
Day 15-30: Eval suite expansion based on real failures. Promote successful patterns to defaults.
The verdict
Production-grade agent deploy: infrastructure (1 day) + observability (1 day) + evals (2-3 days) + guardrails (2-3 days) + cost optimization (1 day) + launch checks (1 day). Total: 1-2 weeks of focused work.
Skip any step and you'll regret it within 30 days of going live.
For more see How to build an AI agent in 2026, AI evals glossary, and LLM observability.