Domain-Specific Agents Beat General Ones

The pitch for general-purpose agents is seductive: one system that plans, calls tools and handles anything a user throws at it. The demo is always excellent. The production incident review, six weeks later, is always the same conversation — the agent chose the wrong tool, on a task nobody had tested, and nothing in the logs explained why.

The teams shipping agents that survive contact with real traffic have mostly gone the other way. They build narrow agents that do one job, with a small tool surface and an evaluation suite that predates the code.

Why Breadth Fails

An agent’s job at every step is a search problem: given the conversation so far, pick the next action. The size of that search space is set by how many tools you exposed and how vaguely you scoped the task.

Add a tool and you have not simply added a capability. You have added a wrong answer to every decision the agent makes from then on. With thirty tools, several of which sound plausible for the same request, tool-selection errors stop being rare — and in a multi-step task they compound. A 95% per-step success rate across eight steps is a 66% success rate for the task.

Breadth breaks evaluation too, which is the more expensive problem. You cannot build a representative test set for “anything a user might ask”. You can build one for “reconcile a supplier invoice against a purchase order”, because that workflow has a finite shape and a filing cabinet of historical examples with known-correct answers.

What “Domain” Actually Means

A domain is a workflow with a definable end state. It is not a job title.

Not a domain	A domain
Finance assistant	Reconcile a supplier invoice against a PO and flag variances
Customer support agent	Triage an inbound ticket and attach the matching account history
DevOps copilot	Diagnose a failed deploy from CI logs and identify the offending commit
Research assistant	Summarise a clinical trial protocol into a structured eligibility table

The test: can you write down what a correct outcome looks like, for fifty real cases, without hedging? If not, you have a category, not a domain, and you should split it further.

Build the Evaluation Harness First

This is the step teams skip, and it is the one that determines whether the project ships.

Before you write the agent, collect real cases from the workflow — historical tickets, invoices, deploys, whatever the domain produces — and record the correct outcome for each. Fifty is enough to start. A hundred is comfortable.

Score on task success, not on how the output reads:

interface EvalCase {
  id: string;
  input: WorkflowInput;
  expected: {
    outcome: 'resolved' | 'escalated';
    // Domain assertions — what must be true of the result
    assertions: Array<(result: AgentResult) => boolean>;
  };
}

const result = await runAgent(testCase.input);
const passed = testCase.expected.assertions.every((assert) => assert(result));

Run the suite on every prompt edit, every tool change, every model upgrade. Prompt changes are not local — tightening the wording for one failure mode regularly breaks three cases that used to pass, and without the suite you find out from a customer.

Track the pass rate over time. That number, not a demo, is what tells you whether the agent is ready.

Design the Tool Surface for the Domain

Tools are the agent’s interface to your system, and they should read like the domain, not like your REST API.

// ❌ Generic wrappers — the model has to know your data model
getRecord(table: string, id: string)
updateRecord(table: string, id: string, patch: object)
runQuery(sql: string)

// ✅ Domain actions — the model has to know the workflow
findPurchaseOrderForInvoice(invoiceId: string)
flagVariance(invoiceId: string, reason: VarianceReason, amount: Money)
escalateToController(invoiceId: string, summary: string)

The generic set is more powerful and much worse. It pushes your schema into the model’s context, invites the agent to invent queries, and gives you no place to enforce rules. The domain set encodes the workflow in the tool names, so the right sequence is the obvious one.

Some practical constraints worth applying to every tool:

• Validate arguments against a schema. Reject rather than coerce. A malformed call the agent can retry beats a plausible-looking wrong action.
• Scope reads server-side. Tenant and permission filters belong in the tool implementation, never in the prompt. The prompt is a suggestion; the query is the boundary.
• Separate reads from writes. Reads run freely; writes are confirmed, rate-limited and audited.
• Return errors the model can act on. "No purchase order found matching supplier ACME and amount £1,240 — try searching by PO number" recovers. "500 Internal Server Error" sends the agent into a retry loop.

Give It a Way Out

Most bad agent behaviour in production is an agent that had no legitimate move available and improvised one. If the data is missing, the request is out of scope, or confidence is low, the correct action is to stop — and that only happens if stopping is an available, explicitly described action.

Make escalation a first-class tool, name the conditions in the system prompt, and treat “escalated” as a passing outcome in your evaluation suite where it is the right answer. An agent that resolves 60% of cases and cleanly hands over the rest is deployable. One that resolves 85% and silently guesses at the remainder is not.

Instrument the Trajectory

Traditional telemetry tells you the request took 4.2 seconds and returned 200. For agents you need the trajectory: every step, the tool chosen, the arguments, the result, and the tokens consumed.

Log enough that a failed run can be replayed. When a case fails in production, the fix is to add it to the evaluation set, reproduce it, change one thing, and re-run the whole suite. Without a full trajectory log, you get to guess instead.

When You Genuinely Need Breadth

Sometimes the work really does span domains. The answer is composition, not a bigger agent: several narrow agents, each with its own tool surface and evaluation suite, and a thin router in front that classifies the request and dispatches.

The router is a classification problem with a small, closed label set — far easier to evaluate and far easier to fix than an agent picking among forty tools. You also keep the property that matters: when something breaks, it breaks inside one domain, with a test suite already covering it.

Summary

Agent reliability is bought with constraint. Pick one workflow with a definable end state, write the evaluation harness before the agent, expose the smallest tool surface that completes the job, and make escalation a real option. Breadth is what you compose towards later — it is not where you start.