AI Agent Performance Metrics: What to Track in 90 Days

The dashboard says the agent finished the work. It does not say the work was right, or that it was worth what you are paying. An agent can complete every task you assigned and still be wrong often enough to create a second job for the person who checks it. Performance, for an operations team, is three layers, and most teams only measure the first.

Task completion is whether the agent finished. That number comes from its own logs, and a 95% completion rate can sit next to a 30% error rate if nobody reads the output. Quality is whether the answer was the right one: a resolution that matches policy, a report that matches the source, a booking that was actually open. Business impact is whether revenue or cost moved. IBM's June 2025 Institute for Business Value study of 2,900 executives found that 83% expect agents to improve process efficiency by 2026. The same study put AI-enabled workflows at 3% today, with a projection of 25% by the end of 2025. Expectation is not a measurement. The weekly owner report is where these layers should land so a finance conversation does not depend on the vendor dashboard.

What does agent performance mean for an operations team?

In the first 30 days, completion is enough to see whether the agent is running. From day 31 to day 60, quality is the leading indicator of what will go wrong if you ignore it. From day 61 to day 90, impact is the number a finance lead and a customer will eventually ask for. MIT Sloan Management Review and BCG (November 2025) found 35% of organizations deploying agentic AI, 44% planning to soon, and 76% treating it more like a coworker than a tool. A coworker still needs a scorecard.

Each layer answers a different month of the deployment.

Completion in the first month tells you the agent is running and where it stops. It does not tell you the finished work was right, and a team that celebrates a high completion rate in week two often discovers the corrections in week six. Quality, from day 31 to day 60, is the early warning. If a sample of finished tasks is wrong, those errors become the complaints, the reopened tickets, and the reports a customer sends back. Impact, from day 61 to day 90, is the only layer that answers the question a finance lead will ask, which is whether the time or the money moved. MIT Sloan Management Review and BCG found leaders already describing these systems as coworkers. A coworker is not measured by whether they stayed busy. They are measured by whether the work they handed back was usable, and then by whether the week got shorter.

Which metrics tell you the agent is doing the job?

Four metrics cover an operations deployment. The targets below are an operating frame, not a client result.

Task success rate asks whether the agent achieved the outcome, not whether it ran. Treat 85% as production readiness. Below 80%, revise the instructions before you add volume.

Escalation rate is the share the agent could not resolve and passed to a person. A rate of 20 to 30% in the first 30 days is normal, because the agent is meeting requests outside its job. It should fall below 10% by day 60 as the instructions get tighter.

Time to complete includes retries and tool calls. Compare it to the human baseline you measured before launch. If the agent is consistently slower than a trained person on a simple task, the workflow is misconfigured.

Error rate is the share of finished tasks with a factual mistake, a policy break, or a made-up detail. It requires a person to read a sample. Review 10% in the first 30 days, then 5% by day 90 as the patterns settle. Hold the error rate under 5% before you scale volume.

Define success for each task before the first real one runs.

Before the first real task, define success for each task type. A booking succeeds when the slot is confirmed and the customer gets the confirmation. A support task succeeds when the ticket stays closed for 48 hours. A report succeeds when the figures match the source and a person does not have to rebuild the file. Without that definition you have nothing to score against.

Write the definition as something a reviewer can mark yes or no on a single task. "The customer was helped" is not a definition, because two reviewers will disagree. "The slot on the confirmation matches an open slot, and the customer received that confirmation" is a definition. "The ticket did not reopen for 48 hours" is a definition. "Every total in the report matches the source export" is a definition. The 85% target, the 80% pause, and the error-rate sample all depend on that yes or no. If you skip it, the dashboard will keep reporting completion, and the team will keep arguing about whether the work was good.

How do you set a baseline before the agent goes live?

A baseline is the human team's performance on the same tasks. Without it, an improvement is a story. Spend two to four weeks recording, for each task type, the average minutes, the error rate from a 10% spot check, and how often a second person had to take the task. After launch, measure the same three. The difference is what the agent contributed.

• Average minutes per task, taken from the people who do the work today, including the time they spend finding the source and fixing their own mistakes.
• Error rate from a 10% spot check of finished work, scored against the same definition you will use for the agent.
• How often a second person had to take the task, which is the human version of an escalation.

These three numbers are the before-state. After launch you measure the same three on the same task types, and the gap is what you can attribute to the agent. A before-state that lives in someone's memory will be rewritten the moment the new numbers look good or bad. Write it down before the agent touches a live task.

When you have no history, run two weeks in parallel.

If you have no logs, run two weeks in parallel. The team handles the tasks and writes down time, errors, and handoffs. The agent runs the same tasks, and its output does not reach a customer yet. That gives you a before and an after before the switch. The structure of that period is in how to test an agent. Store the before-state in a shared sheet: task name, human minutes, error rate, escalation frequency. An agent with no before-state has nothing to say to the person who pays for it.

The sheet is the comparison point for every review in the first 90 days. One row per task type is enough. Columns are the task name, the human minutes, the error rate from the spot check, and how often a second person took over. Update it when you learn that the old timing was wrong, and date the update. Do not let the agent go live on a task that has no row. When someone asks what changed, you open the sheet. You do not reconstruct the old week from memory.

What does a 90-day measurement cycle look like?

Days 1 to 30: track success rate and escalation daily, and read the logs weekly. The goal is the top three failure categories by volume. At day 30, pause scaling if success is below 80%. If escalation is above 30%, either the job is too wide or the handoff is missing.

Days 31 to 60: add quality. Review a 10% sample for accuracy and policy. Meet weekly with the person responsible for each task type. The goal is an error rate under 5% before you add volume.

Days 61 to 90: add impact. Cost per task before and after, tasks per hour against the human baseline, and the hours the team actually got back. Support triage is a concrete place to see what 90-day numbers look like.

The day-30 review decides whether you add volume.

At the first monthly review, answer five questions before you continue. The answers decide whether you add volume, revise the instructions, or pause.

1. What is the success rate, and is it below 80%, which means you stop adding volume and revise the instructions before the next week of traffic?
2. What are the top three failure categories by volume, and which of them is a missing instruction rather than a one-off the agent should keep escalating?
3. Is escalation falling week to week, or is it stuck above 30%, which means the job is too wide or the handoff was never written down?
4. Did any customer-facing error need a person to correct it, and how long did that correction take compared with doing the task by hand?
5. What is the revised 90-day goal for each metric, written down before the next month starts, so the day-60 review has something to beat?

A day-30 review that only reads the vendor chart will add volume on a completion rate. The five questions force the pause. Below 80% success, more volume multiplies the failures you have not named yet. Above 30% escalation that is not falling, you either narrow what the agent is allowed to finish or you write the handoff the current path is missing. Both are decisions. Leaving the rate where it is and calling it learning is how the second month repeats the first.

Why do standard benchmarks mislead an operations team?

Most benchmarks test short tasks with a clear right answer. Operations work is long, ambiguous, and chained, and success is harder to define. Kapoor, Narayanan, and colleagues (arXiv:2407.01502, July 2024) named four shortcomings: benchmarks obsess over accuracy and ignore cost, they mix up what a model lab needs with what a team deploying the model needs, they lack a proper holdout set, and the methods are too inconsistent to compare. A high score does not predict your tasks.

Those four shortcomings show up in a buying conversation. A score that ignores cost can crown an agent that is accurate and too expensive to run on your volume. A score built for a lab answers whether a model improved, not whether your booking rules or your report layout will survive a Tuesday. A missing holdout set means the agent was tuned on the same examples used to grade it, so the grade flatters the demo. Inconsistent methods mean two vendors can both claim a win and you still cannot tell which one would finish your tasks. None of that is a reason to ignore published research. It is a reason to refuse a leaderboard as a substitute for your own test set.

UI-CUBE (Cristescu et al., arXiv:2511.17131, 21 November 2025) measured the gap on 226 tasks. On simple, single-element screen actions, agents reached 67 to 85% success against 97.9% for people. On complex multi-step workflows, the same agents fell to 9 to 19%, against 61.2% for people. The researchers described a sharp drop, not a gentle slope. Vendors demo the simple tasks. Your team lives in the complex ones. Replace the leaderboard with a test set built from your own history, scored against your own definition of success, and refresh it each quarter as the workflow changes.

Score a test set from your own history, then refresh it.

Build the test set from tasks the team already did, not from a public exam. Pull a few dozen finished examples, hide the outcome, and have the agent attempt them. Score each attempt against the definition you wrote before launch. Keep a portion of the examples out of any instruction tuning so you are not grading the agent on cases it was shown. When the workflow changes, a new form field, a new policy, a new source file, replace the stale examples in the same quarter. A test set from last year will keep giving you a comforting score while the live work drifts. The UI-CUBE gap is the warning: simple actions can look fine while the multi-step work your team actually depends on falls off a cliff. Your test set should include the multi-step work, or it will repeat the demo.

How do you turn the metrics into a business case?

Three numbers: the cost of the tasks before the agent, the cost with the agent, and the difference. An illustrative example, not a client result. A 10-person operations team spends 45 minutes on a weekly client report. The agent does the same report in 12 minutes, with a 92% success rate and a 4% error rate, so 4 of every 100 reports need a person. The team gets 33 minutes back per report. At 50 reports a week, that is 27.5 hours. At $35 an hour, the time is worth $963 a week. If the agent's inference and maintenance cost $180 a week, the net is $783 a week. Those are round figures for the arithmetic, not a result from a customer.

Finance hears recovered time, cost per task, or throughput.

The number that survives a review is recovered time, cost per task, or throughput. A 92% success rate does not mean much to a finance lead. Twenty-seven hours a week does. Convert the operational metrics into one of those three before the meeting.

Walk the arithmetic in the order a skeptical reader will check it. Start from the human minutes you wrote down before launch, subtract the agent minutes, and multiply by the number of tasks in a normal week. Price the hours at the blended rate you actually pay, and subtract what the agent costs to run and to maintain that week. The $35, the $963, and the $180 above are round figures so you can see the shape. Swap in your baseline minutes, your volume, and your rate, and label the result as your model. If you cannot name the before-state minutes, you do not have the business case yet. You have a success rate and a hope that someone else will translate it.

What do you need in order to track this?

A log, a sample review, and a weekly sheet. The log is the agent's own output. Export it weekly with task type, success or escalation, time, and a flag for any human correction. That flag is the numerator of the error rate. The review is a weekly block where one person reads 10% of finished tasks against your definition of success. It has to happen every week for 90 days or the error rate is invisible. The sheet has four rows: success rate, escalation rate, error rate, and time to complete. Add cost per task once you have 30 days. Note any move larger than 5 points against the baseline.

The measurement stack is three checkpoints and about two hours a week.

Week one: export daily and score success and escalation. Week four: add the 10% sample. Week twelve: add cost per task and throughput. Hold the scope to completion, quality, and the business number. That is the stack, using the agent's logs and about two hours of review a week. You do not need a separate product to start. You need the export, a person who will actually read the sample, and a sheet that still has the baseline on it when month three arrives. If the weekly read slips, the error rate goes invisible and the only number left is completion, which is the number that misled you at the start.

If you want the tasks and the baselines named for your team before you build the sheet, get your free Systems Plan.

How we know this.

IBM Institute for Business Value, June 2025 (n=2,900 executives, 18 industries, 19 regions), is the source for the 83% efficiency expectation and the 3% to 25% workflow projection. MIT Sloan Management Review and BCG, November 2025, is the source for 35%, 44%, and 76%. Kapoor, Narayanan, et al. (arXiv:2407.01502, July 2024, Princeton and NYU) document the four benchmark shortcomings. UI-CUBE (Cristescu et al., arXiv:2511.17131, November 2025, 226 tasks) is the source for the simple-task rates of 67 to 85% against 97.9% for people, and the complex-task rates of 9 to 19% against 61.2% for people. The 85%, 80%, 20 to 30%, and 10% thresholds, and the $35, $963, and $180 example, are operating frames and illustrative arithmetic, not client data.