Where browser agents actually work, and where they don't
By Anwar Benhamada · August 6, 2026
Browser agents — tools that drive a real browser to complete multi-step tasks — demo better than almost any category in AI. They also have the widest gap between demo and production, and the gap is predictable enough to plan around.
Why the demos work
Demo tasks share properties that real tasks usually don’t:
- The page is clean. No cookie banner, no chat widget, no onboarding tooltip.
- The path is short. Three or four steps.
- The site is popular, so it’s well represented in training data.
- Success is visually obvious. A confirmation page the agent can recognise.
- It was run more than once before you saw it.
Remove any one of these and reliability drops sharply.
The four things that actually break agents
1. Unexpected interstitials. A cookie banner, an A/B-tested modal, a “are you still there?” prompt. Humans dismiss these without noticing. An agent either fails or — worse — clicks something it shouldn’t to get past it.
2. Compounding error rates. This is the mathematical problem and it’s brutal. At 95% reliability per step:
| Steps | Success rate |
|---|---|
| 3 | 86% |
| 5 | 77% |
| 10 | 60% |
| 20 | 36% |
A twenty-step task fails about two times in three even with individually good steps. Long agent tasks are unreliable for arithmetic reasons, not because the model is bad.
3. Silent success. The agent reports completion and nothing happened. This is the expensive failure — it doesn’t error, so nothing alerts, and you find out when someone asks where the data is.
4. Anti-automation defences. Many sites actively detect and block automated browsing. Fighting that is a losing, escalating game, and on sites you don’t own it’s frequently against their terms.
Where they genuinely work
The pattern is consistent — short, verifiable, on pages you control or that are stable:
- Extracting structured data from a page you can check afterwards
- Filling a form on your own internal tool
- Repetitive tasks on stable, unglamorous B2B interfaces
- Testing your own application — the outcome is defined and you own the page
- Anything where you verify the output before acting on it
Where they don’t: long chains, anything transactional you won’t check, anything on a site actively changing or defending against automation.
Design for the failure rate
If you use them, assume they fail and build accordingly:
Split long tasks into short ones with checkpoints. Four five-step tasks with verification between beats one twenty-step task by an enormous margin — 77% per segment with retries dramatically outperforms 36% end-to-end.
Assert the outcome, don’t trust the report. Never accept “task completed”. Check the thing that should have changed actually changed. This is the single highest-value line of code in any agent workflow.
Make it idempotent. You’ll re-run failures. Re-running must not duplicate.
Log the trace. Screenshots or a DOM trail at each step. Without it, “it failed sometimes” is undebuggable.
Cap retries and escalate to a human. Some tasks won’t work today. Silent infinite retry burns quota and hides the problem.
How to evaluate one honestly
Take your tasks, not the demo’s:
- Write ten real tasks — three easy, five typical, two you expect to fail
- Run each three times (they’re non-deterministic; one run tells you nothing)
- Count: completed, failed loudly, failed silently
- Measure how long verification takes you
That third bucket is the number that decides it. A tool with a 70% success rate that fails loudly is usable. A tool with a 90% success rate where half the failures are silent is not — because you now have to check everything, which was the work you were trying to avoid.
The honest summary
Browser agents are genuinely useful for short, checkable tasks on stable pages. They’re not yet reliable for long unattended chains, and the arithmetic above suggests that’s structural rather than a temporary limitation.
Buy accordingly: for the five-step task you verify, not the twenty-step one you hope works overnight.