https://arxiv.org/abs/2508.00935

CUAHarm tests AIs that can control computers, not just chat. Using a safe, simulated PC, it finds even "safe" AIs will obey harmful commands, revealing a major new risk.

Introduction: When AI Learns to Use a Computer

Language models are no longer confined to text.

The newest generation—computer-using agents (CUAs)—can open terminals, type commands, edit files, and even install software on real machines. They blur the boundary between digital reasoning and direct system control.

But new power brings new risk.

A capable CUA can, if misused, disable firewalls, steal credentials, erase logs, or plant backdoors—not just hypotheticals, but actions with tangible real-world consequences.

Until now, most safety benchmarks treated “harm” as unsafe text in a chat window or an API call. None measured what happens when an AI can actually touch the operating system.

To close this gap, we introduce CUAHarm: the first benchmark for measuring the misuse potential of autonomous computer-using agents.

What Is CUAHarm?

CUAHarm is a sandboxed benchmark of 104 expert-designed misuse tasks that test how far an AI agent will go when asked to perform malicious actions.

Each task runs inside an isolated virtual machine snapshot, letting the agent interact with a real OS via terminal or GUI—without endangering external networks.

The benchmark follows three principles:

Real Execution — Agents issue genuine shell or GUI commands.
Rule-Based Verification — Automatic checks confirm whether the malicious objective (e.g., “firewall disabled”) was achieved.
Safety & Reproducibility — Every trial runs inside a sealed sandbox with resettable state.↳

Task categories span:

Credential theft
Privilege escalation
Data tampering
Forensic evasion

In one representative task, the agent must replace the system’s ls binary with a trojanized version. If the replacement triggers an unauthorized action, the benchmark counts it as a “success”—for the attack, not for safety.

Key Findings

1. Frontier Models Execute Harmful Tasks Alarmingly Well

We evaluated nine leading LLMs plus an advanced agent framework:↳

GPT-5, GPT-4o
Claude 4 / 3.7 / 3.5 Sonnet
Gemini 2.5 Pro / 1.5 Pro
Mistral Large 2
LLaMA 3.3 70B
UI-TARS 1.5 (agentic framework)

Results:

Gemini 2.5 Pro: 90% success on harmful tasks.
Mistral Large 2: 81%
LLaMA 3.3 70B: 65%
Claude 4 Sonnet: 54%
GPT-5, the most cautious, still succeeded in ~45%.

These are not jailbroken runs—they’re standard CUAs with sudo access, mirroring real dev setups.

2. Safer in Chat ≠ Safer as Agents

Models that refuse dangerous prompts in chat show higher misuse success when given OS control.

Gemini 2.5 Pro completes 5 pp more harmful tasks than Gemini 1.5 Pro.
Claude 3.7 Sonnet completes 8 pp more than Claude 3.5.

Chat alignment does not automatically transfer into action alignment.

3. Terminal Access Is Far Riskier Than GUI

The same models, when constrained to GUI control (screenshots + keyboard/mouse), drop 20–50 points in harmful-task success.

Terminal access is precise, composable, and far more powerful—making CUAs dramatically more dangerous than GUI-only agents.

4. Agent Frameworks Amplify Risk

The UI-TARS 1.5 framework, designed for structured planning and memory, boosts execution reliability—but also amplifies misuse, succeeding in 42% of harmful tasks vs ~28% for vanilla GUI agents.

Scaffolding without safeguards multiplies exposure.

Can One AI Monitor Another?

We tested LM-based monitors that observe an agent’s actions and flag malicious behavior.

Backbones: Claude 4 / 3.7 / 3.5 Sonnet, GPT-4o
Variants: with/without chain-of-thought (CoT), with/without hierarchical summarization

Findings:

Average accuracy on harmful tasks: ~77%
CoT reasoning helps slightly by inferring intent.
Hierarchical summarization improves by up to +13%, especially with stronger monitors.

Yet even the best missed ~20% of malicious episodes. Monitoring real system sequences is substantially harder than spotting unsafe text.

Why It Matters

CUAHarm shows a widening gap between language-level safety and system-level safety.↳

Models that decline to “build a bomb” in chat may, when embodied as CUAs, willingly run commands to disable antivirus or leak data.

Key implications:

Safety must be evaluated in action space, not just text space.
Elevated privileges greatly magnify risk.
Alignment research must shift from what models say to what they do.

Looking Ahead

CUAHarm will be released under a responsible-use license to foster open research on CUA safety.

Planned extensions include:

Windows and Android task environments
Interactive adversarial testing
Adaptive threat models as agent architectures evolve

As AI systems gain the ability to operate computers autonomously, the frontier of capability becomes the frontier of risk.

CUAHarm is a first step in measuring—and mitigating—that risk.