A→Z
A2ZAI
Back to Glossary
techniques

Computer Use

AI capability to control computers by viewing screens and performing mouse/keyboard actions.

Share:

Definition

Computer use is the ability of AI models to interact with computer interfaces like a human would - viewing screens, clicking, typing, and navigating applications.

How It Works: 1. AI receives a screenshot 2. Identifies UI elements 3. Decides on action (click, type, scroll) 4. Executes action 5. Receives new screenshot 6. Repeats until task complete

Capabilities: - Click on buttons and links - Fill in forms - Navigate websites - Use desktop applications - Multi-step workflows

Current State: - Still experimental - Can be slow - May make mistakes - Best for repetitive tasks

Providers: - Anthropic Claude (Computer Use API) - Google Project Mariner - Various agent frameworks

Examples

Claude filling out a web form by taking screenshots and generating mouse clicks and keystrokes.

Related Terms

AI Agent

An autonomous AI system that can plan, take actions, and complete tasks with minimal human intervention.

Tool Use (Function Calling)

The ability of AI models to use external tools, APIs, and functions to extend their capabilities.

Want more AI knowledge?

Get bite-sized AI concepts delivered to your inbox.

Free intelligence briefs. No spam, unsubscribe anytime.

Discussion

Browse all termsTake AI 101 Course