agent-desktop

AI Community

Repo stars 3,082
Author repo grok-cli

Interpretation is structured for decision-making; original keeps the upstream SKILL.md unchanged.

Decide Fit First

Core job: Use the built-in Computer sub-agent with agent-desktop for macOS desktop automation. Apply when a task needs application lau…
Best fit: Use it when the task has reusable inputs, steps, and validation criteria rather than a one-off answer.
Avoid forcing it: If the source lacks commands, platform support, or external-service evidence, keep those fields unknown instead of guessing.

Design Intent

Structure: The skill is organized around “When to use it”, “Requirements”, “Preferred flow”, “Tool guidance”, showing how the author expects the agent to judge fit, collect context, and produce verifiable output.
Trigger evidence: Prioritize the author’s wording around when to use it, what context to collect, and what output shape to produce.
Evidence boundary: Author text states facts, repository files prove commands and paths, and Fluxly only adds fit, limits, and usage judgment.

How To Use It

Inputs: Provide target material, scope, expected result, forbidden changes, and validation method.
Invocation: Name agent-desktop directly; if the source includes slash commands, start with the command and then add task context.
Validation: Start small and check whether the result follows “When to use it / Requirements / Preferred flow” before expanding.

Boundaries And Review

Dependencies: It usually needs no extra API key, so start with a small validation task.
Permissions: Declared permissions include read / shell-exec; ask the agent to state file, command, and rollback boundaries before acting.
Quality bar: A useful result names the deliverable, evidence, and next action. Generic prose means the task needs tighter context.

agent-desktop

Use this skill when the task involves the host macOS desktop rather than repository files, shell output, or browser-only verification.

This project uses agent-desktop on the host. Prefer the built-in Computer sub-agent for these tasks instead of trying to drive the desktop with plain shell commands.

When to use it

The user wants to inspect or interact with a native macOS application.
The task needs app launch, window focus, accessibility snapshots, semantic element refs, clicks, typing, keypresses, or scrolling.
A browser-only workflow is not enough, or the target is not a web page.

Requirements

agent-desktop must be installed and its native binary available.
On macOS, the terminal app running grok needs Accessibility permission.
Refs from computer_snapshot are only valid until the next snapshot.

Preferred flow

Delegate to task with agent: "computer" unless the current agent already has the computer_* tools and the task is tiny.
Use computer_launch, computer_list_windows, or computer_focus_window to get the target app/window ready.
Start with computer_snapshot.
Pick one ref-based action.
After any UI transition, snapshot again before reusing refs.

Tool guidance

computer_snapshot: primary observation tool; prefer interactive_only.
computer_click: use ref values from the latest snapshot whenever possible.
computer_type: requires a target ref and is preferred over coordinate typing.
computer_press: use for shortcuts and special keys such as Enter, Tab, cmd+space, or cmd+k.
computer_scroll: pass an element ref, not raw coordinates.
computer_get: read text, value, bounds, role, or states from a ref.
computer_screenshot: use for visual confirmation or when the accessibility tree is not enough.

Reliability rules

Use a snapshot -> act -> snapshot loop.
Prefer refs over coordinates.
Refs go stale after the next snapshot; do not reuse them blindly.
Keep actions reversible and low risk unless the user explicitly asked for something destructive.
If the accessibility tree is poor or ambiguous, use computer_screenshot for confirmation and explain the limitation.

Blockers

Stop and report clearly if:

Accessibility permission is missing.
agent-desktop is unavailable or its native binary is missing.
The target app/window cannot be found.
Refs are stale or ambiguous after a UI transition.
The requested action could be destructive and the user did not explicitly ask for it.

Fluxly profile Author and license come from source; runtime, permissions, and network are Fluxly detections or estimates

Fluxly category: AI
Author-declared agents: No explicit declaration found; this is not inferred or tested compatibility
Static check: 88 / 100 · heuristic scan, not runtime safety proof
Author / version / license: @superagent-ai · no license declared
Fluxly token estimate: Lean
Fluxly setup estimate: Guided setup
External API key: No requirement detected
Detected OS requirements: macOS
Runtime requirements: Unspecified
Detected file/system behavior: Read-only
Shell exec
Detected network behavior: Local-only
Install commands: None (reference only)

Profile is derived at build time from SKILL.md and install vectors. Subject to drift from author intent.

Heads up: 未限定 allowed-tools，默认拥有全部工具权限。

Output preview agent-desktop.preview

The current SKILL.md does not define a fixed output example.

SKILL.md SKILL.md

# agent-desktop

Use this skill when the task involves the host macOS desktop rather than repository files, shell output, or browser-only verification.

This project uses `agent-desktop` on the host. Prefer the built-in `Computer` sub-agent for these tasks instead of trying to drive the desktop with plain shell commands.

## When to use it

- The user wants to inspect or interact with a native macOS application.
- The task needs app launch, window focus, accessibility snapshots, semantic element refs, clicks, typing, keypresses, or scrolling.
- A browser-only workflow is not enough, or the target is not a web page.

## Requirements

- `agent-desktop` must be installed and its native binary available.
- On macOS, the terminal app running `grok` needs Accessibility permission.
- Refs from `computer_snapshot` are only valid until the next snapshot.

## Preferred flow

1. Delegate to `task` with `agent: "computer"` unless the current agent already has the `computer_*` tools and the task is tiny.
2. Use `computer_launch`, `computer_list_windows`, or `computer_focus_window` to get the target app/window ready.
3. Start with `computer_snapshot`.
4. Pick one ref-based action.
5. After any UI transition, snapshot again before reusing refs.

## Tool guidance

- `computer_snapshot`: primary observation tool; prefer `interactive_only`.
- `computer_click`: use `ref` values from the latest snapshot whenever possible.
- `computer_type`: requires a target ref and is preferred over coordinate typing.
- `computer_press`: use for shortcuts and special keys such as `Enter`, `Tab`, `cmd+space`, or `cmd+k`.
- `computer_scroll`: pass an element ref, not raw coordinates.
- `computer_get`: read text, value, bounds, role, or states from a ref.

…

Evidence boundary and execution chain

Author text anchors workflow facts; Fluxly only indexes current sections, terms, files, and commands.

sections -> When to use it → Requirements → Preferred flow → Tool guidance → Reliability rules → Blockers
terms    -> Use this skill when the task involves the host macOS desktop rather than repository files, shell output, or browser-only verification. · This project uses agent-desktop on the host. · - The user wants to inspect or interact with a native macOS application. · - agent-desktop must be installed and its native binary available. · 1. Delegate to task with agent: "computer" unless the current agent already has the computer tools and the task is tiny. · - computersnapshot: primary observation tool; prefer interactiveonly. · - Use a snapshot -> act -> snapshot loop. · - Accessibility permission is missing.
files/cmd -> agent-desktop · Computer · grok · computersnapshot · task · agent: "computer" · computer · computerlaunch
body sha256 -> 7d473211e896

Decide Fit First

Design Intent

How To Use It

Boundaries And Review

agent-desktop

When to use it

Requirements

Preferred flow

Tool guidance

Reliability rules

Blockers

Discussion