座席桌面

人工智能 社区

技能解读

当前原文的自动摘译,尚未人工审校。

何时使用它

用户想要检查本机 macOS 应用程序或与之交互。该任务需要应用程序启动、窗口焦点、辅助功能快照、语义元素引用、单击、键入、按键或滚动。仅浏览器工作流程是不够的,或者目标不是网页。

要求

必须安装agent-desktop并且其本机二进制文件可用。在 macOS 上,运行 grok 的终端应用程序需要辅助功能权限。来自computer_snapshot的参考仅在下一个快照之前有效。

首选流量

与agent: "computer"委托给task,除非当前代理已经拥有computer_*工具并且任务很小。使用computer_launch、computer_list_windows或computer_focus_window准备好目标应用程序/窗口。从computer_snapshot开始。选择一项基于参考的行动。任何 UI 转换后,在重用引用之前再次拍摄快照。

工具指导

computer_snapshot:主要观察工具;更喜欢interactive_only。 computer_click:尽可能使用最新快照中的ref值。 computer_type:需要目标参考并且优于坐标输入。 computer_press:用于快捷键和特殊键,例如Enter、Tab、cmd+space或cmd+k。 computer_scroll:传递元素引用,而不是原始坐标。

可靠性规则

使用快照 -> 行为 -> 快照循环。更喜欢参考而不是坐标。下一个快照后,引用就会失效;不要盲目重复使用它们。保持操作可逆且低风险,除非用户明确要求进行破坏性操作。如果可访问性树很差或不明确,请使用computer_screenshot进行确认并解释限制。

流狐档案 运行与权限由流狐静态检查推断,不代表作者声明或实际测试结果。
领域
人工智能
作者声明的智能体
未找到明确声明;不据此推断已兼容或已测试
静态检查
88 / 100 · 启发式扫描,不代表运行安全
运行环境证据
未执行;当前没有隔离运行测试证据
作者 / 版本 / 许可证
@superagent-ai · 未声明许可证
上下文用量评级
低消耗
接入复杂程度
需简单配置
是否需要外部接口密钥
不需要
兼容的系统
macOS
运行环境要求
未检出明确要求
检测到的文件与系统行为
  • 只读
  • 命令执行
网络行为
仅限本地
安装命令数
无(仅作为资料)

需要注意: 未声明工具权限范围;实际权限由运行环境决定。

作者说明文件(原文) SKILL.md

# agent-desktop

Use this skill when the task involves the host macOS desktop rather than repository files, shell output, or browser-only verification.

This project uses `agent-desktop` on the host. Prefer the built-in `Computer` sub-agent for these tasks instead of trying to drive the desktop with plain shell commands.

## When to use it

- The user wants to inspect or interact with a native macOS application.
- The task needs app launch, window focus, accessibility snapshots, semantic element refs, clicks, typing, keypresses, or scrolling.
- A browser-only workflow is not enough, or the target is not a web page.

## Requirements

- `agent-desktop` must be installed and its native binary available.
- On macOS, the terminal app running `grok` needs Accessibility permission.
- Refs from `computer_snapshot` are only valid until the next snapshot.

## Preferred flow

1. Delegate to `task` with `agent: "computer"` unless the current agent already has the `computer_*` tools and the task is tiny.
2. Use `computer_launch`, `computer_list_windows`, or `computer_focus_window` to get the target app/window ready.
3. Start with `computer_snapshot`.
4. Pick one ref-based action.
5. After any UI transition, snapshot again before reusing refs.

## Tool guidance

- `computer_snapshot`: primary observation tool; prefer `interactive_only`.
- `computer_click`: use `ref` values from the latest snapshot whenever possible.
- `computer_type`: requires a target ref and is preferred over coordinate typing.
- `computer_press`: use for shortcuts and special keys such as `Enter`, `Tab`, `cmd+space`, or `cmd+k`.
- `computer_scroll`: pass an element ref, not raw coordinates.
- `computer_get`: read text, value, bounds, role, or states from a ref.
- `computer_screenshot`: use for visual confirmation or when the accessibility tree is not enough.

## Reliability rules

- Use a snapshot -> act -> snapshot loop.
- Prefer refs over coordinates.
- Refs go stale after the next snapshot; do not reuse them blindly.
- Keep actions reversible and low risk unless the user explicitly asked for something destructive.
- If the accessibility tree is poor or ambiguous, use `computer_screenshot` for confirmation and explain the limitation.

## Blockers

Stop and report clearly if:

- Accessibility permission is missing.
- `agent-desktop` is unavailable or its native binary is missing.
- The target app/window cannot be found.
- Refs are stale or ambiguous after a UI transition.
- The requested action could be destructive and the user did not explicitly ask for it.

讨论

基于 GitHub Discussions。登录 GitHub 即可参与讨论、点赞、订阅更新。