Agent审计
- 作者仓库星标 0
- 作者更新于 实时读取
- 作者仓库 skills-registry
- 领域
- AI 智能
- 兼容 Agent
-
- Claude Code
- Cursor
- Cline
- Codex
- Windsurf
- Gemini CLI
- +20
- 信任分
- 88 / 100 · 社区维护
- 作者 / 版本 / 许可
- @tomevault-io · 未声明 license
- Token 消耗评级
- 中等消耗
- 接入复杂程度
- 需手动接入
- 是否需要外部 API Key
- 需要 · Vendor-specific
- 兼容的系统
- macOS · Linux · Windows
- 底层运行要求
- Python
- 文件与系统权限
-
- 只读
- 允许写入 / 修改
- Shell 执行
- 读取环境变量
- 网络行为
- 允许外网请求
- 安装命令数
- 26 条
档案由构建时根据 SKILL.md 与安装命令自动衍生,可能与作者实际意图存在差异。
需要注意: 未限定 allowed-tools,默认拥有全部工具权限。
---
name: instrument
description: Add Opik tracing to an existing codebase. Detects language (Python/TypeScript), identifies LLM f…
category: AI 智能
runtime: Python
---
# instrument 输出预览
## PART A: 任务判断
- 适用问题:提示词、Agent 工作流、模型评估或自动化推理。
- 输入要求:目标材料、限制条件、期望输出和验收方式。
- 证据边界:围绕“Step 1 — Scope / Step 2 — Detect Language & Frameworks / Step 3 — Identify the Call Graph”读取原文规则,不把推断写成作者承诺。
## PART B: 执行结果
- **01** 任务判断:确认你的需求是否属于提示词、Agent 工作流、模型评估或自动化推理,并标出输入、限制和预期结果。
- **02** 执行计划:优先按“Step 1 — Scope / Step 2 — Detect Language & Frameworks / Step 3 — Identify the Call Graph”拆成步骤,说明每一步会读取什么、修改什么、产出什么。
- **03** 交付结果:给出可复制的命令、文件改动、检查清单或内容草稿,并说明如何继续迭代。
- **04** 风险边界:结合 读取文件、写入/修改文件、执行终端命令、读取环境变量、会按任务需要访问外部网络、需要准备 Vendor-specific API Key 给出执行前确认项。
## Running Rules
- 读取文件、写入/修改文件、执行终端命令、读取环境变量;会按任务需要访问外部网络;需要准备 Vendor-specific API Key。
- 先小样例验证,再放大到真实任务。
- 交付时同时给结果、检查口径和下一步迭代建议。 原文出现了 `/opik`、`/api` 这类斜杠命令;如果你的 Agent 支持命令触发,优先用命令开场,再补充目标和边界。
告诉 Agent 目标文件或材料、期望结果、不可改范围、是否允许联网或执行命令。本 Skill 的权限画像是:读取文件、写入/修改文件、执行终端命令、读取环境变量。
先用一个小任务确认它会围绕“Step 1 — Scope / Step 2 — Detect Language & Frameworks / Step 3 — Identify the Call Graph”工作;涉及文件或命令时,先看 diff、日志、预览或测试结果。
检查最终产物是否包含明确结果、必要证据和下一步动作;如果输出泛泛而谈,就补充输入、边界和验收标准后重跑。
---
name: instrument
description: Add Opik tracing to an existing codebase. Detects language (Python/TypeScript), identifies LLM f…
category: AI 智能
source: tomevault-io/skills-registry
---
# instrument
## 什么时候使用
- 把 AI / Agent方向的常用动作沉淀成 Agent 可调用的技能 适合处理AI Agent、提示词、模型评估与自动化推理,核心价值是把输入、判断、执行、验证和交付边界固定下来,避免 Agent 泛泛回答。 把任务拆成可执行、可检查…
- 面向提示词、Agent 工作流、模型评估或自动化推理,优先处理能明确输入、步骤和验收标准的工作。
## 需要提供什么
- 目标材料、目录范围、期望结果和不可改动内容。
- 是否允许联网、执行命令、读写文件或调用外部服务。
## 执行规则
- 围绕「Step 1 — Scope / Step 2 — Detect Language & Frameworks / Step 3 — Identify the Call Graph」组织步骤,不把推断写成作者事实。
- 读取文件、写入/修改文件、执行终端命令、读取环境变量;会按任务需要访问外部网络;需要准备 Vendor-specific API Key。
- 先跑小样例,确认结果可检查后再扩大任务范围。
## 输出要求
- 给出最终产物、关键证据、验证方式和下一步动作。
- 信息不足时标记 unknown,不编造命令、平台或依赖。 作者原文负责流程事实;仓库文件负责来源和命令;流狐只补充适用场景、限制和质量判断。
skill "instrument" {
输入层 -> 用户目标 + 目标文件 + 禁止范围 + 验收标准
上下文层 -> Step 1 — Scope / Step 2 — Detect Language & Frameworks / Step 3 — Identify the Call Graph
规则层 -> SKILL.md 触发条件 / 执行顺序 / 输出格式
运行层 -> Python | 读取文件、写入/修改文件、执行终端命令、读取环境变量 | 会按任务需要访问外部网络
安全层 -> 需要准备 Vendor-specific API Key + 小任务验证 + diff / 日志复核
输出层 -> 可复制结果 + 检查清单 + 下一步迭代
} Instrument — Add Opik Tracing to a Codebase
You are instrumenting an existing codebase with Opik observability. Follow these steps precisely.
Step 1 — Scope
If $ARGUMENTS is provided, scope your work to those files or directories. Otherwise, discover the project root and instrument the main application code.
Step 2 — Detect Language & Frameworks
Scan the codebase to determine:
- Language: Python (look for
*.py,pyproject.toml,requirements.txt) or TypeScript (look for*.ts,*.tsx,package.json) - LLM frameworks in use — search imports for these patterns:
| Import pattern | Framework | Integration |
|---|---|---|
from openai / import OpenAI |
OpenAI | track_openai |
import anthropic |
Anthropic | track_anthropic |
from langchain / @langchain |
LangChain | OpikTracer callback |
from langgraph |
LangGraph | OpikTracer with graph= |
from crewai |
CrewAI | track_crewai |
import dspy |
DSPy | OpikCallback |
from google … genai |
Google Gemini | track_genai |
import boto3 … bedrock |
AWS Bedrock | track_bedrock |
from llama_index |
LlamaIndex | LlamaIndexCallbackHandler |
import litellm |
LiteLLM | OpikLogger callback |
from pydantic_ai |
Pydantic AI | Logfire OTLP bridge |
from opik.integrations.adk / from google.adk |
Google ADK | track_adk_agent_recursive |
import ollama |
Ollama | track_openai with localhost base_url or manual @opik.track |
from agents import / from openai.agents |
OpenAI Agents SDK | OpikTracingProcessor |
from haystack |
Haystack | OpikConnector |
opik-openai / trackOpenAI (TS) |
OpenAI (TS) | trackOpenAI |
opik-vercel / OpikExporter (TS) |
Vercel AI SDK | OpikExporter |
opik-langchain / OpikCallbackHandler (TS) |
LangChain.js | OpikCallbackHandler |
opik-gemini / trackGemini (TS) |
Gemini (TS) | trackGemini |
- Existing Opik usage — check if
opikor@opik.trackis already imported. If so, audit rather than re-instrument.
Step 3 — Identify the Call Graph
Find:
- Entrypoint: the top-level function that kicks off the agent (e.g.,
main,run,agent,handle_message, a route handler, or whatever the user's main orchestration function is) - LLM call sites: functions that call an LLM provider directly
- Tool functions: retrieval, search, API calls, or other tool-like operations
- Existing config classes: dataclasses, Pydantic models, or plain classes holding model names, temperatures, prompts, or other tunable parameters
Entrypoint Parameter Rules
The function marked with entrypoint=True must only accept primitive-typed parameters: str, int, float, bool, and list/dict of primitives. This is because:
- Opik reads the function's type hints to build an input form in the UI
- Users will type these values manually in a text field via the Local Runner
- Complex types (Pydantic models, dataclasses, request objects, custom classes) cannot be entered in a UI input field
If the candidate entrypoint accepts complex types (e.g., a request model, a config object, a dataclass):
- Look higher in the call chain for a function that already accepts primitives
- If none exists, create a thin wrapper function that accepts only primitives, unpacks them, and calls the original function. Move the
entrypoint=Truedecorator to this wrapper.
Example — bad entrypoint (complex parameter):
# ❌ DO NOT mark this as entrypoint — RecommendRequest is a Pydantic model
@app.post("/recommend")
async def recommend(request: RecommendRequest):
summary, tool_results = await run_agent(user_message=build_user_message(request))
return RecommendResponse(city=request.city, recommendations=_extract_recommendations(tool_results), summary=summary)
Example — good entrypoint (primitives only):
@opik.track(name="recommend-agent", entrypoint=True)
async def _run_entrypoint(user_message: str) -> tuple[str, list[dict]]:
"""Opik entrypoint — receives only the user message for Local Runner schema."""
return await run_agent(user_message=user_message)
@app.post("/recommend")
async def recommend(request: RecommendRequest):
summary, tool_results = await _run_entrypoint(user_message=build_user_message(request))
return RecommendResponse(city=request.city, recommendations=_extract_recommendations(tool_results), summary=summary)
The wrapper extracts the primitive values from the complex object and delegates to the existing logic. The HTTP handler calls the wrapper instead of the inner function directly, so the trace captures the full execution.
Step 4 — Add Framework Integrations
For each detected framework, add the appropriate integration at the module level. See the integration table above and references/integrations.md for the exact patterns.
Python examples:
# OpenAI
from opik.integrations.openai import track_openai
client = track_openai(OpenAI()) # wrap existing client
# Anthropic
from opik.integrations.anthropic import track_anthropic
client = track_anthropic(anthropic.Anthropic())
# LangChain / LangGraph
from opik.integrations.langchain import OpikTracer
tracer = OpikTracer()
# pass config={"callbacks": [tracer]} to invoke()
# LiteLLM inside @opik.track — CRITICAL: pass span context
from opik.opik_context import get_current_span_data
# in every litellm.completion() call, add:
# metadata={"opik": {"current_span_data": get_current_span_data()}}
TypeScript examples:
// OpenAI
import { trackOpenAI } from "opik-openai";
const trackedClient = trackOpenAI(openai);
// Vercel AI SDK
import { OpikExporter } from "opik-vercel";
// set up NodeSDK with OpikExporter
Step 5 — Add @opik.track Decorators (Python) or Client Tracing (TypeScript)
Python
Add import opik at the top of each file you instrument.
| Function role | Decorator |
|---|---|
| Entrypoint (top-level agent) | @opik.track(entrypoint=True, name="<agent-name>") |
| LLM call | @opik.track(type="llm") |
| Tool / retrieval | @opik.track(type="tool") |
| Guardrail / validation | @opik.track(type="guardrail") |
| Other helper in the call chain | @opik.track |
- Entrypoint parameters must be primitives only (
str,int,float,bool,list,dict). If the natural entrypoint takes a complex type, create a wrapper — see Step 3 "Entrypoint Parameter Rules". - Config access must happen inside
@opik.track: Any call toclient.get_or_create_config()and subsequent access of config fields must occur inside a@opik.track-decorated function, or in a function called downstream from one. This is how Opik injects config metadata into the current trace. Calling it at module level or outside the traced call stack will raise an error. - Place the decorator above any existing decorators (e.g., above
@app.route) - For async functions,
@opik.trackworks the same way — no changes needed - If the function is a script entrypoint (not a long-running server), add
opik.flush_tracker()after the top-level call
TypeScript
Use the client-based approach:
import { Opik } from "opik";
const client = new Opik({ projectName: "<project-name>" });
// In the entrypoint function:
const trace = client.trace({ name: "<agent-name>", input: { ... } });
const span = trace.span({ name: "<operation>", type: "tool", input: { ... } });
// ... logic
span.end({ output: { ... } });
trace.end({ output: { ... } });
await client.flush();
For entrypoints that should be discoverable by opik connect — note that params must only use primitive types (string, number, boolean) since users enter these values in a UI text field:
import { track } from "opik";
const myAgent = track(
{ name: "<agent-name>", entrypoint: true, params: [{ name: "query", type: "string" }] },
async (query: string) => { /* ... */ }
);
Step 6 — Conversational Agents: Add thread_id
If the agent handles multi-turn conversations (chat bots, support agents, multi-step assistants), wire thread_id:
@opik.track(entrypoint=True)
def handle_message(session_id: str, message: str) -> str:
opik.update_current_trace(thread_id=session_id)
return generate_response(session_id, message)
Skip this for single-shot agents or batch processing.
Step 7 — Environment Config
Follow the setup decision tree from the main opik skill:
- If the project has
.env/.env.local→ appendOPIK_API_KEY,OPIK_WORKSPACE,OPIK_URL_OVERRIDE(if missing) - If no
.envexists → Python: create/update~/.opik.config; TypeScript: create.envor.env.local - Never introduce a second config mechanism
- Never overwrite existing values
- Update
.env.example/.env.sampleif one exists - Set
project_namein code, not in env files
OPIK_URL_OVERRIDE path rules
The URL suffix depends on where Opik is hosted:
| Deployment | URL format | Example |
|---|---|---|
| Opik Cloud / managed | <base>/opik/api |
https://www.comet.com/opik/api |
| Self-hosted (local) | <base>/api |
http://localhost:5173/api |
- Cloud/managed: always append
/opik/api - Self-hosted (typically
localhostor an internal hostname): append only/api— no/opikprefix - When writing or suggesting an
OPIK_URL_OVERRIDEvalue, apply this rule so users don't have to remember it
Step 8 — Install Dependencies
Print the install command but do NOT run it automatically. Let the user decide.
Python:
pip install opik
Plus any integration packages if needed (most are included in opik).
TypeScript:
npm install opik
Plus framework-specific packages: opik-openai, opik-vercel, opik-langchain, opik-gemini as needed.
Step 9 — Verify
After instrumentation, do a quick audit:
- Every LLM call site is traced (via integration wrapper or
@opik.track) - Exactly one function has
entrypoint=True - The entrypoint function accepts only primitive parameters (
str,int,float,bool,list,dict) — no Pydantic models, dataclasses, or custom classes - All
get_or_create_config()calls and config field access happen inside@opik.track-decorated functions (or downstream from one) - Script entrypoints call
opik.flush_tracker()(Python) orawait client.flush()(TypeScript) - LiteLLM calls inside
@opik.trackpasscurrent_span_datavia metadata - No hardcoded API keys were introduced
- Existing tests still import correctly (no circular imports introduced)
Anti-Patterns to Avoid
- Double-wrapping: Don't add
@opik.track(type="llm")to a function that already uses a framework integration (e.g.,track_openai). The integration handles tracing. - Orphaned LiteLLM traces: Always pass
current_span_datawhenOpikLoggeris used inside@opik.trackcode. - Complex entrypoint parameters: The entrypoint function must only accept primitives (
str,int,float,bool,list,dict). Pydantic models, dataclasses, or custom classes can't be typed into a UI input field. If the natural entrypoint takes a complex type, create a thin wrapper that accepts primitives. - Config access outside
@opik.track:get_or_create_config()and config field reads must happen inside a@opik.track-decorated function or downstream from one. Module-level or untraced calls will fail and won't attach config metadata to the trace. - Missing entrypoint: Without
entrypoint=True, Local Runner (opik connect) won't discover the agent. - Missing flush: Scripts that exit without flushing lose trace data.
- Overwriting config: Check before writing to
.envor~/.opik.config.
References
For detailed API signatures and advanced patterns, see:
../opik/references/tracing-python.md— Python SDK reference../opik/references/tracing-typescript.md— TypeScript SDK reference../opik/references/integrations.md— All framework integrations../opik/references/observability.md— Core concepts (traces, spans, threads)
Source: comet-ml/opik-skills — distributed by TomeVault.
先判断是否适合
作者设计意图
作者的方法与取舍
边界和复核