代理会话监视器
- 作者仓库星标 8,455
- 作者仓库 higress
- 领域
- 人工智能
- 作者声明的智能体
- 未找到明确声明;不据此推断已兼容或已测试
- 静态检查
- 88 / 100 · 启发式扫描,不代表运行安全
- 运行环境证据
- 未执行;当前没有隔离运行测试证据
- 作者 / 版本 / 许可证
- @higress-group · 未声明许可证
- 上下文用量评级
- 低消耗
- 接入复杂程度
- 需简单配置
- 是否需要外部接口密钥
- 不需要
- 兼容的系统
- 未声明(默认跨平台)
- 运行环境要求
- 未检出明确要求
- 检测到的文件与系统行为
-
- 只读
- 允许写入 / 修改
- 网络行为
- 仅限本地
- 安装命令数
- 无(仅作为资料)
需要注意: 未声明工具权限范围;实际权限由运行环境决定。
## Overview
Real-time monitoring of Higress access logs, extracting ai_log JSON, grouping multi-turn conversations by session_id, and calculating token costs with visualization.
### Core Features
- **Real-time Log Monitoring**: Monitors Higress access log files, parses new ai_log entries in real-time
- **Log Rotation Support**: Full logrotate support, automatically tracks access.log.1~5 etc.
- **Incremental Parsing**: Inode-based tracking, processes only new content, no duplicates
- **Session Grouping**: Associates multi-turn conversations by session_id (each turn is a separate request)
- **Complete Conversation Tracking**: Records messages, question, answer, reasoning, tool_calls for each turn
- **Token Usage Tracking**: Distinguishes input/output/reasoning/cached tokens
- **Web Visualization**: Browser-based UI with overview and session drill-down
- **Real-time URL Generation**: Clawdbot can generate observation links based on current session ID
- **Background Processing**: Independent process, continuously parses access logs
- **State Persistence**: Maintains parsing progress and session data across runs
## Usage
### 1. Background Monitoring (Continuous)
```bash
# Parse Higress access logs (with log rotation support)
python3 main.py --log-path /var/log/proxy/access.log --output-dir ./sessions
# Filter by session key
python3 main.py --log-path /var/log/proxy/access.log --session-key <session-id>
# Scheduled task (incremental parsing every minute)
* * * * * python3 /path/to/main.py --log-path /var/log/proxy/access.log --output-dir /var/lib/sessions
```
### 2. Start Web UI (Recommended)
```bash
# Start web server
python3 scripts/webserver.py --data-dir ./sessions --port 8888
# Access in browser
open http://localhost:8888
```
Web UI features:
- 📊 Overview: View all session statistics and group by model
- 🔍 Session Details: Click session ID to drill down into complete conversation history
- 💬 Conversation Log: Display messages, question, answer, reasoning, tool_calls for each turn
- 💰 Cost Statistics: Real-time token usage and cost calculation
- 🔄 Auto Refresh: Updates every 30 seconds
### 3. Use in Clawdbot Conversations
When users ask about current session token consumption or conversation history:
1. Get current session_id (from runtime or context)
2. Generate web UI URL and return to user
Example response:
```
Your current session statistics:
- Session ID: agent:main:discord:channel:1465367993012981988
- View details: http://localhost:8888/session?id=agent:main:discord:channel:1465367993012981988
Click the link to see:
✅ Complete conversation history
✅ Token usage breakdown per turn
✅ Tool call records
✅ Cost statistics
```
### 4. CLI Queries (Optional)
```bash
# View specific session details
python3 scripts/cli.py show <session-id>
# List all sessions
python3 scripts/cli.py list --sort-by cost --limit 10
# Statistics by model
python3 scripts/cli.py stats-model
# Statistics by date (last 7 days)
python3 scripts/cli.py stats-date --days 7
# Export reports
python3 scripts/cli.py export finops-report.json
```
## Configuration
### main.py (Background Monitor)
| Parameter | Description | Required | Default |
|-----------|-------------|----------|---------|
| `--log-path` | Higress access log file path | Yes | /var/log/higress/access.log |
| `--output-dir` | Session data storage directory | No | ./sessions |
| `--session-key` | Monitor only specified session key | No | Monitor all sessions |
| `--state-file` | State file path (records read offsets) | No | <output-dir>/.state.json |
| `--refresh-interval` | Log refresh interval (seconds) | No | 1 |
### webserver.py (Web UI)
| Parameter | Description | Required | Default |
|-----------|-------------|----------|---------|
| `--data-dir` | Session data directory | No | ./sessions |
| `--port` | HTTP server port | No | 8888 |
| `--host` | HTTP server address | No | 0.0.0.0 |
## Output Examples
### 1. Real-time Monitor
```
🔍 Session Monitor - Active
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📊 Active Sessions: 3
┌──────────────────────────┬─────────┬──────────┬───────────┐
│ Session ID │ Msgs │ Input │ Output │
├──────────────────────────┼─────────┼──────────┼───────────┤
│ sess_abc123 │ 5 │ 1,250 │ 800 │
│ sess_xyz789 │ 3 │ 890 │ 650 │
│ sess_def456 │ 8 │ 2,100 │ 1,200 │
└──────────────────────────┴─────────┴──────────┴───────────┘
📈 Token Statistics
Total Input: 4240 tokens
Total Output: 2650 tokens
Total Cached: 0 tokens
Total Cost: $0.00127
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
```
### 2. CLI Session Details
```bash
$ python3 scripts/cli.py show agent:main:discord:channel:1465367993012981988
======================================================================
📊 Session Detail: agent:main:discord:channel:1465367993012981988
======================================================================
🕐 Created: 2026-02-01T09:30:00+08:00
🕑 Updated: 2026-02-01T10:35:12+08:00
🤖 Model: Qwen3-rerank
💬 Messages: 5
📈 Token Statistics:
Input: 1,250 tokens
Output: 800 tokens
Reasoning: 150 tokens
Total: 2,200 tokens
💰 Estimated Cost: $0.00126000 USD
📝 Conversation Rounds (5):
──────────────────────────────────────────────────────────────────────
Round 1 @ 2026-02-01T09:30:15+08:00
Tokens: 250 in → 160 out
🔧 Tool calls: Yes
Messages (2):
[user] Check Beijing weather
❓ Question: Check Beijing weather
✅ Answer: Checking Beijing weather for you...
🧠 Reasoning: User wants to know Beijing weather, I need to call weather API.
🛠️ Tool Calls:
- get_weather({"location":"Beijing"})
```
### 3. Statistics by Model
```bash
$ python3 scripts/cli.py stats-model
================================================================================
📊 Statistics by Model
================================================================================
Model Sessions Input Output Cost (USD)
────────────────────────────────────────────────────────────────────────────
Qwen3-rerank 12 15,230 9,840 $ 0.016800
DeepSeek-R1 5 8,450 6,200 $ 0.010600
Qwen-Max 3 4,200 3,100 $ 0.008300
GPT-4 2 2,100 1,800 $ 0.017100
────────────────────────────────────────────────────────────────────────────
TOTAL 22 29,980 20,940 $ 0.052800
================================================================================
```
### 4. Statistics by Date
```bash
$ python3 scripts/cli.py stats-date --days 7
================================================================================
📊 Statistics by Date (Last 7 days)
================================================================================
Date Sessions Input Output Cost (USD) Models
────────────────────────────────────────────────────────────────────────────
2026-01-26 3 2,100 1,450 $ 0.0042 Qwen3-rerank
2026-01-27 5 4,850 3,200 $ 0.0096 Qwen3-rerank, GPT-4
2026-01-28 4 3,600 2,800 $ 0.0078 DeepSeek-R1, Qwen
────────────────────────────────────────────────────────────────────────────
TOTAL 22 29,980 20,940 $ 0.0528
================================================================================
```
### 5. Web UI (Recommended)
Access `http://localhost:8888` to see:
**Home Page:**
- 📊 Total sessions, token consumption, cost cards
- 📋 Recent sessions list (clickable for details)
- 📈 Statistics by model table
**Session Detail Page:**
- 💬 Complete conversation log (messages, question, answer, reasoning, tool_calls per turn)
- 🔧 Tool call history
- 💰 Token usage breakdown and costs
**Features:**
- 🔄 Auto-refresh every 30 seconds
- 📱 Responsive design, mobile-friendly
- 🎨 Clean UI, easy to read
## Session Data Structure
Each session is stored as an independent JSON file with complete conversation history and token statistics:
```json
{
"session_id": "agent:main:discord:channel:1465367993012981988",
"created_at": "2026-02-01T10:30:00Z",
"updated_at": "2026-02-01T10:35:12Z",
"messages_count": 5,
"total_input_tokens": 1250,
"total_output_tokens": 800,
"total_reasoning_tokens": 150,
"total_cached_tokens": 0,
"model": "Qwen3-rerank",
"rounds": [
{
"round": 1,
"timestamp": "2026-02-01T10:30:15Z",
"input_tokens": 250,
"output_tokens": 160,
"reasoning_tokens": 0,
"cached_tokens": 0,
"model": "Qwen3-rerank",
"has_tool_calls": true,
"response_type": "normal",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant..."
},
{
"role": "user",
"content": "Check Beijing weather"
}
],
"question": "Check Beijing weather",
"answer": "Checking Beijing weather for you...",
"reasoning": "User wants to know Beijing weather, need to call weather API.",
"tool_calls": [
{
"index": 0,
"id": "call_abc123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{\"location\":\"Beijing\"}"
}
}
],
"input_token_details": {"cached_tokens": 0},
"output_token_details": {}
}
]
}
```
### Field Descriptions
**Session Level:**
- `session_id`: Unique session identifier (from ai_log's session_id field)
- `created_at`: Session creation time
- `updated_at`: Last update time
- `messages_count`: Number of conversation turns
- `total_input_tokens`: Cumulative input tokens
- `total_output_tokens`: Cumulative output tokens
- `total_reasoning_tokens`: Cumulative reasoning tokens (DeepSeek, o1, etc.)
- `total_cached_tokens`: Cumulative cached tokens (prompt caching)
- `model`: Current model in use
**Round Level (rounds):**
- `round`: Turn number
- `timestamp`: Current turn timestamp
- `input_tokens`: Input tokens for this turn
- `output_tokens`: Output tokens for this turn
- `reasoning_tokens`: Reasoning tokens (o1, etc.)
- `cached_tokens`: Cached tokens (prompt caching)
- `model`: Model used for this turn
- `has_tool_calls`: Whether includes tool calls
- `response_type`: Response type (normal/error, etc.)
- `messages`: Complete conversation history (OpenAI messages format)
- `question`: User's question for this turn (last user message)
- `answer`: AI's answer for this turn
- `reasoning`: AI's thinking process (if model supports)
- `tool_calls`: Tool call list (if any)
- `input_token_details`: Complete input token details (JSON)
- `output_token_details`: Complete output token details (JSON)
## Log Format Requirements
Higress access logs must include ai_log field (JSON format). Example:
```json
{
"__file_offset__": "1000",
"timestamp": "2026-02-01T09:30:15Z",
"ai_log": "{\"session_id\":\"sess_abc\",\"messages\":[...],\"question\":\"...\",\"answer\":\"...\",\"input_token\":250,\"output_token\":160,\"model\":\"Qwen3-rerank\"}"
}
```
Supported ai_log attributes:
- `session_id`: Session identifier (required)
- `messages`: Complete conversation history
- `question`: Question for current turn
- `answer`: AI answer
- `reasoning`: Thinking process (DeepSeek, o1, etc.)
- `reasoning_tokens`: Reasoning token count (from PR #3424)
- `cached_tokens`: Cached token count (from PR #3424)
- `tool_calls`: Tool call list
- `input_token`: Input token count
- `output_token`: Output token count
- `input_token_details`: Complete input token details (JSON)
- `output_token_details`: Complete output token details (JSON)
- `model`: Model name
- `response_type`: Response type
## Implementation
### Technology Stack
- **Log Parsing**: Direct JSON parsing, no regex needed
- **File Monitoring**: Polling-based (no watchdog dependency)
- **Session Management**: In-memory + disk hybrid storage
- **Token Calculation**: Model-specific pricing for GPT-4, Qwen, Claude, o1, etc.
### Privacy and Security
- ✅ Does not record conversation content in logs, only token statistics
- ✅ Session data stored locally, not uploaded to external services
- ✅ Supports log file path allowlist
- ✅ Session key access control
### Performance Optimization
- Incremental log parsing, avoids full scans
- In-memory session data with periodic persistence
- Optimized log file reading (offset tracking)
- Inode-based file identification (handles rotation efficiently)
章节与文件索引
按说明文件中的顺序排列。
概述
核心特点
2. 启动Web UI(推荐)
3. 在 Clawdbot 对话中使用
main.py(后台监视器)
文件/命令 -> --log-path · --output-dir · --session-key · --state-file · --refresh-interval · --data-dir · --port · --host
内容校验值 -> 28d199de26f8
当前原文的自动摘译,尚未人工审校。
概述
实时监控Higress访问日志,提取ai_log JSON,按session_id对多轮对话进行分组,并通过可视化计算令牌成本。
核心特点
实时日志监控:监控 Higress 访问日志文件,实时解析新的 ai_log 日志轮转支持:完整的 logrotate 支持,自动跟踪 access.log.1~5 等增量解析:基于 Inode 的跟踪,仅处理新内容,不重复会话分组:通过 session_id 关联多轮对话(每轮都是一个单独的请求)完整的对话跟踪:记录消息、问题、答案、推理、tool_calls每回合
2. 启动Web UI(推荐)
Web UI 功能: 📊 概述:查看所有会话统计信息并按模型分组 🔍 会话详细信息:单击会话 ID 可深入查看完整的对话历史记录 💬 对话日志:显示每轮的消息、问题、答案、推理、工具调用 💰 成本统计:实时代币使用情况和成本计算
3. 在 Clawdbot 对话中使用
当用户询问当前会话令牌消耗或对话历史记录时: 获取当前 session_id(从运行时或上下文) 生成 Web UI URL 并返回给用户 响应示例:
main.py(后台监视器)
参数 · 描述 · 必需 · 默认
--log-path· Higress 访问日志文件路径 · 是 · /var/log/higress/access.log--output-dir· 会话数据存储目录 · 否 · ./sessions--session-key· 只监控指定会话密钥 · 否 · 监控所有会话--state-file· 状态文件路径(记录读取偏移量) · 否 · <output-dir>/.state.json