加载

写作 社区

技能解读

当前原文的自动摘译,尚未人工审校。

何时调用

用户说“将其加载到 Postgres / MySQL / SQLite”,“将其放入数据库”,“上传到我的 SQL 服务器”。数据集是干净的,用户希望它可以通过 SQL 进行查询。

支持的后端

SQLite — 基于文件、零配置;默认用于快速本地加载。 PostgreSQL — 通过 psycopg2 或 psycopg[binary]。 MySQL / MariaDB — 通过 PyMySQL 或 mysql-connector-python。 Microsoft SQL Server — 通过pyodbc。 DuckDB — 基于文件的分析数据库;最适合 Parquet 原生工作流程。

程序

确定连接:现有连接配置 - 检查$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json以获取已保存的数据库配置文件。列出它们,让用户选择。新连接 - 询问用户:后端、主机/端口或文件路径、数据库名称、用户名以及如何提供密码(环境变量名称、通过 op-vault 的 1Password 参考,或连接时的提示)。切勿在文件中硬编码密码。如果用户想重复使用,请将配置文件的非秘密部分保存到$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json。

连接配置格式

$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json: password_ref 选项: {"type": "env", "name": "PGPASSWORD"} — 在连接时从环境变量中读取。 {"type": "op", "reference": "op://Private/postgres/password"} — 通过 1Password CLI 获取。 {"type": "prompt"} — 每次运行时提示用户。

边缘情况

模式漂移(文件列!=表列)——报告差异;询问用户是否添加列、删除列或失败。字符编码——确保客户端/服务器编码与数据集编码匹配;默认 UTF-8。非常大的文件 - 使用后端本地批量加载器(COPY FROM,LOAD DATA)而不是INSERT。尽可能直接从 Parquet 进行流式传输。事务——将负载包装在单个事务中;如果中途失败,用户保持先前的状态。

流狐档案 运行与权限由流狐静态检查推断,不代表作者声明或实际测试结果。
领域
写作
作者声明的智能体
未找到明确声明;不据此推断已兼容或已测试
静态检查
88 / 100 · 启发式扫描,不代表运行安全
运行环境证据
未执行;当前没有隔离运行测试证据
作者 / 版本 / 许可证
@tomevault-io · 未声明许可证
上下文用量评级
低消耗
接入复杂程度
即装即用
是否需要外部接口密钥
不需要
兼容的系统
未声明(默认跨平台)
运行环境要求
Python
检测到的文件与系统行为
  • 只读
  • 允许写入 / 修改
  • 读取环境变量
网络行为
允许外网请求
安装命令数
无(仅作为资料)

需要注意: 未声明工具权限范围;实际权限由运行环境决定。

作者说明文件(原文) SKILL.md

# SQL Load

Load a flat-file dataset into a SQL database.

## When to invoke

- User says "load this into Postgres / MySQL / SQLite", "put this in the database", "upload to my SQL server".
- Dataset is clean and the user wants it queryable via SQL.

## Supported backends

- **SQLite** — file-based, zero config; default for quick local loads.
- **PostgreSQL** — via `psycopg2` or `psycopg[binary]`.
- **MySQL** / MariaDB — via `PyMySQL` or `mysql-connector-python`.
- **Microsoft SQL Server** — via `pyodbc`.
- **DuckDB** — file-based analytics DB; best for Parquet-native workflows.

## Procedure

1. **Determine the connection**:
   - **Existing connection config** — check `$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json` for saved database profiles. List them, let the user pick.
   - **New connection** — ask the user for: backend, host/port OR file path, database name, username, and how to provide the password (env var name, 1Password reference via `op-vault`, or prompt at connection time). Never hard-code passwords in files. Save the non-secret parts of the profile to `$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json` if the user wants to reuse it.
2. **Confirm the target table**:
   - Table name (default: dataset filename stem, sanitised).
   - Schema (for Postgres / MSSQL).
   - Action if exists: `fail` (default), `append`, `replace`, `upsert` (requires primary key).
3. **Load the dataset** with pandas.
4. **Map dtypes to SQL types** — use the data dictionary if present to pin types; otherwise infer. Default mappings:
   - `int64` → `BIGINT` / `INTEGER`.
   - `float64` → `DOUBLE PRECISION` / `REAL`.
   - `object` (string) → `TEXT` / `VARCHAR(n)` — size from max length observed, padded.
   - `datetime64[ns]` → `TIMESTAMP`.
   - `bool` → `BOOLEAN` / `BIT`.
   Ask user to confirm for columns where inference is uncertain.
5. **Create the table** via `CREATE TABLE IF NOT EXISTS ...` (or `CREATE OR REPLACE` if requested). Respect primary key / not-null / index requests.
6. **Insert** in chunks (default 10k rows) using `to_sql(..., method='multi')` or backend-native bulk loaders (`COPY` for Postgres, `LOAD DATA INFILE` for MySQL) for large files.
7. **Validate** — row count `SELECT COUNT(*)` matches source; sample a few rows and compare.
8. **Report** — table name, row count, load duration, index/PK status.

## Connection config format

`$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json`:

```json
{
  "sql_profiles": {
    "local-postgres": {
      "backend": "postgresql",
      "host": "localhost",
      "port": 5432,
      "database": "analytics",
      "user": "daniel",
      "password_ref": {"type": "env", "name": "PGPASSWORD"}
    },
    "local-sqlite": {
      "backend": "sqlite",
      "path": "~/Documents/data/warehouse.db"
    }
  }
}
```

`password_ref` options:
- `{"type": "env", "name": "PGPASSWORD"}` — read from env var at connect time.
- `{"type": "op", "reference": "op://Private/postgres/password"}` — fetch via 1Password CLI.
- `{"type": "prompt"}` — prompt the user each run.

Never write plaintext passwords into this file.

## Dependencies

```bash
pip install pandas sqlalchemy
# per backend
pip install psycopg[binary]     # postgres
pip install pymysql             # mysql
pip install pyodbc              # mssql
pip install duckdb              # duckdb
```

## Edge cases

- **Schema drift** (file columns != table columns) — report diff; ask user whether to add columns, drop columns, or fail.
- **Character encoding** — ensure client/server encoding matches dataset encoding; default UTF-8.
- **Very large files** — use backend-native bulk loaders (`COPY FROM`, `LOAD DATA`) rather than `INSERT`. Stream from Parquet directly where possible.
- **Transactions** — wrap the load in a single transaction; if it fails mid-way, the user keeps the prior state.
- **PII** — if the dataset hasn't been PII-checked (see `pii-flag`), warn before loading into a shared database.

---
> Source: [danielrosehill/Claude-Data-Wrangler-plugin](https://github.com/danielrosehill/Claude-Data-Wrangler-plugin) — distributed by [TomeVault](https://tomevault.io).
<!-- tomevault:4.0:skill_md:2026-05-22 -->

讨论

基于 GitHub Discussions。登录 GitHub 即可参与讨论、点赞、订阅更新。