加载
- 作者仓库星标 0
- 作者仓库 skills-registry
- 领域
- 写作
- 作者声明的智能体
- 未找到明确声明;不据此推断已兼容或已测试
- 静态检查
- 88 / 100 · 启发式扫描,不代表运行安全
- 运行环境证据
- 未执行;当前没有隔离运行测试证据
- 作者 / 版本 / 许可证
- @tomevault-io · 未声明许可证
- 上下文用量评级
- 低消耗
- 接入复杂程度
- 即装即用
- 是否需要外部接口密钥
- 不需要
- 兼容的系统
- 未声明(默认跨平台)
- 运行环境要求
- Python
- 检测到的文件与系统行为
-
- 只读
- 允许写入 / 修改
- 读取环境变量
- 网络行为
- 允许外网请求
- 安装命令数
- 无(仅作为资料)
需要注意: 未声明工具权限范围;实际权限由运行环境决定。
# SQL Load
Load a flat-file dataset into a SQL database.
## When to invoke
- User says "load this into Postgres / MySQL / SQLite", "put this in the database", "upload to my SQL server".
- Dataset is clean and the user wants it queryable via SQL.
## Supported backends
- **SQLite** — file-based, zero config; default for quick local loads.
- **PostgreSQL** — via `psycopg2` or `psycopg[binary]`.
- **MySQL** / MariaDB — via `PyMySQL` or `mysql-connector-python`.
- **Microsoft SQL Server** — via `pyodbc`.
- **DuckDB** — file-based analytics DB; best for Parquet-native workflows.
## Procedure
1. **Determine the connection**:
- **Existing connection config** — check `$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json` for saved database profiles. List them, let the user pick.
- **New connection** — ask the user for: backend, host/port OR file path, database name, username, and how to provide the password (env var name, 1Password reference via `op-vault`, or prompt at connection time). Never hard-code passwords in files. Save the non-secret parts of the profile to `$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json` if the user wants to reuse it.
2. **Confirm the target table**:
- Table name (default: dataset filename stem, sanitised).
- Schema (for Postgres / MSSQL).
- Action if exists: `fail` (default), `append`, `replace`, `upsert` (requires primary key).
3. **Load the dataset** with pandas.
4. **Map dtypes to SQL types** — use the data dictionary if present to pin types; otherwise infer. Default mappings:
- `int64` → `BIGINT` / `INTEGER`.
- `float64` → `DOUBLE PRECISION` / `REAL`.
- `object` (string) → `TEXT` / `VARCHAR(n)` — size from max length observed, padded.
- `datetime64[ns]` → `TIMESTAMP`.
- `bool` → `BOOLEAN` / `BIT`.
Ask user to confirm for columns where inference is uncertain.
5. **Create the table** via `CREATE TABLE IF NOT EXISTS ...` (or `CREATE OR REPLACE` if requested). Respect primary key / not-null / index requests.
6. **Insert** in chunks (default 10k rows) using `to_sql(..., method='multi')` or backend-native bulk loaders (`COPY` for Postgres, `LOAD DATA INFILE` for MySQL) for large files.
7. **Validate** — row count `SELECT COUNT(*)` matches source; sample a few rows and compare.
8. **Report** — table name, row count, load duration, index/PK status.
## Connection config format
`$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json`:
```json
{
"sql_profiles": {
"local-postgres": {
"backend": "postgresql",
"host": "localhost",
"port": 5432,
"database": "analytics",
"user": "daniel",
"password_ref": {"type": "env", "name": "PGPASSWORD"}
},
"local-sqlite": {
"backend": "sqlite",
"path": "~/Documents/data/warehouse.db"
}
}
}
```
`password_ref` options:
- `{"type": "env", "name": "PGPASSWORD"}` — read from env var at connect time.
- `{"type": "op", "reference": "op://Private/postgres/password"}` — fetch via 1Password CLI.
- `{"type": "prompt"}` — prompt the user each run.
Never write plaintext passwords into this file.
## Dependencies
```bash
pip install pandas sqlalchemy
# per backend
pip install psycopg[binary] # postgres
pip install pymysql # mysql
pip install pyodbc # mssql
pip install duckdb # duckdb
```
## Edge cases
- **Schema drift** (file columns != table columns) — report diff; ask user whether to add columns, drop columns, or fail.
- **Character encoding** — ensure client/server encoding matches dataset encoding; default UTF-8.
- **Very large files** — use backend-native bulk loaders (`COPY FROM`, `LOAD DATA`) rather than `INSERT`. Stream from Parquet directly where possible.
- **Transactions** — wrap the load in a single transaction; if it fails mid-way, the user keeps the prior state.
- **PII** — if the dataset hasn't been PII-checked (see `pii-flag`), warn before loading into a shared database.
---
> Source: [danielrosehill/Claude-Data-Wrangler-plugin](https://github.com/danielrosehill/Claude-Data-Wrangler-plugin) — distributed by [TomeVault](https://tomevault.io).
<!-- tomevault:4.0:skill_md:2026-05-22 -->
章节与文件索引
按说明文件中的顺序排列。
何时调用
支持的后端
程序
连接配置格式
边缘情况
文件/命令 -> psycopg2 · psycopg[binary] · PyMySQL · mysql-connector-python · pyodbc · $CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json · op-vault · fail
内容校验值 -> 15960f525ef4
当前原文的自动摘译,尚未人工审校。
何时调用
用户说“将其加载到 Postgres / MySQL / SQLite”,“将其放入数据库”,“上传到我的 SQL 服务器”。数据集是干净的,用户希望它可以通过 SQL 进行查询。
支持的后端
SQLite — 基于文件、零配置;默认用于快速本地加载。 PostgreSQL — 通过
psycopg2或psycopg[binary]。 MySQL / MariaDB — 通过PyMySQL或mysql-connector-python。 Microsoft SQL Server — 通过pyodbc。 DuckDB — 基于文件的分析数据库;最适合 Parquet 原生工作流程。程序
确定连接:现有连接配置 - 检查
$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json以获取已保存的数据库配置文件。列出它们,让用户选择。新连接 - 询问用户:后端、主机/端口或文件路径、数据库名称、用户名以及如何提供密码(环境变量名称、通过op-vault的 1Password 参考,或连接时的提示)。切勿在文件中硬编码密码。如果用户想重复使用,请将配置文件的非秘密部分保存到$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json。连接配置格式
$CLAUDE_USER_DATA/Claude-Data-Wrangler/config.json:password_ref选项:{"type": "env", "name": "PGPASSWORD"}— 在连接时从环境变量中读取。{"type": "op", "reference": "op://Private/postgres/password"}— 通过 1Password CLI 获取。{"type": "prompt"}— 每次运行时提示用户。边缘情况
模式漂移(文件列!=表列)——报告差异;询问用户是否添加列、删除列或失败。字符编码——确保客户端/服务器编码与数据集编码匹配;默认 UTF-8。非常大的文件 - 使用后端本地批量加载器(
COPY FROM,LOAD DATA)而不是INSERT。尽可能直接从 Parquet 进行流式传输。事务——将负载包装在单个事务中;如果中途失败,用户保持先前的状态。