baoyu-url-to-markdown
- Repo stars 21,713
- Forks 2,492
- Author updated Jun 13, 2026, 05:00 AM
- Author repo baoyu-skills
- Domain
- Documentation
- Compatible agents
-
- Claude Code
- Cursor
- Cline
- Codex
- Windsurf
- Gemini CLI
- +20
- Trust score
- 92 / 100 · audit passed
- Author / version / license
- @JimLiu · v1.61.0 · no license declared
- Token usage
- Lean
- Setup complexity
- Guided setup
- External API key
- Not required
- Operating systems
- macOS · Linux · Windows
- Runtime requirements
- Bun
- Permissions
-
- Read-only
- Write / modify
- Shell exec
- Env read
- Network behavior
- External requests
- Install commands
- 26 variants
Profile is derived at build time from SKILL.md and install vectors. Subject to drift from author intent.
Heads up: 未限定 allowed-tools,默认拥有全部工具权限。
---
name: baoyu-url-to-markdown
description: Fetch any URL and convert to markdown using baoyu-fetch CLI (Chrome CDP with site-specific adapt…
category: documentation
runtime: Bun
---
# baoyu-url-to-markdown output preview
## PART A: Task fit
- Use case: Fetch any URL and convert to markdown using baoyu-fetch CLI (Chrome CDP with site-specific adapters). Built-in adapters for X/Twitter, YouTube transcripts, Hacker News threads, and generic pages via Defuddle. Handles login/CAPTCHA via interaction wait modes. Use when user wants to save a webpage as markdown..
- Inputs: target material, constraints, expected output, and acceptance criteria.
- Evidence boundary: follow “User Input Tools / CLI Setup / Preferences (EXTEND.md)” and do not present inference as author intent.
## PART B: Execution result
- **01** The card summarizes the use case; runtime output centers on “Fetch any URL and convert to markdown using baoyu-fetch CLI (Chrome CDP with site-specific adapters). Built-in adapters for X/Twitter, YouTube transcripts, Hacker News threads, and generic pages via Defuddle. Handles login/CAPTCHA via interaction wait modes. Use when user wants to save a webpage as markdown.”.
- **02** When the source has headings, the agent prioritizes “User Input Tools / CLI Setup / Preferences (EXTEND.md)” so the result follows the author’s structure.
- **03** Typical output includes task judgment, concrete steps, required commands or file edits, validation, and follow-up options.
- **04** Risk context follows the fingerprint: read files, write/modify files, run shell commands, read environment variables; may access external network resources; usually needs no extra API key.
## Running Rules
- read files, write/modify files, run shell commands, read environment variables; may access external network resources; usually needs no extra API key.
- Validate with a small sample before expanding scope.
- Return the result, validation criteria, and next iteration options. The source does not require a stable slash command. After installation, invoke the skill by name and describe the task.
Name target files or source material, expected output, forbidden changes, and whether network or shell access is allowed. Permission fingerprint: read files, write/modify files, run shell commands, read environment variables.
Start with a small task and check whether the result follows “User Input Tools / CLI Setup / Preferences (EXTEND.md)”. Inspect diffs, logs, previews, or tests before expanding scope.
Confirm the final output includes a concrete result, evidence, and next action. If it stays generic, tighten inputs, boundaries, and acceptance criteria.
---
name: baoyu-url-to-markdown
description: Fetch any URL and convert to markdown using baoyu-fetch CLI (Chrome CDP with site-specific adapt…
category: documentation
source: JimLiu/baoyu-skills
---
# baoyu-url-to-markdown
## When to use
- Fetch any URL and convert to markdown using baoyu-fetch CLI (Chrome CDP with site-specific adapters). Built-in adapter…
- Use it when the task has clear inputs, repeatable steps, and validation criteria.
## What to provide
- Target material, scope, expected result, and forbidden changes.
- Whether network, commands, file writes, or external services are allowed.
## Execution rules
- Organize steps around “User Input Tools / CLI Setup / Preferences (EXTEND.md)” and keep inference separate from source facts.
- read files, write/modify files, run shell commands, read environment variables; may access external network resources; usually needs no extra API key.
- Validate with a small sample before expanding the task.
## Output requirements
- Return the deliverable, key evidence, validation method, and next action.
- Mark missing information as unknown; do not invent commands, platforms, or dependencies. The author source anchors workflow facts; repository files anchor sources and commands; Fluxly only adds fit, limitations, and quality judgment.
skill "baoyu-url-to-markdown" {
input -> user goal + target files + boundaries + acceptance criteria
context -> User Input Tools / CLI Setup / Preferences (EXTEND.md)
rules -> SKILL.md triggers / order / output contract
runtime -> Bun | read files, write/modify files, run shell commands, read environment variables | may access external network resources
guardrails -> usually needs no extra API key + small-sample validation + diff/log review
output -> copyable result + checklist + next iteration
} URL to Markdown
Fetches any URL via baoyu-fetch CLI (Chrome CDP + site-specific adapters) and converts it to clean markdown.
User Input Tools
When this skill prompts the user, follow this tool-selection rule (priority order):
- Prefer built-in user-input tools exposed by the current agent runtime — e.g.,
AskUserQuestion,request_user_input,clarify,ask_user, or any equivalent. - Fallback: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question.
- Batching: if the tool supports multiple questions per call, combine all applicable questions into a single call; if only single-question, ask them one at a time in priority order.
Concrete AskUserQuestion references below are examples — substitute the local equivalent in other runtimes.
CLI Setup
Important: The CLI source is vendored in {baseDir}/scripts/lib. scripts/package.json installs only third-party runtime dependencies.
Agent Execution Instructions:
- Determine this SKILL.md file's directory path as
{baseDir} - Resolve
${BUN}runtime: ifbuninstalled →bun; else suggest installing Bun - If
{baseDir}/scripts/node_modulesdoes not exist, run${BUN} install --cwd {baseDir}/scripts ${READER}={baseDir}/scripts/baoyu-fetch- Replace all
${READER}in this document with the resolved value
Preferences (EXTEND.md)
Check EXTEND.md in priority order — the first one found wins:
| Priority | Path | Scope |
|---|---|---|
| 1 | .baoyu-skills/baoyu-url-to-markdown/EXTEND.md |
Project |
| 2 | ${XDG_CONFIG_HOME:-$HOME/.config}/baoyu-skills/baoyu-url-to-markdown/EXTEND.md |
XDG |
| 3 | $HOME/.baoyu-skills/baoyu-url-to-markdown/EXTEND.md |
User home |
| Result | Action |
|---|---|
| Found | Read, parse, apply settings |
| Not found | MUST run first-time setup (see below) — do NOT silently create defaults |
EXTEND.md supports: download media by default, default output directory.
First-Time Setup ⛔ BLOCKING
When EXTEND.md is not found, you MUST use AskUserQuestion to gather preferences before creating EXTEND.md. NEVER create EXTEND.md with silent defaults. Generation is BLOCKED until setup completes. Batch all three questions into a single call:
- Q1 — Media (header "Media"): "How to handle images and videos in pages?"
- "Ask each time (Recommended)" — Prompt after each save
- "Always download" — Download to local
imgs/andvideos/ - "Never download" — Keep remote URLs
- Q2 — Output (header "Output"): "Default output directory?"
- "url-to-markdown (Recommended)" — Save to
./url-to-markdown/{domain}/{slug}.md - User may pick "Other" and type a custom path
- "url-to-markdown (Recommended)" — Save to
- Q3 — Save (header "Save"): "Where to save preferences?"
- "User (Recommended)" —
~/.baoyu-skills/(all projects) - "Project" —
.baoyu-skills/(this project only)
- "User (Recommended)" —
After answers, write EXTEND.md, confirm "Preferences saved to [path]", then continue.
Full template: references/config/first-time-setup.md.
Supported Keys
| Key | Default | Values | Description |
|---|---|---|---|
download_media |
ask |
ask / 1 / 0 |
ask = prompt each time, 1 = always, 0 = never |
default_output_dir |
empty | path or empty | Default output directory (empty = ./url-to-markdown/) |
EXTEND.md → CLI mapping:
| EXTEND.md key | CLI argument | Notes |
|---|---|---|
download_media: 1 |
--download-media |
Requires --output to be set |
default_output_dir: ./posts/ |
Agent constructs --output ./posts/{domain}/{slug}.md |
Agent generates path, not a direct flag |
Value priority: CLI arguments → EXTEND.md → skill defaults.
Usage
# Default: headless capture, markdown to stdout
${READER} <url>
# Save to file
${READER} <url> --output article.md
# Save with media download
${READER} <url> --output article.md --download-media
# Wait for interaction (login/CAPTCHA) — auto-detect and continue
${READER} <url> --wait-for interaction --output article.md
# Wait for interaction — manual control (Enter to continue)
${READER} <url> --wait-for force --output article.md
# JSON output
${READER} <url> --format json --output article.json
# Force specific adapter
${READER} <url> --adapter youtube --output transcript.md
Options
| Option | Description |
|---|---|
<url> |
URL to fetch |
--output <path> |
Output file path (default: stdout) |
--format <type> |
Output format: markdown (default) or json |
--json |
Shorthand for --format json |
--adapter <name> |
Force adapter: x, youtube, hn, or generic (default: auto-detect) |
--headless |
Force headless Chrome (no visible window) |
--wait-for <mode> |
Interaction wait mode: none (default), interaction, or force |
--wait-for-interaction |
Alias for --wait-for interaction |
--wait-for-login |
Alias for --wait-for interaction |
--timeout <ms> |
Page load timeout (default: 30000) |
--interaction-timeout <ms> |
Login/CAPTCHA wait timeout (default: 600000 = 10 min) |
--interaction-poll-interval <ms> |
Poll interval for interaction checks (default: 1500) |
--download-media |
Download images/videos to local imgs/ and videos/, rewrite markdown links. Requires --output |
--media-dir <dir> |
Base directory for downloaded media (default: same as --output directory) |
--cdp-url <url> |
Reuse existing Chrome DevTools Protocol endpoint |
--browser-path <path> |
Custom Chrome/Chromium binary path |
--chrome-profile-dir <path> |
Chrome user data directory (default: BAOYU_CHROME_PROFILE_DIR env or ./baoyu-skills/chrome-profile) |
--debug-dir <dir> |
Write debug artifacts (document.json, markdown.md, page.html, network.json) |
Agent Quality Gate
CRITICAL: treat default headless capture as provisional. Some sites render differently in headless mode and can silently return low-quality content without failing the CLI.
After every headless run, inspect the saved markdown. See references/quality-gate.md for the full checklist, recovery workflow, and capture-mode table. Read it whenever a run looks suspicious or the user asks about login/CAPTCHA handling.
Output Path Generation
The agent must construct the output file path — baoyu-fetch does not auto-generate paths.
Algorithm:
- Determine base directory from EXTEND.md
default_output_diror default./url-to-markdown/ - Extract domain from URL (e.g.,
example.com) - Generate slug from URL path or page title (kebab-case, 2-6 words)
- Construct:
{base_dir}/{domain}/{slug}/{slug}.md— each URL gets its own directory so media files stay isolated - Conflict resolution: append timestamp
{slug}-YYYYMMDD-HHMMSS/{slug}-YYYYMMDD-HHMMSS.md
Pass the constructed path to --output. Media files (--download-media) are saved into subdirectories next to the markdown file, keeping each URL's assets self-contained.
Adapters & Media
See references/adapters.md for the adapter catalog (X, YouTube, Hacker News, generic), per-adapter notes, the media download flow (ask / always / never), and the JSON output schema. Read it before answering adapter-specific questions or handling media prompts.
Environment Variables
| Variable | Description |
|---|---|
BAOYU_CHROME_PROFILE_DIR |
Chrome user data directory (can also use --chrome-profile-dir) |
Troubleshooting: Chrome not found → use --browser-path. Timeout → increase --timeout. Login/CAPTCHA → --wait-for interaction. Debug → --debug-dir to inspect captured HTML and network logs.
Extension Support
Custom configurations via EXTEND.md. See Preferences section above for paths and supported keys.
Decide Fit First
Design Intent
How To Use It
Boundaries And Review