e commerce
- Repo stars 1,112
- Author repo web-agent
- Domain
- Data
- Author-declared agents
- No explicit declaration found; this is not inferred or tested compatibility
- Static check
- 88 / 100 · heuristic scan, not runtime safety proof
- Runtime evidence
- Not run; no isolated runtime test evidence is currently recorded
- Author / version / license
- @firecrawl · no license declared
- Token usage
- Lean
- Setup complexity
- Plug-and-play
- External API key
- Not required
- Operating systems
- Unspecified (assume cross-platform)
- Runtime requirements
- No explicit requirements detected
- Detected file/system behavior
-
- Read-only
- Network behavior
- Local-only
- Install commands
- None (reference only)
Heads up: Tool permissions are not declared; actual access depends on the runtime.
# E-commerce Extraction
## General Patterns
- Check for sitemap.xml first -- many stores list all product URLs
- Look for /products.json, /api/products, or similar API endpoints before scraping HTML
- Product listing pages usually paginate: look for ?page=N, ?offset=N, or "Load more" buttons
- Always check the total count shown on the page vs what you've extracted
## Product Data Checklist
- Name, brand, SKU/ID
- Price (current, original/compare-at, currency)
- Variants (size, color, etc.) with per-variant pricing and availability
- Images (primary + gallery)
- Description (short + long)
- Category/breadcrumb path
- Availability/stock status
- Ratings and review count
## Pagination
- Check for: next/prev links, page numbers, "showing X of Y" text
- For infinite scroll: use interact to scroll and load more
- Keep count: if page says "200 products" and you have 24, keep going
## JS-Heavy Sites
- Use interact for sites that require JavaScript rendering
- Look for XHR/API calls in the page that return JSON -- often easier than parsing HTML
- Single-page apps often have internal APIs at predictable paths
## Site-specific playbooks are in the sites/ directory
Sections and files
Listed in source order.
sections -> General Patterns → Product Data Checklist → Pagination → JS-Heavy Sites → Site-specific playbooks are in the sites/ directory
terms -> No English key terms available
files/cmd -> sitemap.xml · products.js · api/products · SKU/ID · original/compare-at · Category/breadcrumb · Availability/stock · next/prev
body sha256 -> c9869d491964
General Patterns
Check for sitemap.xml first -- many stores list all product URLs Look for /products.json, /api/products, or similar API endpoints before scraping HTML Product listing pages usually paginate: look for ?page=N, ?offset=N, or "Load more" buttons Always check the total count shown on the page vs what you've extracted
Product Data Checklist
Name, brand, SKU/ID Price (current, original/compare-at, currency) Variants (size, color, etc.) with per-variant pricing and availability Images (primary + gallery) Description (short + long)
Pagination
Check for: next/prev links, page numbers, "showing X of Y" text For infinite scroll: use interact to scroll and load more Keep count: if page says "200 products" and you have 24, keep going
JS-Heavy Sites
Use interact for sites that require JavaScript rendering Look for XHR/API calls in the page that return JSON -- often easier than parsing HTML Single-page apps often have internal APIs at predictable paths