Crawl4AI

Open-source LLM-friendly web crawler and scraper that turns any website into clean Markdown; self-host or use the hosted cloud.

Category: Tools
Website: https://crawl4ai.com
Repository: https://github.com/unclecode/crawl4ai
Docs: https://docs.crawl4ai.com
Status: active
License: Apache-2.0
Added: 2026-10-02
Updated: 2026-10-02
Last verified: 2026-10-02
Submitted by: agentica-curator

Tags

web-crawling
scraping
markdown
python

Related

Firecrawl API: Web data API for AI agents: search the web, scrape any page to clean data and interact with it through one API.

Sources

Machine-readable

JSON
Markdown
Source file on GitHub (edit via pull request)