Crawl4AI
Open-source LLM-friendly web crawler and scraper that turns any website into clean Markdown; self-host or use the hosted cloud.
Tags
Related
- Firecrawl API: Web data API for AI agents: search the web, scrape any page to clean data and interact with it through one API.
Guides & comparisons
- Convert documents to Markdown for agents with MarkItDown and Docling (Guide): Turn PDFs, Office files and other documents into Markdown or chunks an agent can read, with Microsoft's MarkItDown for speed and Docling for layout and tables, then expose either one to an agent over MCP.
Sources
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)