Web Scraping MCP for AI Agents
CrawlForge is an open-source MCP server that equips AI agents with 31 web tools at once—scraping, rendering, searching, and structured extraction—so any URL can be fetched in a single call as clean Markdown or typed JSON.
It integrates with Claude, Cursor, and LangChain, and can also be used as an API in your own code. Installation is just one line.npx -y crawlforge-mcp-server..., add your API key, and then choose which tools to enable. You get 1,000 free credits when you sign up, with no credit card required; there’s also an online Playground on the official website you can try first.
For developers building research agents, competitor monitoring, and data collection. When scraping other people's websites, be mindful of their terms of use and robots.txt rules.