← All services

Robots.txt & AI Crawler Checker

MCP server MCP

dev.workers.quietworklabs.robots-check · dev.workers.quietworklabs.robots-check · Web & Scraping

not indexed Provider site ↗

robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.

Price — no price published

Calls · 30d — not reported by any source

Buyers · 30d — Observed sending wallets · Base USDC · partial sender history · purchases unverified

Volume · 30d — Observed Base USDC inflows · not attributable or not measured

Inflow growth · 7d — not comparable (baseline under $1 or window incomplete)

Trx · 30d — Observed transfers · not attributable or not measured

Observed inflows · last 30 days

No inflow observed in this window

No payout address is published, so nothing can be observed.

Observed inflows by window for Robots.txt & AI Crawler Checker.
Window	Observed	Transfers
Last complete day (2026-11-06)	—	—
Last 7 days	—	—
Last 30 days	—	—
Growth, 7d against the 7 before	not comparable

Source: on-chain transfer scan (USDC on Base) Window: 2026-10-08 → 2026-11-06 Scanned: 30/30 days Sample: 0 days with a measurement First seen: never Last update: 2026-11-06

Daily history →. Observed transfers show value arriving at the payout address; they do not establish service revenue.

Competitors & category benchmark

Rank the whole category →

Closest competitors Web & Scraping

Web read 10,345 calls · 0.002 USDC per call —
Domain WHOIS & DNS Lookup 533 calls · 0.02 USDC per call —
Agent Web-Access 429 calls · 0.001 USDC per call —
Domain age check 428 calls · 0.0324 USDC per call —
Automaton Paid API 256 calls · 0.001 USDC per call $12.12

Sorted by source-measured calls, not by our opinion. The right-hand figure is observed inflow, which stays empty for peers whose address is shared.

Category benchmark 6,458 services in this category

With a measured call count	1,862 (28.8 %)
With a published price	2,543 (39.4 %)
With attributable inflows	70 (1.1 %)
Observed inflows · 30d	$1,034.57
Median measured calls	2
Median price per call	$0.01