MARKET MONITOR / CATALOGUE Services 162,062 MCP listed 50,342 x402 111,860 Rails Base / Solana / Polygon / Arbitrum Imported 2026-11-18 · 3 sources
Task

Robots Txt

34 services carry this tag in their source record. Tags come from the provider, not from us — a tag is a claim about what the service does, not proof that it does it well.

Services tagged Robots Txt

NetIntel x402 REST usage

Email deliverability & domain security check — validates SPF, DKIM (multi-selector), and DMARC, detects spoofing and misconfigurations, and returns an A–F email security grade with transparent deductions. Anti-spam / ant...

netintel.dev Security 391 calls / 30d checked 2026-11-18
0.002 USDC per call per call

Agent-Ready Assessment x402 REST usage

Audits a website for AI agent and AI crawler readiness from caller-supplied inputs: robots.txt, HTML, headers, llms.txt, sitemap lastmod, /.well-known. Names the policy per AI token (GPTBot, OAI-SearchBot, ChatGPT-User,...

api.ozdreamtools.de AI Models & Inference 19 calls / 30d checked 2026-11-18
0.01 USDC per call per call

Minifetch Technical SEO API x402 REST usage

Cheap robots.txt preflight check before you call the Minifetch /extract and /run/seo-page-audit endpoints: returns whether Minifetch may fetch a target URL (allow/deny) plus its crawl-delay. If blocked and the site is no...

minifetch.com Business & Growth 117 calls / 30d checked 2026-11-18
0.001 USDC per call per call

Hero HQ Agent Store x402 REST usage

Is this website live and in good shape? HTTP status, valid HTTPS, redirect chain, mobile viewport, page title and description, platform/CMS (WordPress, Shopify, Wix, Squarespace, Webflow, GoDaddy...), load time, and a sh...

api.glowewebsites.com Business & Growth 1 calls / 30d checked 2026-11-18
0.1 USDC per call per call

AGMM x402 REST usage

Domain in, a grade from A to F out: how readable a website is to AI crawlers and answer engines (robots.txt, llms.txt, sitemap, JSON-LD, MCP and x402 readiness). Public files only, no LLM. $0.05 USDC per request.

agmm.co.uk Crypto & Payments 2 calls / 30d checked 2026-11-18
0.05 USDC per call per call

MarketPulse x402 REST usage

Where is this competitor vulnerable? Finds exploitable gaps in a competitor's marketing presence across content, channels and positioning, then returns quick wins and a six-month attack plan for a given competitor and in...

marketpulse.theaslangroupllc.com Business & Growth 1 calls / 30d checked 2026-11-14
0.1 USDC per call per call

Website Audit Snapshot | Lordevo x402 REST usage

Site snapshot (SEO + security + tech + email): All four quick checks in one call for a website: on-page SEO score and issues, security headers/TLS grade, tech stack, and the domain's SPF/DKIM/DMARC email authentication.

shop.lordevo.com Security 4 calls / 30d checked 2026-11-18
0.02 USDC per call per call

Agent Web Tools x402 REST usage

Check if a URL is up: status, latency, redirects, title.

agent-web-tools.agent-web-tools.workers.dev Infrastructure 79 calls / 30d checked 2026-11-18
0.001 USDC per call per call

how.is x402 REST usage

Take a screenshot of any public web page: a 1280-wide JPEG, viewport or full page, with its final URL, HTTP status and title.

how.is Web & Scraping 3 calls / 30d checked 2026-11-18
0.02 USDC per call per call

robots.txt Linter (RFC 9309) x402 REST usage

Lints and validates a caller-supplied robots.txt against RFC 9309 and reports every check, not only the failures: group structure, product token charset, path patterns, special characters, other records, size limit. Give...

api.ozdreamtools.de Dev & Code 1 calls / 30d checked 2026-11-18
0.05 USDC per call per call

Nero Labs Agent Tools x402 REST usage

Scores up to 20 product ideas or keywords against live demand: how many tools on the Apify Store already do it, monthly users of the top tools, the leader and its failure rate, tools that gained most users this month, x4...

x402-agent-tools.nerolabs.workers.dev Crypto & Payments 1 calls / 30d checked 2026-11-18
0.04 USDC per call per call

CrawlWitness x402 REST usage

Use when an agent is about to crawl, train on or ingest a website into RAG: checks the domain's AI consent and licensing signals (robots.txt per AI bot, RSL license, IETF AIPREF, TDMRep, noai meta tags, llms.txt, C2PA),...

api.crawlwitness.com Search & Research 7 calls / 30d checked 2026-11-18
0.005 USDC per call per call

Robots.txt Policy Writer x402 REST usage

Robots.txt Policy Writer — Turns a crawler policy written in words into a correct robots.txt, answered as the file only. Knows the mistakes that make a file do the opposite of what was meant - a crawler obeys only the on...

api.aiskills402.com Search & Research 1 calls / 30d checked 2026-11-18
0.03 USDC per call per call

TariffMonkee x402 REST usage

Health and payment-path check. Returns server time and confirms the x402 payment settled. The cheapest endpoint in the catalog at $0.001, intended for testing a client's payment flow.

tariffmonkee.com Crypto & Payments 29 calls / 30d checked 2026-11-18
0.001 USDC per call per call

Saylor Innovations Guide Library x402 REST usage

CUDA, driver and Ollama compatibility - GPU support, model memory fit and proof inference uses the GPU

saylorinnovations.com AI Models & Inference 2 calls / 30d checked 2026-11-08
0.01 USDC per call per call

CrawlSafe x402 REST usage

Before your agent fetches a blog post, docs page or any public URL, check RFC 9309 robots.txt for your user agent, plus X-Robots-Tag and meta robots. Returns crawl_allowed and index_allowed. SSRF-guarded on every redirec...

crawlsafe.grindfree.net Crypto & Payments 13 calls / 30d checked 2026-11-18
0.001 USDC per call per call

robots.txt Checker | Lordevo x402 REST usage

robots.txt checker for one URL: may a generic agent and 16 named AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended...) fetch it under RFC 9309? Plus Crawl-delay, Content-Signal, X-Robots-Tag noai/noindex and...

shop.lordevo.com Web & Scraping 6 calls / 30d checked 2026-11-18
0.002 USDC per call per call

ah-ai-crawlers x402 REST usage

Given a website domain, reports which AI crawlers may fetch its home page under its robots.txt (RFC 9309): GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, Applebot-Extend...

x402.agentharbor.agency AI Models & Inference 1 calls / 30d checked 2026-11-18
0.01 USDC per call per call

Website Sitemap URLs x402 REST usage

Website sitemap URLs: list every page URL on a site from robots.txt and XML sitemaps (indexes, gzip, text). Up to 5000 URLs with lastmod/changefreq/priority, optional keyword filter; homepage-link fallback. Charged only...

ryumoon-x402-tools.ryumoon.workers.dev Geo & Maps 2 calls / 30d checked 2026-11-18
0.003 USDC per call per call

Grokzilla x402 Micro Pay x402 REST usage

Resolve ENS (.eth) on Ethereum and Basenames (.base.eth) on Base, forward or reverse. Returns address, name, source, forward_verified, optional avatar.

grokzilla.shop Crypto & Payments 2 calls / 30d checked 2026-11-18
0.003 USDC per call per call

Robots.txt check x402 REST usage

Robots.txt check. Read a site's robots.txt and say whether a crawler may fetch a URL, by RFC 9309 rules as Google applies them (most specific user-agent group, longest matching Allow or Disallow, Allow on ties, * and $ w...

agent-data-tools.revmesh2074.workers.dev Geo & Maps 5 calls / 30d checked 2026-11-18
0.001 USDC per call per call

Forest Web x402 REST usage

Convert one bounded fiat amount using the latest successfully fetched European Central Bank daily foreign-exchange reference table. Forest returns the ECB observation date and deterministic cross-rates through EUR. Fresh...

http--forest-gas-station-mainnet--lcjl27p8lmjs.code.run Crypto & Payments 4 calls / 30d checked 2026-11-18
0.001 USDC per call per call

Robots x402 REST usage

robots.txt parser: rules, sitemaps, AI-crawler blocks, and is-this-path-allowed check — Genesis402 / UnyKorn Operator Network

twin.unykorn.org Search & Research 1 calls / 30d checked 2026-11-18
0.003 USDC per call per call

x402 Forge x402 REST checked

How many working days between two dates in the Netherlands? Working days (NL): Counts business days in the Netherlands, skipping weekends and public holidays: for deadlines and delivery dates. $0.002 per call in USDC, ch...

forge.fizzl.eu Crypto & Payments 1 calls / 30d checked 2026-11-15
0.002 USDC per call per call checked

Related tasks