SeismologBot

SeismologBot is the crawler of Seismolog, a public, append-only log of how websites publish their standards for agent discovery (DNS-AID, UCP, A2A agent cards, MCP server cards, Agent Skills, OAuth metadata, Web Bot Auth, payment declarations, DNSSEC, DANE). It records what a site publishes and when that changes. It does not score, rank or judge anyone.

It identifies itself like this:

User-Agent: Mozilla/5.0 (compatible; SeismologBot/1.0; +https://seismolog.org/bot)

What it fetches

• Plain HTTPS GET requests for a small, fixed set of public resources, such as /robots.txt, /.well-known/ucp, agent cards, MCP server cards and OAuth metadata. A visit is 28 requests to a site that publishes none of these, a few more to one that lists MCP server cards or paid routes, and 56 to one that redirects its domain to its own www host (see below). Most domains get one visit a week (about 30 requests, about 60 with that redirect); domains with a DNSSEC-signed zone daily (about 200 a week, about 400); sites that publish agent signals every six hours (about 800 a week, about 1,600).
• Never anything else: no login, no cookies, no JavaScript, no crawling of links, and no POST except the read-only MCP handshake. For MCP endpoints a site declares itself it sends initialize and tools/list and never calls a tool. Registration and payment endpoints are never called.
• It follows no redirects, except for robots.txt as RFC 9309 requires (at most five), and a domain's redirect to its own www host: if https://example.com/path answers 301, 302, 307 or 308 with exactly https://www.example.com/path (same path and query), it makes one more request there, after reading the robots.txt of www.example.com and only if that allows the path. It follows nothing else and never a second redirect.
• Sites that publish agent signals are visited every six hours at the most, others daily or weekly, in random order, and never faster than two requests per second to one host.
• It also asks the public DNS about the same domains (for example SVCB and DNSSEC records).

What it keeps

When what a site publishes changes, Seismolog adds an entry to its public log. The entry holds only metadata and hashes, never the content itself. The exact request and response behind the entry are kept as evidence (a WARC file) so that anyone can check the entry later.

How to block it

Use robots.txt only, as specified in RFC 9309. The product token is SeismologBot:

User-agent: SeismologBot
Disallow: /

• If no group names SeismologBot, the * group applies. Rules for single paths apply to those paths only.
• robots.txt is fetched at most once a day. A 4xx answer means there is no file and everything is allowed. If it cannot be read (5xx, timeout, too many redirects), SeismologBot treats everything as disallowed and fetches nothing else from that site in that round.
• Blocking ends all HTTP requests except for robots.txt itself. DNS lookups continue, because they are queries to public DNS and not requests to your server. History already in the log stays, and the block itself is recorded there.

IP addresses

SeismologBot sends all requests from fixed addresses. Our DNS resolver and our transparency log (its witness requests) use the same addresses; those are not SeismologBot requests and carry no signature. The list is machine-readable at https://seismolog.org/bot/ranges.json (format of the IETF draft JAFAR, refreshed at least daily):

2.31.48.98/32

The reverse DNS name of these addresses is egress1.seismolog.org, which resolves back to them.

Signed requests

Every request of SeismologBot is signed with Web Bot Auth (HTTP Message Signatures, RFC 9421), so nobody else can pose as SeismologBot. A request carries:

Signature-Agent: sig1="https://seismolog.org"
Signature-Input: sig1=("@authority" "signature-agent";key="sig1");created=...;keyid="Swvaun1uOgF8ET7PziTOJPfO_Ag9wqHnykySJMup_CQ";alg="ed25519";expires=...;nonce="...";tag="web-bot-auth"
Signature: sig1=:...:

To verify one, fetch the key directory at https://seismolog.org/.well-known/http-message-signatures-directory (media type application/http-message-signatures-directory+json), take the key whose thumbprint (RFC 7638) is the keyid and check the signature over the @authority and the Signature-Agent member as RFC 9421 describes. The directory response is itself signed by its key. The current key id is Swvaun1uOgF8ET7PziTOJPfO_Ag9wqHnykySJMup_CQ. Signatures are valid for five minutes.

Contact

Questions, complaints and requests concerning personal data: ops@seismolog.org.