Skip to main content
All Seymour guides

How Seymour identifies itself, and how to opt out or slow it down

Seymour tells you who it is, asks permission first and keeps its pace gentle. Here is how to recognise it and how to change what it does.

How Seymour identifies itself

Every request carries this User-Agent:

SeymourBot/1.0 (+https://seemeplease.com/bot)
Code block

The address in brackets leads to the crawler information page. Seymour reads robots.txt before it reads anything else, and keeps a copy for 24 hours.

Opt out entirely

Add this to the robots.txt at the root of your site. Seymour will not visit.

User-agent: SeymourBot
Disallow: /
Code block

Block part of a site

Name the path you want left alone.

User-agent: SeymourBot
Disallow: /private/
Code block

Slow it down

Use Crawl-delay, in seconds. Seymour honours delays up to 2 minutes.

User-agent: SeymourBot
Crawl-delay: 30
Code block

Default limits

  • At most one request every 2 seconds per site, starting at one every 5 seconds and slowing down at once if the site responds slowly or returns errors.
  • One request in flight to a site at a time.
  • If a site asks Seymour to wait with Retry-After, it waits, up to 1 hour.
  • A kill switch can stop every crawl at once.

Get in touch

Questions, or a finding you want looked at again? Use the review request form or email hello@seemeplease.com.