Kampalo
العربية

KampaloBot

Short answer

KampaloBot is the crawler behind Kampalo's SEO reports. It reads public pages on sites our customers track, their own site and competitors they add, to spot new pages and changed titles. It follows robots.txt, waits between requests, visits each site about once a week and stops if a site asks it to slow down.

Last reviewed

How to recognise it

KampaloBot sends this user agent with every request:

KampaloBot/1.0 (+https://kampalo.com/bot)

In robots.txt it answers to the token KampaloBot.

What it does

Kampalo customers use our SEO suite to watch their own website and the competitor websites they choose. KampaloBot visits those sites so we can report things like:

  • New pages and pages that were removed
  • Changes to page titles, meta descriptions, H1 headings and noindex settings
  • Which topics a site covers, from its page titles and headings

It only requests public HTML pages, robots.txt and sitemaps. It does not log in, fill in forms, run JavaScript, or download images, PDFs or other files.

How it behaves

  • Follows robots.txt (RFC 9309). If robots.txt cannot be read because of a server error, it treats the whole site as off-limits and tries again on a later visit.
  • One request at a time per site, at least 1.5 seconds apart. If your robots.txt sets a Crawl-delay, it waits that long instead, up to 30 seconds.
  • Backs off. A 429 or 503 response stops the visit straight away, and so do repeated 403 responses.
  • Visits rarely. Each site is crawled about once a week. A customer can ask for a fresh crawl, but we allow that at most once a day per site.
  • Keeps visits small. At most 300 pages per visit. It sends If-None-Match and If-Modified-Since, so pages that haven't changed cost a short 304 response.

What we keep

For each page we store its URL, title, meta description, headings, word count, canonical URL, language and structured-data types. We do not store the full text of your pages; we keep a fingerprint of the text only to notice when it changes. These details are shown only to the Kampalo customers who track your site.

How to block or limit it

To stop KampaloBot visiting any page on your site, add this to your robots.txt:

User-agent: KampaloBot
Disallow: /

To keep it out of one section and slow it down:

User-agent: KampaloBot
Disallow: /private/
Crawl-delay: 10

KampaloBot picks up robots.txt changes at the start of its next visit.

Contact us

If KampaloBot is causing problems on your site, or you want it to stop visiting, email us and include your domain. We will look into it.

Tekreign (parent company of KAMPALO)

Email: connect@kampalo.com

Privacy Policy