ZeekleBot

ZeekleBot is the small, focused crawler behind Zeekle, a directory of AI tools. It looks for AI products on the public web and checks that listed tools are still online.

User agent

ZeekleBot/1.0 (+https://zeekle.ai/bot)

How it behaves

• Obeys robots.txt, including Crawl-delay.
• At most 1 request per second to any one site, in small batches.
• Only public pages: no logins, accounts, checkouts or private areas.
• Never crawls search engine result pages.
• Stores a page's title, a short text excerpt, its links and the fetch date — never personal data.
• Re-checks listed tools about once a week.

How to opt out

Add this to your robots.txt. ZeekleBot reads it before crawling your site and stops on its next visit:

User-agent: ZeekleBot
Disallow: /

Want your tool's listing corrected or removed? Use "Report a problem" on its page.