CONTACT EXTRACTOR / CRAWLER
About the ContactExtractor crawler
Requests with the user agent ContactExtractor/1.0 come from Contact Extractor, a tool that reads the contact details businesses publish on their own websites (contact, imprint and about pages) on behalf of our users.
- We fetch a handful of pages per site (8 at most by default), a few at a time, with timeouts and size limits.
- We honour
robots.txt. To opt out, addUser-agent: ContactExtractorwithDisallow: /. - We do not log in, submit forms, solve challenges or bypass bot protection. If a site blocks us, we stop.
- We never fetch private or internal network addresses.