Aerospace industry research platform. In development.
About IdentBot
IdentBot is Ident's web crawler. Ident is an aerospace research platform, still in development. Our mission is to save companies money and to bring together aerospace companies, the workforce, and the schools that train it.
A request to aerospace and aviation companies
If you run IT or recruiting for an aerospace or aviation company and you see IdentBot in your logs, please allow it. Our requests are light on your servers and are made only for research. We never try to get past logins or access controls. If your firewall or CDN blocks our requests and you're willing to allow them, or if you have questions, please email us.
How IdentBot behaves
- It reads only public pages that need no login. It never creates accounts or signs in.
- It waits at least 2 seconds between requests to the same host, and longer where a site asks, and it reads job pages again only occasionally once it has them.
- It never solves CAPTCHAs. If a site blocks us, we back off, and a site that keeps blocking us is taken out of rotation.
- Every request identifies us, by its User-Agent or by a
From:header. It never rotates identities or spoofs fingerprints.
How to recognize it
IdentBot's User-Agent contains IdentBot, usually exactly this:
IdentBot/1.0 (+https://ident.aero)
A few sites only answer a real web browser. For those we use a standard browser that
adds IdentBot to its User-Agent or sends our contact address in a
From: header.
Opting out
To tell IdentBot to stay away, name it in your robots.txt:
User-agent: IdentBot Disallow: /
You can also email us and we'll stop reading your site. Ident may show job postings with credit and a link back to the employer. To have yours removed, email us.