Details for the Falcon Internet robot. Details for this robot include owner, description, HTTP user agent and whether this robot adheres to the robot exclusion standard.
| Owner |
|
| Country |
|
| Description |
|
| IP Addresses |
209.133.117.28 |
| HTTP User Agent |
falcon/1.0 |
| Exclusion |
It is not known whether this robot supports the robots.txt exclusion standard, which is described at http://www.robotstxt.org/wc/exclusion.html#robotstxt. If this robot did support the protocol then we would expect to include something like the following in our robots.txt file, to stop the robot from visiting the /cgi-bin directory:
User-agent: falcon
Disallow: /cgi-bin/ |
| Further Info. |
|
NAT August 2005 (Revised)
| |
This article viewed: 1596 times |
Back |
Copyright © 2004-2009 Janet Systems Ltd.