Identification of the Wget Internet robot. These include owner, description, HTTP user agent and whether this robot adheres to the robot exclusion standard.
| Owner |
|
| Country |
User dependant |
| Description |
Wget is a computer program which is used to retrieve files from the Internet. It is able to work non-interactively and may be run from a script or scheduled cron job. the original work was linux/unix based however, windows binaries are available. The program in itself is not malicious and may be a valuable tool. |
| IP Addresses |
User dependant |
| HTTP User Agent |
Wget/1.9+cvs-stable (Red Hat modified)
Wget/1.8.2 |
| Exclusion |
This robot supports the robots.txt exclusion standard, which is described at http://www.robotstxt.org/wc/exclusion.html#robotstxt. To stop the robot from visiting the /cgi-bin directory:
User-agent: wget
Disallow: /cgi-bin/
|
| Further Info. |
www.gnu.org/software/wget/wget.html |
NAT August 2005 (Revised)
| |
This article viewed: 2794 times |
Back |
Copyright © 2004-2009 Janet Systems Ltd.