Details for the Asterias Internet robot. Details for this robot include owner, description, HTTP user agent and whether this robot adheres to the robot exclusion standard.
| Owner |
Singingfish.com |
| Country |
USA |
| Description |
The robot gathers multimedia information for the web site Singing Fish. Some of its information os fed into the Real Player search. In the early days of this search engine the robot gained a poor reputation. You may feel that it is worth accepting the robot to promote your site content. |
| IP Addresses |
|
| HTTP User Agent |
asterias/2.0 asterias Crawler v4; +http://www.singingfish.com/help/spider.html |
| Exclusion |
It looks like their this robot doesn't support the robots.txt exclusion standard, which is described at http://www.robotstxt.org/wc/exclusion.html#robotstxt. You may want to try setting an entry in the robots.txt file. To stop the robot from visiting the /cgi-bin directory:
User-agent: asterias Disallow: /cgi-bin
|
| Further Info. |
www.singingfish.com/help/spider.html |
NAT August 2005 (Revised)
| |
This article viewed: 5145 times |
Back |
Copyright © 2004-2009 Janet Systems Ltd.