Identification of the Pipeliner Internet robot. These include owner, description, HTTP user agent and whether this robot adheres to the robot exclusion standard.
| Owner |
|
| Country |
USA |
| Description |
Pipeline is a search engine and web portal. The result of searches are derived frrom the relevance of a PageRank style link algorithm, together with results clustering. The web sites found in the DMOZ directory are used to populate the Pipliner directory. |
| IP Addresses |
24.106.39.250 |
| HTTP User Agent |
pipeLiner/0.3a (PipeLine Spider; http://www.pipeline-search.com/webmaster.html; webmaster@pipeline-search.com) pipeLiner/0.12 (PipeLine Spider; http://www.pipeline-search.com/webmaster.html) |
| Exclusion |
The web site states that Pipeliner supports the robots.txt exclusion standard, which is described at http://www.robotstxt.org/wc/exclusion.html#robotstxt. For example to stop the robot from visiting the /cgi-bin directory :
User-agent: pipeLiner Disallow: /cgi-bin/ |
| Further Info. |
www.pipeline-search.com/webmaster.html
| |
NAT August 2005 (Revised)
| |
This article viewed: 1571 times |
Back |
Copyright © 2004-2009 Janet Systems Ltd.