spacer
Janet Systems DotNetNuke Websites, Hosting, Custom Servers
Tag Words
spacer
Web Robot Articles

Identification of the Pipeliner Internet robot. These include owner, description, HTTP user agent and whether this robot adheres to the robot exclusion standard.

Owner
Country USA
Description Pipeline is a search engine and web portal. The result of searches are derived frrom the relevance of a PageRank style link algorithm, together with results clustering. The web sites found in the DMOZ directory are used to populate the Pipliner directory.
IP Addresses 24.106.39.250
HTTP User Agent pipeLiner/0.3a (PipeLine Spider; http://www.pipeline-search.com/webmaster.html; webmaster@pipeline-search.com)
pipeLiner/0.12 (PipeLine Spider; http://www.pipeline-search.com/webmaster.html)
Exclusion

The web site states that Pipeliner supports the robots.txt exclusion standard, which is described at http://www.robotstxt.org/wc/exclusion.html#robotstxt. For example to stop the robot from visiting the /cgi-bin directory :

User-agent: pipeLiner
Disallow: /cgi-bin/

Further Info. www.pipeline-search.com/webmaster.html

NAT August 2005 (Revised)

  This article viewed: 1571 times Back

Copyright © 2004-2009 Janet Systems Ltd.

    spacer
    Print