spacer
Janet Systems DotNetNuke Websites, Hosting, Custom Servers
Tag Words
spacer
Web Robot Articles

Identification of the MSNBot Internet robot. These include owner, description, HTTP user agent and whether this robot adheres to the robot exclusion standard.

Owner MSN Search
Country USA
Description

MSNBot is a prototype web-crawling robot. It is used to collect documents from the web to build a searchable index. Currently MSN still use Inktomi for their search content.

IP Addresses 65.54.188.143
65.54.188.146 
207.46.98.42
HTTP User Agent msnbot/0.11 (+http://search.msn.com/msnbot.htm)
msnbot/0.3 (+http://search.msn.com/msnbot.htm)
msnbot/1.0 (+http://search.msn.com/msnbot.htm)
Exclusion This robot supports the robots.txt exclusion standard, which is described at http://www.robotstxt.org/wc/exclusion.html#robotstxt. To stop the robot from visiting the /cgi-bin directory:

User-agent: MSNBot
Disallow: /cgi-bin/

Note, it does not look at the robots.txt file at each visit, but rather downloads the details every day. For this reason it may take a while for the robot to conform to the robots.txt file if changes have been made.

Further Info. www.msnsearch.com/msnbot.htm

NAT August 2005 (Revised)

  This article viewed: 2447 times Back

Copyright © 2004-2009 Janet Systems Ltd.

    spacer
    Print