RE: Web bots - blocking the bad

["Horace Pinker" <webappsec () the86r com>]

Bob,
        Check this website out, as it offers up plenty of information on the
numerous bots out there and their primary functions.

http://www.robotstxt.org/wc/active/html/index.html

Horace

p.s. I may have inadvertently directly sent you this too, apologies



-----Original Message-----
From: Kelley Jr, Robert [mailto:rkelley () ritaohio com] 
Sent: Tuesday, August 19, 2003 2:30 AM
To: 'security-basics () securityfocus com'
Subject: Web bots - blocking the bad

Hello...In reviewing our IIS logs we are seeing many occurrences of web
robots scanning our site.  Like all things, I am seeing there are good
robots and bad robots. I'd like to know your thoughts on spending time
implementing measures to block the bad robots from scanning and indexing the
web-site. Your opinions with a short explanation are appreciated. 

bob

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.509 / Virus Database: 306 - Release Date: 8/12/2003
 


---------------------------------------------------------------------------
----------------------------------------------------------------------------