[NSE] Robots rethink

["Eddie Bell" <ejlbell () gmail com>]

Hi all,

I've had this new robot script lying around for a while. The current
version prints out all disallowed entries in a site's robots.txt file
which takes up a lot of screen estate[1]. The attached version just
prints a message if a site has a robots.txt file with disallowed
entries[2].

What do you think?

 - eddie

[1] 80/tcp open  http
     |  robots.txt: /news?output=xhtml& /search /groups /images
     |  /catalogs /catalogues /news /nwshp /? /addurl/image?
     |  /pagead/ /relpage/ /relcontent /sorry/ /imgres
     < ... this goes on for another 26 lines ... >

[2] 80/tcp open  http
     |_ robots.txt: has disallowed entries

Attachment: robots.nse
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org