Home page logo

pen-test logo Penetration Testing mailing list archives

RE: Spidering
From: "Thor (Hammer of God)" <thor () hammerofgod com>
Date: Fri, 18 Jan 2008 11:58:45 -0800

This doesn't answer your question, but in regard to the validity of the technique, understand that an easy way to 
thwart these efforts (as far as IIS is concerned) is to edit your ISAPI extensions and have the ASP.DLL parse your 
.HTM(L) pages as well and just rename your .asp pages to .htm -- that way they will provide dynamic content the same 
way, but will have .htm extensions (in IIS7, you'll use a handler mappings) so that people will not (immediately) know 
they are dynamic pages by extention...


-----Original Message-----
From: listbounce () securityfocus com
[mailto:listbounce () securityfocus com] On Behalf Of me me
Sent: Thursday, January 17, 2008 9:59 AM
To: pen-test () securityfocus com
Subject: Spidering

Hi All

One of the most important aspects of assessing a web application is
separating the dynamic code from the static HTML. I tend to do this
spidering, and use a number of tools including Paros and a commercial

Basically, I like to know:

The names and places of the scripts
The variables they take
The Comments etc in HTML
Hidden form values

All the usual.

Whilst I don't expect it to get everything (JavaScript etc is going to
manual intervention, so is a number of other possible technologies), I
never really found a tool that I consider to be the defacto spidering
from this perspective. One of the biggest problems is a lot of the
seem to choke on really big sites, or go into infinite loops  etc etc.

Has anyone got a reliable spider that they've used time and again (even
big sites) that they could recommend? Or are most people WGETing the
and then regex'ing the local mirror?

Thanks in advance

This list is sponsored by: Cenzic

Need to secure your web apps NOW?
Cenzic finds more, "real" vulnerabilities fast.
Click to try it, buy it or download a solution FREE today!


  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]