Re: Web App URL Scanner

[Jon Hart <warchild () spoofed org>]

On Mon, Oct 13, 2003 at 09:34:37PM -0500, Jimi Thompson wrote:
> All,
>
> I'm currently seeking some software that will test all possible URL's 
> on an web application, much like a dictionary attack against a 
> password.  I could probably write it but I'd rather just download 
> something if I can.  I'd like to see if I'm able to discover URL's 
> that aren't normally accessible.  If anyone has ideas, I'd be 
> grateful.

In addition to the responses you've already gotten (nikto, webscarab,
and spikeproxy), you might want to try out a script I wrote when I was
in a similar situation:

        http://spoofed.org/files/termite.pl

termite requires that you provide it with some potentially interesting
names, then it will do the leg work for you and see if it can discover
where a file similar to that might be hiding on a website by doing
things like checking common directory names (i.e., /cgi-bin/, /bin/),
common CGI extensions (i.e., .pl, .asp, .cgi), file renaming and more.
It can be fairly nosey, so you may want to tweak it to your specific
needs.

What tool you use really depends on what you mean by "test all possible
URLs on a web application", but I think with the tools you've been
pointed at so far you'll be well on your way.  I think you'll also find
that there may not be a single tool that does everything you need, so
it'd be best if you try to find the best combination of tools and keep
them at the ready.

Hope that helps,

-jon