WebApp Sec: Re: How to list all the URLs on a web server

[tie <tie () einet bg>]

Another way to find the URLs of hidden files is to gain access to the 
server logs or to some statistics derived from these logs.
More often than not webmasters generate web site statistics (HTML 
documents with graphics) and publish these statistics on the same site, 
using some easy to remember/guess name (for example,  
www.websites.com/stats/). Sometimes, these are password-protected, but 
attacking the passwords would be usually easier than trying to 
bruteforce filenames that you have no information of. These statistics 
usually contain a list of all accessed URLs.
Also, if you can get the application code, it will be fairly easy to get 
the exact URLs. Another option would be to run a sniffer on the same 
network segment where the application works.
Anyway, automated brute-forcing should be your last resort.

Regards,
tie

> > -----Original Message-----
> > From: Lists [mailto:sakaba () alexandria cc]
> > Sent: Friday, January 07, 2005 6:35 PM
> > To: webappsec () securityfocus com
> > Subject: How to list all the URLs on a web server
> >
> > Hi Everyone,
> >
> > I am auditing a system where files are stored on a web server and
> > accessed without authentication directly by an application that knows
> > each file URL.  I don't like it but the app owner wants me to
> > demonstrate that someone could guess the URLs.  I have tried a number
> > of spider tools but they are based on links so they don't pull up
> > anything.
> >
> > I am wondering if there is a tool or another method where I could find
> > out all the URLs on the web site.   The funny thing is I saw this same
> > kind of system with the same explanation just the other week at
> >    
> another
>  
>
> > company.  Maybe its a new trend...
> >
> > Regards,
> > sakaba
> >    
>
>
>
>