Home page logo
/

nmap-dev logo Nmap Development mailing list archives

Re: Web crawling library proposal
From: Paulino Calderon <paulino () calderonpale com>
Date: Mon, 07 Nov 2011 11:16:47 -0800

Hi list,

Thank you all for the feedback. You've pointed out some very important issues. I've started working on the following changes:
* OO design.
* crawl() must return a iterator.
* Allow multiple crawlers to run at the same time.
* Remove anchors from links by default instead of having to set the variable httpspider.ignoreParams.
* Queue will be removed.
* Implement a better uri filter.

Cheers!

--
Paulino Calderón Pale
Web: http://calderonpale.com
Twitter: http://www.twitter.com/paulinocaIderon

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/


  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]
AlienVault