mailing list archives
Re: Web crawling library proposal
From: Paulino Calderon <paulino () calderonpale com>
Date: Mon, 07 Nov 2011 11:16:47 -0800
Thank you all for the feedback. You've pointed out some very important
issues. I've started working on the following changes:
* OO design.
* crawl() must return a iterator.
* Allow multiple crawlers to run at the same time.
* Remove anchors from links by default instead of having to set the
* Queue will be removed.
* Implement a better uri filter.
Paulino Calderón Pale
Sent through the nmap-dev mailing list
Archived at http://seclists.org/nmap-dev/