mailing list archives
[Positive Technologies Research] Open Source WebEngine and Web Crawler v.0.2 is out!
From: aanisimov () ptsecurity com
Date: Wed, 25 Aug 2010 03:23:26 -0600
============= Positive Technologies Research Lab =============
Open Source WebEngine and Web Crawler (Beta)
Web Crawler is a utility designed for testing and demonstration of the WebEngine open source library features.
This program gathers information about the resources of a specified web server by analyzing references in the HTML
information about the analyzed site. This check demonstrates analysis of web application vulnerabilities.
The main features provided by the application are listed below:
- Access to the contents of web servers via HTTP
- Support of the Basic, Digest, and NTLM authorization schemes
- Operation via proxy servers with various authorization schemes
- A wide variety of options to describe the scan target (lists of scanned domains, restriction of scanning to a
host, a domain, or a web server directory, etc.)
- Modular structure, which allows one to implement plug-ins
WebEngine LGPL library and other projects:
---[ Package Structure ]
The package consists of two main components: the crawler utility and a XUL-based GUI. To display the GUI, one
can use the Firefox browser or a specialized application (e.g. xulrunner or prism).
The application root directory contains the utility binary files and the XUL configuration file
(application.ini). The nested-directories structure is defined by the rules of formation of applications based on XUL.
A user may be interested in the chrome/skin directory, which contains files describing the application appearance. The
package offers several pre-installed themes. To change the appearance, it is sufficiently to replace the contents of
the chrome/skin/classic directory with the chosen theme. A new theme can be created on the basis of an existing one or
by modifying themes from the site http://jqueryui.com/themeroller/. The themes downloaded from this site should be
supplemented with some images and CSS descriptions by analogy with the existing ones.
---[ ToDo ]
The Crawler utility and WebEngine library:
- Tests for web application vulnerabilities
- Improve the operation stability and performance
- And a number of others
- Broaden the settings on the Settings tab to avoid necessity of manual utility configuration
- Implement the feature of viewing log files with filtering and highlighting of the message levels
---[ About ]
This utility was designed by the Positive Technologies Research Lab team within the bounds of development of a
web application analyzer for the MaxPatrol system. The product is developed as open-source software according to the
terms of the GNU Lesser General Public License. You can find the source code of the program and its components at
You can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published
by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
WebEngine library and Crawler utility is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with this software. If not, see
---[ About Positive Technologies ]
Positive Technologies is one of the leading companies in information security in Russia. The principle
company's activities are: information security monitoring systems development (XSpider, MaxPatrol); consulting and
services in IT security, SecurityLab special portal development.
Positive Technologies products are certified by Ministry of Defense of the Russian Federation and Federal
Service for Technical and Export Control (FSTEK Russia). Positive Technologies clients are more than 40 state
institutes, more than 50 banks and financial structures, 20 telecommunication companies, more than 40 industrial
enterprises, IT companies, service and retail companies from Russia, CIS, Baltic states, and also from the Great
Britain, Germany, Holland, Israel, Iran, China, Mexico, USA, Thailand, Turkey, Ecuador, South African Republic, Japan.
Positive Technologies is a team of highly qualified developers, consultants and experts with great practical
experience that have professional titles and certificates, are the members of international organizations and actively
take part in industry development.
- [Positive Technologies Research] Open Source WebEngine and Web Crawler v.0.2 is out! aanisimov (Aug 25)