Home page logo
/

bugtraq logo Bugtraq mailing list archives

[Positive Technologies Research] Open Source WebEngine and Web Crawler v.0.2 is out!
From: aanisimov () ptsecurity com
Date: Wed, 25 Aug 2010 03:23:26 -0600

==============================================================

============= Positive Technologies Research Lab =============

==============================================================

         Open Source WebEngine and Web Crawler (Beta)

==============================================================


---[   Introduction

        Web Crawler is a utility designed for testing and demonstration of the WebEngine open source library features. 
This program gathers information about the resources of a specified web server by analyzing references in the HTML 
markup, text, and JavaScript code. Additionally, a query is sent to the Web Of Trust knowledge base to obtain 
information about the analyzed site. This check demonstrates analysis of web application vulnerabilities.

The main features provided by the application are listed below:

        - JavaScript analysis aimed at receiving references with simulation of a DOM structure
        - Access to the contents of web servers via HTTP
        - Support of the Basic, Digest, and NTLM authorization schemes
        - Operation via proxy servers with various authorization schemes
        - A wide variety of options to describe the scan target (lists of scanned domains, restriction of scanning to a 
host, a domain, or a web server directory, etc.)
        - Modular structure, which allows one to implement plug-ins


Download URL:
        http://code.google.com/p/webapptools/downloads/detail?name=Crawler_v0_2.zip

WebEngine LGPL library and other projects:
        http://webapptools.googlecode.com

Blog:
        http://www.ptresearch.ru
        http://ptresearch.blogspot.com



---[ Package Structure ]

        The package consists of two main components: the crawler utility and a XUL-based GUI. To display the GUI, one 
can use the Firefox browser or a specialized application (e.g. xulrunner or prism).
        The application root directory contains the utility binary files and the XUL configuration file 
(application.ini). The nested-directories structure is defined by the rules of formation of applications based on XUL. 
A user may be interested in the chrome/skin directory, which contains files describing the application appearance. The 
package offers several pre-installed themes. To change the appearance, it is sufficiently to replace the contents of 
the chrome/skin/classic directory with the chosen theme. A new theme can be created on the basis of an existing one or 
by modifying themes from the site http://jqueryui.com/themeroller/. The themes downloaded from this site should be 
supplemented with some images and CSS descriptions by analogy with the existing ones.



---[ ToDo ]

The Crawler utility and WebEngine library:

        - Tests for web application vulnerabilities
        - Improve the operation stability and performance
        - Broaden the JavaScript support
        - And a number of others

GUI:
        - Broaden the settings on the Settings tab to avoid necessity of manual utility configuration
        - Implement the feature of viewing log files with filtering and highlighting of the message levels



---[ About ]

        This utility was designed by the Positive Technologies Research Lab team within the bounds of development of a 
web application analyzer for the MaxPatrol system. The product is developed as open-source software according to the 
terms of the GNU Lesser General Public License. You can find the source code of the program and its components at 
http://webapptools.googlecode.com/.
        You can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published 
by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
        WebEngine library and Crawler utility is distributed in the hope that it will be useful, but WITHOUT ANY 
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU 
General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License along with this software.  If not, see 
http://www.gnu.org/licenses/.



---[ About Positive Technologies ]

        http://www.ptsecurity.com

        Positive Technologies is one of the leading companies in information security in Russia. The principle 
company's activities are: information security monitoring systems development (XSpider, MaxPatrol); consulting and 
services in IT security, SecurityLab special portal development.
        Positive Technologies products are certified by Ministry of Defense of the Russian Federation and Federal 
Service for Technical and Export Control (FSTEK Russia). Positive Technologies clients are more than 40 state 
institutes, more than 50 banks and financial structures, 20 telecommunication companies, more than 40 industrial 
enterprises, IT companies, service and retail companies from Russia, CIS, Baltic states, and also from the Great 
Britain, Germany, Holland, Israel, Iran, China, Mexico, USA, Thailand, Turkey, Ecuador, South African Republic, Japan.
        Positive Technologies is a team of highly qualified developers, consultants and experts with great practical 
experience that have professional titles and certificates, are the members of international organizations and actively 
take part in industry development.


  By Date           By Thread  

Current thread:
  • [Positive Technologies Research] Open Source WebEngine and Web Crawler v.0.2 is out! aanisimov (Aug 25)
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]
AlienVault