mailing list archives
From: Jacek Wielemborek <d33tah () gmail com>
Date: Sun, 05 Jan 2014 16:13:48 +0100
01/01/2014 19:15:11 Jacek Wielemborek <d33tah () gmail com>:
On the CCC I was talking with David about rewriting Ncat test engine so that
the tests would run concurrently, speeding up the testing process. I
suggested a rewrite to Python and heard that it's a good idea. I already
created the branch and I wanted to ask you guys if you have some special
expectations for the testing engine, apart from concurrency. For one, I
have an idea to make all tests have an indentifier (and/or a group) to
allow stuff like:
What do you think?
I did a bit of experimentation and wrote a first prototype of ncat-test.py.
It's currently just a proof-of-concept and has some kludges, but I wanted to
ask you guys if it's generally the direction we'd like to go for.
In case you wanted to take a look at the code before reading the rest of the
e-mail, it's in the SVN (nmap-exp/d33tah/ncat-test-py). You can also see it on
Github as I keep these two repositories in sync - the core (and currently the
only) file can be found here:
My Python demo has currently two tests implemented - one being the first one
from ncat-test.pl, which spawns "ncat -l" and tries to connect to it over IPv4
and IPv6, also using Ncat. The second test tests UDP IPv6 in pretty much the
same way. These two tests run concurrently. Tested under Python 2.6.6 (which
AFAIK can still be found on latest CentOS Linux), 2.7.5 and 3.3.2 under Fedora
19 (Linux). It seems to work under FreeBSD as well.
There's no Windows/Cygwin support at the moment - do_read function would have
to be rewritten to support non-blocking operations there and apparently Cygwin
doesn't support Posix version well (switching subprocess's stdout to a non-
blocking mode prevented me from terminating the process later). Windows
support is currently not a priority for me, please let me know if you consider
this a show-stopper.
As for the code, I wanted to make the tests look similar to ncat-test.pl. The
general idea is that we create separate functions for each test, adding the
@ncat_test generator which is supposed to register the test to the TESTS
global variable and add some metadata, such as the test name and whether it's
expected to fail.
Then, once run_tests(), the main function, is run, we spawn a fixed number of
worker threads that share a queue. On the main thread, we distribute the tests
on the queue and the threads will receive them. This has the nice advantage
that we can configure the maximum number of tests that can be run
simultaneously. Since the tests_worker() and handle_test(), it will be trivial
to convert this code to a single-threaded mode, which could prove useful while
My main question is whether you like this approach. Before I'll start porting
all the test cases to this new program, I'd prefer to find out whether there's
anything potentially fundamental that should be changed. For example, folks
from the #python IRC channel frowned upon my ScopedPopen class and suggested
that I should use Twisted instead. I don't know Twisted very well and while it
would probably solve some problems related to non-blocking reading, it would
also require basically a complete rewrite of this program. This is why I'd
like to know your opinion on this.
For the ones that would like to help me a bit and test it, currently you just
need ncat somewhere in your $PATH and grab ncat-test.py and run it:
Here's the expected output:
$ python ncat-test.py
SUCC: Server default listen address --udp IPV6
SUCC: Server default listen address and port IPv4
2 tests ran, 100.00% success rate (2 SUCC, 0 FAIL, 0 UNEXSUCC, 0 XFAIL)
So, does it work for you? Do you have any comments on your mind? I'd be happy
to hear any feedback :)
Description: This is a digitally signed message part.
Sent through the dev mailing list
Archived at http://seclists.org/nmap-dev/
- ncat-test.py? Jacek Wielemborek (Jan 01)
- Re: ncat-test.py? Jacek Wielemborek (Jan 05)