There is a "somewhat" academic paper on the topic of IDS testing and
automating it:
http://thor.cryptojail.net
The paper is at: http://thor.cryptojail.net/thor.ps.gz
Let me know if you have any questions...
-raffy
> On 11/15/06, Joseph McCray <joe_at_learnsecurityonline.com> wrote:
> >Have any of you ever taken the time to develop a list signatures and
> >their corresponding tools and/or exploits that actually trigger every
> >individual signature the IDS has?
>
> I have not seen a comprehensive open toolkit. I know of the Mucus
> project that is based on sneeze to generate false positives in snort
> by creating attacks from snort rules. (sensorynetworks.com) I'm
> personally not in favor of an automated approach to generating attacks
> as I feel they miss too much, but this is the best way today to cover
> lots of the attacks. I have not played with the whole coremark toolkit
> that mucus is a part of, but it is supposed to do some of what you are
> asking.
>
> I am also hopeful that the openpacket.org project will be able to
> collect traffic samples that can be manipulated with tcpreplay to
> provide testing in many situations.
>
> >I'm really looking for input, tips, ideas of ways to automate a lot of
> >the testing. I'm looking for this especially for exploits - how did you
> >systematically handle things like:
>
> I think that the proper way to handle this is through software testing
> mechanisms. What is required is to extend the typical unit test into a
> more functional realm. There are already some tools heading in this
> direction within perl's Test::Module space.
>
> > 1. running a specific tool/exploit with varying parameters passed
> > to it
> perl.
> > 2. verifying that the system was actually exploited
> Perl's Test::Tail::Multi on CPAN.
> > 3. verifying that the IDS alert actually triggered
>
> Perl's Test::Net::Snort, caught in my employer's open source release
> process, but looks good to be cleared once all the signoffs are
> accomplished.
>
> > 4. verifying that the service is still running after the attempted
> >exploit (restart the service/reboot the box if needed)
>
> Perl's Test::Net::Connect can do rudimentary checking.
> Test::Tail::Multi may also be useful.
>
> > 5. correlating 1, 2 and 3 above
>
> Perl's Test::Distributed, a module that does not exist but I've
> sketched an outline for and have several of the necessary components
> working.
>
> My focus has really been on the IDS testing and not the exploit
> itself. My needs so far have not taken me to the point where the reply
> to an exploit would be the signature response.
>
> I think that if testing the IDS is what one is after, that one may be
> better off setting up the victim to fake being exploited (or not)
> instead of actually providing an exploitable system. Otherwise, one
> needs a whole bunch of VM's running all different kinds of OSs,
> configurations, patch levels and applications.
>
> >I'm thinking a ton of scripting is going to be the secret here. Any
> >ideas and feedback would be greatly appreciated.
>
> Whatever means that you decide to use, here are some important
> considerations:
>
> 1) It should be modular.
> 2) Borrow as much as possible from existing testing frameworks such as
> JUnit or Perl.
> 2a) Spend time writing tests and not reporting systems.
> 3) Tag each test with CVE or something similarly standardized.
> 4) XML makes data portable.
> 5) Realtime is better than time synchronization and post attack log
> matching. I'm using Jabber for an out-of-band control channel.
>
> I hope that give you some useful ideas.
>
> Peace,
> Eric Hacker, CISSP
>
> aptronym (AP-troh-NIM) noun
> A name that is especially suited to the profession of its owner
>
> I _can_ leave well enough alone, but my criteria for well enough is
> pretty darn high.
>
> ------------------------------------------------------------------------
> This List Sponsored by: Cenzic
>
> Need to secure your web apps?
> Cenzic Hailstorm finds vulnerabilities fast.
> Click the link to buy it, try it or download Hailstorm for FREE.
> http://www.cenzic.com/products_services/download_hailstorm.php?camp=701600000008bOW
> ------------------------------------------------------------------------
>
--
Raffael Marty, GCIA, CISSP raffael.marty_at_arcsight.com
Manager Strategic Application Solutions
ArcSight, Inc. +1 (408) 864 2662
------------------------------------------------------------------------
This List Sponsored by: Cenzic
Need to secure your web apps?
Cenzic Hailstorm finds vulnerabilities fast.
Click the link to buy it, try it or download Hailstorm for FREE.
http://www.cenzic.com/products_services/download_hailstorm.php?camp=701600000008bOW
------------------------------------------------------------------------
Received on Nov 20 2006
Intro
