Re: BlackICE IDS

[robert_david_graham () yahoo com (Robert Graham)]

> This is not a Windows NT vs UNIX religious debate.  You'll note that in
> my post I did not mention UNIX even once.  

Sorry. I misinterpreted your comment: 
> > > "I mean, if the NFR IDA can't do 140k packets a second, how do you expect
some Windows system to perform?" 
I assumed you were talking about "Windows", when you really meant "IDA" vs.
"off the shelf". My appologies -- I come from a UNIX background where the
religious issue is common.

> ... deploying a host based IDS across their
> enterprise.

Again, to clarify a misconception: BlackICE is both host-based and
network-based. Most customers mix the two, using desktop agents where
appropriate, and putting "sentry" probes in traditional places. The host-based
agent runs in non-promiscuous mode and never processes high traffic rates;
indeed, it has a smaller CPU/memory footprint that competing personal
firewalls.

> I will maintain one point:  Any Network Security Appliance should
> outperform any operating system + software solution.  ...

Appliance vs. generic is an interesting debate, one that should probably be put
off onto a different thread.

Traditionally, the packet capture component has been the primary bottleneck in
network IDS. One of the problems is how often you have to copy the packet.
Today's PCs have less than 400-MBytes/second memory bandwidth, so a single
frame copy at 100-mbps speeds is about 25-MBytes/second (12.5-meg in/out). I've
heard that Linux does 3-copies, which eats up about a quarter of your
CPU/memory doing just copying. At the same time, the cache gets flushed (which
is why the newer Pentium-IIIs have "streaming" extensions to avoid that).

It is my understanding (Marcus?) that the NFR "appliance" does NO copying by
the CPU, and just has the adapter DMA the packets into memory. In contrast,
Network ICE does a single copy into its own buffers. This is 12.5-MBytes/sec
vs. 37.5-MBytes/sec memory bandwidth eaten up, but is even worse when you
consider the fact that it's the CPU doing the copying, rather than  going off
doing its own stuff out of cache while the adapter DMAs in the background.

Thus, you have a clear demonstration of the benefits of an appliance over a
generic OS. However, as I demonstrated in my previous e-mail, packet capture
wasn't the limiting factor in our dual-CPU configuration. Appliance style
point-optimizations wouldn't make much difference in our case, unless we were
running on a single-CPU system.

In any case, the Nov. 15 Networking Computing has a performance chart of
BlackICE v1.0 vs. the NFR v4.0 IDA:
http://www.nwc.com/1023/1023f19.html
Your original statement was "I mean, if the NFR IDA can't do 140k packets a
second, how do you expect some Windows system to perform?" with the implicit
assumption that Windows couldn't possibly be faster than an imbedded appliance,
but this test showed the opposite. Now lots of errors creep into magazine
reviews, so I'm not claiming BlackICE is significantly faster than NFR (I
haven't run it myself to be sure), but I'll bet it isn't any slower to any
significant degree. (Also note: this test was on a single CPU system, not my
dual-CPU tweaked test-bed).

> Further, do you care to share your traffic file so we can all start
> talking around the same data?

I would if I could. I should have saved it.

>  I for one would like to see how other
> technologies perform with the same set of traffic.  Surely you, as one
> of the founders, don't mind sharing your data so we can all start
> comparing apples to apples.  If anyone want's to look at a point of
> comparison, my company offers some standard tcpdump formatted logs from
> SANS NS'99s IDNET contest.  They can be found at
> http://www.hiverworld.com/snortlogs -- although, there's a lot of really
> strange stuff in these logs, so YMMV.

You can run the "blackd.exe" program directly from tracefiles. The blackice
daemon that is the executable for all the products, so you can use the $40
Defender (home-user) version from our website. Simple do "blackd -r
snort-1.log" (where "snort-1.log" is the uncompressed tracefile from your
website). Unfortunately, figuring out the performance info from this is
problematic because:
1. it'll take longer to initialize and shutdown than read the files; they're
only about 150-megabytes in size.
2. unless you have a high-speed DMA RAID system, disk speed will be the
dominating factor
3. the files contain almost pure attacks, which means your testing eventlogging
speed more than packet capture / analysis speed. (Interesting by itself, but
not the point).

> It's
> just that I'm having a really hard time with host based performance 
> being sold as a neat little package that runs on the desktop in 
> addition to all of the productivity applications that are also running
> and are required for the desktop user to perform his/her job properly.

Sorry about the confusion. As I clarified above, the host-based agent uses so
few resources you can hardly tell it's there. At the last Interop, one of our
home users came up and raved about it. He is a die-hard Quake fan, and they are
VERY concerned about even slight variations in performance.

Regards,
Rob.

=====
Robert Graham
"Anxiously awaiting the millenium so I can start programming
dates with 2-digits again."
__________________________________________________
Do You Yahoo!?
Thousands of Stores.  Millions of Products.  All in one place.
Yahoo! Shopping: http://shopping.yahoo.com