Home page logo

firewall-wizards logo Firewall Wizards mailing list archives

Re: firewall stress testing tool
From: "Dave Diehl" <thogogorog () gmail com>
Date: Fri, 19 May 2006 23:13:24 -0500

My experience as a network manager and later a firewall developer is that
the only "stress testing tool" that can tell you much about how a system
will hold up under load is something designed to mimic that load as closely
as possible.  How much traffic are you expecting?  How much data per session
on average, and with what distribution?  Spread out over how much time?
Evenly distributed or in surges?  Do you expect to see a large number of
idling persistent connections?  What kind of variety and distribution of
source and destination addresses will be involved?  All of the issues I
mention here and more will have an effect on the performance of even a
trivial stateful inspection firewall (and I didn't even start on the
charateristics of the undesired traffic).  The number of relevant variables
goes up quickly when you instead consider a firewall that does something
more than let you check the "firewall" box in your annual audit.

Of course, as Marcus implies, since the firewall you describe isn't really
doing much of anything, it has a pretty good chance of being stable under


On 5/19/06, Marcus J. Ranum <mjr () ranum com> wrote:

pavan shah wrote:
I have configured windows 2003 server to allow only traffic to port
80.I want to check for the stability of the firewall under heavy load.
Could any one suggest any firewall stress testing tool?

There aren't any decent firewall stress testing tools out there.
real network traffic would be the ideal test-bed. Second to that would
be replays of packets captured at a real firewall installation.

Using something like a smartbits is pointless because they're generating
synthetic traffic, which would make the firewalls that do any layer 7
processing look worse (from a performance standpoint) than the firewalls
that are doing only "stateful inspection" or "deep inspection"  We saw
a lot of cooked benchmarks early in the IDS days where unscrupulous
vendors posted unrealistically high performance numbers for IDS packet
capture by using synthetic traffic that the IDS "knew" to discard. There
was the famous intrusion.com benchmark done by Meir Communications
in which intrusion.com demonstrated gigabit speed IDS with no packet
loss - as long as you threw 1 gb/s of 100K packets at TCP port 0. If
you have a firewall that (for example) is trying to do protocol state
parsing for SMTP, it'll look much worse under a synthetic test than
one that simply goes "wow, that's port 25! let it through if you see a
HELO!" Under synthetic testing a "stateful" firewall will fare extremely
well, from a performance standpoint, if all the packets are directed at
an un-established flow.

One of the ironies of "stress testing" security products is that the
ones that do LESS security processing almost always do better
under a load test. Furthermore, the ones that do LESS processing
appear to do better in terms of (let's loosely call it)"reliability" since
they will favor letting things through. I saw this back in 1995 when
one of my customers chose a "stateful" firewall over a proxy because
in synthetic testing the proxy kept terminating a non-standards-compliant
FTP command stream, whereas the "stateful" firewall was just looking
for "PORT" commands and letting everything else through. The
customer felt that the "stateful" firewall was "better" because it was
"more reliable" - meaning "easier to get through." So, if you're stress
testing for "reliability" you need to ask yourself, first, what exactly
you mean by "reliable." It keeps coming back to the eternal trade-off
between performance and accessibility on one hand versus conservative
design and security on the other.


firewall-wizards mailing list
firewall-wizards () listserv icsalabs com

firewall-wizards mailing list
firewall-wizards () listserv icsalabs com

  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]