-----Original Message-----
From: johnadams [mailto:johnadams () apple com]
Sent: Monday, November 03, 2003 3:56 PM
To: Rob Shein
Cc: 'No Man'; pen-test () securityfocus com
Subject: Re: Cisco LEAP
On Monday, November 3, 2003, at 11:59 AM, Rob Shein wrote:
It's not a question of peak performance as much as
consistency. Flat
files
aren't meant to work this way; that's largely why database
applications work
the way they do in the first place. If something like
paging competes
for
drive access just long enough, the whole thing can go to
hell. When
you're
opening a graphic or text file completely into memory to
view or edit
it?
For that, sure, a flat file is faster. But when you're streaming
through a
flat file that's dozens of gigs in size, over an extended period of
time
while running the data into a memory and
processor-intensive program
at the
same time? Try it, and just see how quickly that works over the
length of
the entire file compared to a database :)
The real issue here is the right tool for the job -- we're talking
about a file with many passwords in it, which ostensibly
would be under
a few megabytes in size. You could mmap() the entire thing
into memory
and get consistent access without the use of a database. Memory is
cheap these days.
One thing that I see much of in software design is an overwhelming
desire to put everything into a database with complete disregard for
performance,
I used to work at Inktomi, and we used very little in the way of
databases to hold massive datasets (all web pages on the
Internet.) We
avoided databases for performance reasons, and saw serious gains
because of customized code that read flat files filled with
structures.
I guess the thing to remember here is that eventually the
database has
to write your data out to disk, and when that happens, it'll
be placed
on the disk in a file, using an fwrite() and a modicum of
indexes into
the data. Even programs like mysql eventually write their data out as
BerkeleyDB files.
-john
(posting far outside the scope of pen-test now)
Intro
