Re: [Snort-devel] To discuss: FLoP and missing database (fwd)

[Dirk Geschke <Dirk_Geschke () genua de>]

Hi Chris,

> > (Further we check the endianess. These must now be
> > identical on server and remote sensor. This is the
> > first step towards a mixed environment but is not
> > related to the database...)
>
> Just as an FYI, you should fix that in this future version. That has
> been a PITA w/ barnyard for a while for no good reason other than not
> being careful in the beginning.

I think this problem could be more compicated than it seems
to be. I am not sure where I can run in boundary problems.

Therefore the access is now restricted to similar systems.
If I find the time (and appropiate machines) I will try to
get fix for this problem. (Maybe at one time the sensors
will be running a diskless Linux on x86 whereas the central
server runs on Solaris/Sparc. But I guess the most will use
the same architecture for all machines. So this problem has
a low priority.)

> > Version FLoP-1.xxx: (not started yet)
>
> [xxxx]
>
> > Another solution would be in writing the data binary
> > to a file. This data can be read in if the client
> > connects and the database is available. Thus we
> > first read in the data from the file, remove this
> > file and then start the normal, threaded, process.
>
> That's barnyard basically.

Hmm, I am not sure. I never worked with barnyard but I
think it is not comparable in this point. The problem
is to react on the fact of a missing database. The 
database can be killed or be maintained/replaced and 
therefore the access is not possible.

So what will happen?

Normal INSERTs will fail, this seems to be clear. But
how to react on this? If I understand the database
plugin of barnyard correct then the used solution is
the worst one. Barnyard seems to exit:

int MysqlInsert(MYSQL *mysql, char *sql, unsigned int *row_id)
{
    if(MysqlExecuteQuery(mysql, sql) != 0)
    {
        /* XXX: should really just return up the chain */
        FatalError("Error (%s) executing query: %s\n", mysql_error(mysql), sql);
        return -1;
    }

....

So barnyard will die, has to be restarted by another process
and will die again until the database is back? (BTW: The
database plugin op_acid_db.c looks still a little bit
expermintal?)

I would like to avoid this problem. The system should recover
the problem by itself. Therefore I was thinking about of swapping
out the alerts temporarily or writing the INSERT statements 
to a file for later processing.

Barnyard decouples the output from snort like FLoP/sockserv do.
Barnyard uses files on the filesystem whereas sockserv uses a
unix domain socket. 

But the differences are:

+ FLoP only stores alerts to the database (in addition an e-mail
   can be send but this is in conjunction with the database insert).
+ The ouput processing is done on the central server. The alert
  packets (with payload) are simply forwarded to the central server.
  On the central server the data is inserted in the database (again
  via an unix domain socket)
+ Barnyard handles the database access like the snort database 
  ouput-plugin, all INSERTs and SELECTs are send via TCP/IP to
  the central server (I assume the use this, the database should 
  not sit on the sensor.)
+ FLoP spools the data to the central server. So no alerts remain
  on the sensor (ideally, otherwise only a few alerts are still on
  the sensor). All major processing should be done on this central
  machine. With barnyard all alerts are on the sensor until they 
  are inserted.
+ With FLoP I can take a better care about which alerts are there
  and waiting for processing. With barnyard I have to check all
  remote sensors.

So although both systems have the same roots (decoupling of
the output from snort) there are still some differences.

Even with barnyard I do not know how to handel this problem.
Of course there is no need to write the data to a file as
they are already on the disk. But how to handle a missing
database? You have to close the old connection and have 
still to try to connect again and wait for the database.

Of course, you can work as now, simply restart barnyard
so many times until it can connect the database...

And yes, even FLoP is not perfect. Under some circumstances
it is still necessary to check the sensors for problems. But
the idea is to avoid this as much as possible.

Best regards

Dirk



-------------------------------------------------------
This SF.net email is sponsored by: IBM Linux Tutorials.
Become an expert in LINUX or just sharpen your skills.  Sign up for IBM's
Free Linux Tutorials.  Learn everything from the bash shell to sys admin.
Click now! http://ads.osdn.com/?ad_id78&alloc_id371&op=click
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users