File Preprocessor: Features developed (ExtraData fields in events and S3 storage)

[Pablo Cantos Polaino <pcantos () redborder org>]

Hello all,

My colleague Eugenio and I have been working on including new features to
the File Preprocessor. We have added additional instructions in README.file
that you can find under every comment line that starts with the keyword
"redBorder". These features can be summarized as follows:

   - File Preprocessor was already able to capture files to memory, to disk
   and to network. Sending files to a S3 storage has been added to this
   capture feature.
      - Conf example:
      include file_magic.conf
      preprocessor file_inspect:\
          type_id, \
          capture_queue_size 5000, \
          signature, \
          capture_disk /var/log/snort/files/ 5000, \
          s3_bucket *bucket*, \
          s3_cluster *S3 server*, \
          s3_access_key *access key*, \
          s3_secret_key *secret key*
   - File Preprocessor was already able to send an alert every time it
   detects a specific file type. Inclusion of ExtraData fields in these events
   has been added to this feature. Until now, the ExtraData fields included
   are SHA256, file size, hostname and URI. Since Barnyard2 v2.1.13 doesn't
   take into account the ExtraData fields, we've also changed drastically it,
   altering the way the spooler analyzes events and records.
      - Conf examples:
      include file_magic.conf
      preprocessor file_inspect:\
          type_id, \
          capture_queue_size 5000, \
          signature, \
          capture_disk /var/log/snort/files/ 5000, \
          track_extradata
      include snort_files.rules

These features have been developed over Snort v2.9.7.3 and Barnyard2
v2.1.13 and are available in our github server. Please follow the links
below:

   - Snort features:
      - https://github.com/redBorder/snort/tree/feature/file_extradata
      - https://github.com/redBorder/snort/tree/feature/file_s3
      - Barnyard2 changes:
      -
      https://github.com/redBorder/barnyard2/tree/Feature/Managing_ExtraData_fields
      (just needed if you're interested on Snort ExtraData feature)

Please take into account that this is a very early version that could
contain some bugs, so we will be glad to receive any feedback and
suggestion.

This publication follows the general redBorder principles of divulging new
features and enhancements in Snort in appreciation for the enormous
collective effort of this community. So we hope this can be useful to you.

Best Regards,

Pablo Cantos
redborder.org / pcantos () redborder org

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y

_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!