Home page logo
/

snort logo Snort mailing list archives

Re: [snort-devel] Chainning pre-processors
From: Emiliano Fausto <emiliano.fausto () gmail com>
Date: Thu, 5 Dec 2013 17:19:06 -0200


Hello Hui,

 the "-k none" did the trick, I really really thank you for all your help.

 As a summary (in case anyone else has the some doubts as me), correct me
if I'm wrong:

 1) It's possible to chain preprocessors and dynamic-preprocesors in
SNORT. In fact, they are chained automatically by SNORT by default. The
order will be given for the priority set when they are registered (for
example with the function AddPreproc()).

 2) The priorities are set in the file preprocids.h.

 3) Using the stream5 preprocessor is as explained in the official
Stream5.README documentation, we can add that if you are seeing too much
discarded TCP packets in the statistics, you may want to add the "-k none"
parameter when starting the snort process (tells snort to ignore checksum
errors in the TCP packets).

 4) Inside the preprocessor code, you may use the filter: (SFSnortPacket*)
mypacket->flags & FLAG_REBUILT_STREAM, to be sure that this is a rebuilt
packet.

 My preprocessor and the TCP reassembling is working perfectly now with
your help.

 Again Hui, thanks a lot for everything!
 Emiliano.



2013/12/5 Hui Cao <hcao () sourcefire com>

 Did you see many discards in snort exit. If so, you need to disable
checksum. Use -k none when you run the command.

Best,
Hui.

On 12/05/2013 01:40 PM, Emiliano Fausto wrote:

Hui,

 I was really trying to get Stream5 working and I couldn't, I'm pretty
sure that the problem is not related to Stream5, but I just can't get the
preprocessor to rebuilt TCP segments.

 In wireshark I can see there are lot of TCP segmented packets and if I
select to reassemble the HTTP ones, it does it well. Backing to SRNOT, for
some reason the statistics that the Stream5 are showing are that there
where some TCP sessions, there were discarded TCP packets, etc. But in the
line: "TCP Rebuilt packets" shows a 0.

 I tried with HTTP so that I can have a known protocol to start with as
you suggested me, and tried to download a wallpaper, which (due to its
size) was delivered to me in several packets.

 I changed the snort.conf configuration file, to look like this:

 preprocessor stream5_global: track_tcp yes, \
    track_udp no, \
    track_icmp no, \
     max_tcp 262144, \
    max_active_responses 2, \
     min_response_seconds 5, \
    show_rebuilt_packets
preprocessor stream5_tcp: policy linux, \
    overlap_limit 10, small_segments 3 bytes 150, timeout 180, \
    ports both all

 I also played a little bit with my own preprocessor to print the Flags
that the packet brings, and they change based on the priority I assign to
the AddPreproc(). But I tried different ones, from FIRST to LAST and none
of them matched the & FLAG_REBUILT_STREAM flag.

 I was looking at the stream5 documentation (the README file), also
searched different threads in the snort users and devel community, but none
of them helped me.

 Do you know where can I get more information or examples on how to use
properly the Stream5 preprocess, or if I'm missing something else to make
it works? maybe create a generic rule, or something like that?

 I really thank you in advance for all your help on this,
 Emiliano.



2013/12/5 Hui Cao <hcao () sourcefire com>

 Make sure you have frag3 enabled. You can try the application
preprocessors (such as http, smtp, ssl, sip etc) to see how it works.  If
the packets are reassembled, you can see that from snort exit stats for
stream etc.

Stream will reassemble the payloads of TCP.

Best,
Hui.


On 12/05/2013 08:26 AM, Emiliano Fausto wrote:

Hello Hui,

 you are totally right. I was confused, I didn't have to assemble
fragmented IP packets.

 So, I configured the Stream5 preprocessor in order it to take into
account the port of my application (it's 9090).

 I let my AddPreproc() with priority set to PRIORITY_APPLICATION so
that it's registered after the Stream5.

 Then, I tried to get any packet with (p->flags & FLAG_REBUILT_STREAM),
but none of them matched, it seems that the Stream5 preprocessor isn't
reassembling my TCP segments.

 The SNORT configuration file is set like this:

 preprocessor stream5_global: track_tcp yes, \
    track_udp yes, \
    track_icmp no, \
    max_tcp 262144, \
    max_udp 131072, \
    max_active_responses 2, \
    min_response_seconds 5
preprocessor stream5_tcp: policy windows, detect_anomalies, require_3whs
180, \
    overlap_limit 10, small_segments 3 bytes 150, timeout 180, \
    ports client 21 22, \
    ports both 9090
preprocessor stream5_udp: timeout 180

 My app works on port 9090.

 Maybe, the problem is not related to stream5, maybe the problem is
originated because the one in charge of doing the segments reassembly is my
preprocessor, at the application level, and not the stream5 preprocessor at
the TCP level.

 More on this, into wireshark I can see all the packets of the protocol
TCP segmented, and if I go to preferences-->protocols-->TCP it has an
option that says: "allow dissectors to reassemble TCP". It means that the
reassemble isn't being done at the TCP layer, but into the proper
application.

 Do you know if SNORT has something of this? Or the Stream5 should
reassemble the payloads of the TCP packets?

 Thanks a lot in advance,
Emiliano.


2013/12/4 Hui Cao <hcao () sourcefire com>

 Frag3 will deal with IP fragmentation. You might want reassembled data
from TCP. This means your preprocessor should be after stream and you
should check
(SFSnortPacket*)ipacketp)->flags & FLAG_REBUILT_STREAM. Snort will only
reassemble up to the flush point defined (typically 16K). You can't expect
snort will give all the data from one reassembled packet.

Fragmentation from wireshark might mean TCP segmentation. You might get
full IP packets.

Best,
Hui.
 On 12/04/2013 03:17 PM, Emiliano Fausto wrote:

Hui,

 yes, I understood what you told me about filtering just the
de-fragmented packets, that's why I added these 3 lines:

 SFSnortPacket *p = (SFSnortPacket *)pkt;
int fla = (p->flags & FLAG_REBUILT_FRAG);
_dpd.logMsg("Flags set? %d",fla);

 But I generate fragmented packets (which for instance Wireshark
detects and show as fragmented and the possibility of reassembly them) and
I'm always getting with each one the output: "Flags set? 0".

 I receive all the fragmented packets, but not the last one
reassembled.

 Regards,
Emiliano.


2013/12/4 Hui Cao <hcao () sourcefire com>

It will still get all packets including the raw packets. You need to
use packet flags to filter them.

Best,
Hui.

On Wed, Dec 4, 2013 at 3:00 PM, Emiliano Fausto
 <emiliano.fausto () gmail com> wrote:
Yes,

that's what I thought, but for some reason the TCP packets keep on
coming
fragmented to my preprocessor.

No I took off all the preprocessors and just let the frag3 and
mines, I'll
try to figure out if they are called in the correct order, but they
are not
being chained.

Thanks,
Emiliano.


2013/12/4 Hui Cao <hcao () sourcefire com>

It looks good to me.

Best,
Hui.

On Wed, Dec 4, 2013 at 2:44 PM, Emiliano Fausto
<emiliano.fausto () gmail com> wrote:
Hi Hui,

I've seen that I was using PRIORITY_TRANSPORT, which is lower
than the
PRIORITY_NETWORK that uses frag3.

Anyway, I put the priority: PRIORITY_LAST to my own preprocessor,
but
when
the TCP packets keep arriving fragmented to my preprocessor.

Is there anything else I should take into account?

Thanks in advance,
Emiliano.


2013/12/4 Hui Cao <hcao () sourcefire com>

In sr/preprocids.h

Best,
Hui.

On 12/04/2013 02:36 PM, Emiliano Fausto wrote:

Great,

so, the pre-processors are "chained" by default, and the order
that
SNORT
follows to call them is set by the PRIORITY variable.

Do you know where's defined this PRIORITY variable? Because I
saw that
the
frag3 is being registered with PRIORITY_NETWORK, so I'd like to
set the
priority of my own preprocessor as (PRIORITY_NETWORK -1).

Thanks in advance,
Emiliano


2013/12/4 Hui Cao <hcao () sourcefire com>

sc means snort configuration. We use PRIORITY to sort the
processing.
All
processors enabled will be called and processed based on
priority. You
have
to rely on the code to figure out what exactly snort does.

The checking is correct. You will only process rebuilt packets.

Best,
Hui.
On 12/04/2013 02:19 PM, Emiliano Fausto wrote:

Hello Hui,

thanks a lot for your answer.

Right now I have registered my preprocessor (let's call it
examplePreprocess as you said, because right now I'm using the
one
provided
with the DPX) with this line:

_dpd.addPreproc(ExampleProcess, PRIORITY_TRANSPORT, 10000,
PROTO_BIT__TCP);

So, the only change is to add previous to the parameter
ExampleProcess,
the "sc". What does it mean? Do you know if there's any
documentation
about
this chaining preprocesses?

So, checking the flags, should be:

(SFSnortPacket*)tcppacket)->flags & FLAG_REBUILT_FRAG

right?

Thanks again!
Emiliano.



Then, I'll have to register my own preprocessor where?


2013/12/4 Hui Cao <hcao () sourcefire com>

Yes, it is possible. You can register you preprocessor like
this:

_dpd.addPreproc( sc, ExampleProcess, PRIORITY_TRANSPORT,
You_PP_ID,
PROTO_BIT__IP );

Remember check the following flag in your ExampleProcess:

(SFSnortPacket*)ipacketp)->flags & FLAG_REBUILT_FRAG

Best,
Hui.


On 12/04/2013 12:52 PM, Emiliano Fausto wrote:

Hi everybody,

I'm creating a new preprocessor which needs to have the whole
content
in
a packet which was fragmented.

So I thought of using the frag3 preprocessor to re-assembly the
packets,
and then, when this reassembly is done, sent it to my own
preprocessor.

Do you know if this is possible? May I have the output of
frag3 being
the input of my own preprocessor?

Regards,
Emiliano.





------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single
code
base.
Download it for free now!



http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk



_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:

http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about
Snort!






------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single
code
base.
Download it for free now!



http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:

http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about
Snort!


















------------------------------------------------------------------------------
Sponsored by Intel(R) XDK 
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!

  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]