Re: Snort and Nessus Signature

[Vikram Phatak <vphatak () lucidsecurity com>]

I think we got off on the wrong foot.  I am not intending to disparage 
the current snort signature set.

Jason wrote:

> I did not think that response indicated offense and I am not offended. I
> though the response was a factual response based on the information I have.
>
> Vikram Phatak wrote:
>  
>
> > Wow.  I did not expect that kind of response...  I offered to discuss in
> > detail with Crux or anyone else interested.  Have I somehow offended you?
> >
> > Jason wrote:
> >
> >    
> >
> > > Vikram Phatak wrote:
> > >
> > >      
> > >
> > > > Hi Crux,
> > > >
> > > > It is not a simple matter to integrate Nessus & Snort since there are
> > > > quite a few errors in the snort signatures, or in the supporting
> > > > information for many of the snort signatures (CVE, BID, descriptions,
> > > > etc.).  
> > > >        
> > > How so? Please provide a little more information. 
> > >      
> > First, if you look at the non-VRT certified signatures from Snort, or
> > historically at Snort signatures in general, they do not consistently
> > have references, and in many cases needed to be updated to support
> > functions available in newer versions of Snort (like PCRE) that provide
> > better accuracy and lower false positives.  Sourcefire recognized this -
> > which is why they rewrote over 1000 signatures (may be more now) which
> > can be purchased as part of their VRT Certifiied signature program.
> >    
>
> I think it is an absolute injustice to generalize this statement to
> Snort as a whole. There are many groups that produce snort rules of
> varying quality but the official snort.org rulesets are not as described.
>  
I am surprised to hear you say that.  I am not saying that Snort 
signatures are poor.  In fact, I am saying that Sourcefire invested a 
lot of resources to make sure that their signatures are of HIGH 
quality.  However, Sourcefire themselves have seperate signature sets by 
version so that people get the optimal set for the version they are 
running.  And Snort's signature set was not built from the ground up 
with correlation to vulnerabilities & VA tools in mind and therefore has 
some shortcomings that need to be addressed before correlation is a 
simple matter of cross-referencing data.

(I just looked (briefly) for the press release by Sourcefire re: 
signature QA & updates but was unable to find it.  If I find it later I 
will forward to you.)


> > Having said that, we have found that there can be multiple CVE entries
> > for the same vulnerability, signatures that have no CVE, multiple nessus
> > plugins that have the same CVE (sometimes correct, sometimes not) etc. 
> > You cannot simply assume that everything is correct, and correlate
> > automatically and expect to have any confidence in your results - which
> > means you need to inspect every single correlation (which takes time). 
> >    
>
> Agreed. There is a lot of room for improvement in the vuln DB space.
> That there are several reference points for any one issue is
> problematic. That each reference point handles them differently further
> complicates this.
>
> [...]
>
>  
>
> > > > Also, many snort signatures do not have CVE, BID references since
> > > > historically they have written based upon packet captures of specific
> > > > exploits,  (such as "Sasser") as opposed to vulnerabilities
> > > > (LSASS), which is how CVE entries are sorted.
> > > >        
> > > Absolutely incorrect. LSASS is the detect method and the rules detect
> > > exploitation of a vulnerability not an exploit.
> > >      
> > Um.  Please check your facts.  Sourcefire did not provide its VRT
> > service until Feb/March '05, while LSASS vulnerability that the SASSER
> > worm exploited came out in April/May '04.   The only signatures publicly
> > available at the time were exploit signatures based upon PCAPs of the
> > different worm variants...  Despite any marketing that would have you
> > believe otherwise.
> >    
>
> Here are your facts.
>
> http://securityresponse.symantec.com/avcenter/venc/data/w32.sasser.worm.html
>
> - Discovered on: April 30, 2004
>
> [...]
>
> http://www.sourcefire.com/services/advisories/sa050204.html
>
> [...]
>
> released on April 27, 2004, contained SID 2514 that will detect infected
> hosts attempting to compromise other hosts via the LSASS vulnerability.
> This rule will generate multiple events due to Sasser worm activity.
>
> [...]
>
> These rules were in the snort.org set before there was a worm and the
> rules were quite effective in detecting the worm and variants as well as
> non worm exploitation attempts.
>
> [...]
>  
Interesting.  I have a feeling that SID 2514 is more of an anomaly 
detection rule than a vulnerability based signature, but I have not 
examined it yet...  I remember at the time that was the case, but my 
memory may be suspect.

>  
>
> > But to your point - assuming 100% accuracy of references and that most
> > eventually point back to something that can be correllated with a Nessus
> > plugin - which 70% are covered?  How do I, as an admin, know I am not
> > missing something?  There needs to be some examination of effectiveness
> > before simply saying "lets correlate".  Or rather, correlate - but audit
> > your results.  That was the point of my prior posting.
> >    
>
> Apologies if I got the context wrong there. The shortcomings of text
> based interactions. I absolutely agree that there are challenges with
> the various vuln data sources.
>
> IMHO there are no absolutes in security so any correlation done is
> ultimately additional data that can be taken into account. Even a 10%
> coverage would provide better value than the current 0% coverage.
>  
I agree with that wholeheartedly.  It is not an all or nothing 
situation.  More visibility is better than less, etc. 

I believe very strongly that it is important not to "drink one's own 
cool-aid" when examining a solution.  (I am not saying you are doing 
that.)  I like to get all the facts (good and bad) and make decisions 
accordingly - Which is why I pointed out some of the faults with 
correlating using current data sources.  It was never my intent to say 
"Don't do it" but rather to say "Know what the limitations are" so that 
those limitations can be addressed (as opposed to ignored).  Old habits 
die hard :-)

>  
>
> > > > We (Lucid Security) have found that it was far more efficient (and
> > > > reliable) to choose the OS & Application versions that we want to
> > > > protect (MSFT, Linux, Solaris, Apache, IIS, SQL, etc.) and prioritize
> > > > accordingly.  We then chose the appropriate CVE entries that met the
> > > > requirements of our "filter" and wrote and tested signatures based
> > > > upon the vulnerability accordingly.  If there was an existing
> > > > signature that met our requirements, then great! But we found that
> > > > was rarely the case.  
> > > >        
> > > I take it you are not in the spirit of the community and as such are
> > > either selling your wares and saying screw the rest of the community
> > > or you are simply spreading FUD. Which is it?
> > >
> > >      
> > Nice.  Do you attack Ron or Marty with this kind of nonsense?  I am
> > offering advice from experiences that we have gone through and you are
> > asking me if I am saying "screw the community" or "spreading FUD"?  It
> > is like asking, "Do you still beat your wife?"
> >    
>
> Well do you? ;-)
>  
;-)

> I think I addressed this above. Your advice is certainly welcome. Your
> results are even more welcome. _That_ would be in the spirit of the
> community.
>
> From what I saw you indirectly challenged the quality of the community
> with incorrect data while promoting your own product as superior. I
> found your statements to be incorrect. No offense intended.
>  
I am very willing to discuss what has and has not worked for us.  As far 
as the results... I cannot discuss Lucid Security's plans in that area 
at this time.  I'm sorry I cannot say more, but it would be improper for 
me to do so.

As far as challenging the quality of the community - that was not my 
intent.  As I said before, many (if not most) signatures were not 
written with correlation to VA tools in mind.  I do not mean to imply 
any lacking on the part of the community for not doing so, it simply has 
not been part of the scope for writing an effective signature.  As far 
as the effectiveness of signatures - again it is a moving target with 
newer versions providing better ways to capture malicious traffic.  
Writing signatures against the underlying vulnerability is a relatively 
new practice and not all signatures are written that way.  In many cases 
there is no need to write signatures that way, since nobody is writing 
attacks against older vulnerabilities (the assumption being that they 
are patched already).


   -Vik

--
Vikram Phatak
CTO, Lucid Security
http://www.lucidsecurity.com



------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
------------------------------------------------------------------------