jordan posted this semi-anonymously

[Dave Aitel <dave () immunityinc com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dave,
~    I think that you're more referring to Hidden Markov Models, rather
than Markov Chains. Markov Chains are typically only mentioned in
theoretical discussions, HMMs are used in real tools.

I've long wondered why I haven't seen any security tools that use
these powerful tools (or Dynamic Bayes Nets; even more powerful,
easier to inject domain knowledge). Artificial Neural Nets (ANNs) are
often the wrong tool/hammer for problems involving time-series data,
and are better for classification when you just have a bag of
features. Trying to represent things like "this happened before this",
or "these things happened in this order" is messy, and while there are
things like recurrent ANNs that handle temporal data, they are
incredibly difficult to design, and usually exhibit poor performance.

ANNs are also horribly un-Bayesian in most applications that I've
seen, and so they don't give any sense of the confidence in their
hypotheses. There are good methods for getting error-bars, and the
theory of Bayesian Neural Networks is fairly well understood [1], but
Bayesian ANNs don't seem to be used in practice, from my experience. I
think that the math behind them seems intimidating, and so they are
avoided. Lastly, ANNs with more than a few hidden layers and a few
hundred nodes have been practically infeasible to train until the last
year or two, based on breakthroughs by Hinton [2].

HMM's and DBN's are much more powerful for the problems that these
applications seem to be trying to solve.

Then again I'm a machine learning researcher, not a security
researcher, so I may be way out of line even proffering an opinion on
this issue.

Also, I consider myself a mathematician, and my skull is still intact.

Jordan

[1] http://www.inference.phy.cam.ac.uk/mackay/Bayes_FAQ.html

[2] Hinton, G. E., Osindero, S. and Teh, Y. (2006),
A fast learning algorithm for deep belief nets.
Neural Computation, 18, pp 1527-1554.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHutQmtehAhL0gheoRAnHXAJ9t1eXmJbsAMd1pR/bDiXc8hMqeJgCfdFcx
qlhs6ZYe95zLTIlcMh5+rII=
=3Eup
-----END PGP SIGNATURE-----

_______________________________________________
Dailydave mailing list
Dailydave () lists immunitysec com
http://lists.immunitysec.com/mailman/listinfo/dailydave