Re: Ncat + Lua - GSOC feedback request

[Jacek Wielemborek <wielemborekj1 () gmail com>]

Hi guys,

Since I recently got accepted to the GSoC, I'd like to ask you again
for your feedback on Ncat-Lua project. I would be really grateful if
everybody that has any opinion on the subject spoke out.

Here's the IRC log of my talk with Patrick and bonsaiviking:

23:27:48  batrick$ it's hard to give you recommendations
23:28:02  batrick$ i read over your proposal and I'm still not
completely sure what it will be doing
23:28:26  d33tah$ honestly, i wanted some decisions to be made once i
hear some feedback from you guys
23:28:42  batrick$ well, decisions can't be made until we have use-cases :)
23:28:44  d33tah$ have you read the lengthy email i wrote on dev () nmap org?
23:28:46  batrick$ yes
23:28:54  d33tah$ so there are a few use cases
23:28:58  batrick$ i assumed you were working through these details with david
23:29:04  d33tah$ not yet.
23:29:41  d33tah$ i actually asked him yesterday, he didn't really
have much time to reply.
23:30:01  batrick$ issues i have are
23:30:13  batrick$ adapting NSE libraries to ncat will be hard
23:30:33  batrick$ many assumptions are made
23:31:04  batrick$ I'd like to see an example for NSE testing, which
you also brought up
23:31:10  batrick$ i think that will be difficult to do?
23:31:19  d33tah$ what kind of assumptions? supposing i only wanted to
port a part regarding TCP and UDP?
23:31:55  batrick$ nmap libraries available, concurrency
considerations, availability of NSE specific libraries
23:32:12  batrick$ some of that can be worked out but it'd be a big job
23:32:21  batrick$ and what would you do with those libraries once you
made them available?
23:32:39  d33tah$ hm, one sec. which libraries?
23:32:43  batrick$ ssh libraries are really just bare bones,
engineered to get specific information from ssh daemons
23:32:47  batrick$ not generally useful
23:32:51  batrick$ especially not for somehting like ncat
23:32:54  d33tah$ oh, i see.
23:32:56  batrick$ http library is the same way
23:33:15  batrick$ mutexes and condition variables
23:33:19  batrick$ any of the stuff in nmap.*
23:33:48  batrick$ the embedded stuff for ncat is nice but not a use-case
23:34:02  batrick$ I'm looking at (2) for your post
23:34:13  d33tah$ hmmm, i kinda assumed that NSE supports bigger parts
of the protocols
23:34:20  batrick$ no
23:34:36  batrick$ some may have more complete implementations
23:34:45  batrick$ but even then, they are more client driven
23:34:52  batrick$ none will have server type stuff
23:35:18  d33tah$ which could be a good reason to employ ncat.
23:35:39  batrick$ maybe, but why not just deploy the actual daemon?
23:35:42  batrick$ why use ncat?
23:35:54  d33tah$ hm
23:36:27  d33tah$ my first thought is that you could, for example,
test parts of the protocols. switch simulated states as you wish
23:36:33  batrick$ so one example use case I was thinking of
23:36:46  batrick$ Lua controls how Ncat sets up sockets. You could
setup a pipeline in a single call to ncat without needing something
like:
23:36:48  batrick$ ncat --sh-exec "ncat example.org 80" -l 8080 --keep-open
23:36:58  batrick$ (copy paste from a draft email reply i've been
sitting on for a few weeks)
23:36:59  d33tah$ to do that with the actual daemon would require
quite a tricky set of hooks
23:37:54  d33tah$ so, you mean it would be nice to add a few shortcuts
to use cases like this tunnel?
23:38:06  batrick$ maybe
23:38:08  batrick$ sure
23:38:36  batrick$ i think you need to have a mindset of "how do I
make ncat even better at what it does"
23:39:07  bonsaiviking$ "nmap -sn -Pn --script +foo -p 80 host" could
become "ncat --script foo host 80", but...
23:39:08  d33tah$ application-layer protocols' support was one of my first ideas
23:39:32  bonsaiviking$ then you've only duplicated existing
capability (however complicated it was before)
23:40:01  d33tah$ which starts to feel that it should be a separate library.
23:40:13  bonsaiviking$ I think a strength could be to move some of
the app-layer stuff that's already in ncat into Lua from C
23:40:30  bonsaiviking$ Like the proxy support, the chat server, etc.
23:40:32  d33tah$ hm, that was the plan A.
23:40:52  d33tah$ i like it, but couldn't resist bringing up the plan
B as well ;)
23:41:05  AlexWebr$ The chat server is neat, but srsly, who actually uses it?
23:41:21  d33tah$ AlexWebr: which was exactly my point in the proposal :D
23:41:50  bonsaiviking$ I think once you have some of those in, you'll
start to see other ways to use the infrastructure you've created.
23:42:15  d33tah$ sounds like a good plan.
23:42:40  bonsaiviking$ Being able to replace --sh-exec "external
program" with --script X where X is a lua script that does what
"external program" did would be nice.
23:43:23  bonsaiviking$ or to use lua to generate input instead of stdin
23:43:42  d33tah$ yeah, that was also one of my ideas
23:44:21  d33tah$ but then i was thinking - how much else would i have
to add to be able to run NSE scripts as well? at least the basic ones?
23:44:45  d33tah$ judging by what batrick said, gsoc might prove not
enough for that
23:45:05  bonsaiviking$ This can be done without worrying about parallelization.
23:45:15  d33tah$ not exactly.
23:45:22  d33tah$ consider support for listen mode as well
23:45:46  bonsaiviking$ well, ok
23:45:55  batrick$ so that could be something you improve, make it so
each new connection doesn't require a fork
23:46:04  batrick$ but again, you need a use-case that makes it worth adding
23:47:06  bonsaiviking$ And it may depend on hatchee's
nsock-server-mode implementation
23:47:33  d33tah$ well, I guess that coding up a small server in Lua
that can handle multiple clients simultaneously doesn't seem like
enough of a use case for you, batrick?
23:47:47  batrick$ honestly no, because I could do the same thing with luasocket
23:47:50  batrick$ in 10 minutes
23:50:29  d33tah$ batrick: luasocket doesn't support ssl, does it?
23:50:48  batrick$ I don't know
23:51:52  d33tah$ couldn't find any sources that say it does. and i
guess it wouldn't be much work to add ssl support to ncat-lua.
23:52:51  d33tah$ afk
00:16:20  d33tah$ back
00:16:48  d33tah$ so, the ssl sounded like an argument to use ncat
over luasocket to me.
00:17:21  d33tah$ what do you think, batrick?
00:21:00  batrick$ it can be
00:21:18  batrick$ again, I recommend the mindset "how do I make ncat
even better at what it does"
00:21:24  batrick$ try to put a lot of thought into that
00:21:31  batrick$ one problem is ncat isn't that wide-spread in usage
00:21:34  batrick$ over nc anyway
00:21:41  batrick$ very much a niche hackers tool I think
00:21:50  batrick$ so expanding on its functionality is good
00:22:11  batrick$ but making it a compelling tool to use is what you
want to aim for
00:22:38  batrick$ david has posted numerous times to the list with
some impressive ncat magic
00:22:45  batrick$ maybe look at that and  it will giv eyou some ideas
00:23:12  d33tah$ hm, good idea, thanks.
00:32:45  d33tah$ i just had an idea.
00:34:01  d33tah$ suppose we found some new protocol that allows to
tunnel TCP/UDP over application layer
00:35:34  d33tah$ it would be cool to run ncat
--script="some-weird-tor-clone" nmap.org 80 and use some lua overrides
that provide abstractions for connecting, sending and
                  receiving packets
00:55:19  d33tah$ batrick: i looked through 32 pages of google results
so far and only hit one thread with David's ncat magic so far. could
you give me a clue?
00:59:00  batrick$ did you use something like
site:seclists.org/archive/nmap-dev or w/e?
00:59:04  batrick$ should help narrow results
00:59:35  batrick$ I have seen david do literally 3*80 characters
worth of ncat magic in one command
00:59:47  batrick$ i think that was in a mailing list posting
00:59:56  batrick$ may have been something like on his wiki or private mail
01:04:55  d33tah$ batrick: yeah, i specified the nmap-dev archive there
01:05:02  d33tah$ secwiki, right?
01:05:17  batrick$ ya... harder to navigate via google unfortunately
01:05:21  batrick$ i'm looking through my mail
01:05:28  batrick$ no seclists
01:05:30  batrick$ ?
01:05:37  batrick$ oh
01:05:42  batrick$ no david has a wiki on his website
01:05:47  batrick$ bamsoftware
01:06:13  batrick$ anyway
01:06:23  batrick$ you can always ask david if he has any memorable
ncat commands
01:06:24  d33tah$ didn't know about it. i wonder how much more's
hidden on the domain :P
01:06:30  batrick$ he may have a few scripts that do some crazy shit
01:06:53  batrick$ i can never find anything interesting when i search
through my mail
01:06:56  batrick$ i've looked before

Yours,
Jacek Wielemborek
_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/