Home page logo

nmap-dev logo Nmap Development mailing list archives

Re: Ncat and windows line endings
From: David Fifield <david () bamsoftware com>
Date: Wed, 3 Jun 2009 19:30:02 -0600

On Wed, Jun 03, 2009 at 06:11:07PM -0600, David Fifield wrote:
On Fri, May 08, 2009 at 08:42:25PM -0700, Fyodor wrote:
On Thu, May 07, 2009 at 10:48:46PM +0100, jah wrote:
I felt sure that something like this has been raised before, but haven't
found anything pertinent.  Is this a known issue and, more to the point,
can we afford to modify ncat to do something about it given that \r\n
might have been sent as part of a HTTP GET, for example?

Hi Jah.  Excellent point.  I wish we there was a way we could
automatically detect and do "the right thing".  But as you noted,
passing \r\n straight through may be desirable in some cases
(e.g. writing HTTP requests) while changing \r\n to \n is desirable in
others (talking to a UNIX shell).  We should probably at least provide
options for both approaches.  We currently offer:

First of all, maybe -C should format all bare \n in the buffer to
\r\n.  Is there a good reason (other than ease of implementation) for
only handling bare \n which happens to appear at the end of the
buffer?  Of course it needs to make sure that the \n chars in the
buffer are bare (not preceeded by a \r already).  We don't want to end
up with \r\r\n.

Second, we should probably have an option which does the opposite and
converts \r\n in input to plain \n.  Maybe it could be -N/--newline or
-L/--lf or something.  It should check the whole buffer for \r\n too,

Does anyone have a different idea or should we just implement these
two changes?  Anyone want to write a patch?

I found a way to retain the _O_TEXT setting after duplicating the file
descriptor. This is like a permanent -L setting, and is the way Ncat
used to work. There's no extra work on Ncat's part; as far as it can
tell, the bare \n it sees comes straight from the terminal. -C works to
convert the \n that Ncat sees into \r\n, which is nice because the
command line usage will be exactly the same on Windows and Unix; on
Windows we can simply pretend that the terminal sends \n. Windows users
don't have to remember to use -C to talk to a web server when they're on
Unix, and Unix users don't have to remember to use -L to talk to a shell
when they're on Windows. You always use -C when talking to protocols
that require it, and otherwise your newlines are \n on any platform.

A problem with this is that the newline translation applies to all forms
of input: pipes and files, not just the console. That will break binary
file transfers, for example. So this is what I propose: Set _O_TEXT on
Windows when stdin is a terminal, and set _O_BINARY otherwise. Set
_O_BINARY on stdout unconditionally.

With this, all terminals would appear the same, so there would be no -L
option. We would apply half of jah's patch, the part that fixes -C to
replace \n with \r\n anywhere in an input buffer. I have a patch for
_O_TEXT that I'll submit in a little bit.

Here is the patch. It sets _O_TEXT mode on stdin when the input is from
the console (checked using the _isatty function), and _O_BINARY
otherwise. It sets _O_BINARY on stdout. So the terminal sends \n for
line endings, and no newline translation is done anywhere else, just
like on Unix.

The only problem I can envision is if it's somehow possible to fool
_isatty. I can confirm that it returns true for
        ncat target
and false for
        ncat target < infile
        cat infile | ncat target
but there might be other cases. Documentation is here:

Setting _O_BINARY on stdout fixes another bug that was not discovered
until now. Ncat on Windows was doing newline translation on output as
well, breaking binary transfers.
        unix$ md5sum /etc/services
        aca587f0966a6f14536dae8fa15f095a  /etc/services
        C:\> ncat.exe -l > services
        unix$ ncat win --send-only < /etc/services
        C:\> C:\cygwin\bin\md5sum.exe services
        4458bb84b47c93cfa7f5e2432f21ca68 *services

What do you think?

David Fifield

Attachment: ncat-lf-mode.diff

Sent through the nmap-dev mailing list
Archived at http://SecLists.Org

  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]