mailing list archives
Options to replace select in Nsock
From: David Fifield <david () bamsoftware com>
Date: Fri, 19 Jun 2009 23:48:44 -0600
Nsock, Nmap's parallel sockets library, uses the select function to
determine when a socket is ready to read or write and avoid blocking.
select has a built-in limit on the number of sockets it can handle,
which is FD_SETSIZE. On Linux this is 1024. Brandon Enright tracked down
a memory corruption bug to the fact that more than 1024 sockets were
being used, overflowing the fd_set buffer.
There are alternatives to select that could fix this deficiency. I'm new
to all this, so I spent some time today researching options and I want
to post some links I found. A good overview of the options on different
operating systems is
poll is a function that is similar to select but it doesn't have a limit
on the number of descriptors it can follow. poll is pretty portable,
except it isn't available on Windows before Windows Vista. poll has the
same scalability problem as select, namely that you have to iterate over
the entire descriptor list to find out which ones are ready, but that
hasn't been a problem for us.
Once you get past poll it seems all the APIs are platform-specific.
There is epoll on Linux and kqueue on BSD. Windows has something called
completion ports. All of these have some level of kernel-level support
for tracking state between calls and are considered high-performance
We need to do benchmarking to see if any of the platform-specific APIs
are worth implementing. The FD_SETSIZE limit is a real problem though,
and so I think it makes sense to port Nsock to use poll if possible.
Sent through the nmap-dev mailing list
Archived at http://SecLists.Org
- Options to replace select in Nsock David Fifield (Jun 20)