Re: hostmap.nse improved! Added new "ip to hosts" service provider

[Fyodor <fyodor () insecure org>]

On Thu, Jun 02, 2011 at 12:52:09AM -0700, Paulino Calderon wrote:
> Long story short, I wrote http://www.whataremyhosts.com, an 'ip to 
> hosts' service provider that uses Bing results and I added support to it 
> in hostmap.nse.

Thanks for sending this proof of concept.  So far we have not included
any NSE scripts which use services that we ourselves host.  We may
have to revisit that de facto policy if we can't find other approaches
for features we really want.  Here are the main reasons we have so far
avoided doing this:

o Administrative resources - Running services ourselves can consume a
  lot of technical resources, and it gets worse as we add
  more and more services.  For any given script, we may have to deal
  with issues like:

  o If the 3rd party API (Bing in this case) changes, the script may
    break and we need to debug the problem and fix it.

  o If spammers or other parties abuse the service by sending huge
    numbers of queries, we need to figure out and implement a way to stop
    them

  o If a 3rd party API limits the query rate they will handle, we may
    exceed that just from normal legitimate usage and then have to figure
    out what to do.

  o Things can break for bizarre reasons.  The recent VA Module Alert
    service failure was tracked down to Nessus considering its license key
    invalid after our host changed its MAC address.

  Of course, we also have to deal with the administrative hassles of
  dealing with the host OS, networking, etc.  Also, the service have
  to all be written in the same programming language or maintenance
  becomes an even greater hassle.

o Security - The more self-hosted services we add, the greater the
  chances are that at least one of them has an exploitable security
  hole.  At a minimum, we will have to create a new Linode virtual
  machine for self-hosted services which does nothing else.

o Privacy - The queries people make are effectively data about the
  scan being sent back to our servers.  Of course this is similar to the
  problem with queries sent to 3rd parties and is the reason we have the
  'external' category and never include those scripts in the 'default'
  category.  We could probably do the same for self-hosted scripts.

o Costs - It costs us money for bandwidth, CPU time, and other
  resources used to host scripts.  Since Nmap is free, we need to be
  very frugal.

All this being said, we may want to seriously consider self-hosting
some services if we can't find a better solution.  For example,
geolocation would be particularly useful.  But databases such as
Maxmind are probably too large to ship with Nmap, and we haven't yet
found a good 3rd party service alternative.  But Nmap could do a lot
with that IP-to-location data if it had it.

Cheers,
Fyodor
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/