Home page logo
/

nmap-dev logo Nmap Development mailing list archives

hadoop and hbase information gathering
From: John Bond <john.r.bond () gmail com>
Date: Sat, 8 Oct 2011 15:48:14 +0200

I have written a couple of scripts to scrape various hadoop and hbase
status pages.  would be great to get feed back on wether this works
with different version this is tested on hadoop 0.20.  All scripts
also implement newtargets where appropriate so you can run a command
like the following to discover an entire hadoop cluster

nmap  --script 
hadoop-datanode-info,hbase-master-info,hadoop-jobtracker-info,hadoop-namenode-info,hadoop-tasktracker-info,hbase-region-info.nse
--script-args hadoop-jobtracker-info.userinfo,newtargets -p
60010,50030,50070,50075,50060,60030 master-hadoop-server.example.com



hadoop-jobtracker-info.nse

information gathered:
 * State of the jobstracker
 * Date/Time the service was started
 * Hadoop Version
 * Hadoop Complie date
 * Jobtracker ID
 * Log Directory (reletive to the http://host:port/)
 * Associated Tasktrackers
 * Optionally also show the user activity history

---
-- @usage
-- nmap --script hadoop-jobtracker-info -p 50030 host
-- 
-- @output
-- 50030/tcp open  hadoop-jobtracker
-- | hadoop-jobtracker-info:
-- |   State: RUNNING
-- |   Started: Wed May 11 22:33:44 PDT 2011, bob
-- |   Version: 0.20.2 (f415ef415ef415ef415ef415ef415ef415ef415e)
-- |   Compiled: Wed May 11 22:33:44 PDT 2011 by bob from unknown
-- |   Identifier: 201111031342
-- |   Log Files: logs/
-- |   Tasktrackers:
-- |     tracker1.example.com:50060
-- |    tracker2.example.com:50060
-- |   Userhistory:
-- |     User: bob (Wed Sep 07 12:14:33 CEST 2011)
-- |_    User: bob (Wed Sep 07 12:14:33 CEST 2011)
-- ---

hadoop-tasktracker-info.nse

information gathered:
 * Hadoop Version
 * Hadoop Complie date
 * Log Directory (reletive to the http://host:port/)

---
-- @usage
-- nmap --script hadoop-tasktracker-info -p 50060 host
--
-- @output
-- PORT      STATE SERVICE            REASON
-- 50060/tcp open  hadoop-tasktracker syn-ack
-- | hadoop-tasktracker-info:
-- |   Version: 0.20.1 (f415ef415ef415ef415ef415ef415ef415ef415e)
-- |   Compiled: Wed May 11 22:33:44 PDT 2011 by bob from unknown
-- |_  Logs: /logs/
---

hadoop-namenode-info.nse
information gathered:
 * Date/Time the service was started
 * Hadoop Version
 * Hadoop Complie date
 * Upgrades status
 * Filesystem Directory (reletive to the http://host:port/)
 * Log Directory (reletive to the http://host:port/)
 * Associated Datanodes

---
-- @usage
-- nmap --script hadoop-namenode-info -p 50070 host
--
-- @output
-- PORT      STATE SERVICE         REASON
-- 50070/tcp open  hadoop-namenode syn-ack
-- | hadoop-namenode-info:
-- |   Started:  Wed May 11 22:33:44 PDT 2011
-- |   Version:  0.20.2-cdh3u1, f415ef415ef415ef415ef415ef415ef415ef415e
-- |   Compiled:  Wed May 11 22:33:44 PDT 2011 by bob from unknown
-- |   Upgrades:  There are no upgrades in progress.
-- |   Filesystem: /nn_browsedfscontent.jsp
-- |   Logs: /logs/
-- |   Storage:
-- |   Total       Used (DFS)      Used (Non DFS)  Remaining
-- |   100 TB      85 TB           500 GB          14.5 TB
-- |   Datanodes (Live):
-- |     Datanode: datanode1.example.com:50075
-- |     Datanode: datanode2.example.com:50075

hadoop-datanode-info.nse

information gathered:
 * Log Directory (reletive to the http://host:port/)

---
-- @usage
-- nmap --script hadoop-datanode-info.nse -p 50075 host
--
-- @output
-- PORT      STATE SERVICE         REASON
-- 50075/tcp open  hadoop-datanode syn-ack
-- | hadoop-datanode-info:
-- |_  Logs: /logs/


hbase-master-info.nse

information gathered:
 * Hbase Version
 * Hbase Complie date
 * Hbase Root Directory
 * Hadoop Version
 * Hadoop Complie date
 * Average Load
 * Zookeeper Quorum Server
 * Associated Region Servers

---
-- @usage
-- nmap --script hbase-master-info -p 60010 host
--
-- @output
-- | hbase-master-info:
-- |   Hbase Version: 0.90.1
-- |   Hbase Compiled: Wed May 11 22:33:44 PDT 2011, bob
-- |   HBase Root Directory: hdfs://master.example.com:8020/hbase
-- |   Hadoop Version: 0.20  f415ef415ef415ef415ef415ef415ef415ef415e
-- |   Hadoop Compiled: Wed May 11 22:33:44 PDT 2011, bob
-- |   Average Load: 0.12
-- |   Zookeeper Quorum: zookeeper.example.com:2181
-- |   Region Servers:
-- |     region1.example.com:60030
-- |_    region2.example.com:60030

hbase-region-info.nse

information gathered:
 * Hbase Version
 * Hbase Complie date
 * A bunch of metrics about the state of the region server
 * Zookeeper Quorum Server

---
-- @usage
-- nmap --script hbase-region-info -p 60030 host
--
-- @output
-- PORT      STATE SERVICE      REASON
-- 60030/tcp open  hbase-region syn-ack
-- | hbase-region-info:
-- |   Hbase Version: 0.90.1
-- |   Hbase Compiled: Wed May 11 22:33:44 PDT 2011, bob
-- |   Metrics requests=0, regions=0, stores=0, storefiles=0,
storefileIndexSize=0, memstoreSize=0,
-- |   compactionQueueSize=0, flushQueueSize=0, usedHeap=0, maxHeap=0,
blockCacheSize=0,
-- |   blockCacheFree=0, blockCacheCount=0, blockCacheHitCount=0,
blockCacheMissCount=0,
-- |   blockCacheEvictedCount=0, blockCacheHitRatio=0,
blockCacheHitCachingRatio=0
-- |_  Zookeeper Quorum: zookeeper.example.com:2181

Attachment: hbase-region-info.nse
Description:

Attachment: hbase-master-info.nse
Description:

Attachment: hadoop-tasktracker-info.nse
Description:

Attachment: hadoop-namenode-info.nse
Description:

Attachment: hadoop-datanode-info.nse
Description:

Attachment: hadoop-jobtracker-info.nse
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/

  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]
AlienVault