Hi, I am trying to setup Hadoop on Demand on a cluster running Condor. I downloaded hod-open-4.tar.gz from here:
http://issues.apache.org/jira/browse/HADOOP-1301 First of all I am not sure if this is the most recent version to use or if this the appropriate place to retrieve it. The website for Hadoop does not seem to have a download link for hod even though it contains documentation for hod. The version I download seems to have some support for condor but it is not documented. I tryed to setup the hodrc as best I could but with not much sucesses. I have included a copy of my config file below if someone could help me to get hod working I would be grateful. When I run the following command: ./hod -c ../conf/hodrc -m 3 -a "run dfs -ls /" I recieve the following errors: Failed to start hod shell: <type 'exceptions.AttributeError'> NodePoolDesc instance has no attribute 'getPkgDirs' Traceback (most recent call last): File "hod", line 463, in <module> exitStatus = main(hodConfig) File "/home/ugrad/ddriddle/src/hod/hodlib/Hod/hod.py", line 120, in main if np: np.finalize() File "/home/ugrad/ddriddle/src/hod/hodlib/NodePools/condor.py", line 73, in finalize self.freeNodeSet(ns) File "/home/ugrad/ddriddle/src/hod/hodlib/NodePools/condor.py", line 64, in freeNodeSet self._delNodeSet(nodeSet) File "/home/ugrad/ddriddle/src/hod/hodlib/NodePools/condor.py", line 219, in _delNodeSet pkgdirs = npd.getPkgDirs() AttributeError: NodePoolDesc instance has no attribute 'getPkgDirs' Thanks, David Riddle # Template hodrc file # # -- hod shell specific configuration [hod] temp-dir = /tmp/hod-ddriddle #java-home = ${JAVA_HOME} java-home = /home/ugrad/ddriddle/lib/java debug = 3 stream = True non-interactive = false allocate-timeout = 3600 #cluster = ${CLUSTER_NAME} cluster = hal9000 #hadoop-install = ${HADOOP_HOME} hadoop-install = /home/ugrad/ddriddle/src/hadoop-0.15.3 #log-dir = /var/hod/logs-${RM_USER} #log-collection-dir = /var/hod/logs-${RM_USER} #syslog-address = localhost:8814 #cluster-factor = 1.8 # -- resource manager configuration [resource_manager] #id = ${RM_ID} id = condor #pbs-user = ${RM_USER} pbs-user = ddriddle pbs-server = hal9000-server.cs.uiuc.edu queue = parallel batch-home = /home/hal/condor/condor env-vars = HOD_PYTHON_HOME=/home/ugrad/ddriddle/bin # -- serive_registry configuration [service_registry] debug = 2 http-address = :8002 xrs-address = :8879 #stream = False #log-dir = /var/hod/logs-${RM_USER} #xrs-port-range = 32768-65536 # -- ringmaster configuration [ringmaster] temp-dir = /tmp/hod-ddriddle work-dirs = /tmp/hod-ddriddle/1,/tmp/hod-ddriddle/2,/tmp/hod-ddriddle/3,/tmp/hod-ddriddle/4 register = true xrs-port-range = 32768-65536 http-port-range = 8000-9000 debug = 2 #stream = False #syslog-address = localhost:8814 #log-dir = /var/hod/logs-${RM_USER} #svcrgy-addr = :8879 # -- hod ring configuration [hodring] temp-dir = /tmp/hod-ddriddle register = true #java-home = ${JAVA_HOME} java-home = /home/ugrad/ddriddle/lib/java debug = 2 xrs-port-range = 32768-65536 http-port-range = 8000-9000 #log-dir = /var/hod/logs-${RM_USER} #log-destination-uri = local://var/log-${RM_USER} #stream = false #syslog-address = localhost:8814 #svcrgy-addr = :8879 [gridservice-mapred] external = false tracker_port = 8030 info_port = 50080 #pkgs = ${HADOOP_HOME} pkgs = /home/ugrad/ddriddle/src/hadoop-0.15.3/ #host = localhost [gridservice-hdfs] external = false fs_port = 8020 info_port = 50070 pkgs = /home/ugrad/ddriddle/src/hadoop-0.15.3/ #pkgs = ${HADOOP_HOME} #host = localhost -- Always code as if the (person) who ends up maintaining your code will be a violent psychopath who knows where you live. - M. Golding
