Maybe its just a configuration thing. I’ve tried running in giraph.SplitMasterWorker mode and its seems like hadoop is missing the worker nodes:
Here is my command: $ hadoop jar giraphs-and-balloons-computation-0.0.1-for-hadoop-2.5.1-and-giraph-1.1.0-RC1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner\ de.unipassau.fim.dimis.nolte.computation.FindSupertypeClustersComputation\ -eif de.unipassau.fim.dimis.nolte.io.NTriplesToDirectedGraphInputFormat\ -eip /user/hduser/input/equivalence.nt\ -vof de.unipassau.fim.dimis.nolte.io.SupertypeClustersVertexValueOutputFormat\ -op /user/hduser/output/equivalence\ -w 3\ -ca mapred.job.tracker=master:5431\ -ca giraph.SplitMasterWorker=true And it then hangs on: INFO mapred.LocalJobRunner: MASTER_ZOOKEEPER_ONLY checkWorkers: Only found 0 responses of 3 needed to start superstep -1 > map If I don’t use SplitMasterWorker, I get INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map That is why I assumed, I need to start the task tracker on the worker nodes, but task trackers are no longer available in hadoop 2.5.1. My nodes’ NodeManagers are connected to the ResourceManager, so I don’t quite understand the error message. Any ideas? Thanks in advance! Am 15.12.2014 um 03:31 schrieb Roman Shaposhnik <ro...@shaposhnik.org>: > On Sun, Dec 14, 2014 at 4:09 PM, Philipp Nolte <p...@daslaboratorium.de> > wrote: >> I’ve had a look at the assembled giraph-core jar file and it does not >> contain any GiraphYarnTask. How so? >> >> Running my application using the GiraphRunner works fine as long as I only >> have one worker (local mode). To use the other workers, I need to start MR >> TaskTrackers on the machines - which aren’t available on hadoop 2.5.1. >> >> Thats why I need the GiraphYarnTask. > > Looks like we're talking past each other. What I am saying is that running > pure MR-based Giraph job on a fully distributed YARNized cluster is perfectly > valid and works fine. You don't *have* to use YARN, even though it is > available on your cluster. > > Thanks, > Roman.
signature.asc
Description: Message signed with OpenPGP using GPGMail
