This looks like YARN cluster is misconfigured. Alternately, you need to
configure it to allow a few more worker tasks. Giraph on YARN at minimum
needs one Application Master, one Master, and one Worker (so 3 YARN
containers) I have a feeling this could be the issue.
On Sat, Mar 1, 2014 at 9:18 PM, Jyotirmoy Sundi <sundi...@gmail.com> wrote:
> Hi Folks,
>
> The job was working properly in MR1 without any issue. I am trying to run
> a simple CC sample Giraph job on YARN. . I have attached the stacktrace and a
> few errors. Any pointers will be really helpful for the below errors.
>
> *1. BspServiceMaster (YARN profile) is FAILING this task, throwing exception
> to end job run.*
>
> *2. java.lang.IllegalStateException: Not enough healthy workers to create
> input splits*
>
>
>
>
> *StackTrace:*
>
> 2014-03-02 04:53:24,646 INFO org.apache.giraph.master.BspServiceMaster:
> logMissingWorkersOnSuperstep: No response from partition 2 (could be master)
> 2014-03-02 04:53:24,646 ERROR org.apache.giraph.master.BspServiceMaster:
> checkWorkers: Did not receive enough processes in time (only 1 of 2 required)
> after waiting 600000msecs). This occurs if you do not have enough map tasks
> available simultaneously on your Hadoop instance to fulfill the number of
> requested workers.
> 2014-03-02 04:53:24,649 INFO org.apache.giraph.master.BspServiceMaster:
> setJobState:
> {"_stateKey":"FAILED","_applicationAttemptKey":-1,"_superstepKey":-1} on
> superstep -1
> 2014-03-02 04:53:24,653 FATAL org.apache.giraph.master.BspServiceMaster:
> failJob: Killing job job_201402281650_0019
> 2014-03-02 04:53:24,654 FATAL org.apache.giraph.master.BspServiceMaster:
> failJob: exception java.lang.IllegalStateException: Not enough healthy
> workers to create input splits
> 2014-03-02 04:53:24,654 ERROR org.apache.giraph.master.MasterThread:
> masterThread: Master algorithm failed with RuntimeException
> java.lang.RuntimeException: BspServiceMaster (YARN profile) is FAILING this
> task, throwing exception to end job run.
> at
> org.apache.giraph.master.BspServiceMaster.failJob(BspServiceMaster.java:349)
> at
> org.apache.giraph.master.BspServiceMaster.setJobStateFailed(BspServiceMaster.java:297)
> at
> org.apache.giraph.master.BspServiceMaster.createInputSplits(BspServiceMaster.java:616)
> at
> org.apache.giraph.master.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:692)
> at org.apache.giraph.master.MasterThread.run(MasterThread.java:100)
> Caused by: java.lang.IllegalStateException: Not enough healthy workers to
> create input splits
> ... 4 more
> 2014-03-02 04:53:24,656 FATAL org.apache.giraph.graph.GraphMapper:
> uncaughtException: OverrideExceptionHandler on thread
> org.apache.giraph.master.MasterThread, msg = java.lang.RuntimeException:
> BspServiceMaster (YARN profile) is FAILING this task, throwing exception to
> end job run., exiting...
> java.lang.IllegalStateException: java.lang.RuntimeException: BspServiceMaster
> (YARN profile) is FAILING this task, throwing exception to end job run.
> at org.apache.giraph.master.MasterThread.run(MasterThread.java:181)
> Caused by: java.lang.RuntimeException: BspServiceMaster (YARN profile) is
> FAILING this task, throwing exception to end job run.
> at
> org.apache.giraph.master.BspServiceMaster.failJob(BspServiceMaster.java:349)
> at
> org.apache.giraph.master.BspServiceMaster.setJobStateFailed(BspServiceMaster.java:297)
> at
> org.apache.giraph.master.BspServiceMaster.createInputSplits(BspServiceMaster.java:616)
> at
> org.apache.giraph.master.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:692)
> at org.apache.giraph.master.MasterThread.run(MasterThread.java:100)
> Caused by: java.lang.IllegalStateException: Not enough healthy workers to
> create input splits
> ... 4 more
>
> ------------------------------
>
>
