Well I have a test app where it just reads the input, which is:
1 2 3 4 5
2 1 3
3 1 2 4
4 1 3
5 1
and I just send a message but it doesn't even start. The error is:
13/10/19 18:56:27 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
java.lang.NullPointerException
at
org.apache.hama.bsp.PartitioningRunner.bsp(PartitioningRunner.java:207)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:256)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
13/10/19 18:56:27 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
java.util.concurrent.ExecutionException: java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hama.bsp.PartitioningRunner.bsp(PartitioningRunner.java:207)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:256)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
... 1 more
13/10/19 18:56:30 INFO bsp.BSPJobClient: Current supersteps number: 1
13/10/19 18:56:30 INFO bsp.BSPJobClient: Job failed.
13/10/19 18:56:30 ERROR bsp.BSPJobClient: Error partitioning the input path.
Exception in thread "main" java.io.IOException: Runtime partition
failed for the job.
at org.apache.hama.bsp.BSPJobClient.partition(BSPJobClient.java:465)
at
org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:333)
at org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293)
at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:228)
at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203)
at org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:235)
at hamatest.HamaTest.main(HamaTest.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hama.util.RunJar.main(RunJar.java:146)
What is this NullPointerException? I've seen it when NumBspTasks
wasn't specified, but I have job.setNumBspTasks(1) and still happens.
I am also running in local mode and the only thing specified in
hama-site.xml is the hdfs.
Another issue with the same jar.
When I run it again on local mode and nothing is mentioned in
hama-site.xml, the output is this:
13/10/19 19:12:48 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes
where applicable
13/10/19 19:12:48 INFO bsp.FileInputFormat: Total input paths to process : 1
13/10/19 19:12:48 WARN snappy.LoadSnappy: Snappy native library not loaded
13/10/19 19:12:49 INFO bsp.FileInputFormat: Total input paths to process : 1
13/10/19 19:12:49 INFO bsp.BSPJobClient: Running job: job_localrunner_0001
13/10/19 19:12:49 INFO bsp.LocalBSPRunner: Setting up a new barrier
for 1 tasks!
13/10/19 19:12:52 INFO bsp.BSPJobClient: Current supersteps number: 1
13/10/19 19:12:52 INFO bsp.BSPJobClient: The total number of supersteps: 1
13/10/19 19:12:52 INFO bsp.BSPJobClient: Counters: 6
13/10/19 19:12:52 INFO bsp.BSPJobClient:
org.apache.hama.bsp.JobInProgress$JobCounter
13/10/19 19:12:52 INFO bsp.BSPJobClient: SUPERSTEPS=1
13/10/19 19:12:52 INFO bsp.BSPJobClient: LAUNCHED_TASKS=1
13/10/19 19:12:52 INFO bsp.BSPJobClient:
org.apache.hama.bsp.BSPPeerImpl$PeerCounter
13/10/19 19:12:52 INFO bsp.BSPJobClient: SUPERSTEP_SUM=2
13/10/19 19:12:52 INFO bsp.BSPJobClient: IO_BYTES_READ=35
13/10/19 19:12:52 INFO bsp.BSPJobClient: TIME_IN_SYNC_MS=0
13/10/19 19:12:52 INFO bsp.BSPJobClient: TASK_INPUT_RECORDS=6
Exception in thread "main"
org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist: /tmp/hama-parts/job_localrunner_0001
at
org.apache.hama.bsp.FileInputFormat.listStatus(FileInputFormat.java:163)
at
org.apache.hama.bsp.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:45)
at
org.apache.hama.bsp.FileInputFormat.getSplits(FileInputFormat.java:180)
at
org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:337)
at org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293)
at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:228)
at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203)
at org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:235)
at hamatest.HamaTest.main(HamaTest.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hama.util.RunJar.main(RunJar.java:146)
I don't understand again why this is happening. The input path I have
specified is not "/tmp/hama-parts/job_localrunner_0001". I didn't
repsond immediately cause I wanted to search it more by myself but I
give up.
Hope you can help.
Ilias
Quoting Anastasis Andronidis <[email protected]>:
Hey,
I suggest you try the following as a general debug approach:
1) Can you test your input alone with a simple job that is just
parsing it? You can be sure at this point that something is not
wrong inside you code. Like lib dependences with problems, etc.
2) Check your job config. You might have keys and values to your
classes that are not fine represented, so Hama can not parse it.
3) Try to run Hama in local mode, just to be sure it's not a network problem.
If you still have problem, can you send a part of your input that
you just checked that fails? So we can search for what is wrong.
Cheers,
Anastasis
On 17 Οκτ 2013, at 9:34 μ.μ., Ηλίας Καπουράνης <[email protected]> wrote:
Hey,
I only have 1 job. I don't know why it starts 2 jobs.
Στις 17/10/2013 9:08 μμ, ο/η Anastasis Andronidis έγραψε:
Hi,
this seams to be a problem with the input. If you say that your
first job produces nothing, then the second job is getting null
input and this is probably why you get this error. I think your
problem might be in your first job.
Can you separate the jobs and investigate why this is happening?
Or can you create a custom input for your second job and run it
with a specific input to be sure that the problem is there?
Cheers,
Anastasis
On 17 Οκτ 2013, at 6:44 μ.μ., [email protected] wrote:
Hello there,
i just downloaded the version 0.6.3, did the configurations and
wanted to run my programs. Previously I was using hama 0.6.0. The
examples are working fine but when I give one jar of mine to be
run, I have the following issue:
- It starts 2 jobs and after the first is completed without doing
anything, the second hangs and prints this:
attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO
ipc.Server: Starting Socket Reader #1 for port 52820
attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO
ipc.Server: IPC Server Responder: starting
attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO
ipc.Server: IPC Server listener on 52820: starting
attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO
ipc.Server: IPC Server handler 0 on 52820: starting
attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO
message.HadoopMessageManagerImpl: BSPPeer address:master
port:52820
attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO
sync.ZKSyncClient: Initializing ZK Sync Client
attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO
sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
master/192.168.1.2:52820
attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 ERROR
bsp.BSPTask: Error running bsp setup and bsp function.
attempt_201310172123_0009_000000_0: java.lang.NullPointerException
attempt_201310172123_0009_000000_0: at
org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:430)
attempt_201310172123_0009_000000_0: at
org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:114)
attempt_201310172123_0009_000000_0: at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176)
attempt_201310172123_0009_000000_0: at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
attempt_201310172123_0009_000000_0: at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1246)
attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 INFO
ipc.Server: Stopping server on 52820
attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 INFO
ipc.Server: IPC Server handler 0 on 52820: exiting
attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 INFO
ipc.Server: Stopping IPC Server listener on 52820
attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 INFO
ipc.Server: Stopping IPC Server Responder
attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 ERROR
bsp.BSPTask: Shutting down ping service.
attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 FATAL
bsp.GroomServer: Error running child
attempt_201310172123_0009_000000_0: java.lang.NullPointerException
attempt_201310172123_0009_000000_0: at
org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:430)
attempt_201310172123_0009_000000_0: at
org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:114)
attempt_201310172123_0009_000000_0: at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176)
attempt_201310172123_0009_000000_0: at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
attempt_201310172123_0009_000000_0: at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1246)
attempt_201310172123_0009_000000_0: java.lang.NullPointerException
attempt_201310172123_0009_000000_0: at
org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:430)
attempt_201310172123_0009_000000_0: at
org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:114)
attempt_201310172123_0009_000000_0: at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176)
attempt_201310172123_0009_000000_0: at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
attempt_201310172123_0009_000000_0: at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1246)
13/10/17 21:38:44 INFO bsp.BSPJobClient: Job failed.
From what I see, it has to do something either with the setup and
bsp functions or with the reader because there is
NullPointerException at loadVertices.
I don't think it is the first one cause I am using hama Graph.
Any ideas?
Thanks
