1. How to check the logs for partition job failure?
Also based on my earlier partition failure, I wanted to check the
logs for the job "job_201308091403_0002"
however, when I change directory under "tasklogs" I do not see a
directory with above name.
2. Also the partition failure comes only the first time I run the examples
jar. That is to reproduce I have to stop/start bspd so that I get partition
error
3.
On Fri, Aug 9, 2013 at 2:10 PM, Mahesh Babu <[email protected]> wrote:
> Hi Anastasis,
>
> Thanks for the reply.
>
> I believe the input is okay. I am able to run the input files in
> standalone (eclipse) mode as unit-test. This time I copied them into HDFS
> and running through hama-examples<x>.jar file.
>
> No i do not have hama.check.missing.vertex in my configurations.
>
> After your email I set the hama-site.xml with
> <property>
> <name>hama.check.missing.vertex</name>
> <value>true</value>
> </property>
>
> Now I get partition error: (with hama.check.missing.vertex set in
> hama-site.xml)
> 13/08/09 14:03:49 INFO bsp.FileInputFormat: Total input paths to process :
> 1
> 13/08/09 14:03:50 INFO bsp.BSPJobClient: Running job: job_201308091403_0002
> 13/08/09 14:03:53 INFO bsp.BSPJobClient: Current supersteps number: 0
> *13/08/09 14:03:53 INFO bsp.BSPJobClient: Job failed.
> 13/08/09 14:03:53 ERROR bsp.BSPJobClient: Error partitioning the input
> path.*
> *java.io.IOException: Runtime partition failed for the job.*
> * at org.apache.hama.bsp.BSPJobClient.partition(BSPJobClient.java:465)*
> at
> org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:333)
> at org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293)
> at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:229)
> at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203)
> at org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:236)
> at org.apache.hama.examples.SSSP.main(SSSP.java:166)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> Any clues.. It is quite large file, so I have to find a way to identify
> missing vertex.
>
> Regards,
> Mahesh Babu
>
>
>
>
> On Fri, Aug 9, 2013 at 1:08 PM, andronat_asf <[email protected]>wrote:
>
>> Hello Mahesh,
>>
>> 1) Do you have hama.check.missing.vertex = true flag in any of your
>> configurations?
>> 2) Are you sure that your input is ok? e.g. edges to non existing
>> vertices?
>>
>> Cheers,
>> Anastasis Andronidis
>>
>> On 9 Αυγ 2013, at 9:43 π.μ., Mahesh Babu <[email protected]> wrote:
>>
>> > Hi,
>> >
>> > adding more details about the environment.
>> >
>> > When I run hama-examples-0.6.2.jar with tutorial example/input (berlin
>> example) file it works without any error.
>> >
>> > Now when I try my own input file which has approximately 10k nodes and
>> 39k edges but it fails with:
>> > java.lang.IllegalArgumentException: Messages must never be
>> behind the vertex in ID! Current Message ID: 2 vs. 20
>> >
>> > What am I doing wrong ?
>> >
>> > Any help/explanation is appreciated.
>> >
>> > Regards,
>> > Mahesh Babu
>> >
>> >
>> >
>> >
>> > On Thu, Aug 8, 2013 at 4:25 PM, Mahesh Babu <[email protected]> wrote:
>> > Hi,
>> >
>> > I followed the SSSP example, GraphJob example and getting following
>> error:
>> >
>> > 13/08/08 16:01:57 INFO bsp.BSPJobClient: Current supersteps number: 0
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 INFO mortbay.log:
>> Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
>> org.mortbay.log.Slf4jLog
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 INFO
>> sync.ZKSyncClient: Initializing ZK Sync Client
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 INFO
>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At localhost/
>> 127.0.0.1:61001
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 WARN
>> message.MessageTransferQueueFactory: Message queue is configured on
>> deprecated parameter:hama.messenger.queue.class
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 WARN
>> message.MessageTransferQueueFactory: Message queue is configured on
>> deprecated parameter:hama.messenger.queue.class
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 INFO ipc.Server:
>> Starting SocketReader
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 INFO ipc.Server:
>> IPC Server Responder: starting
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 INFO ipc.Server:
>> IPC Server listener on 61001: starting
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 INFO
>> message.HadoopMessageManagerImpl: BSPPeer address:localhost port:61001
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 INFO ipc.Server:
>> IPC Server handler 0 on 61001: starting
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 INFO
>> graph.GraphJobRunner: 4935 vertices are loaded into localhost:61001
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 WARN
>> message.MessageTransferQueueFactory: Message queue is configured on
>> deprecated parameter:hama.messenger.queue.class
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 WARN
>> message.MessageTransferQueueFactory: Message queue is configured on
>> deprecated parameter:hama.messenger.queue.class
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 WARN
>> message.MessageTransferQueueFactory: Message queue is configured on
>> deprecated parameter:hama.messenger.queue.class
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 WARN
>> message.MessageTransferQueueFactory: Message queue is configured on
>> deprecated parameter:hama.messenger.queue.class
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 WARN
>> message.MessageTransferQueueFactory: Message queue is configured on
>> deprecated parameter:hama.messenger.queue.class
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 ERROR
>> bsp.BSPTask: Error running bsp setup and bsp function.
>> > attempt_201308081548_0005_000000_0: java.lang.IllegalArgumentException:
>> Messages must never be behind the vertex in ID! Current Message ID: 2 vs. 20
>> > attempt_201308081548_0005_000000_0: at
>> org.apache.hama.graph.GraphJobRunner.iterate(GraphJobRunner.java:284)
>> > attempt_201308081548_0005_000000_0: at
>> org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:229)
>> > attempt_201308081548_0005_000000_0: at
>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133)
>> > attempt_201308081548_0005_000000_0: at
>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:177)
>> > attempt_201308081548_0005_000000_0: at
>> org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
>> > attempt_201308081548_0005_000000_0: at
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1262)
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 WARN
>> message.MessageTransferQueueFactory: Message queue is configured on
>> deprecated parameter:hama.messenger.queue.class
>> > attempt_201308081548_0005_000000_0: 13/08/08 16:00:49 INFO ipc.Server:
>> Stopping server on 61001
>> >
>> >
>> > What could be the reason I am getting this error.
>> >
>> > I am able to run the code in eclipse/standalong environment. However,
>> when I try to run with hama/hadoop pseudo distribution mode it fails.
>> >
>> > Thanks,
>> > Mahesh Babu
>> >
>> >
>> > <hamaError.txt>
>>
>>
>
