Hi ,
I got the connected component working for 1B nodes, but when I run
the job again, it fails with the below error. Aprt form this in
zookeeper the data is not cleared in the data directory. For
successful jobs the data in zookeper from giraph is cleared.
The following errors seems to be coming because the node tries to
connect to the zookeeper with a session id which is cleared as seens
in
"Client session timed out, have not heard from server in 68845ms for
sessionid 0x3415cc6ce930059, closing socket connection and attempting
reconnect" , Any idea if increasing the session time out will be good
?
2013-09-27 00:57:11,748 WARN org.apache.giraph.bsp.BspService:
process: Got unknown null path event WatchedEvent state:Expired
type:None path:null
2013-09-27 00:57:11,748 INFO org.apache.zookeeper.ClientCnxn: Unable
to reconnect to ZooKeeper service, session 0x3415cc6ce930059 has
expired, closing socket connection
2013-09-27 00:57:11,748 WARN
org.apache.giraph.worker.InputSplitsHandler: process: Problem with
zookeeper, got event with path null, state Expired, event type None
2013-09-27 00:57:11,748 INFO org.apache.zookeeper.ClientCnxn:
EventThread shut down
2013-09-27 00:57:11,925 INFO
org.apache.giraph.worker.InputSplitsCallable: loadFromInputSplit:
Finished loading
/_hadoopBsp/job_201309260044_0116/_vertexInputSplitDir/89 (v=258127,
e=1792906)
2013-09-27 00:57:11,926 ERROR
org.apache.giraph.utils.LogStacktraceCallable: Execution of callable
failed
java.lang.IllegalStateException: markInputSplitPathFinished:
KeeperException on
/_hadoopBsp/job_201309260044_0116/_vertexInputSplitDir/89/_vertexInputSplitFinished
at
org.apache.giraph.worker.InputSplitsHandler.markInputSplitPathFinished(InputSplitsHandler.java:168)
at
org.apache.giraph.worker.InputSplitsCallable.loadInputSplit(InputSplitsCallable.java:226)
at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:161)
at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:58)
at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for
/_hadoopBsp/job_201309260044_0116/_vertexInputSplitDir/89/_vertexInputSplitFinished
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.giraph.zk.ZooKeeperExt.createExt(ZooKeeperExt.java:152)
at
org.apache.giraph.worker.InputSplitsHandler.markInputSplitPathFinished(InputSplitsHandler.java:159)
... 9 more
--
Best Regards,
Jyotirmoy Sundi
Data Engineer,
Admobius
San Francisco, CA 94158
