Sergey Edunov created GIRAPH-992:
------------------------------------
Summary: Zookeeper logs have too many NodeExists exceptions
Key: GIRAPH-992
URL: https://issues.apache.org/jira/browse/GIRAPH-992
Project: Giraph
Issue Type: Bug
Reporter: Sergey Edunov
There are several places in our code where we do not check properly if zk node
already exists before attempting to create it. As a result ZK logs are full of
these exceptions.
Biggest offender is recursive path creation in ZooKeeperExt.createExt().
Obviously if part of the path already exists we don't need to create it.
Second biggest offender is writing input splits from master. Here we lunch
multiple threads each of them attempting to create the same path.
INFO 2015-02-04 14:32:39,730 [ProcessThread(sid:0 cport:-1):]
org.apache.zookeeper.server.PrepRequestProcessor - Got user-level
KeeperException when processing sessionid:0x14b56b9176f0001 type:create
cxid:0x1 zxid:0x19 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201411061513.83344_0001/_masterJobState
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201411061513.83344_0001/_masterJobState
INFO 2015-02-04 14:32:39,740 [ProcessThread(sid:0 cport:-1):]
org.apache.zookeeper.server.PrepRequestProcessor - Got user-level
KeeperException when processing sessionid:0x14b56b9176f0001 type:create
cxid:0x3 zxid:0x1a txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir
INFO 2015-02-04 14:32:39,742 [ProcessThread(sid:0 cport:-1):]
org.apache.zookeeper.server.PrepRequestProcessor - Got user-level
KeeperException when processing sessionid:0x14b56b9176f0001 type:create
cxid:0x5 zxid:0x1b txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir/0/_superstepDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir/0/_superstepDir
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
