[jira] [Created] (HDDS-721) NullPointerException thrown while trying to read a file when datanode restarted
Nilotpal Nandi created HDDS-721:
---
Summary: NullPointerException thrown while trying to read a file
when datanode restarted
Key: HDDS-721
URL: https://issues.apache.org/jira/browse/HDDS-721
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Datanode
Affects Versions: 0.3.0
Reporter: Nilotpal Nandi
steps taken :
---
# Put few files and directories using ozonefs
# stopped all services of cluster.
# started the scm, om and then datanodes.
While datanodes were starting up, tried to read a file. Null pointer Exception
was thrown.
{noformat}
[root@ctr-e138-1518143905142-53-01-03 ~]#
/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/bin/ozone fs -ls -R /
2018-10-24 04:48:00,703 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
drwxrwxrwx - root root 0 2018-10-24 04:12 /testdir1
-rw-rw-rw- 1 root root 5368709120 1970-02-25 15:29 /testdir1/5GB
-rw-rw-rw- 1 root root 4798 1970-02-25 15:22 /testdir1/passwd
drwxrwxrwx - root root 0 2018-10-24 04:46 /testdir3
[root@ctr-e138-1518143905142-53-01-03 ~]#
/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/bin/ozone fs -cat
o3fs://fs-bucket.fs-volume/testdir1/passwd
2018-10-24 04:49:24,955 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
cat: Exception getting XceiverClient:
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.NullPointerException{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-722) ozone datanodes failed to start on few nodes
Nilotpal Nandi created HDDS-722:
---
Summary: ozone datanodes failed to start on few nodes
Key: HDDS-722
URL: https://issues.apache.org/jira/browse/HDDS-722
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Datanode
Affects Versions: 0.3.0
Reporter: Nilotpal Nandi
steps taken :
--
# put few keys using ozonefs.
# stopped all services of the cluster.
# started om and scm.
# After sometime , started datanodes.
All datanodes failed to start . Out of 12 datanodes, 4 datanodes failed to
start.
Here is the datanode log snippet :
{noformat}
2018-10-24 04:49:30,594 ERROR org.apache.ratis.server.impl.StateMachineUpdater:
Terminating with exit status 2:
StateMachineUpdater-9524f4e2-9031-4852-ab7c-11c2da3460db: the
StateMachineUpdater hits Throwable
org.apache.ratis.server.storage.RaftLogIOException: java.io.IOException:
Premature EOF from inputStream
at org.apache.ratis.server.storage.LogSegment.loadCache(LogSegment.java:299)
at
org.apache.ratis.server.storage.SegmentedRaftLog.get(SegmentedRaftLog.java:192)
at
org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:142)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Premature EOF from inputStream
at org.apache.ratis.util.IOUtils.readFully(IOUtils.java:100)
at org.apache.ratis.server.storage.LogReader.decodeEntry(LogReader.java:250)
at org.apache.ratis.server.storage.LogReader.readEntry(LogReader.java:155)
at
org.apache.ratis.server.storage.LogInputStream.nextEntry(LogInputStream.java:128)
at
org.apache.ratis.server.storage.LogSegment.readSegmentFile(LogSegment.java:110)
at org.apache.ratis.server.storage.LogSegment.access$400(LogSegment.java:43)
at
org.apache.ratis.server.storage.LogSegment$LogEntryLoader.load(LogSegment.java:167)
at
org.apache.ratis.server.storage.LogSegment$LogEntryLoader.load(LogSegment.java:161)
at org.apache.ratis.server.storage.LogSegment.loadCache(LogSegment.java:295)
... 3 more
2018-10-24 04:49:30,598 INFO org.apache.hadoop.ozone.HddsDatanodeService:
SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down HddsDatanodeService at
ctr-e138-1518143905142-541661-01-03.hwx.site/172.27.57.0
/
2018-10-24 04:49:30,598 WARN org.apache.hadoop.fs.CachingGetSpaceUsed: Thread
Interrupted waiting to refresh disk information: sleep interrupted
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-723) CloseContainerCommandHandler throwing NullPointerException
Nilotpal Nandi created HDDS-723:
---
Summary: CloseContainerCommandHandler throwing NullPointerException
Key: HDDS-723
URL: https://issues.apache.org/jira/browse/HDDS-723
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: SCM
Affects Versions: 0.3.0
Reporter: Nilotpal Nandi
Seeing NullPointerException error while CloseContainerCommandHandler is trying
to close container.
{noformat}
2018-10-24 04:22:04,699 INFO org.apache.ratis.server.storage.RaftLogWorker:
8a61160b-8985-412e-9f25-9e65ceafa824-RaftLogWorker got closed and hit exception
java.io.IOException: java.lang.InterruptedException
at org.apache.ratis.util.IOUtils.asIOException(IOUtils.java:51)
at
org.apache.ratis.server.storage.RaftLogWorker.flushWrites(RaftLogWorker.java:232)
at
org.apache.ratis.server.storage.RaftLogWorker.access$600(RaftLogWorker.java:51)
at
org.apache.ratis.server.storage.RaftLogWorker$WriteLog.execute(RaftLogWorker.java:309)
at org.apache.ratis.server.storage.RaftLogWorker.run(RaftLogWorker.java:179)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.InterruptedException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:347)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at
org.apache.ratis.server.storage.RaftLogWorker.flushWrites(RaftLogWorker.java:230)
... 4 more
2018-10-24 04:22:04,712 INFO org.apache.ratis.server.storage.RaftLogWorker:
8a61160b-8985-412e-9f25-9e65ceafa824-RaftLogWorker close()
2018-10-24 04:22:31,293 ERROR
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler:
Can't close container 18
java.lang.NullPointerException
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler.handle(CloseContainerCommandHandler.java:78)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CommandDispatcher.handle(CommandDispatcher.java:93)
at
org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$initCommandHandlerThread$1(DatanodeStateMachine.java:381)
at java.lang.Thread.run(Thread.java:745)
2018-10-24 04:22:31,293 ERROR
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler:
Can't close container 10
java.lang.NullPointerException
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler.handle(CloseContainerCommandHandler.java:78)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CommandDispatcher.handle(CommandDispatcher.java:93)
at
org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$initCommandHandlerThread$1(DatanodeStateMachine.java:381)
at java.lang.Thread.run(Thread.java:745)
2018-10-24 04:22:31,293 ERROR
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler:
Can't close container 14
java.lang.NullPointerException
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler.handle(CloseContainerCommandHandler.java:78)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CommandDispatcher.handle(CommandDispatcher.java:93)
at
org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$initCommandHandlerThread$1(DatanodeStateMachine.java:381)
at java.lang.Thread.run(Thread.java:745){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-725) Exception thrown in loop while trying to write a file in ozonefs
Nilotpal Nandi created HDDS-725:
---
Summary: Exception thrown in loop while trying to write a file in
ozonefs
Key: HDDS-725
URL: https://issues.apache.org/jira/browse/HDDS-725
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Manager
Affects Versions: 0.3.0
Environment: Ran the following command :
ozone fs -put 2GB /testdir5/
Exceptions are thrown continuously in loop. Please note that there are 8
datanodes alive in the cluster.
{noformat}
root@ctr-e138-1518143905142-53-01-08 logs]# /root/allssh.sh 'jps -l |
grep Datanode'
Host::172.27.20.96
411564 org.apache.hadoop.ozone.HddsDatanodeService
Host::172.27.20.91
472897 org.apache.hadoop.ozone.HddsDatanodeService
Host::172.27.38.9
351139 org.apache.hadoop.ozone.HddsDatanodeService
Host::172.27.24.90
314304 org.apache.hadoop.ozone.HddsDatanodeService
Host::172.27.15.139
324820 org.apache.hadoop.ozone.HddsDatanodeService
Host::172.27.10.199
Host::172.27.15.131
Host::172.27.57.0
Host::172.27.23.139
627053 org.apache.hadoop.ozone.HddsDatanodeService
Host::172.27.68.65
557443 org.apache.hadoop.ozone.HddsDatanodeService
Host::172.27.19.74
Host::172.27.85.64
508121 org.apache.hadoop.ozone.HddsDatanodeService{noformat}
{noformat}
2018-10-24 09:49:47,093 INFO org.apache.ratis.server.impl.LeaderElection:
7c3b2fb1-cf16-4e5f-94dc-8a089492ad57: Election REJECTED; received 0 response(s)
[] and 2 exception(s); 7c3b2fb1-cf16-4e5f-94dc-8a089492ad57:t16296,
leader=null, voted=7c3b2fb1-cf16-4e5f-94dc-8a089492ad57, raftlog=[(t:37,
i:271)], conf=271: [7c3b2fb1-cf16-4e5f-94dc-8a089492ad57:172.27.85.64:9858,
86f9e313-ae49-4675-95d7-27856641aee1:172.27.15.131:9858,
9524f4e2-9031-4852-ab7c-11c2da3460db:172.27.57.0:9858], old=null
2018-10-24 09:49:47,093 INFO org.apache.ratis.server.impl.LeaderElection: 0:
java.util.concurrent.ExecutionException:
org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: UNAVAILABLE: io
exception
2018-10-24 09:49:47,093 INFO org.apache.ratis.server.impl.LeaderElection: 1:
java.util.concurrent.ExecutionException:
org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: UNAVAILABLE: io
exception
2018-10-24 09:49:47,093 INFO org.apache.ratis.server.impl.RaftServerImpl:
7c3b2fb1-cf16-4e5f-94dc-8a089492ad57 changes role from CANDIDATE to FOLLOWER at
term 16296 for changeToFollower
2018-10-24 09:49:47,093 INFO org.apache.ratis.server.impl.RoleInfo:
7c3b2fb1-cf16-4e5f-94dc-8a089492ad57: shutdown LeaderElection
2018-10-24 09:49:47,093 INFO org.apache.ratis.server.impl.RoleInfo:
7c3b2fb1-cf16-4e5f-94dc-8a089492ad57: start FollowerState
2018-10-24 09:49:48,171 INFO org.apache.ratis.server.impl.FollowerState:
7c3b2fb1-cf16-4e5f-94dc-8a089492ad57 changes to CANDIDATE, lastRpcTime:1078,
electionTimeout:1078ms
2018-10-24 09:49:48,171 INFO org.apache.ratis.server.impl.RoleInfo:
7c3b2fb1-cf16-4e5f-94dc-8a089492ad57: shutdown FollowerState
2018-10-24 09:49:48,171 INFO org.apache.ratis.server.impl.RaftServerImpl:
7c3b2fb1-cf16-4e5f-94dc-8a089492ad57 changes role from FOLLOWER to CANDIDATE at
term 16296 for changeToCandidate
2018-10-24 09:49:48,172 INFO org.apache.ratis.server.impl.RoleInfo:
7c3b2fb1-cf16-4e5f-94dc-8a089492ad57: start LeaderElection
2018-10-24 09:49:48,173 INFO org.apache.ratis.server.impl.LeaderElection:
7c3b2fb1-cf16-4e5f-94dc-8a089492ad57: begin an election in Term 16297
2018-10-24 09:49:48,174 INFO org.apache.ratis.server.impl.LeaderElection:
7c3b2fb1-cf16-4e5f-94dc-8a089492ad57 got exception when requesting votes: {}
java.util.concurrent.ExecutionException:
org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: UNAVAILABLE: io
exception
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.ratis.server.impl.LeaderElection.waitForResults(LeaderElection.java:214)
at
org.apache.ratis.server.impl.LeaderElection.askForVotes(LeaderElection.java:146)
at org.apache.ratis.server.impl.LeaderElection.run(LeaderElection.java:102)
Caused by: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
UNAVAILABLE: io exception
at
org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:222)
at
org.apache
[jira] [Created] (HDDS-727) ozone.log is not getting created in logs directory
Nilotpal Nandi created HDDS-727: --- Summary: ozone.log is not getting created in logs directory Key: HDDS-727 URL: https://issues.apache.org/jira/browse/HDDS-727 Project: Hadoop Distributed Data Store Issue Type: Bug Components: Ozone Manager Affects Versions: 0.3.0 Reporter: Nilotpal Nandi ozone.log is no more present in logs directory of datanodes. Need to be added back. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-736) scm UI conf page throwing error 'HTTP ERROR 500'
Nilotpal Nandi created HDDS-736: --- Summary: scm UI conf page throwing error 'HTTP ERROR 500' Key: HDDS-736 URL: https://issues.apache.org/jira/browse/HDDS-736 Project: Hadoop Distributed Data Store Issue Type: Bug Components: SCM Affects Versions: 0.3.0 Reporter: Nilotpal Nandi Attachments: error_scm_ui.png scm UI conf page throwing error 'HTTP ERROR 500' . Please refer the screenshot. !error_scm_ui.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-740) datanode failed to start due 'Corrupted log header'
Nilotpal Nandi created HDDS-740:
---
Summary: datanode failed to start due 'Corrupted log header'
Key: HDDS-740
URL: https://issues.apache.org/jira/browse/HDDS-740
Project: Hadoop Distributed Data Store
Issue Type: Bug
Affects Versions: 0.3.0
Reporter: Nilotpal Nandi
steps taken :
--
# wrote 5GB files through ozonefs
# stopped datanodes, scm , om.
# started all services.
# Tried to read the file.
One of the datanodes failed to start. Throwing
"java.lang.IllegalStateException: Corrupted log header"
{noformat}
2018-10-26 10:26:01,317 ERROR org.apache.ratis.server.storage.LogInputStream:
caught exception initializing log_inprogress_293
java.lang.IllegalStateException: Corrupted log header: ^@^@^@^@^@^@^@^@
at org.apache.ratis.util.Preconditions.assertTrue(Preconditions.java:60)
at org.apache.ratis.server.storage.LogInputStream.init(LogInputStream.java:93)
at
org.apache.ratis.server.storage.LogInputStream.nextEntry(LogInputStream.java:120)
at
org.apache.ratis.server.storage.LogSegment.readSegmentFile(LogSegment.java:111)
at org.apache.ratis.server.storage.LogSegment.loadSegment(LogSegment.java:133)
at
org.apache.ratis.server.storage.RaftLogCache.loadSegment(RaftLogCache.java:110)
at
org.apache.ratis.server.storage.SegmentedRaftLog.loadLogSegments(SegmentedRaftLog.java:151)
at
org.apache.ratis.server.storage.SegmentedRaftLog.open(SegmentedRaftLog.java:120)
at org.apache.ratis.server.impl.ServerState.initLog(ServerState.java:191)
at org.apache.ratis.server.impl.ServerState.(ServerState.java:114)
at org.apache.ratis.server.impl.RaftServerImpl.(RaftServerImpl.java:106)
at
org.apache.ratis.server.impl.RaftServerProxy.lambda$newRaftServerImpl$2(RaftServerProxy.java:196)
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at
java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
2018-10-26 10:26:03,671 INFO
org.apache.hadoop.ozone.web.netty.ObjectStoreRestHttpServer: Listening HDDS
REST traffic on /0.0.0.0:9880
2018-10-26 10:26:03,672 INFO org.apache.hadoop.ozone.HddsDatanodeService:
Started plug-in org.apache.hadoop.ozone.web.OzoneHddsDatanodeService@1e411d81
2018-10-26 10:26:03,676 INFO
org.apache.hadoop.ozone.container.ozoneimpl.OzoneContainer: Attempting to start
container services.
2018-10-26 10:26:03,676 INFO
org.apache.hadoop.ozone.container.common.transport.server.ratis.XceiverServerRatis:
Starting XceiverServerRatis 0d7f5327-df16-40fe-ac88-7ed06e76a20f at port 9858
2018-10-26 10:26:03,702 ERROR
org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine:
Unable to start the DatanodeState Machine
java.io.IOException: java.lang.IllegalStateException: Corrupted log header:
^@^@^@^@^@^@^@^@
at org.apache.ratis.util.IOUtils.asIOException(IOUtils.java:51)
at
org.apache.ratis.server.storage.LogInputStream.nextEntry(LogInputStream.java:123)
at
org.apache.ratis.server.storage.LogSegment.readSegmentFile(LogSegment.java:111)
at org.apache.ratis.server.storage.LogSegment.loadSegment(LogSegment.java:133)
at
org.apache.ratis.server.storage.RaftLogCache.loadSegment(RaftLogCache.java:110)
at
org.apache.ratis.server.storage.SegmentedRaftLog.loadLogSegments(SegmentedRaftLog.java:151)
at
org.apache.ratis.server.storage.SegmentedRaftLog.open(SegmentedRaftLog.java:120)
at org.apache.ratis.server.impl.ServerState.initLog(ServerState.java:191)
at org.apache.ratis.server.impl.ServerState.(ServerState.java:114)
at org.apache.ratis.server.impl.RaftServerImpl.(RaftServerImpl.java:106)
at
org.apache.ratis.server.impl.RaftServerProxy.lambda$newRaftServerImpl$2(RaftServerProxy.java:196)
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at
java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.IllegalStateException: Corrupted log header:
^@^@^@^@^@^@^@^@
at org.apache.ratis.util.Preconditions.assertTrue(Preconditions.java:60)
at org.apache.ratis.server.storage.LogInputStream.init(LogInputStream.java:93)
at
org.apache.ratis.server.storage.LogInputStream.nextEntry(LogInputStream.java:120)
... 15 more
2018-10-26 10:26:03,703 ERROR org.apache.hadoop.ozone.HddsDatanodeService:
Except
[jira] [Created] (HDDS-741) all pipelines are in 'CLOSING' state after service restart
Nilotpal Nandi created HDDS-741:
---
Summary: all pipelines are in 'CLOSING' state after service restart
Key: HDDS-741
URL: https://issues.apache.org/jira/browse/HDDS-741
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: SCM
Affects Versions: 0.3.0
Reporter: Nilotpal Nandi
all pipelines are in closing state after service restart. None of the writes
are working after restart.
The cluster contains 11 live datanodes.
**
{noformat}
[
{
"nodeType": "OM",
"hostname": "ctr-e138-1518143905142-53-01-08.hwx.site",
"ports": {
"RPC": 9889,
"HTTP": 9874
}
},
{
"nodeType": "SCM",
"hostname": "ctr-e138-1518143905142-53-01-03.hwx.site",
"ports": {
"RPC": 9860
}
},
{
"nodeType": "DATANODE",
"hostname": "ctr-e138-1518143905142-541661-01-03.hwx.site",
"ports": {
"HTTP": 9880
}
},
{
"nodeType": "DATANODE",
"hostname": "ctr-e138-1518143905142-541661-01-07.hwx.site",
"ports": {
"HTTP": 9880
}
},
{
"nodeType": "DATANODE",
"hostname": "ctr-e138-1518143905142-53-01-03.hwx.site",
"ports": {
"HTTP": 9880
}
},
{
"nodeType": "DATANODE",
"hostname": "ctr-e138-1518143905142-541661-01-04.hwx.site",
"ports": {
"HTTP": 9880
}
},
{
"nodeType": "DATANODE",
"hostname": "ctr-e138-1518143905142-53-01-04.hwx.site",
"ports": {
"HTTP": 9880
}
},
{
"nodeType": "DATANODE",
"hostname": "ctr-e138-1518143905142-53-01-08.hwx.site",
"ports": {
"HTTP": 9880
}
},
{
"nodeType": "DATANODE",
"hostname": "ctr-e138-1518143905142-541661-01-02.hwx.site",
"ports": {
"HTTP": 9880
}
},
{
"nodeType": "DATANODE",
"hostname": "ctr-e138-1518143905142-53-01-05.hwx.site",
"ports": {
"HTTP": 9880
}
},
{
"nodeType": "DATANODE",
"hostname": "ctr-e138-1518143905142-541661-01-06.hwx.site",
"ports": {
"HTTP": 9880
}
},
{
"nodeType": "DATANODE",
"hostname": "ctr-e138-1518143905142-53-01-07.hwx.site",
"ports": {
"HTTP": 9880
}
},
{
"nodeType": "DATANODE",
"hostname": "ctr-e138-1518143905142-53-01-06.hwx.site",
"ports": {
"HTTP": 9880
}
}
]{noformat}
error thrown while write :
{noformat}
[root@ctr-e138-1518143905142-541661-01-07 test_files]# ozone fs -put
/etc/passwd /testdir5/
2018-10-26 12:09:43,822 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2018-10-26 12:09:47,882 ERROR io.ChunkGroupOutputStream: Try to allocate more
blocks for write failed, already allocated 0 blocks for this write.
put: Allocate block failed, error:INTERNAL_ERROR{noformat}
pipelines in the cluster :
{noformat}
[root@ctr-e138-1518143905142-541661-01-07 test_files]# ozone scmcli
listPipelines
Pipeline[ Id: 29b68cc2-2d18-4db0-a11a-587ae4abc715, Nodes:
e3d89961-fe38-4ed0-8a32-cd1849c58e0c{ip: 172.27.20.96, host:
ctr-e138-1518143905142-53-01-08.hwx.site}b33a30d9-f1e2-448e-aabb-61a970445cea{ip:
172.27.85.64, host: ctr-e138-1518143905142-541661-01-07.hwx.site},
Type:RATIS, Factor:THREE, State:CLOSING]
Pipeline[ Id: 05061f87-4c68-443b-ae27-984da2d0a2cd, Nodes:
dc002a73-fc63-4e76-be3e-3c6d16ede5f6{ip: 172.27.38.9, host:
ctr-e138-1518143905142-53-01-04.hwx.site}4e6bd2a2-6802-4e67-9710-612a2cdb9dc1{ip:
172.27.24.90, host:
ctr-e138-1518143905142-53-01-05.hwx.site}be3f0db4-3a19-44a5-bd6e-0da47d2ed92e{ip:
172.27.20.91, host: ctr-e138-1518143905142-53-01-03.hwx.site},
Type:RATIS, Factor:THREE, State:CLOSING]
Pipeline[ Id: 80893f87-5e73-49a2-8f38-2adb2b13140a, Nodes:
63833540-bf93-410c-b081-243a56f93c88{ip: 172.27.10.199, host:
ctr-e138-1518143905142-53-01-07.hwx.site}6e8b7129-8615-45fe-81e0-848a2e0ba520{ip:
172.27.15.139, host:
ctr-e138-1518143905142-53-01-06.hwx.site}aab1f2e5-1cf0-430d-b1bf-04be8630a8ee{ip:
172.27.57.0, host: ctr-e138-1518143905142-541661-01-03.hwx.site},
Type:RATIS, Factor:THREE, State:CLOSING]
Pipeline[ Id: f0a14cb9-d37a-4c7c-b3e6-b7e3830dfd5f, Nodes:
61e271bf-68ad-435e-8a6e-582be90ebb6f{ip: 172.27.19.74, host:
ctr-e138-1518143905142-541661-01-06.hwx.site}3622352c-b136-4c74-b952-34e938cbda94{ip:
172.27.15.131, host:
ctr-e138-1518143905142-541661-01-02.hwx.site}cb2b1e95-e803-48d3-bdf2-bf878cae62cf{ip:
172.27.23.139, host: ctr-e138-1518143905142-541661-01-04.hwx.site},
Type:RATIS, Factor:THREE, State:CLOSING]
{noformat}
datanode.log :
---
{noformat}
2018-10-26 12:17:23,697 INFO org.apache.ratis.server.impl.LeaderElection:
e3d89961-fe38-4ed0-8a32-cd1849c58e0c: Election REJECTED; received 1 response(s)
[e3d89961-fe38-4ed0-8a32-cd1849c58e0c<-b33a30d9-f1e2-448e-aabb-61a970445cea#0:FAIL-t1019]
and 1 exception(s); e3d89961-fe38-4ed0-8a32-cd1849c58e0c:t1019, leader=null,
voted=e3d89961-fe38-4ed0-8a32-cd1849c58e0c,
raftlog=e3d89961-fe38-4ed0-8a32-cd1849c58e0c-Segmen
[jira] [Created] (HDDS-768) writeStateMachineData times out
Nilotpal Nandi created HDDS-768:
---
Summary: writeStateMachineData times out
Key: HDDS-768
URL: https://issues.apache.org/jira/browse/HDDS-768
Project: Hadoop Distributed Data Store
Issue Type: Bug
Affects Versions: 0.3.0
Reporter: Nilotpal Nandi
datanode stopped due to following error :
datanode.log
{noformat}
2018-10-31 09:12:04,517 INFO org.apache.ratis.server.impl.RaftServerImpl:
9fab9937-fbcd-4196-8014-cb165045724b: set configuration 169:
[9fab9937-fbcd-4196-8014-cb165045724b:172.27.15.131:9858,
ce0084c2-97cd-4c97-9378-e5175daad18b:172.27.15.139:9858,
f0291cb4-7a48-456a-847f-9f91a12aa850:172.27.38.9:9858], old=null at 169
2018-10-31 09:12:22,187 ERROR org.apache.ratis.server.storage.RaftLogWorker:
Terminating with exit status 1:
9fab9937-fbcd-4196-8014-cb165045724b-RaftLogWorker failed.
org.apache.ratis.protocol.TimeoutIOException: Timeout: WriteLog:182: (t:10,
i:182), STATEMACHINELOGENTRY, client-611073BBFA46, cid=127-writeStateMachineData
at org.apache.ratis.util.IOUtils.getFromFuture(IOUtils.java:87)
at
org.apache.ratis.server.storage.RaftLogWorker$WriteLog.execute(RaftLogWorker.java:310)
at org.apache.ratis.server.storage.RaftLogWorker.run(RaftLogWorker.java:182)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.TimeoutException
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.apache.ratis.util.IOUtils.getFromFuture(IOUtils.java:79)
... 3 more{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-769) temporary file ._COPYING_ is not deleted after put command failure
Nilotpal Nandi created HDDS-769:
---
Summary: temporary file ._COPYING_ is not deleted after put
command failure
Key: HDDS-769
URL: https://issues.apache.org/jira/browse/HDDS-769
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Manager
Affects Versions: 0.3.0
Reporter: Nilotpal Nandi
steps taken :
-
# stopped all datanodes.
# Ran ozonefs put command. Command execution failed.
{noformat}
[root@ctr-e138-1518143905142-53-01-08 ~]# ozone fs -put /etc/passwd
/testdir5/
2018-10-31 08:42:12,711 [main] ERROR - Try to allocate more blocks for write
failed, already allocated 0 blocks for this write.
put: Allocate block failed, error:INTERNAL_ERROR{noformat}
But , the temporary file was not deleted from OM.
{noformat}
[root@ctr-e138-1518143905142-53-01-03 logs]# ozone fs -ls
/testdir5/passwd._COPYING_
-rw-rw-rw- 1 root root 0 2018-10-31 08:42 /testdir5/passwd._COPYING_{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-770) ozonefs client warning exception logs should not be displayed on console
Nilotpal Nandi created HDDS-770:
---
Summary: ozonefs client warning exception logs should not be
displayed on console
Key: HDDS-770
URL: https://issues.apache.org/jira/browse/HDDS-770
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Affects Versions: 0.3.0
Reporter: Nilotpal Nandi
steps taken :
-
# ran ozonefs cp command - "ozone fs -cp /testdir2/2GB /testdir2/2GB_111"
# command execution was successful and file was successfully copied.
But , the warning logs/exceptions are displayed on console :
{noformat}
[root@ctr-e138-1518143905142-53-01-03 ~]# ozone fs -cp /testdir2/2GB
/testdir2/2GB_111
2018-10-31 09:12:35,052 WARN scm.XceiverClientGrpc: Failed to execute command
cmdType: GetBlock
traceID: "b73d7d2d-232a-40d7-b0b6-478e3d40ed6a"
containerID: 17
datanodeUuid: "ce0084c2-97cd-4c97-9378-e5175daad18b"
getBlock {
blockID {
containerID: 17
localID: 100989077200109583
}
blockCommitSequenceId: 60
}
on datanode 9fab9937-fbcd-4196-8014-cb165045724b
java.util.concurrent.ExecutionException:
org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: UNAVAILABLE: io
exception
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at
org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommandWithRetry(XceiverClientGrpc.java:167)
at
org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommand(XceiverClientGrpc.java:146)
at
org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.getBlock(ContainerProtocolCalls.java:105)
at
org.apache.hadoop.ozone.client.io.ChunkGroupInputStream.getFromOmKeyInfo(ChunkGroupInputStream.java:301)
at org.apache.hadoop.ozone.client.rpc.RpcClient.getKey(RpcClient.java:493)
at org.apache.hadoop.ozone.client.OzoneBucket.readKey(OzoneBucket.java:272)
at org.apache.hadoop.fs.ozone.OzoneFileSystem.open(OzoneFileSystem.java:178)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:950)
at
org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:341)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:277)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:262)
at org.apache.hadoop.fs.shell.Command.processPathInternal(Command.java:367)
at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:331)
at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:304)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.java:257)
at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:286)
at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:270)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:228)
at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
at org.apache.hadoop.fs.shell.Command.run(Command.java:177)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:327)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:390)
Caused by: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
UNAVAILABLE: io exception
at
org.apache.ratis.thirdparty.io.grpc.Status.asRuntimeException(Status.java:526)
at
org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:420)
at
org.apache.ratis.thirdparty.io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at
org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at
org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at
org.apache.ratis.thirdparty.io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
at
org.apache.ratis.thirdparty.io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at
org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at
org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at
org.apache.ratis.thirdparty.io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:403)
at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:459)
at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
at
org.apache.r
[jira] [Created] (HDDS-772) ratis retries infinitely and does not timeout when datanode goes down
Nilotpal Nandi created HDDS-772:
---
Summary: ratis retries infinitely and does not timeout when
datanode goes down
Key: HDDS-772
URL: https://issues.apache.org/jira/browse/HDDS-772
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Affects Versions: 0.3.0
Reporter: Nilotpal Nandi
steps taken :
-
# Ran ozonefs client operations.
# Some of the datanodes were down.
# client operations did not fail and are in waiting/hung state.
reason: RATIS retries infinitely.
datanode.log
{noformat}
2018-10-31 11:13:28,423 WARN
org.apache.ratis.grpc.server.GrpcServerProtocolService:
046351fe-bb76-4f86-b296-c682746981c4: Failed requestVote
54026017-a738-45f5-92f9-c50a0fc24a9f->046351fe-bb76-4f86-b296-c682746981c4#0
org.apache.ratis.protocol.GroupMismatchException:
046351fe-bb76-4f86-b296-c682746981c4: group-FF58136AA1BA not found.
at
org.apache.ratis.server.impl.RaftServerProxy$ImplMap.get(RaftServerProxy.java:114)
at
org.apache.ratis.server.impl.RaftServerProxy.getImplFuture(RaftServerProxy.java:257)
at
org.apache.ratis.server.impl.RaftServerProxy.getImpl(RaftServerProxy.java:266)
at
org.apache.ratis.server.impl.RaftServerProxy.getImpl(RaftServerProxy.java:261)
at
org.apache.ratis.server.impl.RaftServerProxy.requestVote(RaftServerProxy.java:428)
at
org.apache.ratis.grpc.server.GrpcServerProtocolService.requestVote(GrpcServerProtocolService.java:54)
at
org.apache.ratis.proto.grpc.RaftServerProtocolServiceGrpc$MethodHandlers.invoke(RaftServerProtocolServiceGrpc.java:319)
at
org.apache.ratis.thirdparty.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)
at
org.apache.ratis.thirdparty.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283)
at
org.apache.ratis.thirdparty.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:707)
at
org.apache.ratis.thirdparty.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at
org.apache.ratis.thirdparty.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-10-31 11:13:29,574 WARN
org.apache.ratis.grpc.server.GrpcServerProtocolService:
046351fe-bb76-4f86-b296-c682746981c4: Failed requestVote
54026017-a738-45f5-92f9-c50a0fc24a9f->046351fe-bb76-4f86-b296-c682746981c4#0
org.apache.ratis.protocol.GroupMismatchException:
046351fe-bb76-4f86-b296-c682746981c4: group-FF58136AA1BA not found.
at
org.apache.ratis.server.impl.RaftServerProxy$ImplMap.get(RaftServerProxy.java:114)
at
org.apache.ratis.server.impl.RaftServerProxy.getImplFuture(RaftServerProxy.java:257)
at
org.apache.ratis.server.impl.RaftServerProxy.getImpl(RaftServerProxy.java:266)
at
org.apache.ratis.server.impl.RaftServerProxy.getImpl(RaftServerProxy.java:261)
at
org.apache.ratis.server.impl.RaftServerProxy.requestVote(RaftServerProxy.java:428)
at
org.apache.ratis.grpc.server.GrpcServerProtocolService.requestVote(GrpcServerProtocolService.java:54)
at
org.apache.ratis.proto.grpc.RaftServerProtocolServiceGrpc$MethodHandlers.invoke(RaftServerProtocolServiceGrpc.java:319)
at
org.apache.ratis.thirdparty.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)
at
org.apache.ratis.thirdparty.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283)
at
org.apache.ratis.thirdparty.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:707)
at
org.apache.ratis.thirdparty.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at
org.apache.ratis.thirdparty.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-10-31 11:13:30,772 WARN
org.apache.ratis.grpc.server.GrpcServerProtocolService:
046351fe-bb76-4f86-b296-c682746981c4: Failed requestVote
54026017-a738-45f5-92f9-c50a0fc24a9f->046351fe-bb76-4f86-b296-c682746981c4#0
org.apache.ratis.protocol.GroupMismatchException:
046351fe-bb76-4f86-b296-c682746981c4: group-FF58136AA1BA not found.
at
org.apache.ratis.server.impl.RaftServerProxy$ImplMap.get(RaftServerProxy.java:114)
at
org.apache.ratis.server.impl.RaftServerProxy.getImplFuture(RaftServerProxy.java:257)
at
org.apache.ratis.server.impl.RaftServerProxy.getImpl(RaftServerProxy.java:266)
at
org.apache.ratis.server
[jira] [Created] (HDDS-874) ozonefs write operation failed
Nilotpal Nandi created HDDS-874:
---
Summary: ozonefs write operation failed
Key: HDDS-874
URL: https://issues.apache.org/jira/browse/HDDS-874
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Filesystem
Reporter: Nilotpal Nandi
Here is the error thrown on console while trying to write a file to ozone
filesystem
{noformat}
[root@ctr-e139-1542663976389-11261-01-05 test_files]# ozone fs
-copyFromLocal 5GB /
2018-11-27 04:24:14,117 WARN scm.XceiverClientRatis: 3 way commit failed
java.util.concurrent.TimeoutException
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at
org.apache.hadoop.hdds.scm.XceiverClientRatis.watchForCommit(XceiverClientRatis.java:162)
at
org.apache.hadoop.hdds.scm.storage.ChunkOutputStream.watchForCommit(ChunkOutputStream.java:303)
at
org.apache.hadoop.hdds.scm.storage.ChunkOutputStream.handleFullBuffer(ChunkOutputStream.java:289)
at
org.apache.hadoop.hdds.scm.storage.ChunkOutputStream.write(ChunkOutputStream.java:193)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream$ChunkOutputStreamEntry.write(ChunkGroupOutputStream.java:695)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleWrite(ChunkGroupOutputStream.java:292)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.write(ChunkGroupOutputStream.java:251)
at
org.apache.hadoop.fs.ozone.OzoneFSOutputStream.write(OzoneFSOutputStream.java:47)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:57)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:96)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:68)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:129)
at
org.apache.hadoop.fs.shell.CommandWithDestination$TargetFileSystem.writeStreamToFile(CommandWithDestination.java:485)
at
org.apache.hadoop.fs.shell.CommandWithDestination.copyStreamToTarget(CommandWithDestination.java:407)
at
org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:342)
at
org.apache.hadoop.fs.shell.CopyCommands$CopyFromLocal.copyFile(CopyCommands.java:357)
at
org.apache.hadoop.fs.shell.CopyCommands$CopyFromLocal.copyFileToTarget(CopyCommands.java:365)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:277)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:262)
at org.apache.hadoop.fs.shell.Command.processPathInternal(Command.java:367)
at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:331)
at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:304)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.java:257)
at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:286)
at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:270)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:228)
at
org.apache.hadoop.fs.shell.CopyCommands$Put.processArguments(CopyCommands.java:295)
at
org.apache.hadoop.fs.shell.CopyCommands$CopyFromLocal.processArguments(CopyCommands.java:385)
at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
at org.apache.hadoop.fs.shell.Command.run(Command.java:177)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:327)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:390)
2018-11-27 04:24:17,497 INFO scm.XceiverClientRatis: Could not commit 146 to
all the nodes.Committed by majority.
2018-11-27 04:24:55,867 WARN scm.XceiverClientRatis: 3 way commit failed
java.util.concurrent.TimeoutException
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at
org.apache.hadoop.hdds.scm.XceiverClientRatis.watchForCommit(XceiverClientRatis.java:162)
at
org.apache.hadoop.hdds.scm.storage.ChunkOutputStream.watchForCommit(ChunkOutputStream.java:303)
at
org.apache.hadoop.hdds.scm.storage.ChunkOutputStream.handleFullBuffer(ChunkOutputStream.java:289)
at
org.apache.hadoop.hdds.scm.storage.ChunkOutputStream.write(ChunkOutputStream.java:193)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream$ChunkOutputStreamEntry.write(ChunkGroupOutputStream.java:695)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleWrite(ChunkGroupOutputStream.java:292)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.write(ChunkGroupOutputStream.java:251)
at
org.apache.hadoop.fs.ozone.OzoneFSO
[jira] [Created] (HDDS-893) pipeline status is ALLOCATED in scmcli listPipelines command
Nilotpal Nandi created HDDS-893:
---
Summary: pipeline status is ALLOCATED in scmcli listPipelines
command
Key: HDDS-893
URL: https://issues.apache.org/jira/browse/HDDS-893
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: SCM
Reporter: Nilotpal Nandi
Pipeline status cannot be allocated , It should be either OPEN or CLOSING.
{noformat}
[root@ctr-e139-1542663976389-11261-01-05 test_files]# ozone scmcli
listPipelines
Pipeline[ Id: 202f7208-6977-4f65-b070-c1e7e57cb2ed, Nodes:
06e074f7-67b4-4dde-8f20-a437ca60b7a1{ip: 172.27.20.97, host:
ctr-e139-1542663976389-11261-01-07.hwx.site}c5bf9a9f-d471-4cef-aae4-61cb387ea9e3{ip:
172.27.79.145, host:
ctr-e139-1542663976389-11261-01-06.hwx.site}96c18fe3-5520-4941-844b-ff7186a146a6{ip:
172.27.55.132, host: ctr-e139-1542663976389-11261-01-03.hwx.site},
Type:RATIS, Factor:THREE, State:ALLOCATED]{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-897) all healthy datanodes are not involved in pipeline allocation
Nilotpal Nandi created HDDS-897:
---
Summary: all healthy datanodes are not involved in pipeline
allocation
Key: HDDS-897
URL: https://issues.apache.org/jira/browse/HDDS-897
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Datanode, SCM
Reporter: Nilotpal Nandi
Attachments: active_datanode1.png, active_datanode2.png
Please find the attached screenshot. There are 6 active datanodes , but there
is only 1 pipeline created. All new writes are now done on a single pipeline
even when there are 3 more free and active datanodes.
!active_datanode2.png!
!active_datanode1.png!
{noformat}
[root@ctr-e139-1542663976389-11261-01-07 meta]# ozone scmcli listPipelines
Pipeline[ Id: 202f7208-6977-4f65-b070-c1e7e57cb2ed, Nodes:
06e074f7-67b4-4dde-8f20-a437ca60b7a1{ip: 172.27.20.97, host:
ctr-e139-1542663976389-11261-01-07.hwx.site}c5bf9a9f-d471-4cef-aae4-61cb387ea9e3{ip:
172.27.79.145, host:
ctr-e139-1542663976389-11261-01-06.hwx.site}96c18fe3-5520-4941-844b-ff7186a146a6{ip:
172.27.55.132, host: ctr-e139-1542663976389-11261-01-03.hwx.site},
Type:RATIS, Factor:THREE, State:ALLOCATED]{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-899) Include pipeline info and leader info in scm UI metric page
Nilotpal Nandi created HDDS-899: --- Summary: Include pipeline info and leader info in scm UI metric page Key: HDDS-899 URL: https://issues.apache.org/jira/browse/HDDS-899 Project: Hadoop Distributed Data Store Issue Type: Improvement Components: SCM Reporter: Nilotpal Nandi Open pipeline info and corresponding leader information should be included in SCM UI metric page ( HTTP port : 9876). These information would be helpful while debugging the issues live. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-900) scmcli info cannot find container if the container does not contain any key
Nilotpal Nandi created HDDS-900:
---
Summary: scmcli info cannot find container if the container does
not contain any key
Key: HDDS-900
URL: https://issues.apache.org/jira/browse/HDDS-900
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: SCM
Reporter: Nilotpal Nandi
Please check the commands below. scmcli 'list' command shows the container but
the 'info' command does not show it .
{noformat}
[root@ctr-e139-1542663976389-22144-01-02 test_files]# ozone scmcli list
--start=19
{
"state" : "OPEN",
"replicationFactor" : "THREE",
"replicationType" : "RATIS",
"usedBytes" : 0,
"numberOfKeys" : 0,
"lastUsed" : 6719234213,
"stateEnterTime" : 9829697449,
"owner" : "16b9d1db-cfca-4d46-bdfa-7ee136ba626a",
"containerID" : 20,
"deleteTransactionId" : 0,
"sequenceId" : 0,
"open" : true
}{noformat}
[root@ctr-e139-1542663976389-22144-01-02 test_files]# ozone scmcli info 20
ContainerID 20 does not exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-904) RATIS group not found thrown on datanodes while leader election
Nilotpal Nandi created HDDS-904:
---
Summary: RATIS group not found thrown on datanodes while leader
election
Key: HDDS-904
URL: https://issues.apache.org/jira/browse/HDDS-904
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Datanode, SCM
Reporter: Nilotpal Nandi
Attachments: datanode_1.log, datanode_2.log, datanode_3.log, scm.log
Following exception seen in datanode.log of one the docker nodes
-
{noformat}
2018-12-06 09:32:11 INFO LeaderElection:127 -
0e3aa95d-ab51-4b20-9bff-3f7bd7df0500: begin an election in Term 1
2018-12-06 09:32:12 INFO LeaderElection:46 -
0e3aa95d-ab51-4b20-9bff-3f7bd7df0500: Election TIMEOUT; received 0 response(s)
[] and 0 exception(s); 0e3aa95d-ab51-4b20-9bff-3f7bd7df0500:t1, leader=null,
voted=0e3aa95d-ab51-4b20-9bff-3f7bd7df0500,
raftlog=0e3aa95d-ab51-4b20-9bff-3f7bd7df0500-SegmentedRaftLog:OPENED, conf=-1:
[76153aab-4681-40b6-bc32-cc9ed5ef1daf:192.168.0.7:9858,
79ca7251-7514-4c53-968c-ade59d6df07b:192.168.0.6:9858,
0e3aa95d-ab51-4b20-9bff-3f7bd7df0500:192.168.0.4:9858], old=null
2018-12-06 09:32:13 INFO LeaderElection:127 -
0e3aa95d-ab51-4b20-9bff-3f7bd7df0500: begin an election in Term 2
2018-12-06 09:32:13 INFO LeaderElection:230 -
0e3aa95d-ab51-4b20-9bff-3f7bd7df0500 got exception when requesting votes: {}
java.util.concurrent.ExecutionException:
org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: INTERNAL:
76153aab-4681-40b6-bc32-cc9ed5ef1daf: group-41B8C34A6DE4 not found.
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.ratis.server.impl.LeaderElection.waitForResults(LeaderElection.java:214)
at
org.apache.ratis.server.impl.LeaderElection.askForVotes(LeaderElection.java:146)
at org.apache.ratis.server.impl.LeaderElection.run(LeaderElection.java:102)
Caused by: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
INTERNAL: 76153aab-4681-40b6-bc32-cc9ed5ef1daf: group-41B8C34A6DE4 not found.
at
org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:222)
at
org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:203)
at
org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:132)
at
org.apache.ratis.proto.grpc.RaftServerProtocolServiceGrpc$RaftServerProtocolServiceBlockingStub.requestVote(RaftServerProtocolServiceGrpc.java:265)
at
org.apache.ratis.grpc.server.GrpcServerProtocolClient.requestVote(GrpcServerProtocolClient.java:63)
at org.apache.ratis.grpc.server.GrpcService.requestVote(GrpcService.java:150)
at
org.apache.ratis.server.impl.LeaderElection.lambda$submitRequests$0(LeaderElection.java:188)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-12-06 09:32:14 INFO LeaderElection:46 -
0e3aa95d-ab51-4b20-9bff-3f7bd7df0500: Election TIMEOUT; received 0 response(s)
[] and 1 exception(s); 0e3aa95d-ab51-4b20-9bff-3f7bd7df0500:t2, leader=null,
voted=0e3aa95d-ab51-4b20-9bff-3f7bd7df0500,
raftlog=0e3aa95d-ab51-4b20-9bff-3f7bd7df0500-SegmentedRaftLog:OPENED, conf=-1:
[76153aab-4681-40b6-bc32-cc9ed5ef1daf:192.168.0.7:9858,
79ca7251-7514-4c53-968c-ade59d6df07b:192.168.0.6:9858,
0e3aa95d-ab51-4b20-9bff-3f7bd7df0500:192.168.0.4:9858], old=null{noformat}
cc - [~ljain]
all logs attached.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-932) Add blockade Tests for Network partition
Nilotpal Nandi created HDDS-932: --- Summary: Add blockade Tests for Network partition Key: HDDS-932 URL: https://issues.apache.org/jira/browse/HDDS-932 Project: Hadoop Distributed Data Store Issue Type: Bug Components: Ozone Datanode Affects Versions: 0.4.0 Reporter: Nilotpal Nandi Assignee: Nilotpal Nandi Fix For: 0.4.0 Blockade tests need to be added pertaining to network partition. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-934) freon run hung and did not terminate when run on non-functional pipeline
Nilotpal Nandi created HDDS-934:
---
Summary: freon run hung and did not terminate when run on
non-functional pipeline
Key: HDDS-934
URL: https://issues.apache.org/jira/browse/HDDS-934
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Reporter: Nilotpal Nandi
Attachments: jstack.txt
steps taken:
# created docker cluster with 3 datanodes where datanodes are not able to
communicate with each other.
# ran freon to write 1 key
freon run threw following exception but it did not terminate.
exception:
--
{noformat}
2018-12-18 10:38:06 INFO RandomKeyGenerator:227 - Number of Threads: 10
2018-12-18 10:38:06 INFO RandomKeyGenerator:233 - Number of Volumes: 1.
2018-12-18 10:38:06 INFO RandomKeyGenerator:234 - Number of Buckets per Volume:
1.
2018-12-18 10:38:06 INFO RandomKeyGenerator:235 - Number of Keys per Bucket: 1.
2018-12-18 10:38:06 INFO RandomKeyGenerator:236 - Key size: 10240 bytes
2018-12-18 10:38:06 INFO RandomKeyGenerator:266 - Starting progress bar Thread.
0.00% |█ | 0/1 Time: 0:00:002018-12-18 10:38:06 INFO RpcClient:250 - Creating
Volume: vol-0-74492, with hadoop as owner and quota set to 1152921504606846976
bytes.
2018-12-18 10:38:06 INFO RpcClient:379 - Creating Bucket:
vol-0-74492/bucket-0-16002, with Versioning false and Storage Type set to DISK
0.00% |█ | 0/1 Time: 0:02:402018-12-18 10:40:46 ERROR
ChunkGroupOutputStream:275 - Try to allocate more blocks for write failed,
already allocated 0 blocks for this write.
2018-12-18 10:40:46 ERROR RandomKeyGenerator:624 - Exception while adding key:
key-0-28925 in bucket: org.apache.hadoop.ozone.client.OzoneBucket@1675c402 of
volume: org.apache.hadoop.ozone.client.OzoneVolume@5b6bfafd.
java.io.IOException: Allocate block failed, error:INTERNAL_ERROR
at
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.allocateBlock(OzoneManagerProtocolClientSideTranslatorPB.java:620)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.allocateNewBlock(ChunkGroupOutputStream.java:437)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleWrite(ChunkGroupOutputStream.java:272)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleException(ChunkGroupOutputStream.java:377)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleFlushOrClose(ChunkGroupOutputStream.java:473)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleFlushOrClose(ChunkGroupOutputStream.java:474)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleFlushOrClose(ChunkGroupOutputStream.java:474)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleFlushOrClose(ChunkGroupOutputStream.java:474)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.handleWrite(ChunkGroupOutputStream.java:309)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.write(ChunkGroupOutputStream.java:255)
at
org.apache.hadoop.ozone.client.io.OzoneOutputStream.write(OzoneOutputStream.java:49)
at java.io.OutputStream.write(OutputStream.java:75)
at
org.apache.hadoop.ozone.freon.RandomKeyGenerator$OfflineProcessor.run(RandomKeyGenerator.java:606)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
0.00% |█ | 0/1 Time: 0:11:01
0.00% |█ | 0/1 Time: 0:11:19
{noformat}
Here is the jstack for the freon process :
[^jstack.txt]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-953) put key operation throws exception when run via ozone sh cli
Nilotpal Nandi created HDDS-953:
---
Summary: put key operation throws exception when run via ozone sh
cli
Key: HDDS-953
URL: https://issues.apache.org/jira/browse/HDDS-953
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Reporter: Nilotpal Nandi
steps taken :
# created docker cluster consisting of 3 datanodes.
# created volume and bucket and then ran put key operation.
Put key operation throws following exception but the operation succeeds.
{noformat}
hadoop@4214b6a267fc:~$ ozone sh key put test-1/bbb-1/passwd /etc/passwd
2019-01-02 10:30:06 ERROR BlockOutputStream:530 - Unexpected Storage Container
Exception:
org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException:
ContainerID 2 does not exist
at
org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.validateContainerResponse(ContainerProtocolCalls.java:495)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.validateResponse(BlockOutputStream.java:528)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.lambda$writeChunkToContainer$5(BlockOutputStream.java:605)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-01-02 10:30:06 ERROR BlockOutputStream:530 - Unexpected Storage Container
Exception:
org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException:
ContainerID 2 does not exist
at
org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.validateContainerResponse(ContainerProtocolCalls.java:495)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.validateResponse(BlockOutputStream.java:528)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.lambda$handlePartialFlush$2(BlockOutputStream.java:386)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-954) get key operation is failing
Nilotpal Nandi created HDDS-954:
---
Summary: get key operation is failing
Key: HDDS-954
URL: https://issues.apache.org/jira/browse/HDDS-954
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Datanode
Reporter: Nilotpal Nandi
steps taken :
---
# created 3 datanodes docker cluster.
# created volume, bucket and keys.
get key operations are failing.
{noformat}
hadoop@4214b6a267fc:~$ ozone fs -ls o3fs://bbb-1.test-1/
Found 21 items
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:30 o3fs://bbb-1.test-1/passwd
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:37 o3fs://bbb-1.test-1/passwd_1
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_10
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_11
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_12
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_13
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_14
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_15
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_16
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_17
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_18
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_19
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:37 o3fs://bbb-1.test-1/passwd_2
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_20
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:37 o3fs://bbb-1.test-1/passwd_3
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:37 o3fs://bbb-1.test-1/passwd_4
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:37 o3fs://bbb-1.test-1/passwd_5
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:37 o3fs://bbb-1.test-1/passwd_6
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:37 o3fs://bbb-1.test-1/passwd_7
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:37 o3fs://bbb-1.test-1/passwd_8
-rw-rw-rw- 1 hadoop hadoop 1010 2019-01-02 10:38 o3fs://bbb-1.test-1/passwd_9
hadoop@4214b6a267fc:~$ ozone fs -cat o3fs://bbb-1.test-1/passwd_9
cat: ContainerID 11 does not exist
hadoop@4214b6a267fc:~$ ozone fs -cat o3fs://bbb-1.test-1/passwd_8
cat: ContainerID 10 does not exist
hadoop@4214b6a267fc:~/logs$ ozone sh key get test-1/bbb-1/passwd_9 abc
ContainerID 11 does not exist
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-988) containers remain in CLOSING state in one of the datanodes when there datanodes are isolated in docker cluster
Nilotpal Nandi created HDDS-988:
---
Summary: containers remain in CLOSING state in one of the
datanodes when there datanodes are isolated in docker cluster
Key: HDDS-988
URL: https://issues.apache.org/jira/browse/HDDS-988
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Datanode, SCM
Reporter: Nilotpal Nandi
Attachments: datanode_1, datanode_2, datanode_3, om, scm
steps taken :
---
# Created 3 datanodes docker cluster.
# wrote some data to create a pipeline.
# Then, isolated all datanodes , i.e, datanodes coud not communicate with each
other . (datanodes can communicate with scm and om).
# Tried to write some data again, write failed as expected.
# After waiting for 'ozone.scm.stale.node.interval' and
'ozone.scm.dead.node.interval' , the container replicas are still in CLOSING
state. Containers failed to get CLOSED.
{noformat}
hadoop@8876c7214ee5:~$ cat
/data/hdds/hdds/40bb080a-1a9f-42c8-9e20-8257ed567e46/current/containerDir0/*/metadata/*.container
!
checksum: 7ee8f706cf215a5fa4b7e9a195529c15147823ceea302ab4998c7476ee64ebf4
chunksPath:
/data/hdds/hdds/40bb080a-1a9f-42c8-9e20-8257ed567e46/current/containerDir0/2/chunks
containerDBType: RocksDB
containerID: 2
containerType: KeyValueContainer
layOutVersion: 1
maxSize: 5368709120
metadata: {}
metadataPath:
/data/hdds/hdds/40bb080a-1a9f-42c8-9e20-8257ed567e46/current/containerDir0/2/metadata
originNodeId: 6e077f73-9fd9-4f4e-930f-578c9857912c
originPipelineId: ee5f9e7a-0d63-412a-839a-77af2cf7ca93
state: CLOSING{noformat}
Expectation :
-
The container should have at least two closed replicas .
scm, om datanodes log attached.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-997) Add blockade Tests for scm isolation and mixed node isolation
Nilotpal Nandi created HDDS-997: --- Summary: Add blockade Tests for scm isolation and mixed node isolation Key: HDDS-997 URL: https://issues.apache.org/jira/browse/HDDS-997 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Nilotpal Nandi -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1027) Add blockade Tests for datanode isolation and scm failures
Nilotpal Nandi created HDDS-1027: Summary: Add blockade Tests for datanode isolation and scm failures Key: HDDS-1027 URL: https://issues.apache.org/jira/browse/HDDS-1027 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Nilotpal Nandi -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1036) container replica state in datanode should be QUASI-CLOSED if the datanode is isolated from other two datanodes in 3 datanode cluster
Nilotpal Nandi created HDDS-1036: Summary: container replica state in datanode should be QUASI-CLOSED if the datanode is isolated from other two datanodes in 3 datanode cluster Key: HDDS-1036 URL: https://issues.apache.org/jira/browse/HDDS-1036 Project: Hadoop Distributed Data Store Issue Type: Bug Components: Ozone Datanode, SCM Reporter: Nilotpal Nandi steps taken : --- # created a 3 datanode docker cluster. # wrote some data to create a pipeline. # Then, one of the datanodes is isolated from other two datanodes. All datanodes can communicate with SCM. # Tried to write new data , write failed. # Wait for 900 seconds. Observation: container state is CLOSED in all three replicas. Expectation: --- container state in isolated datanode should be QUASI-CLOSED. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1040) Add blockade Tests for client failures
Nilotpal Nandi created HDDS-1040: Summary: Add blockade Tests for client failures Key: HDDS-1040 URL: https://issues.apache.org/jira/browse/HDDS-1040 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Nilotpal Nandi -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1057) get key operation fails when client cannot communicate with 2 of the datanodes in 3 node cluster
Nilotpal Nandi created HDDS-1057:
Summary: get key operation fails when client cannot communicate
with 2 of the datanodes in 3 node cluster
Key: HDDS-1057
URL: https://issues.apache.org/jira/browse/HDDS-1057
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Reporter: Nilotpal Nandi
steps taken :
--
# created 3 node docker cluster.
# wrote a key
# created partition such that 2 out of 3 datanodes cannot communicate with any
other node.
# Third datanode can communicate with all other nodes.
# Tried to read the key
Exception seen :
{noformat}
Failed to execute command cmdType: GetBlock
E traceID: "9b3ebd93-e598-4ca2-a6f4-2389f2d35f63"
E containerID: 22
E datanodeUuid: "15345663-15c9-4fe3-9b8f-a46123ba8a6e"
E getBlock {
E blockID {
E containerID: 22
E localID: 101545011736215553
E blockCommitSequenceId: 5
E }
E }
E on datanode 15345663-15c9-4fe3-9b8f-a46123ba8a6e
E java.util.concurrent.ExecutionException:
org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: UNAVAILABLE: io
exception
E at
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
E at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
E at
org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommandWithRetry(XceiverClientGrpc.java:220)
E at
org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommand(XceiverClientGrpc.java:201)
E at
org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.getBlock(ContainerProtocolCalls.java:118)
E at
org.apache.hadoop.ozone.client.io.KeyInputStream.getFromOmKeyInfo(KeyInputStream.java:305)
E at org.apache.hadoop.ozone.client.rpc.RpcClient.getKey(RpcClient.java:608)
E at org.apache.hadoop.ozone.client.OzoneBucket.readKey(OzoneBucket.java:284)
E at
org.apache.hadoop.ozone.web.ozShell.keys.GetKeyHandler.call(GetKeyHandler.java:95)
E at
org.apache.hadoop.ozone.web.ozShell.keys.GetKeyHandler.call(GetKeyHandler.java:48)
E at picocli.CommandLine.execute(CommandLine.java:919)
E at picocli.CommandLine.access$700(CommandLine.java:104)
E at picocli.CommandLine$RunLast.handle(CommandLine.java:1083)
E at picocli.CommandLine$RunLast.handle(CommandLine.java:1051)
E at
picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:959)
E at picocli.CommandLine.parseWithHandlers(CommandLine.java:1242)
E at picocli.CommandLine.parseWithHandler(CommandLine.java:1181)
E at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:61)
E at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:52)
E at org.apache.hadoop.ozone.web.ozShell.Shell.main(Shell.java:83)
E Caused by: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
UNAVAILABLE: io exception
E at
org.apache.ratis.thirdparty.io.grpc.Status.asRuntimeException(Status.java:526)
E at
org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:434)
E at
org.apache.ratis.thirdparty.io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
E at
org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
E at
org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
E at
org.apache.ratis.thirdparty.io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:678)
E at
org.apache.ratis.thirdparty.io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
E at
org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
E at
org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
E at
org.apache.ratis.thirdparty.io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:397)
E at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:459)
E at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
E at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:546)
E at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:467)
E at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:584)
E at
org.apache.ratis.thirdparty.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
E at
org.apache.ratis.thirdparty.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
E at
java.util.concurrent.ThreadPoolExecutor.r
[jira] [Created] (HDDS-1067) freon run on client gets hung when two of the datanodes are down in 3 datanode cluster
Nilotpal Nandi created HDDS-1067: Summary: freon run on client gets hung when two of the datanodes are down in 3 datanode cluster Key: HDDS-1067 URL: https://issues.apache.org/jira/browse/HDDS-1067 Project: Hadoop Distributed Data Store Issue Type: Bug Components: Ozone Client Reporter: Nilotpal Nandi steps taken : # created 3 node docker cluster. # wrote a key # created partition such that 2 out of 3 datanodes cannot communicate with any other node. # Third datanode can communicate with scm, om and the client. # ran freon to write key Observation : - freon run is hung. There is no timeout. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1079) java.lang.RuntimeException: ManagedChannel allocation site exception seen on client cli when datanode restarted in one of the pipelines
Nilotpal Nandi created HDDS-1079:
Summary: java.lang.RuntimeException: ManagedChannel allocation
site exception seen on client cli when datanode restarted in one of the
pipelines
Key: HDDS-1079
URL: https://issues.apache.org/jira/browse/HDDS-1079
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Reporter: Nilotpal Nandi
steps taken :
# created 12 datanode cluster.
# started put key operation with size 100GB.
# Restarted one of the datanodes from one of the pipelines.
exception seen on cli :
{noformat}
[root@ctr-e139-1542663976389-62237-01-06 ~]# time ozone sh key put
volume1/bucket1/key1 /root/100G
Feb 11, 2019 9:12:49 AM
org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference
cleanQueue
SEVERE: *~*~*~ Channel ManagedChannelImpl{logId=61, target=172.27.10.133:9858}
was not shutdown properly!!! ~*~*~*
Make sure to call shutdown()/shutdownNow() and wait until awaitTermination()
returns true.
java.lang.RuntimeException: ManagedChannel allocation site
at
org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.(ManagedChannelOrphanWrapper.java:103)
at
org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper.(ManagedChannelOrphanWrapper.java:53)
at
org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper.(ManagedChannelOrphanWrapper.java:44)
at
org.apache.ratis.thirdparty.io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:411)
at
org.apache.ratis.grpc.client.GrpcClientProtocolClient.(GrpcClientProtocolClient.java:116)
at
org.apache.ratis.grpc.client.GrpcClientRpc.lambda$new$0(GrpcClientRpc.java:54)
at
org.apache.ratis.util.PeerProxyMap$PeerAndProxy.lambda$getProxy$0(PeerProxyMap.java:60)
at org.apache.ratis.util.LifeCycle.startAndTransition(LifeCycle.java:191)
at
org.apache.ratis.util.PeerProxyMap$PeerAndProxy.getProxy(PeerProxyMap.java:59)
at org.apache.ratis.util.PeerProxyMap.getProxy(PeerProxyMap.java:106)
at
org.apache.ratis.grpc.client.GrpcClientRpc.sendRequestAsync(GrpcClientRpc.java:69)
at
org.apache.ratis.client.impl.RaftClientImpl.sendRequestAsync(RaftClientImpl.java:324)
at
org.apache.ratis.client.impl.RaftClientImpl.sendRequestWithRetryAsync(RaftClientImpl.java:286)
at
org.apache.ratis.util.SlidingWindow$Client.sendOrDelayRequest(SlidingWindow.java:243)
at org.apache.ratis.util.SlidingWindow$Client.retry(SlidingWindow.java:259)
at
org.apache.ratis.client.impl.RaftClientImpl.lambda$null$10(RaftClientImpl.java:293)
at
org.apache.ratis.util.TimeoutScheduler.lambda$onTimeout$0(TimeoutScheduler.java:85)
at
org.apache.ratis.util.TimeoutScheduler.lambda$onTimeout$1(TimeoutScheduler.java:104)
at org.apache.ratis.util.LogUtils.runAndLog(LogUtils.java:50)
at org.apache.ratis.util.LogUtils$1.run(LogUtils.java:91)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Feb 11, 2019 9:12:49 AM
org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference
cleanQueue
SEVERE: *~*~*~ Channel ManagedChannelImpl{logId=29, target=172.27.10.133:9858}
was not shutdown properly!!! ~*~*~*
Make sure to call shutdown()/shutdownNow() and wait until awaitTermination()
returns true.
java.lang.RuntimeException: ManagedChannel allocation site
at
org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.(ManagedChannelOrphanWrapper.java:103)
at
org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper.(ManagedChannelOrphanWrapper.java:53)
at
org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper.(ManagedChannelOrphanWrapper.java:44)
at
org.apache.ratis.thirdparty.io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:411)
at
org.apache.ratis.grpc.client.GrpcClientProtocolClient.(GrpcClientProtocolClient.java:116)
at
org.apache.ratis.grpc.client.GrpcClientRpc.lambda$new$0(GrpcClientRpc.java:54)
at
org.apache.ratis.util.PeerProxyMap$PeerAndProxy.lambda$getProxy$0(PeerProxyMap.java:60)
at org.apache.ratis.util.LifeCycle.startAndTransition(LifeCycle.java:191)
at
org.apache.ratis.util.PeerProxyMap$PeerAndProxy.getProxy(PeerPr
[jira] [Created] (HDDS-1082) OutOfMemoryError while reading key
Nilotpal Nandi created HDDS-1082:
Summary: OutOfMemoryError while reading key
Key: HDDS-1082
URL: https://issues.apache.org/jira/browse/HDDS-1082
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken :
# put key with size 100GB
# Tried to read back the key.
error thrown:
--
{noformat}
java.lang.OutOfMemoryError: Java heap space
Dumping heap to /tmp/heapdump.bin ...
Heap dump file created [3883178021 bytes in 10.667 secs]
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at
org.apache.ratis.thirdparty.com.google.protobuf.ByteString.toByteArray(ByteString.java:643)
at org.apache.hadoop.ozone.common.Checksum.verifyChecksum(Checksum.java:217)
at
org.apache.hadoop.hdds.scm.storage.BlockInputStream.readChunkFromContainer(BlockInputStream.java:227)
at
org.apache.hadoop.hdds.scm.storage.BlockInputStream.prepareRead(BlockInputStream.java:188)
at
org.apache.hadoop.hdds.scm.storage.BlockInputStream.read(BlockInputStream.java:130)
at
org.apache.hadoop.ozone.client.io.KeyInputStream$ChunkInputStreamEntry.read(KeyInputStream.java:232)
at
org.apache.hadoop.ozone.client.io.KeyInputStream.read(KeyInputStream.java:126)
at
org.apache.hadoop.ozone.client.io.OzoneInputStream.read(OzoneInputStream.java:49)
at java.io.InputStream.read(InputStream.java:101)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:100)
at
org.apache.hadoop.ozone.web.ozShell.keys.GetKeyHandler.call(GetKeyHandler.java:98)
at
org.apache.hadoop.ozone.web.ozShell.keys.GetKeyHandler.call(GetKeyHandler.java:48)
at picocli.CommandLine.execute(CommandLine.java:919)
at picocli.CommandLine.access$700(CommandLine.java:104)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1083)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1051)
at
picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:959)
at picocli.CommandLine.parseWithHandlers(CommandLine.java:1242)
at picocli.CommandLine.parseWithHandler(CommandLine.java:1181)
at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:61)
at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:52)
at org.apache.hadoop.ozone.web.ozShell.Shell.main(Shell.java:83){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1088) Add blockade Tests to test Replica Manager
Nilotpal Nandi created HDDS-1088: Summary: Add blockade Tests to test Replica Manager Key: HDDS-1088 URL: https://issues.apache.org/jira/browse/HDDS-1088 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Nilotpal Nandi We need to add tests for testing Replica Manager for scenarios like loss of node, adding new nodes, under-replicated containers -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1102) docker datanode stopped when new datanodes are added to the cluster
Nilotpal Nandi created HDDS-1102:
Summary: docker datanode stopped when new datanodes are added to
the cluster
Key: HDDS-1102
URL: https://issues.apache.org/jira/browse/HDDS-1102
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken:
# created 5 datanode cluster.
# shutdown 2 datanodes
# started the datanodes again.
One of the datanodes was shut down.
exception seen :
{noformat}
2019-02-14 07:37:26 INFO LeaderElection:230 -
6a0522ba-019e-4b77-ac1f-a9322cd525b8 got exception when requesting votes: {}
java.util.concurrent.ExecutionException:
org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: INTERNAL:
a3d1dd2d-554e-4e87-a2cf-076a229af352: group-FD6FA533F1FB not found.
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.ratis.server.impl.LeaderElection.waitForResults(LeaderElection.java:214)
at
org.apache.ratis.server.impl.LeaderElection.askForVotes(LeaderElection.java:146)
at org.apache.ratis.server.impl.LeaderElection.run(LeaderElection.java:102)
Caused by: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
INTERNAL: a3d1dd2d-554e-4e87-a2cf-076a229af352: group-FD6FA533F1FB not found.
at
org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:233)
at
org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:214)
at
org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:139)
at
org.apache.ratis.proto.grpc.RaftServerProtocolServiceGrpc$RaftServerProtocolServiceBlockingStub.requestVote(RaftServerProtocolServiceGrpc.java:265)
at
org.apache.ratis.grpc.server.GrpcServerProtocolClient.requestVote(GrpcServerProtocolClient.java:83)
at org.apache.ratis.grpc.server.GrpcService.requestVote(GrpcService.java:187)
at
org.apache.ratis.server.impl.LeaderElection.lambda$submitRequests$0(LeaderElection.java:188)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-02-14 07:37:26 INFO LeaderElection:46 -
6a0522ba-019e-4b77-ac1f-a9322cd525b8: Election PASSED; received 1 response(s)
[6a0522ba-019e-4b77-ac1f-a9322cd525b8<-61ad3bf3-e9b1-48e5-90e3-3b78c8b5bba5#0:OK-t7]
and 1 exception(s); 6a0522ba-019e-4b77-ac1f-a9322cd525b8:t7, leader=null,
voted=6a0522ba-019e-4b77-ac1f-a9322cd525b8,
raftlog=6a0522ba-019e-4b77-ac1f-a9322cd525b8-SegmentedRaftLog:OPENED, conf=3:
[61ad3bf3-e9b1-48e5-90e3-3b78c8b5bba5:172.20.0.8:9858,
6a0522ba-019e-4b77-ac1f-a9322cd525b8:172.20.0.6:9858,
0f377918-aafa-4d8a-972a-6ead54048fba:172.20.0.3:9858], old=null
2019-02-14 07:37:26 INFO LeaderElection:52 - 0:
java.util.concurrent.ExecutionException:
org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: INTERNAL:
a3d1dd2d-554e-4e87-a2cf-076a229af352: group-FD6FA533F1FB not found.
2019-02-14 07:37:26 INFO RoleInfo:130 - 6a0522ba-019e-4b77-ac1f-a9322cd525b8:
shutdown LeaderElection
2019-02-14 07:37:26 INFO RaftServerImpl:161 -
6a0522ba-019e-4b77-ac1f-a9322cd525b8 changes role from CANDIDATE to LEADER at
term 7 for changeToLeader
2019-02-14 07:37:26 INFO RaftServerImpl:258 -
6a0522ba-019e-4b77-ac1f-a9322cd525b8: change Leader from null to
6a0522ba-019e-4b77-ac1f-a9322cd525b8 at term 7 for becomeLeader, leader elected
after 1066ms
2019-02-14 07:37:26 INFO RaftServerConfigKeys:43 -
raft.server.staging.catchup.gap = 1000 (default)
2019-02-14 07:37:26 INFO RaftServerConfigKeys:43 - raft.server.rpc.sleep.time =
25ms (default)
2019-02-14 07:37:26 INFO RaftServerConfigKeys:43 - raft.server.watch.timeout =
10s (default)
2019-02-14 07:37:26 INFO RaftServerConfigKeys:43 -
raft.server.watch.timeout.denomination = 1s (default)
2019-02-14 07:37:26 INFO RaftServerConfigKeys:43 -
raft.server.log.appender.snapshot.chunk.size.max = 16MB (=16777216) (default)
2019-02-14 07:37:26 INFO RaftServerConfigKeys:43 -
raft.server.log.appender.buffer.byte-limit = 33554432 (custom)
2019-02-14 07:37:26 INFO RaftServerConfigKeys:43 -
raft.server.log.appender.buffer.element-limit = 1 (custom)
2019-02-14 07:37:26 INFO GrpcConfigKeys$Server:43 -
raft.grpc.server.leader.outstanding.appends.max = 128 (default)
2019-02-14 07:37:26 INFO RaftServerConfigKeys:43 -
raft.server.rpc.request.timeout = 3000ms (default)
2019-02-14 07:37:26 INFO RaftServerConfigKeys:43 -
raft.server.log.appender.snapshot.chunk.size.max = 16MB (=16777216) (default)
2019-02-14 07:37:26 INFO RaftServerConfigKeys:43 -
raft.server.log.appe
[jira] [Created] (HDDS-1124) java.lang.IllegalStateException exception in datanode log
Nilotpal Nandi created HDDS-1124:
Summary: java.lang.IllegalStateException exception in datanode log
Key: HDDS-1124
URL: https://issues.apache.org/jira/browse/HDDS-1124
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken :
# created 12 datanodes cluster and running workload on all the nodes
exception seen :
---
{noformat}
2019-02-15 10:15:53,355 INFO org.apache.ratis.server.storage.RaftLogWorker:
943007c8-4fdd-4926-89e2-2c8c52c05073-RaftLogWorker: Rolled log segment from
/data/disk1/ozone/meta/ratis/01d3ef2a-912c-4fc0-80b6-012343d76adb/current/log_inprogress_3036
to
/data/disk1/ozone/meta/ratis/01d3ef2a-912c-4fc0-80b6-012343d76adb/current/log_3036-3047
2019-02-15 10:15:53,367 INFO org.apache.ratis.server.impl.RaftServerImpl:
943007c8-4fdd-4926-89e2-2c8c52c05073: set configuration 3048:
[a40a7b01-a30b-469c-b373-9fcb20a126ed:172.27.54.212:9858,
8c77b16b-8054-49e3-b669-1ff759cfd271:172.27.23.196:9858,
943007c8-4fdd-4926-89e2-2c8c52c05073:172.27.76.72:9858], old=null at 3048
2019-02-15 10:15:53,523 INFO org.apache.ratis.server.storage.RaftLogWorker:
943007c8-4fdd-4926-89e2-2c8c52c05073-RaftLogWorker: created new log segment
/data/disk1/ozone/meta/ratis/01d3ef2a-912c-4fc0-80b6-012343d76adb/current/log_inprogress_3048
2019-02-15 10:15:53,580 ERROR org.apache.ratis.grpc.server.GrpcLogAppender:
Failed onNext serverReply {
requestorId: "943007c8-4fdd-4926-89e2-2c8c52c05073"
replyId: "a40a7b01-a30b-469c-b373-9fcb20a126ed"
raftGroupId {
id: "\001\323\357*\221,O\300\200\266\001#C\327j\333"
}
success: true
}
term: 3
nextIndex: 3049
followerCommit: 3047
java.lang.IllegalStateException: reply's next index is 3049, request's previous
is term: 1
index: 3047
at org.apache.ratis.util.Preconditions.assertTrue(Preconditions.java:60)
at
org.apache.ratis.grpc.server.GrpcLogAppender.onSuccess(GrpcLogAppender.java:285)
at
org.apache.ratis.grpc.server.GrpcLogAppender$AppendLogResponseHandler.onNextImpl(GrpcLogAppender.java:230)
at
org.apache.ratis.grpc.server.GrpcLogAppender$AppendLogResponseHandler.onNext(GrpcLogAppender.java:215)
at
org.apache.ratis.grpc.server.GrpcLogAppender$AppendLogResponseHandler.onNext(GrpcLogAppender.java:197)
at
org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onMessage(ClientCalls.java:421)
at
org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33)
at
org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33)
at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:519)
at
org.apache.ratis.thirdparty.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at
org.apache.ratis.thirdparty.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-02-15 10:15:56,442 INFO org.apache.ratis.server.storage.RaftLogWorker:
943007c8-4fdd-4926-89e2-2c8c52c05073-RaftLogWorker: Rolling segment
log-3048_3066 to index:3066
2019-02-15 10:15:56,442 INFO org.apache.ratis.server.storage.RaftLogWorker:
943007c8-4fdd-4926-89e2-2c8c52c05073-RaftLogWorker: Rolled log segment from
/data/disk1/ozone/meta/ratis/01d3ef2a-912c-4fc0-80b6-012343d76adb/current/log_inprogress_3048
to
/data/disk1/ozone/meta/ratis/01d3ef2a-912c-4fc0-80b6-012343d76adb/current/log_3048-3066
2019-02-15 10:15:56,564 INFO org.apache.ratis.server.storage.RaftLogWorker:
943007c8-4fdd-4926-89e2-2c8c52c05073-RaftLogWorker: created new log segment
/data/disk1/ozone/meta/ratis/01d3ef2a-912c-4fc0-80b6-012343d76adb/current/log_inprogress_3067
2019-02-15 10:16:45,420 INFO org.apache.ratis.server.storage.RaftLogWorker:
943007c8-4fdd-4926-89e2-2c8c52c05073-RaftLogWorker: Rolling segment
log-3067_3077 to index:3077
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1125) java.lang.InterruptedException seen in datanode logs
Nilotpal Nandi created HDDS-1125:
Summary: java.lang.InterruptedException seen in datanode logs
Key: HDDS-1125
URL: https://issues.apache.org/jira/browse/HDDS-1125
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken :
# created 12 datanodes cluster and running workload on all the nodes
exception seen :
-
{noformat}
2019-02-15 10:16:48,713 ERROR org.apache.ratis.server.impl.LogAppender:
943007c8-4fdd-4926-89e2-2c8c52c05073: Failed readStateMachineData for (t:3,
i:3084), STATEMACHINELOGENTRY, client-632E77ADA885, cid=6232
java.lang.InterruptedException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:347)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at
org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at
org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at
org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748)
2019-02-15 10:16:48,714 ERROR org.apache.ratis.server.impl.LogAppender:
GrpcLogAppender(943007c8-4fdd-4926-89e2-2c8c52c05073 ->
8c77b16b-8054-49e3-b669-1ff759cfd271) hit IOException while loading raft log
org.apache.ratis.server.storage.RaftLogIOException:
943007c8-4fdd-4926-89e2-2c8c52c05073: Failed readStateMachineData for (t:3,
i:3084), STATEMACHINELOGENTRY, client-632E77ADA885, cid=6232
at
org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:440)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at
org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at
org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.InterruptedException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:347)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at
org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
... 6 more
2019-02-15 10:16:48,715 ERROR org.apache.ratis.server.impl.LogAppender:
943007c8-4fdd-4926-89e2-2c8c52c05073: Failed readStateMachineData for (t:3,
i:3084), STATEMACHINELOGENTRY, client-632E77ADA885, cid=6232
java.lang.InterruptedException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:347)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at
org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at
org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at
org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748)
2019-02-15 10:16:48,715 ERROR org.apache.ratis.server.impl.LogAppender:
GrpcLogAppender(943007c8-4fdd-4926-89e2-2c8c52c05073 ->
a40a7b01-a30b-469c-b373-9fcb20a126ed) hit IOException while loading raft log
org.apache.ratis.server.storage.RaftLogIOException:
943007c8-4fdd-4926-89e2-2c8c52c05073: Failed readStateMachineData for (t:3,
i:3084), STATEMACHINELOGENTRY, client-632E77ADA885, cid=6232
at
org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:440)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at
org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at
org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.InterruptedException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:347)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at
org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
... 6 more
2019-02-15 10:16:48,723 WARN
org.apache.ratis.grpc.client.GrpcClientProtocolService:
943007c8-4fdd-4926-89e2-2c8c52c05073-5: onError:
org.apache.ratis.thir
[jira] [Created] (HDDS-1126) datanode is trying to qausi-close a container which is already closed
Nilotpal Nandi created HDDS-1126:
Summary: datanode is trying to qausi-close a container which is
already closed
Key: HDDS-1126
URL: https://issues.apache.org/jira/browse/HDDS-1126
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken :
# created 12 datanodes cluster and running workload on all the nodes
# running failure injection/restart on 1 datanode at a time periodically and
randomly.
Error seen in ozone.log :
--
{noformat}
2019-02-18 06:06:32,780 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:176) - Executing cycle Number : 30
2019-02-18 06:06:32,784 [Command processor thread] DEBUG
(CloseContainerCommandHandler.java:71) - Processing Close Container command.
2019-02-18 06:06:32,785 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:176) - Executing cycle Number : 31
2019-02-18 06:06:32,785 [Command processor thread] ERROR
(CloseContainerCommandHandler.java:118) - Can't close container #37
org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException:
Cannot quasi close container #37 while in CLOSED state.
at
org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.quasiCloseContainer(KeyValueHandler.java:903)
at
org.apache.hadoop.ozone.container.ozoneimpl.ContainerController.quasiCloseContainer(ContainerController.java:93)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler.handle(CloseContainerCommandHandler.java:110)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CommandDispatcher.handle(CommandDispatcher.java:93)
at
org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$initCommandHandlerThread$1(DatanodeStateMachine.java:413)
at java.lang.Thread.run(Thread.java:748)
2019-02-18 06:06:32,785 [Command processor thread] DEBUG
(CloseContainerCommandHandler.java:71) - Processing Close Container command.
2019-02-18 06:06:32,788 [Command processor thread] DEBUG
(CloseContainerCommandHandler.java:71) - Processing Close Container command.
2019-02-18 06:06:32,788 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:176) - Executing cycle Number : 32
2019-02-18 06:06:34,430 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:36,608 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:38,876 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:41,084 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:43,297 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:45,469 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:47,684 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:49,958 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:52,124 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:54,344 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:56,499 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:58,764 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:07:00,969 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:07:02,788 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:176) - Executing cycle Number : 33
2019-02-18 06:07:03,240 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:07:05,486 [main] DEBUG (OzoneClientFactory.java:287) - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1131) destroy pipeline failed with PipelineNotFoundException
Nilotpal Nandi created HDDS-1131:
Summary: destroy pipeline failed with PipelineNotFoundException
Key: HDDS-1131
URL: https://issues.apache.org/jira/browse/HDDS-1131
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken :
# created 12 datanodes cluster and running workload on all the nodes
exceptions seen in scm log
{noformat}
2019-02-18 07:17:51,112 INFO
org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils: destroying
pipeline:PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb with
group-012343D76ADB:[a40a7b01-a30b-469c-b373-9fcb20a126ed:172.27.54.212:9858,
8c77b16b-8054-49e3-b669-1ff759cfd271:172.27.23.196:9858,
943007c8-4fdd-4926-89e2-2c8c52c05073:172.27.76.72:9858]
2019-02-18 07:17:51,112 INFO
org.apache.hadoop.hdds.scm.container.CloseContainerEventHandler: Close
container Event triggered for container : #40
2019-02-18 07:17:51,113 INFO
org.apache.hadoop.hdds.scm.container.CloseContainerEventHandler: Close
container Event triggered for container : #41
2019-02-18 07:17:51,114 INFO
org.apache.hadoop.hdds.scm.container.CloseContainerEventHandler: Close
container Event triggered for container : #42
2019-02-18 07:22:51,127 WARN
org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils: Pipeline destroy failed
for pipeline=PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb
dn=a40a7b01-a30b-469c-b373-9fcb20a126ed{ip: 172.27.54.212, host:
ctr-e139-1542663976389-62237-01-07.hwx.site}
2019-02-18 07:22:51,139 WARN
org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils: Pipeline destroy failed
for pipeline=PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb
dn=8c77b16b-8054-49e3-b669-1ff759cfd271{ip: 172.27.23.196, host:
ctr-e139-1542663976389-62237-01-15.hwx.site}
2019-02-18 07:22:51,149 WARN
org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils: Pipeline destroy failed
for pipeline=PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb
dn=943007c8-4fdd-4926-89e2-2c8c52c05073{ip: 172.27.76.72, host:
ctr-e139-1542663976389-62237-01-06.hwx.site}
2019-02-18 07:22:51,150 ERROR
org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils: Destroy pipeline failed
for pipeline:PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb with
group-012343D76ADB:[a40a7b01-a30b-469c-b373-9fcb20a126ed:172.27.54.212:9858,
8c77b16b-8054-49e3-b669-1ff759cfd271:172.27.23.196:9858,
943007c8-4fdd-4926-89e2-2c8c52c05073:172.27.76.72:9858]
org.apache.hadoop.hdds.scm.pipeline.PipelineNotFoundException:
PipelineID=01d3ef2a-912c-4fc0-80b6-012343d76adb not found
at
org.apache.hadoop.hdds.scm.pipeline.PipelineStateMap.getPipeline(PipelineStateMap.java:112)
at
org.apache.hadoop.hdds.scm.pipeline.PipelineStateMap.removePipeline(PipelineStateMap.java:247)
at
org.apache.hadoop.hdds.scm.pipeline.PipelineStateManager.removePipeline(PipelineStateManager.java:90)
at
org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.removePipeline(SCMPipelineManager.java:261)
at
org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils.destroyPipeline(RatisPipelineUtils.java:103)
at
org.apache.hadoop.hdds.scm.pipeline.RatisPipelineUtils.lambda$finalizeAndDestroyPipeline$1(RatisPipelineUtils.java:133)
at
org.apache.ratis.util.TimeoutScheduler.lambda$onTimeout$0(TimeoutScheduler.java:85)
at
org.apache.ratis.util.TimeoutScheduler.lambda$onTimeout$1(TimeoutScheduler.java:104)
at org.apache.ratis.util.LogUtils.runAndLog(LogUtils.java:50)
at org.apache.ratis.util.LogUtils$1.run(LogUtils.java:91)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1164) Add New blockade Tests to test Replica Manager
Nilotpal Nandi created HDDS-1164: Summary: Add New blockade Tests to test Replica Manager Key: HDDS-1164 URL: https://issues.apache.org/jira/browse/HDDS-1164 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Nilotpal Nandi -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1206) need to handle in the client when one of the datanode disk goes out of space
Nilotpal Nandi created HDDS-1206:
Summary: need to handle in the client when one of the datanode
disk goes out of space
Key: HDDS-1206
URL: https://issues.apache.org/jira/browse/HDDS-1206
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
Assignee: Shashikant Banerjee
steps taken :
# create 40 datanode cluster.
# one of the datanodes has less than 5 GB space.
# Started writing key of size 600MB.
operation failed:
Error on the client:
{noformat}
Fri Mar 1 09:05:28 UTC 2019 Ruuning
/root/hadoop_trunk/ozone-0.4.0-SNAPSHOT/bin/ozone sh key put
testvol172275910-1551431122-1/testbuck172275910-1551431122-1/test_file24
/root/test_files/test_file24
original md5sum a6de00c9284708585f5a99b0490b0b23
2019-03-01 09:05:39,142 ERROR storage.BlockOutputStream: Unexpected Storage
Container Exception:
org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException:
ContainerID 79 creation failed
at
org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.validateContainerResponse(ContainerProtocolCalls.java:568)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.validateResponse(BlockOutputStream.java:535)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.lambda$writeChunkToContainer$5(BlockOutputStream.java:613)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-03-01 09:05:39,578 ERROR storage.BlockOutputStream: Unexpected Storage
Container Exception:
org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException:
ContainerID 79 creation failed
at
org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.validateContainerResponse(ContainerProtocolCalls.java:568)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.validateResponse(BlockOutputStream.java:535)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.lambda$writeChunkToContainer$5(BlockOutputStream.java:613)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-03-01 09:05:40,368 ERROR storage.BlockOutputStream: Unexpected Storage
Container Exception:
org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException:
ContainerID 79 creation failed
at
org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.validateContainerResponse(ContainerProtocolCalls.java:568)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.validateResponse(BlockOutputStream.java:535)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.lambda$writeChunkToContainer$5(BlockOutputStream.java:613)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-03-01 09:05:40,450 ERROR storage.BlockOutputStream: Unexpected Storage
Container Exception:
org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException:
ContainerID 79 creation failed
at
org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.validateContainerResponse(ContainerProtocolCalls.java:568)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.validateResponse(BlockOutputStream.java:535)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.lambda$writeChunkToContainer$5(BlockOutputStream.java:613)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20
[jira] [Created] (HDDS-1251) all chunks are not deleted by block deletion even when all keys are deleted and all containers are closed
Nilotpal Nandi created HDDS-1251: Summary: all chunks are not deleted by block deletion even when all keys are deleted and all containers are closed Key: HDDS-1251 URL: https://issues.apache.org/jira/browse/HDDS-1251 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Nilotpal Nandi steps taken : --- # created 40 nodes cluster, wrote data on all datanodes. # deleted all keys from the cluster and all containers are closed. block deletion triggered and deleted most of the chunks from all datanodes. But , it could not delete all chunks even after several days. expectations : all chunks should be deleted if there is no key present in the cluster and all containers are closed -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1289) get Key failed on SCM restart
Nilotpal Nandi created HDDS-1289:
Summary: get Key failed on SCM restart
Key: HDDS-1289
URL: https://issues.apache.org/jira/browse/HDDS-1289
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
Attachments: hadoop-hdfs-scm-ctr-e139-1542663976389-86524-01-03.log
Seeing ContainerNotFoundException in scm log when get key operation tried after
scm restart.
scm.log:
[^hadoop-hdfs-scm-ctr-e139-1542663976389-86524-01-03.log]
{noformat}
2019-03-13 17:00:54,348 ERROR container.ContainerReportHandler
(ContainerReportHandler.java:processContainerReplicas(173)) - Received
container report for an unknown container 22 from datanode
80f046cb-6fe2-4a05-bb67-9bf46f48723b{ip: 172.27.69.155, host:
ctr-e139-1542663976389-86524-01-05.hwx.site} {}
org.apache.hadoop.hdds.scm.container.ContainerNotFoundException: #22 at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.checkIfContainerExist(ContainerStateMap.java:543)
at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.updateContainerReplica(ContainerStateMap.java:230)
at
org.apache.hadoop.hdds.scm.container.ContainerStateManager.updateContainerReplica(ContainerStateManager.java:565)
at
org.apache.hadoop.hdds.scm.container.SCMContainerManager.updateContainerReplica(SCMContainerManager.java:393)
at
org.apache.hadoop.hdds.scm.container.ReportHandlerHelper.processContainerReplica(ReportHandlerHelper.java:74)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.processContainerReplicas(ContainerReportHandler.java:159)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:110)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:51)
at
org.apache.hadoop.hdds.server.events.SingleThreadExecutor.lambda$onMessage$1(SingleThreadExecutor.java:85)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 2019-03-13 17:00:54,349 ERROR
container.ContainerReportHandler
(ContainerReportHandler.java:processContainerReplicas(173)) - Received
container report for an unknown container 23 from datanode
80f046cb-6fe2-4a05-bb67-9bf46f48723b{ip: 172.27.69.155, host:
ctr-e139-1542663976389-86524-01-05.hwx.site} {}
org.apache.hadoop.hdds.scm.container.ContainerNotFoundException: #23 at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.checkIfContainerExist(ContainerStateMap.java:543)
at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.updateContainerReplica(ContainerStateMap.java:230)
at
org.apache.hadoop.hdds.scm.container.ContainerStateManager.updateContainerReplica(ContainerStateManager.java:565)
at
org.apache.hadoop.hdds.scm.container.SCMContainerManager.updateContainerReplica(SCMContainerManager.java:393)
at
org.apache.hadoop.hdds.scm.container.ReportHandlerHelper.processContainerReplica(ReportHandlerHelper.java:74)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.processContainerReplicas(ContainerReportHandler.java:159)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:110)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:51)
at
org.apache.hadoop.hdds.server.events.SingleThreadExecutor.lambda$onMessage$1(SingleThreadExecutor.java:85)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 2019-03-13 17:01:24,230 ERROR
container.ContainerReportHandler
(ContainerReportHandler.java:processContainerReplicas(173)) - Received
container report for an unknown container 22 from datanode
076fd0d8-ab5f-4fbe-ad10-b71a1ccb19bf{ip: 172.27.39.88, host:
ctr-e139-1542663976389-86524-01-04.hwx.site} {}
org.apache.hadoop.hdds.scm.container.ContainerNotFoundException: #22 at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.checkIfContainerExist(ContainerStateMap.java:543)
at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.updateContainerReplica(ContainerStateMap.java:230)
at
org.apache.hadoop.hdds.scm.container.ContainerStateManager.updateContainerReplica(ContainerStateManager.java:565)
at
org.apache.hadoop.hdds.scm.container.SCMContainerManager.updateContainerReplica(SCMContainerManager.java:393)
at
org.apache.hadoop.hdds.scm.container.ReportHandlerHelper.processContainerReplica(ReportHandlerHelper.java:74)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.processContainerReplicas(ContainerReportHandler.java:159)
at
org.apache.hadoop.hdds.scm.container.ContainerR
[jira] [Created] (HDDS-1290) ozone.log is not getting created in logs directory
Nilotpal Nandi created HDDS-1290: Summary: ozone.log is not getting created in logs directory Key: HDDS-1290 URL: https://issues.apache.org/jira/browse/HDDS-1290 Project: Hadoop Distributed Data Store Issue Type: Bug Components: Ozone Client Reporter: Nilotpal Nandi ozone.log is getting created in the log directory of the client or any other nodes of ozone cluster. ozone version : Source code repository g...@github.com:hortonworks/ozone.git -r 67b7c4fd071b3f557bdb54be2a266b8a611cbce6 Compiled by jenkins on 2019-03-06T22:02Z Compiled with protoc 2.5.0 >From source with checksum 65be9a337d178cd3855f5c5a2f111 Using HDDS 0.4.0.3.0.100.0-348 Source code repository g...@github.com:hortonworks/ozone.git -r 67b7c4fd071b3f557bdb54be2a266b8a611cbce6 Compiled by jenkins on 2019-03-06T22:01Z Compiled with protoc 2.5.0 >From source with checksum 324109cb3e8b188c1b89dc0b328c3a [root@ctr-e139-1542663976389-86524-01-06 hdfs]# hadoop version Hadoop 3.1.1.3.0.100.0-348 Source code repository g...@github.com:hortonworks/hadoop.git -r 484434b1c2480bdc9314a7ee1ade8a0f4db1758f Compiled by jenkins on 2019-03-06T22:14Z Compiled with protoc 2.5.0 >From source with checksum ba6aad94c14256ef3ad8634e3b5086 This command was run using /usr/hdp/3.0.100.0-348/hadoop/hadoop-common-3.1.1.3.0.100.0-348.jar -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1298) blockade tests failing as the nodes are not able to communicate with Ozone Manager
Nilotpal Nandi created HDDS-1298:
Summary: blockade tests failing as the nodes are not able to
communicate with Ozone Manager
Key: HDDS-1298
URL: https://issues.apache.org/jira/browse/HDDS-1298
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken:
# started 3 datanodes docker cluster.
# freon run fails with error : "No such service: ozoneManager"
{noformat}
om_1 | STARTUP_MSG: build = https://github.com/apache/hadoop.git -r
e97acb3bd8f3befd27418996fa5d4b50bf2e17bf; compiled by 'sunilg' on
2019-01-15T17:34Z
om_1 | STARTUP_MSG: java = 11.0.1
om_1 | /
om_1 | 2019-03-18 06:31:41 INFO OzoneManager:51 - registered UNIX signal
handlers for [TERM, HUP, INT]
om_1 | 2019-03-18 06:31:41 WARN ScmUtils:77 - ozone.om.db.dirs is not
configured. We recommend adding this setting. Falling back to
ozone.metadata.dirs instead.
om_1 | 2019-03-18 06:31:41 INFO OzoneManager:484 - OM Service ID is not set.
Setting it to the default ID: omServiceIdDefault
om_1 | 2019-03-18 06:31:41 INFO OzoneManager:490 - OM Node ID is not set.
Setting it to the OmStorage's OmID: 25501758-f7f6-42d5-8196-52a885af7e23
om_1 | 2019-03-18 06:31:41 INFO OzoneManager:441 - Found matching OM address
with OMServiceId: null, OMNodeId: null, RPC Address: om:9862 and Ratis port:
9872
om_1 | 2019-03-18 06:31:42 WARN ScmUtils:77 - ozone.om.db.dirs is not
configured. We recommend adding this setting. Falling back to
ozone.metadata.dirs instead.
om_1 | 2019-03-18 06:31:42 INFO log:192 - Logging initialized @4061ms
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
table: userTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
profile:DBProfile.DISK for Table:userTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
table: volumeTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
profile:DBProfile.DISK for Table:volumeTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
table: bucketTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
profile:DBProfile.DISK for Table:bucketTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
table: keyTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
profile:DBProfile.DISK for Table:keyTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
table: deletedTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
profile:DBProfile.DISK for Table:deletedTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
table: openKeyTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
profile:DBProfile.DISK for Table:openKeyTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
table: s3Table
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
profile:DBProfile.DISK for Table:s3Table
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
table: multipartInfoTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
profile:DBProfile.DISK for Table:multipartInfoTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
table: s3SecretTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
profile:DBProfile.DISK for Table:s3SecretTable
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
table: default
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:158 - Using default column
profile:DBProfile.DISK for Table:default
om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:189 - Using default options.
DBProfile.DISK
om_1 | 2019-03-18 06:31:42 INFO CallQueueManager:84 - Using callQueue: class
java.util.concurrent.LinkedBlockingQueue, queueCapacity: 2000, scheduler: class
org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
om_1 | 2019-03-18 06:31:42 INFO Server:1074 - Starting Socket Reader #1 for
port 9862
om_1 | 2019-03-18 06:31:43 WARN ScmUtils:77 - ozone.om.db.dirs is not
configured. We recommend adding this setting. Falling back to
ozone.metadata.dirs instead.
om_1 | 2019-03-18 06:31:43 INFO OzoneManager:1129 - OzoneManager RPC server is
listening at om/172.21.0.3:9862
om_1 | 2019-03-18 06:31:43 INFO MetricsConfig:118 - Loaded properties from
hadoop-metrics2.properties
om_1 | 2019-03-18 06:31:43 INFO MetricsSystemImpl:374 - Scheduled Metric
snapshot period at 10 second(s).
om_1 | 2019-03-18 06:31:43 INFO MetricsSystemImpl:191 - OzoneManager metrics
system started
om_1 | 2019-03-18 06:31:43 INFO Server:1314 - IPC Server Responder: starting
om_1 | 2019-03-18 06:31:43
[jira] [Created] (HDDS-1325) Exception thrown while initializing ozoneClientAdapter
Nilotpal Nandi created HDDS-1325:
Summary: Exception thrown while initializing ozoneClientAdapter
Key: HDDS-1325
URL: https://issues.apache.org/jira/browse/HDDS-1325
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
ozone version :
{noformat}
Source code repository https://gitbox.apache.org/repos/asf/hadoop.git -r
568d3ab8b65d1348dec9c971feffe200e6cba2ef
Compiled by nnandi on 2019-03-19T03:54Z
Compiled with protoc 2.5.0
>From source with checksum c44d339e20094d3054754078afbf4c
Using HDDS 0.5.0-SNAPSHOT
Source code repository https://gitbox.apache.org/repos/asf/hadoop.git -r
568d3ab8b65d1348dec9c971feffe200e6cba2ef
Compiled by nnandi on 2019-03-19T03:53Z
Compiled with protoc 2.5.0
>From source with checksum b354934fb1352f4d5425114bf8dce11
{noformat}
steps taken :
---
# Add ozone libs in hadoop classpath.
# Tried to run s3dupdo workload ([https://github.com/t3rmin4t0r/s3dupdo])
Here is the exception thrown :
{noformat}
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.fs.ozone.OzoneClientAdapterFactory.lambda$createAdapter$1(OzoneClientAdapterFactory.java:65)
at
org.apache.hadoop.fs.ozone.OzoneClientAdapterFactory.createAdapter(OzoneClientAdapterFactory.java:105)
at
org.apache.hadoop.fs.ozone.OzoneClientAdapterFactory.createAdapter(OzoneClientAdapterFactory.java:61)
at
org.apache.hadoop.fs.ozone.OzoneFileSystem.initialize(OzoneFileSystem.java:167)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3303)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352)
at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:3326)
at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:532)
at org.notmysock.repl.Works$CopyWorker.run(Works.java:243)
at org.notmysock.repl.Works$CopyWorker.call(Works.java:279)
at org.notmysock.repl.Works$CopyWorker.call(Works.java:204)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.LinkageError: loader constraint violation: loader
(instance of org/apache/hadoop/fs/ozone/FilteredClassLoader) previously
initiated loading for a different type with name
"org/apache/hadoop/security/token/Token"
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at
org.apache.hadoop.fs.ozone.FilteredClassLoader.loadClass(FilteredClassLoader.java:71)
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetPublicMethods(Class.java:2902)
at java.lang.Class.getMethods(Class.java:1615)
at sun.misc.ProxyGenerator.generateClassFile(ProxyGenerator.java:451)
at sun.misc.ProxyGenerator.generateProxyClass(ProxyGenerator.java:339)
at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:639)
at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:557)
at java.lang.reflect.WeakCache$Factory.get(WeakCache.java:230)
at java.lang.reflect.WeakCache.get(WeakCache.java:127)
at java.lang.reflect.Proxy.getProxyClass0(Proxy.java:419)
at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:719)
at
org.apache.hadoop.ozone.client.OzoneClientFactory.getClient(OzoneClientFactory.java:264)
at
org.apache.hadoop.ozone.client.OzoneClientFactory.getRpcClient(OzoneClientFactory.java:169)
at
org.apache.hadoop.fs.ozone.OzoneClientAdapterImpl.(OzoneClientAdapterImpl.java:140)
at
org.apache.hadoop.fs.ozone.OzoneClientAdapterImpl.(OzoneClientAdapterImpl.java:104)
at
org.apache.hadoop.fs.ozone.OzoneClientAdapterImpl.(OzoneClientAdapterImpl.java:75)
... 20 more{noformat}
[jira] [Created] (HDDS-1326) putkey operation failed with java.lang.ArrayIndexOutOfBoundsException
Nilotpal Nandi created HDDS-1326:
Summary: putkey operation failed with
java.lang.ArrayIndexOutOfBoundsException
Key: HDDS-1326
URL: https://issues.apache.org/jira/browse/HDDS-1326
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken :
---
# trying to write key in 40 node cluster.
# write failed.
client output
---
{noformat}
e530-491c-ab03-3b1c34d1a751:c80390,
974a806d-bf7d-4f1b-adb4-d51d802d368a:c80390,
469bd8c4-5da2-43bb-bc4b-7edd884931e5:c80390]
2019-03-22 10:56:19,592 [main] WARN - Encountered exception {}
java.io.IOException: Unexpected Storage Container Exception:
java.util.concurrent.ExecutionException:
java.util.concurrent.CompletionException:
org.apache.ratis.protocol.StateMachineException:
org.apache.hadoop.hdds.scm.container.common.helpers.ContainerNotOpenException
from Server 5d3eb91f-e530-491c-ab03-3b1c34d1a751: Container 1269 in CLOSED state
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.close(BlockOutputStream.java:511)
at
org.apache.hadoop.ozone.client.io.BlockOutputStreamEntry.close(BlockOutputStreamEntry.java:144)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.handleFlushOrClose(KeyOutputStream.java:565)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.handleWrite(KeyOutputStream.java:329)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.write(KeyOutputStream.java:273)
at
org.apache.hadoop.ozone.client.io.OzoneOutputStream.write(OzoneOutputStream.java:49)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:96)
at
org.apache.hadoop.ozone.web.ozShell.keys.PutKeyHandler.call(PutKeyHandler.java:111)
at
org.apache.hadoop.ozone.web.ozShell.keys.PutKeyHandler.call(PutKeyHandler.java:53)
at picocli.CommandLine.execute(CommandLine.java:919)
at picocli.CommandLine.access$700(CommandLine.java:104)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1083)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1051)
at
picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:959)
at picocli.CommandLine.parseWithHandlers(CommandLine.java:1242)
at picocli.CommandLine.parseWithHandler(CommandLine.java:1181)
at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:61)
at org.apache.hadoop.ozone.web.ozShell.Shell.execute(Shell.java:82)
at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:52)
at org.apache.hadoop.ozone.web.ozShell.Shell.main(Shell.java:93)
Caused by: java.util.concurrent.ExecutionException:
java.util.concurrent.CompletionException:
org.apache.ratis.protocol.StateMachineException:
org.apache.hadoop.hdds.scm.container.common.helpers.ContainerNotOpenException
from Server 5d3eb91f-e530-491c-ab03-3b1c34d1a751: Container 1269 in CLOSED state
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.waitOnFlushFutures(BlockOutputStream.java:529)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.handleFlush(BlockOutputStream.java:481)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.close(BlockOutputStream.java:496)
... 19 more
Caused by: java.util.concurrent.CompletionException:
org.apache.ratis.protocol.StateMachineException:
org.apache.hadoop.hdds.scm.container.common.helpers.ContainerNotOpenException
from Server 5d3eb91f-e530-491c-ab03-3b1c34d1a751: Container 1269 in CLOSED state
at
org.apache.ratis.client.impl.RaftClientImpl.handleStateMachineException(RaftClientImpl.java:402)
at
org.apache.ratis.client.impl.RaftClientImpl.lambda$sendAsync$3(RaftClientImpl.java:198)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.apache.ratis.client.impl.RaftClientImpl$PendingAsyncRequest.setReply(RaftClientImpl.java:95)
at
org.apache.ratis.client.impl.RaftClientImpl$PendingAsyncRequest.setReply(RaftClientImpl.java:75)
at
org.apache.ratis.util.SlidingWindow$RequestMap.setReply(SlidingWindow.java:127)
at
org.apache.ratis.util.SlidingWindow$Client.receiveReply(SlidingWindow.java:279)
at
org.apache.ratis.client.impl.RaftClientImpl.lambda$sendRequestAsync$13(RaftClientImpl.java:344)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.apache.rati
[jira] [Created] (HDDS-1338) ozone shell commands are throwing InvocationTargetException
Nilotpal Nandi created HDDS-1338:
Summary: ozone shell commands are throwing
InvocationTargetException
Key: HDDS-1338
URL: https://issues.apache.org/jira/browse/HDDS-1338
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
ozone version
{noformat}
Source code repository g...@github.com:hortonworks/ozone.git -r
310ebf5dc83b6c9e68d09246ed6c6f7cf6370fde
Compiled by jenkins on 2019-03-21T22:06Z
Compiled with protoc 2.5.0
>From source with checksum 9c367143ad43b81ca84bfdaafd1c3f
Using HDDS 0.4.0.3.0.100.0-388
Source code repository g...@github.com:hortonworks/ozone.git -r
310ebf5dc83b6c9e68d09246ed6c6f7cf6370fde
Compiled by jenkins on 2019-03-21T22:06Z
Compiled with protoc 2.5.0
>From source with checksum f3297cbd3a5f59fb4e5fd551afa05ba9
{noformat}
Here is the ozone volume create failure output :
{noformat}
hdfs@ctr-e139-1542663976389-91321-01-02 ~]$ ozone sh volume create
testvolume11
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/hdp/3.0.100.0-388/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/hdp/3.0.100.0-388/hadoop-ozone/share/ozone/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
19/03/26 17:31:37 ERROR client.OzoneClientFactory: Couldn't create protocol
class org.apache.hadoop.ozone.client.rpc.RpcClient exception:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.ozone.client.OzoneClientFactory.getClientProtocol(OzoneClientFactory.java:291)
at
org.apache.hadoop.ozone.client.OzoneClientFactory.getRpcClient(OzoneClientFactory.java:169)
at
org.apache.hadoop.ozone.web.ozShell.OzoneAddress.createClient(OzoneAddress.java:111)
at
org.apache.hadoop.ozone.web.ozShell.volume.CreateVolumeHandler.call(CreateVolumeHandler.java:70)
at
org.apache.hadoop.ozone.web.ozShell.volume.CreateVolumeHandler.call(CreateVolumeHandler.java:38)
at picocli.CommandLine.execute(CommandLine.java:919)
at picocli.CommandLine.access$700(CommandLine.java:104)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1083)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1051)
at
picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:959)
at picocli.CommandLine.parseWithHandlers(CommandLine.java:1242)
at picocli.CommandLine.parseWithHandler(CommandLine.java:1181)
at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:61)
at org.apache.hadoop.ozone.web.ozShell.Shell.execute(Shell.java:82)
at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:52)
at org.apache.hadoop.ozone.web.ozShell.Shell.main(Shell.java:93)
Caused by: java.lang.VerifyError: Cannot inherit from final class
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.(OzoneManagerProtocolClientSideTranslatorPB.java:169)
at org.apache.hadoop.ozone.client.rpc.RpcClient.(RpcClient.java:142)
... 20 more
Couldn't create protocol class org.apache.hadoop.ozone.client.rpc.RpcClient
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-1298) blockade tests failing as the nodes are not able to communicate with Ozone Manager
[
https://issues.apache.org/jira/browse/HDDS-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nilotpal Nandi resolved HDDS-1298.
--
Resolution: Duplicate
> blockade tests failing as the nodes are not able to communicate with Ozone
> Manager
> --
>
> Key: HDDS-1298
> URL: https://issues.apache.org/jira/browse/HDDS-1298
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Nilotpal Nandi
>Assignee: Nilotpal Nandi
>Priority: Critical
> Attachments: alllogs.log
>
>
> steps taken:
>
> # started 3 datanodes docker cluster.
> # freon run fails with error : "No such service: ozoneManager"
>
> {noformat}
> om_1 | STARTUP_MSG: build = https://github.com/apache/hadoop.git -r
> e97acb3bd8f3befd27418996fa5d4b50bf2e17bf; compiled by 'sunilg' on
> 2019-01-15T17:34Z
> om_1 | STARTUP_MSG: java = 11.0.1
> om_1 | /
> om_1 | 2019-03-18 06:31:41 INFO OzoneManager:51 - registered UNIX signal
> handlers for [TERM, HUP, INT]
> om_1 | 2019-03-18 06:31:41 WARN ScmUtils:77 - ozone.om.db.dirs is not
> configured. We recommend adding this setting. Falling back to
> ozone.metadata.dirs instead.
> om_1 | 2019-03-18 06:31:41 INFO OzoneManager:484 - OM Service ID is not set.
> Setting it to the default ID: omServiceIdDefault
> om_1 | 2019-03-18 06:31:41 INFO OzoneManager:490 - OM Node ID is not set.
> Setting it to the OmStorage's OmID: 25501758-f7f6-42d5-8196-52a885af7e23
> om_1 | 2019-03-18 06:31:41 INFO OzoneManager:441 - Found matching OM address
> with OMServiceId: null, OMNodeId: null, RPC Address: om:9862 and Ratis port:
> 9872
> om_1 | 2019-03-18 06:31:42 WARN ScmUtils:77 - ozone.om.db.dirs is not
> configured. We recommend adding this setting. Falling back to
> ozone.metadata.dirs instead.
> om_1 | 2019-03-18 06:31:42 INFO log:192 - Logging initialized @4061ms
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
> table: userTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
> profile:DBProfile.DISK for Table:userTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
> table: volumeTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
> profile:DBProfile.DISK for Table:volumeTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
> table: bucketTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
> profile:DBProfile.DISK for Table:bucketTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
> table: keyTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
> profile:DBProfile.DISK for Table:keyTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
> table: deletedTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
> profile:DBProfile.DISK for Table:deletedTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
> table: openKeyTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
> profile:DBProfile.DISK for Table:openKeyTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
> table: s3Table
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
> profile:DBProfile.DISK for Table:s3Table
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
> table: multipartInfoTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
> profile:DBProfile.DISK for Table:multipartInfoTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
> table: s3SecretTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:152 - Using default column
> profile:DBProfile.DISK for Table:s3SecretTable
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:101 - using custom profile for
> table: default
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:158 - Using default column
> profile:DBProfile.DISK for Table:default
> om_1 | 2019-03-18 06:31:42 INFO DBStoreBuilder:189 - Using default options.
> DBProfile.DISK
> om_1 | 2019-03-18 06:31:42 INFO CallQueueManager:84 - Using callQueue: class
> java.util.concurrent.LinkedBlockingQueue, queueCapacity: 2000, scheduler:
> class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
> om_1 | 2019-03-18 06:31:42 INFO Server:1074 - Starting Socket Reader #1 for
> port 9862
> om_1 | 2019-03-18 06:31:43 WARN ScmUtils:77 - ozone.om.db.dirs is not
> configured. We recommend adding this setting. Falling back to
> ozone.metadata.dirs instead.
> om_1 | 2019-03-18 06:31:43 INF
[jira] [Created] (HDDS-1497) Refactor blockade Tests
Nilotpal Nandi created HDDS-1497: Summary: Refactor blockade Tests Key: HDDS-1497 URL: https://issues.apache.org/jira/browse/HDDS-1497 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Nilotpal Nandi Assignee: Nilotpal Nandi -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1534) freon should return non-zero exit code on failure
Nilotpal Nandi created HDDS-1534: Summary: freon should return non-zero exit code on failure Key: HDDS-1534 URL: https://issues.apache.org/jira/browse/HDDS-1534 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Nilotpal Nandi Assignee: Nilotpal Nandi Currently freon does not return any non-zero exit code even on failure. The status shows as "Failed" but the exit code is always zero. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-2043) "VOLUME_NOT_FOUND" exception thrown while listing volumes
Nilotpal Nandi created HDDS-2043: Summary: "VOLUME_NOT_FOUND" exception thrown while listing volumes Key: HDDS-2043 URL: https://issues.apache.org/jira/browse/HDDS-2043 Project: Hadoop Distributed Data Store Issue Type: Bug Components: Ozone CLI, Ozone Manager Reporter: Nilotpal Nandi ozone list volume command throws OMException /opt/cloudera/parcels/CDH/bin/ozone sh volume list --user root VOLUME_NOT_FOUND org.apache.hadoop.ozone.om.exceptions.OMException: Volume info not found for vol-test-putfile-1566902803 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-2350) NullPointerException seen in datanode log while writing data
Nilotpal Nandi created HDDS-2350:
Summary: NullPointerException seen in datanode log while writing
data
Key: HDDS-2350
URL: https://issues.apache.org/jira/browse/HDDS-2350
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
NullPointerException exception seen in datanode log while writing 10GB data.
There is one pipelinee with factor 3 while writing data.
{noformat}
2019-10-23 11:25:45,674 ERROR
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: Error getting metrics
from source
ratis_core.ratis_leader.a23fb300-4c1e-420f-a21e-7e73d0c22cbe@group-4CA404C938C2
java.lang.NullPointerException
at
org.apache.ratis.server.impl.RaftLeaderMetrics.lambda$null$2(RaftLeaderMetrics.java:86)
at
com.github.joshelser.dropwizard.metrics.hadoop.HadoopMetrics2Reporter.snapshotAllMetrics(HadoopMetrics2Reporter.java:239)
at
com.github.joshelser.dropwizard.metrics.hadoop.HadoopMetrics2Reporter.getMetrics(HadoopMetrics2Reporter.java:219)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:200)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.snapshotMetrics(MetricsSystemImpl.java:419)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.sampleMetrics(MetricsSystemImpl.java:406)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.onTimerEvent(MetricsSystemImpl.java:381)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$4.run(MetricsSystemImpl.java:368)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
2019-10-23 11:25:55,673 ERROR
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: Error getting metrics
from source
ratis_core.ratis_leader.a23fb300-4c1e-420f-a21e-7e73d0c22cbe@group-4CA404C938C2
java.lang.NullPointerException
at
org.apache.ratis.server.impl.RaftLeaderMetrics.lambda$null$2(RaftLeaderMetrics.java:86)
at
com.github.joshelser.dropwizard.metrics.hadoop.HadoopMetrics2Reporter.snapshotAllMetrics(HadoopMetrics2Reporter.java:239)
at
com.github.joshelser.dropwizard.metrics.hadoop.HadoopMetrics2Reporter.getMetrics(HadoopMetrics2Reporter.java:219)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:200)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.snapshotMetrics(MetricsSystemImpl.java:419)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.sampleMetrics(MetricsSystemImpl.java:406)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.onTimerEvent(MetricsSystemImpl.java:381)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$4.run(MetricsSystemImpl.java:368)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
2019-10-23 11:26:05,674 ERROR
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: Error getting metrics
from source
ratis_core.ratis_leader.a23fb300-4c1e-420f-a21e-7e73d0c22cbe@group-4CA404C938C2
java.lang.NullPointerException
at
org.apache.ratis.server.impl.RaftLeaderMetrics.lambda$null$2(RaftLeaderMetrics.java:86)
at
com.github.joshelser.dropwizard.metrics.hadoop.HadoopMetrics2Reporter.snapshotAllMetrics(HadoopMetrics2Reporter.java:239)
at
com.github.joshelser.dropwizard.metrics.hadoop.HadoopMetrics2Reporter.getMetrics(HadoopMetrics2Reporter.java:219)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:200)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.snapshotMetrics(MetricsSystemImpl.java:419)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.sampleMetrics(MetricsSystemImpl.java:406)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.onTimerEvent(MetricsSystemImpl.java:381)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$4.run(MetricsSystemImpl.java:368)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505){noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-14980) diskbalancer query command always tries to contact to port 9867
Nilotpal Nandi created HDFS-14980:
-
Summary: diskbalancer query command always tries to contact to
port 9867
Key: HDFS-14980
URL: https://issues.apache.org/jira/browse/HDFS-14980
Project: Hadoop HDFS
Issue Type: Bug
Components: diskbalancer
Reporter: Nilotpal Nandi
disbalancer query commands always tries to connect to port 9867 even when
datanode IPC port is different.
In this setup , datanode IPC port is set to 20001.
diskbalancer report command works fine and connects to IPC port 20001
{noformat}
hdfs diskbalancer -report -node 172.27.131.193
19/11/12 08:58:55 INFO command.Command: Processing report command
19/11/12 08:58:57 INFO balancer.KeyManager: Block token params received from
NN: update interval=10hrs, 0sec, token lifetime=10hrs, 0sec
19/11/12 08:58:57 INFO block.BlockTokenSecretManager: Setting block keys
19/11/12 08:58:57 INFO balancer.KeyManager: Update block keys every 2hrs,
30mins, 0sec
19/11/12 08:58:58 INFO command.Command: Reporting volume information for
DataNode(s). These DataNode(s) are parsed from '172.27.131.193'.
Processing report command
Reporting volume information for DataNode(s). These DataNode(s) are parsed from
'172.27.131.193'.
[172.27.131.193:20001] - : 3
volumes with node data density 0.05.
[DISK: volume-/dataroot/ycloud/dfs/NEW_DISK1/] - 0.15 used:
39343871181/259692498944, 0.85 free: 220348627763/259692498944, isFailed:
False, isReadOnly: False, isSkip: False, isTransient: False.
[DISK: volume-/dataroot/ycloud/dfs/NEW_DISK2/] - 0.15 used:
39371179986/259692498944, 0.85 free: 220321318958/259692498944, isFailed:
False, isReadOnly: False, isSkip: False, isTransient: False.
[DISK: volume-/dataroot/ycloud/dfs/dn/] - 0.19 used: 49934903670/259692498944,
0.81 free: 209757595274/259692498944, isFailed: False, isReadOnly: False,
isSkip: False, isTransient: False.
{noformat}
But diskbalancer query command fails and tries to connect to port 9867
(default port).
{noformat}
hdfs diskbalancer -query 172.27.131.193
19/11/12 06:37:15 INFO command.Command: Executing "query plan" command.
19/11/12 06:37:16 INFO ipc.Client: Retrying connect to server:
/172.27.131.193:9867. Already tried 0 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
19/11/12 06:37:17 INFO ipc.Client: Retrying connect to server:
/172.27.131.193:9867. Already tried 1 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
..
..
..
19/11/12 06:37:25 ERROR tools.DiskBalancerCLI: Exception thrown while running
DiskBalancerCLI.
{noformat}
Expectation :
diskbalancer query command should work fine without explicitly mentioning
datanode IPC port address
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-2604) scmcli pipeline deactivate command not working
Nilotpal Nandi created HDDS-2604:
Summary: scmcli pipeline deactivate command not working
Key: HDDS-2604
URL: https://issues.apache.org/jira/browse/HDDS-2604
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: SCM Client
Reporter: Nilotpal Nandi
Assignee: Nilotpal Nandi
scmcli pipeline deactivate not working
output :
{noformat}
ozone scmcli pipeline deactivate 212e1f47-4890-49c2-a950-4d0b3a70cbfd
Unknown command type: DeactivatePipeline
root@st-ozone-kg2qce-l2ltm:/ansible# echo $?
255{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-15619) Metric for ordered snapshot deletion GC thread
Nilotpal Nandi created HDFS-15619: - Summary: Metric for ordered snapshot deletion GC thread Key: HDFS-15619 URL: https://issues.apache.org/jira/browse/HDFS-15619 Project: Hadoop HDFS Issue Type: Task Components: hdfs Reporter: Nilotpal Nandi Assignee: Nilotpal Nandi Following info should be captured and shown in JMX for garbage collection thread of ordered snapshot deletion * metric for all pending snapshots to be GCed * Number of times GC thread ran * Number of Snapshots already GCed * Average time taken by each GC run * Thread running Status * metric for failed deletion of GC thread -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-206) default port number taken by ksm is 9862 while listing the volumes
Nilotpal Nandi created HDDS-206:
---
Summary: default port number taken by ksm is 9862 while listing
the volumes
Key: HDDS-206
URL: https://issues.apache.org/jira/browse/HDDS-206
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Manager
Reporter: Nilotpal Nandi
Fix For: 0.2.1
Here is the output of ozone -listVolume command without any port mentioned .
By default, it chooses the port number as 9862 which is not mentioned in the
ozone-site.xml
{noformat}
[root@ozone-vm bin]# ./ozone oz -listVolume o3://127.0.0.1/
2018-06-29 04:42:20,652 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2018-06-29 04:42:21,914 INFO ipc.Client: Retrying connect to server:
localhost/127.0.0.1:9862. Already tried 0 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-06-29 04:42:22,915 INFO ipc.Client: Retrying connect to server:
localhost/127.0.0.1:9862. Already tried 1 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-06-29 04:42:23,917 INFO ipc.Client: Retrying connect to server:
localhost/127.0.0.1:9862. Already tried 2 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-06-29 04:42:24,925 INFO ipc.Client: Retrying connect to server:
localhost/127.0.0.1:9862. Already tried 3 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-06-29 04:42:25,928 INFO ipc.Client: Retrying connect to server:
localhost/127.0.0.1:9862. Already tried 4 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-06-29 04:42:26,931 INFO ipc.Client: Retrying connect to server:
localhost/127.0.0.1:9862. Already tried 5 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-06-29 04:42:27,932 INFO ipc.Client: Retrying connect to server:
localhost/127.0.0.1:9862. Already tried 6 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-06-29 04:42:28,934 INFO ipc.Client: Retrying connect to server:
localhost/127.0.0.1:9862. Already tried 7 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-06-29 04:42:29,935 INFO ipc.Client: Retrying connect to server:
localhost/127.0.0.1:9862. Already tried 8 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-06-29 04:42:30,938 INFO ipc.Client: Retrying connect to server:
localhost/127.0.0.1:9862. Already tried 9 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-06-29 04:42:31,075 [main] ERROR - Couldn't create protocol class
org.apache.hadoop.ozone.client.rpc.RpcClient exception:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.ozone.client.OzoneClientFactory.getClientProtocol(OzoneClientFactory.java:292)
at
org.apache.hadoop.ozone.client.OzoneClientFactory.getRpcClient(OzoneClientFactory.java:172)
at
org.apache.hadoop.ozone.client.OzoneClientFactory.getRpcClient(OzoneClientFactory.java:156)
at
org.apache.hadoop.ozone.client.OzoneClientFactory.getRpcClient(OzoneClientFactory.java:111)
at org.apache.hadoop.ozone.web.ozShell.Handler.verifyURI(Handler.java:96)
at
org.apache.hadoop.ozone.web.ozShell.volume.ListVolumeHandler.execute(ListVolumeHandler.java:80)
at org.apache.hadoop.ozone.web.ozShell.Shell.dispatch(Shell.java:395)
at org.apache.hadoop.ozone.web.ozShell.Shell.run(Shell.java:135)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.ozone.web.ozShell.Shell.main(Shell.java:114)
Caused by: java.net.ConnectException: Call From ozone-vm/10.200.5.166 to
localhost:9862 failed on connection exception: java.net.ConnectException:
Connection refused; For more details see:
http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Co
[jira] [Created] (HDDS-207) ozone listVolume command accepts random values as argument
Nilotpal Nandi created HDDS-207:
---
Summary: ozone listVolume command accepts random values as argument
Key: HDDS-207
URL: https://issues.apache.org/jira/browse/HDDS-207
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Reporter: Nilotpal Nandi
Fix For: 0.2.1
When argument from listVolume is provided, it complains.
But a random argument is provided for listVolume command, it accepts and
displays all the volumes.
{noformat}
[root@ozone-vm bin]# ./ozone oz -listVolume
Missing argument for option: listVolumeERROR: null
[root@ozone-vm bin]# ./ozone oz -listVolume abcdefghijk
2018-06-29 07:09:43,451 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
[ {
"owner" : {
"name" : "root"
},
"quota" : {
"unit" : "TB",
"size" : 1048576
},
"volumeName" : "nnvolume1",
"createdOn" : "Sun, 18 Sep +50444 15:12:11 GMT",
"createdBy" : "root"
}, {
"owner" : {
"name" : "root"
},
"quota" : {
"unit" : "TB",
"size" : 1048576
},
"volumeName" : "nnvolume2",
"createdOn" : "Tue, 27 Sep +50444 13:05:43 GMT",
"createdBy" : "root"
} ]
{noformat}
expectation:
It should not accept random values as argument
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-208) ozone createVolume command ignores the first character of the "volume name" given as argument
Nilotpal Nandi created HDDS-208:
---
Summary: ozone createVolume command ignores the first character of
the "volume name" given as argument
Key: HDDS-208
URL: https://issues.apache.org/jira/browse/HDDS-208
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
Fix For: 0.2.1
createVolume command ran to create volume "testvolume123".
Volume created with name "estvolume123" instead of "testvolume123". It ignores
the first character of the volume name
{noformat}
[root@ozone-vm bin]# ./ozone oz -createVolume testvolume123 -user root
2018-07-02 05:33:35,510 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2018-07-02 05:33:36,093 [main] INFO - Creating Volume: estvolume123, with root
as owner and quota set to 1152921504606846976 bytes.
{noformat}
ozone listVolume command :
{noformat}
[root@ozone-vm bin]# ./ozone oz -listVolume /
2018-07-02 05:36:47,835 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
[ {
"owner" : {
"name" : "root"
},
"quota" : {
"unit" : "TB",
"size" : 1048576
},
"volumeName" : "nnvolume1",
"createdOn" : "Sun, 18 Sep +50444 15:12:11 GMT",
"createdBy" : "root"
..
..
}, {
"owner" : {
"name" : "root"
},
"quota" : {
"unit" : "TB",
"size" : 1048576
},
"volumeName" : "estvolume123",
"createdOn" : "Sat, 17 May +50470 08:01:41 GMT",
"createdBy" : "root"
} ]
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-209) createVolume command throws error when user is not present locally but creates the volume
Nilotpal Nandi created HDDS-209:
---
Summary: createVolume command throws error when user is not
present locally but creates the volume
Key: HDDS-209
URL: https://issues.apache.org/jira/browse/HDDS-209
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Reporter: Nilotpal Nandi
Fix For: 0.2.1
user "test_user3" does not exist locally.
When -createVolume command is ran for the user "test_user3", it throws error on
standard output but successfully creates the volume.
The exit code for the command execution is non-zero.
{noformat}
[root@ozone-vm bin]# ./ozone oz -createVolume /testvolume121 -user test_user3
2018-07-02 06:01:37,020 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2018-07-02 06:01:37,605 WARN security.ShellBasedUnixGroupsMapping: unable to
return groups for user test_user3
PartialGroupNameException The user name 'test_user3' is not found. id:
test_user3: no such user
id: test_user3: no such user
at
org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:294)
at
org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:207)
at
org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:97)
at
org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:51)
at
org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupList(Groups.java:384)
at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:319)
at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:269)
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
at
com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:227)
at
org.apache.hadoop.security.UserGroupInformation.getGroups(UserGroupInformation.java:1547)
at
org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1535)
at
org.apache.hadoop.ozone.client.rpc.RpcClient.createVolume(RpcClient.java:190)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.ozone.client.OzoneClientInvocationHandler.invoke(OzoneClientInvocationHandler.java:54)
at com.sun.proxy.$Proxy11.createVolume(Unknown Source)
at org.apache.hadoop.ozone.client.ObjectStore.createVolume(ObjectStore.java:77)
at
org.apache.hadoop.ozone.web.ozShell.volume.CreateVolumeHandler.execute(CreateVolumeHandler.java:98)
at org.apache.hadoop.ozone.web.ozShell.Shell.dispatch(Shell.java:395)
at org.apache.hadoop.ozone.web.ozShell.Shell.run(Shell.java:135)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.ozone.web.ozShell.Shell.main(Shell.java:114)
2018-07-02 06:01:37,611 [main] INFO - Creating Volume: testvolume121, with
test_user3 as owner and quota set to 1152921504606846976 bytes.
{noformat}
{noformat}
[root@ozone-vm bin]# ./ozone oz -listVolume / -user test_user3
2018-07-02 06:02:20,385 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
[ {
"owner" : {
"name" : "test_user3"
},
"quota" : {
"unit" : "TB",
"size" : 1048576
},
"volumeName" : "testvolume121",
"createdOn" : "Thu, 05 Jun +50470 19:07:00 GMT",
"createdBy" : "test_user3"
} ]
{noformat}
Expectation :
--
Volume should not present if local user is not present.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-210) ozone getKey command always expects the filename to be present along with file-path in "-file" argument
Nilotpal Nandi created HDDS-210:
---
Summary: ozone getKey command always expects the filename to be
present along with file-path in "-file" argument
Key: HDDS-210
URL: https://issues.apache.org/jira/browse/HDDS-210
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Environment: ozone getKey command always expects the filename to be
present along with the file-path for the "-file" argument.
It throws error if filename is not provided.
{noformat}
[root@ozone-vm bin]# ./ozone oz -getKey /nnvolume1/bucket123/passwd -file
/test1/
2018-07-02 06:45:27,355 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
Command Failed : {"httpCode":0,"shortMessage":"/test1/exists. Download will
overwrite an existing file.
Aborting.","resource":null,"message":"/test1/exists. Download will overwrite an
existing file. Aborting.","requestID":null,"hostName":null}
[root@ozone-vm bin]# ./ozone oz -getKey /nnvolume1/bucket123/passwd -file
/test1/passwd
2018-07-02 06:45:39,722 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2018-07-02 06:45:40,354 INFO conf.ConfUtils: raft.rpc.type = GRPC (default)
2018-07-02 06:45:40,366 INFO conf.ConfUtils: raft.grpc.message.size.max =
33554432 (custom)
2018-07-02 06:45:40,372 INFO conf.ConfUtils: raft.client.rpc.retryInterval =
300 ms (default)
2018-07-02 06:45:40,374 INFO conf.ConfUtils:
raft.client.async.outstanding-requests.max = 100 (default)
2018-07-02 06:45:40,374 INFO conf.ConfUtils:
raft.client.async.scheduler-threads = 3 (default)
2018-07-02 06:45:40,507 INFO conf.ConfUtils: raft.grpc.flow.control.window =
1MB (=1048576) (default)
2018-07-02 06:45:40,507 INFO conf.ConfUtils: raft.grpc.message.size.max =
33554432 (custom)
2018-07-02 06:45:40,814 INFO conf.ConfUtils: raft.client.rpc.request.timeout =
3000 ms (default){noformat}
Expectation :
--
ozone getKey should work even when only file-path is provided (without
filename). It should create a file in the given file-path with its key's name
as its name.
i.e,
given , /test1 is a directory .
if ./ozone oz -getKey /nnvolume1/bucket123/passwd -file /test1 is run,
file 'passwd' should be created in the directory /test1 .
Reporter: Nilotpal Nandi
Fix For: 0.2.1
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-264) 'oz' subcommand reference is not present in 'ozone' command help
Nilotpal Nandi created HDDS-264:
---
Summary: 'oz' subcommand reference is not present in 'ozone'
command help
Key: HDDS-264
URL: https://issues.apache.org/jira/browse/HDDS-264
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Reporter: Nilotpal Nandi
Fix For: 0.2.1
'oz' subcommand is not present in ozone help.
ozone help:
{noformat}
hadoop@8ceb8dfccb36:~/bin$ ./ozone
Usage: ozone [OPTIONS] SUBCOMMAND [SUBCOMMAND OPTIONS]
OPTIONS is none or any of:
--buildpaths attempt to add class files from build tree
--config dir Hadoop config directory
--daemon (start|status|stop) operate on a daemon
--debug turn on shell script debug mode
--help usage information
--hostnames list[,of,host,names] hosts to use in worker mode
--hosts filename list of hosts to use in worker mode
--loglevel level set the log4j level for this command
--workers turn on worker mode
SUBCOMMAND is one of:
Admin Commands:
jmxget get JMX exported values from NameNode or DataNode.
Client Commands:
classpath prints the class path needed to get the hadoop jar and the
required libraries
envvars display computed Hadoop environment variables
freon runs an ozone data generator
genconf generate minimally required ozone configs and output to
ozone-site.xml in specified path
genesis runs a collection of ozone benchmarks to help with tuning.
getozoneconf get ozone config values from configuration
noz ozone debug tool, convert ozone metadata into relational data
o3 command line interface for ozone
scmcli run the CLI of the Storage Container Manager
version print the version
Daemon Commands:
datanode run a HDDS datanode
om Ozone Manager
scm run the Storage Container Manager service
SUBCOMMAND may print help when invoked w/o parameters or with -h.
{noformat}
'oz' subcommand example :
{noformat}
hadoop@8ceb8dfccb36:~/bin$ ./ozone oz -listVolume /
2018-07-19 14:51:25 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
[ {
"owner" : {
"name" : "hadoop"
},
"quota" : {
"unit" : "TB",
"size" : 1048576
},
"volumeName" : "vol-0-01597",
"createdOn" : "Sat, 20 Feb +50517 10:11:35 GMT",
"createdBy" : "hadoop"
}, {
"owner" : {
"name" : "hadoop"
},
"quota" : {
"unit" : "TB",
"size" : 1048576
},
"volumeName" : "vol-0-19478",
"createdOn" : "Thu, 03 Jun +50517 22:23:12 GMT",
"createdBy" : "hadoop"
}, {
"owner" : {
"name" : "hadoop"
},
"quota" : {
"unit" : "TB",
"size" : 1048576
}
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-275) DeleteVolume command does not give any confirmation message on successful execution
Nilotpal Nandi created HDDS-275:
---
Summary: DeleteVolume command does not give any confirmation
message on successful execution
Key: HDDS-275
URL: https://issues.apache.org/jira/browse/HDDS-275
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Reporter: Nilotpal Nandi
Fix For: 0.2.1
On deleting a Volume and successful execution , there is no confirmation
message saying that the "volume is deleted".
For example :
{noformat}
hadoop@d4eb7d83cdee:~/bin$ for i in $(seq 201 1000)
> do
> ./ozone oz -deleteVolume /vol1${i}
> done
2018-07-20 09:11:50 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
2018-07-20 09:11:52 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
..
..
..
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-281) Need container size distribution metric in OzoneManager UI
Nilotpal Nandi created HDDS-281: --- Summary: Need container size distribution metric in OzoneManager UI Key: HDDS-281 URL: https://issues.apache.org/jira/browse/HDDS-281 Project: Hadoop Distributed Data Store Issue Type: Improvement Components: Ozone Manager Reporter: Nilotpal Nandi It would be good if we have some metric/histogram in OzoneManager UI indicating the different container size range and corresponding percentages for the same created in the cluster. For example : 0-2 GB 10% 2-4 GB . 20% 4-5 GB 70% 5+ GB 0% -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-283) Need an option to list all volumes created in the cluster
Nilotpal Nandi created HDDS-283: --- Summary: Need an option to list all volumes created in the cluster Key: HDDS-283 URL: https://issues.apache.org/jira/browse/HDDS-283 Project: Hadoop Distributed Data Store Issue Type: Bug Components: Ozone Client Reporter: Nilotpal Nandi Fix For: 0.2.1 Currently , listVolume command either gives : 1) all the volumes created by a particular user , using -user argument. 2) or , all the volumes created by the logged in user , if no -user argument is provided. We need an option to list all the volumes created in the cluster -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-301) ozone does not contain subcommand to run ozoneFS commands
Nilotpal Nandi created HDDS-301: --- Summary: ozone does not contain subcommand to run ozoneFS commands Key: HDDS-301 URL: https://issues.apache.org/jira/browse/HDDS-301 Project: Hadoop Distributed Data Store Issue Type: Bug Components: Ozone Filesystem Reporter: Nilotpal Nandi Fix For: 0.2.1 Unlike hdfs , currently ozone does not have the sub-command to run ozoneFS commands. In hdfs, we have dfs subcommand to run hdfs filesystem commands. Similar, subcommand needs to be added for ozone -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-314) ozoneShell putKey command overwrites the existing key having same name
Nilotpal Nandi created HDDS-314:
---
Summary: ozoneShell putKey command overwrites the existing key
having same name
Key: HDDS-314
URL: https://issues.apache.org/jira/browse/HDDS-314
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Reporter: Nilotpal Nandi
Fix For: 0.2.1
steps taken :
1) created a volume root-volume and a bucket root-bucket.
2) Ran following command to put a key with name 'passwd'
{noformat}
hadoop@08315aa4b367:~/bin$ ./ozone oz -putKey /root-volume/root-bucket/passwd
-file /etc/services -v
2018-08-02 09:20:17 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Volume Name : root-volume
Bucket Name : root-bucket
Key Name : passwd
File Hash : 567c100888518c1163b3462993de7d47
2018-08-02 09:20:18 INFO ConfUtils:41 - raft.rpc.type = GRPC (default)
2018-08-02 09:20:18 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-02 09:20:18 INFO ConfUtils:41 - raft.client.rpc.retryInterval = 300 ms
(default)
2018-08-02 09:20:18 INFO ConfUtils:41 -
raft.client.async.outstanding-requests.max = 100 (default)
2018-08-02 09:20:18 INFO ConfUtils:41 - raft.client.async.scheduler-threads = 3
(default)
2018-08-02 09:20:18 INFO ConfUtils:41 - raft.grpc.flow.control.window = 1MB
(=1048576) (default)
2018-08-02 09:20:18 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-02 09:20:18 INFO ConfUtils:41 - raft.client.rpc.request.timeout = 3000
ms (default)
Aug 02, 2018 9:20:18 AM
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl detectProxy
{noformat}
3) Ran following command to put a key with name 'passwd' again.
{noformat}
hadoop@08315aa4b367:~/bin$ ./ozone oz -putKey /root-volume/root-bucket/passwd
-file /etc/passwd -v
2018-08-02 09:20:41 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Volume Name : root-volume
Bucket Name : root-bucket
Key Name : passwd
File Hash : b056233571cc80d6879212911cb8e500
2018-08-02 09:20:41 INFO ConfUtils:41 - raft.rpc.type = GRPC (default)
2018-08-02 09:20:42 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-02 09:20:42 INFO ConfUtils:41 - raft.client.rpc.retryInterval = 300 ms
(default)
2018-08-02 09:20:42 INFO ConfUtils:41 -
raft.client.async.outstanding-requests.max = 100 (default)
2018-08-02 09:20:42 INFO ConfUtils:41 - raft.client.async.scheduler-threads = 3
(default)
2018-08-02 09:20:42 INFO ConfUtils:41 - raft.grpc.flow.control.window = 1MB
(=1048576) (default)
2018-08-02 09:20:42 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-02 09:20:42 INFO ConfUtils:41 - raft.client.rpc.request.timeout = 3000
ms (default)
Aug 02, 2018 9:20:42 AM
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl detectProxy{noformat}
key 'passwd' was overwritten with new content and it did not throw any saying
that the key is already present.
Expectation :
---
key overwrite with same name should not be allowed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-315) ozoneShell infoKey does not work for directories created as key and throws 'KEY_NOT_FOUND' error
Nilotpal Nandi created HDDS-315:
---
Summary: ozoneShell infoKey does not work for directories created
as key and throws 'KEY_NOT_FOUND' error
Key: HDDS-315
URL: https://issues.apache.org/jira/browse/HDDS-315
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
Fix For: 0.2.1
infoKey for directories created using ozoneFs does not work and throws
'KEY_NOT_FOUND' error. However, it shows up in the 'listKey' command.
Here in this example, 'dir1' was created using ozoneFS , infoKey for the
directory throws error.
{noformat}
hadoop@08315aa4b367:~/bin./ozone oz -infoKey /root-volume/root-bucket/dir1
2018-08-02 11:34:06 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Command Failed : Lookup key failed, error:KEY_NOT_FOUND
hadoop@08315aa4b367:~/bin$ ./ozone oz -infoKey /root-volume/root-bucket/dir1/
2018-08-02 11:34:16 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Command Failed : Lookup key failed, error:KEY_NOT_FOUND
hadoop@08315aa4b367:~/bin$ ./ozone oz -listKey /root-volume/root-bucket/
2018-08-02 11:34:21 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
[ {
"version" : 0,
"md5hash" : null,
"createdOn" : "Wed, 07 May +50555 12:44:16 GMT",
"modifiedOn" : "Wed, 07 May +50555 12:44:30 GMT",
"size" : 0,
"keyName" : "dir1/"
}, {
"version" : 0,
"md5hash" : null,
"createdOn" : "Wed, 07 May +50555 14:14:06 GMT",
"modifiedOn" : "Wed, 07 May +50555 14:14:19 GMT",
"size" : 0,
"keyName" : "dir2/"
}, {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 08 May +50555 21:40:55 GMT",
"modifiedOn" : "Thu, 08 May +50555 21:40:59 GMT",
"size" : 0,
"keyName" : "dir2/b1/"{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-318) ratis INFO logs should not shown during ozoneFs command-line execution
Nilotpal Nandi created HDDS-318:
---
Summary: ratis INFO logs should not shown during ozoneFs
command-line execution
Key: HDDS-318
URL: https://issues.apache.org/jira/browse/HDDS-318
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Filesystem
Reporter: Nilotpal Nandi
Fix For: 0.2.1
ratis INFOs should not be shown during ozoneFS CLI execution.
Please find the snippet from one othe execution :
{noformat}
hadoop@08315aa4b367:~/bin$ ./ozone fs -put /etc/passwd /p2
2018-08-02 12:17:18 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
2018-08-02 12:17:19 INFO ConfUtils:41 - raft.rpc.type = GRPC (default)
2018-08-02 12:17:19 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-02 12:17:19 INFO ConfUtils:41 - raft.client.rpc.retryInterval = 300 ms
(default)
2018-08-02 12:17:19 INFO ConfUtils:41 -
raft.client.async.outstanding-requests.max = 100 (default)
2018-08-02 12:17:19 INFO ConfUtils:41 - raft.client.async.scheduler-threads = 3
(default)
2018-08-02 12:17:19 INFO ConfUtils:41 - raft.grpc.flow.control.window = 1MB
(=1048576) (default)
2018-08-02 12:17:19 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-02 12:17:20 INFO ConfUtils:41 - raft.client.rpc.request.timeout = 3000
ms (default)
Aug 02, 2018 12:17:20 PM
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl detectProxy
WARNING: Failed to construct URI for proxy lookup, proceeding without proxy
..
..
..
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-321) ozoneFS put/copyFromLocal command does not work for a directory when the directory contains file(s) as well as subdirectories
Nilotpal Nandi created HDDS-321:
---
Summary: ozoneFS put/copyFromLocal command does not work for a
directory when the directory contains file(s) as well as subdirectories
Key: HDDS-321
URL: https://issues.apache.org/jira/browse/HDDS-321
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
Fix For: 0.2.1
Steps taken :
-
# Created a local directory 'TEST_DIR1' which contains directory "SUB_DIR1"
and a file "test_file1".
# Ran "./ozone fs -put TEST_DIR1/ /" . The command kept on running , throwing
error on console.
stack trace of the error thrown on the console :
{noformat}
2018-08-02 12:55:46 INFO ConfUtils:41 - raft.grpc.flow.control.window = 1MB
(=1048576) (default)
2018-08-02 12:55:46 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-02 12:55:46 INFO ConfUtils:41 - raft.client.rpc.request.timeout = 3000
ms (default)
Aug 02, 2018 12:55:46 PM
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl detectProxy
WARNING: Failed to construct URI for proxy lookup, proceeding without proxy
java.net.URISyntaxException: Illegal character in hostname at index 13:
https://ozone_datanode_3.ozone_default:9858
at java.net.URI$Parser.fail(URI.java:2848)
at java.net.URI$Parser.parseHostname(URI.java:3387)
at java.net.URI$Parser.parseServer(URI.java:3236)
at java.net.URI$Parser.parseAuthority(URI.java:3155)
at java.net.URI$Parser.parseHierarchical(URI.java:3097)
at java.net.URI$Parser.parse(URI.java:3053)
at java.net.URI.(URI.java:673)
at
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl.detectProxy(ProxyDetectorImpl.java:128)
at
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl.proxyFor(ProxyDetectorImpl.java:118)
at
org.apache.ratis.shaded.io.grpc.internal.InternalSubchannel.startNewTransport(InternalSubchannel.java:207)
at
org.apache.ratis.shaded.io.grpc.internal.InternalSubchannel.obtainActiveTransport(InternalSubchannel.java:188)
at
org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$SubchannelImpl.requestConnection(ManagedChannelImpl.java:1130)
at
org.apache.ratis.shaded.io.grpc.PickFirstBalancerFactory$PickFirstBalancer.handleResolvedAddressGroups(PickFirstBalancerFactory.java:79)
at
org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$NameResolverListenerImpl$1NamesResolved.run(ManagedChannelImpl.java:1032)
at
org.apache.ratis.shaded.io.grpc.internal.ChannelExecutor.drain(ChannelExecutor.java:73)
at
org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$4.get(ManagedChannelImpl.java:403)
at
org.apache.ratis.shaded.io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:238)
at
org.apache.ratis.shaded.io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1.start(CensusTracingModule.java:386)
at
org.apache.ratis.shaded.io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1.start(CensusStatsModule.java:679)
at
org.apache.ratis.shaded.io.grpc.stub.ClientCalls.startCall(ClientCalls.java:293)
at
org.apache.ratis.shaded.io.grpc.stub.ClientCalls.asyncStreamingRequestCall(ClientCalls.java:283)
at
org.apache.ratis.shaded.io.grpc.stub.ClientCalls.asyncBidiStreamingCall(ClientCalls.java:92)
at
org.apache.ratis.shaded.proto.grpc.RaftClientProtocolServiceGrpc$RaftClientProtocolServiceStub.append(RaftClientProtocolServiceGrpc.java:208)
at
org.apache.ratis.grpc.client.RaftClientProtocolClient.appendWithTimeout(RaftClientProtocolClient.java:139)
at
org.apache.ratis.grpc.client.GrpcClientRpc.sendRequest(GrpcClientRpc.java:109)
at
org.apache.ratis.grpc.client.GrpcClientRpc.sendRequest(GrpcClientRpc.java:88)
at
org.apache.ratis.client.impl.RaftClientImpl.sendRequest(RaftClientImpl.java:302)
at
org.apache.ratis.client.impl.RaftClientImpl.sendRequestWithRetry(RaftClientImpl.java:256)
at org.apache.ratis.client.impl.RaftClientImpl.send(RaftClientImpl.java:192)
at org.apache.ratis.client.impl.RaftClientImpl.send(RaftClientImpl.java:173)
at org.apache.ratis.client.RaftClient.send(RaftClient.java:80)
at
org.apache.hadoop.hdds.scm.XceiverClientRatis.sendRequest(XceiverClientRatis.java:218)
at
org.apache.hadoop.hdds.scm.XceiverClientRatis.sendCommand(XceiverClientRatis.java:235)
at
org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.writeChunk(ContainerProtocolCalls.java:219)
at
org.apache.hadoop.hdds.scm.storage.ChunkOutputStream.writeChunkToContainer(ChunkOutputStream.java:220)
at
org.apache.hadoop.hdds.scm.storage.ChunkOutputStream.close(ChunkOutputStream.java:150)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream$ChunkOutputStreamEntry.close(ChunkGroupOutputStream.java:486)
at
org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.close(ChunkGroupOutputStream.java:326)
at
org.apache.hadoop.fs.ozone.OzoneFSOutputStream.close(OzoneFSOutputStream.java:57)
at
org.apache.hadoop.fs.FSDataO
[jira] [Created] (HDDS-337) keys created with key name having special character/wildcard should not allowed
Nilotpal Nandi created HDDS-337:
---
Summary: keys created with key name having special
character/wildcard should not allowed
Key: HDDS-337
URL: https://issues.apache.org/jira/browse/HDDS-337
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Reporter: Nilotpal Nandi
Fix For: 0.2.1
Please find the snippet of command execution. Here , the keys are created with
wildcard special character in its key name.
Expectation :
wildcard special characters should not be allowed.
{noformat}
hadoop@1a1fa8a11332:~/bin$ ./ozone oz -putKey root-volume/root-bucket/d++ -file
/etc/services -v
2018-08-08 13:17:48 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Volume Name : root-volume
Bucket Name : root-bucket
Key Name : d++
File Hash : 567c100888518c1163b3462993de7d47
Key Name : d++ does not exist, creating it
2018-08-08 13:17:48 INFO ConfUtils:41 - raft.rpc.type = GRPC (default)
2018-08-08 13:17:48 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-08 13:17:48 INFO ConfUtils:41 - raft.client.rpc.retryInterval = 300 ms
(default)
2018-08-08 13:17:48 INFO ConfUtils:41 -
raft.client.async.outstanding-requests.max = 100 (default)
2018-08-08 13:17:48 INFO ConfUtils:41 - raft.client.async.scheduler-threads = 3
(default)
2018-08-08 13:17:49 INFO ConfUtils:41 - raft.grpc.flow.control.window = 1MB
(=1048576) (default)
2018-08-08 13:17:49 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-08 13:17:49 INFO ConfUtils:41 - raft.client.rpc.request.timeout = 3000
ms (default)
Aug 08, 2018 1:17:49 PM
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl detectProxy
WARNING: Failed to construct URI for proxy lookup, proceeding without proxy
java.net.URISyntaxException: Illegal character in hostname at index 13:
https://ozone_datanode_1.ozone_default:9858
at java.net.URI$Parser.fail(URI.java:2848)
at java.net.URI$Parser.parseHostname(URI.java:3387)
at java.net.URI$Parser.parseServer(URI.java:3236)
at java.net.URI$Parser.parseAuthority(URI.java:3155)
at java.net.URI$Parser.parseHierarchical(URI.java:3097)
at java.net.URI$Parser.parse(URI.java:3053)
at java.net.URI.(URI.java:673)
at
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl.detectProxy(ProxyDetectorImpl.java:128)
at
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl.proxyFor(ProxyDetectorImpl.java:118)
at
org.apache.ratis.shaded.io.grpc.internal.InternalSubchannel.startNewTransport(InternalSubchannel.java:207)
at
org.apache.ratis.shaded.io.grpc.internal.InternalSubchannel.obtainActiveTransport(InternalSubchannel.java:188)
at
org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$SubchannelImpl.requestConnection(ManagedChannelImpl.java:1130)
at
org.apache.ratis.shaded.io.grpc.PickFirstBalancerFactory$PickFirstBalancer.handleResolvedAddressGroups(PickFirstBalancerFactory.java:79)
at
org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$NameResolverListenerImpl$1NamesResolved.run(ManagedChannelImpl.java:1032)
at
org.apache.ratis.shaded.io.grpc.internal.ChannelExecutor.drain(ChannelExecutor.java:73)
at
org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$LbHelperImpl.runSerialized(ManagedChannelImpl.java:1000)
at
org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$NameResolverListenerImpl.onAddresses(ManagedChannelImpl.java:1044)
at
org.apache.ratis.shaded.io.grpc.internal.DnsNameResolver$1.run(DnsNameResolver.java:201)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
hadoop@1a1fa8a11332:~/bin$ ./ozone oz -putKey root-volume/root-bucket/d** -file
/etc/passwd -v
2018-08-08 13:18:13 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Volume Name : root-volume
Bucket Name : root-bucket
Key Name : d**
File Hash : b056233571cc80d6879212911cb8e500
Key Name : d** does not exist, creating it
2018-08-08 13:18:14 INFO ConfUtils:41 - raft.rpc.type = GRPC (default)
2018-08-08 13:18:14 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-08 13:18:14 INFO ConfUtils:41 - raft.client.rpc.retryInterval = 300 ms
(default)
2018-08-08 13:18:14 INFO ConfUtils:41 -
raft.client.async.outstanding-requests.max = 100 (default)
2018-08-08 13:18:14 INFO ConfUtils:41 - raft.client.async.scheduler-threads = 3
(default)
2018-08-08 13:18:14 INFO ConfUtils:41 - raft.grpc.flow.control.window = 1MB
(=1048576) (default)
2018-08-08 13:18:14 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-08 13:18:14 INFO ConfUtils:41 - raft.client.rpc.request.timeout
[jira] [Created] (HDDS-338) ozoneFS allows to create file key and directory key with same keyname
Nilotpal Nandi created HDDS-338:
---
Summary: ozoneFS allows to create file key and directory key with
same keyname
Key: HDDS-338
URL: https://issues.apache.org/jira/browse/HDDS-338
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Filesystem
Reporter: Nilotpal Nandi
Fix For: 0.2.1
steps taken :
--
1. created a directory through ozoneFS interface.
{noformat}
hadoop@1a1fa8a11332:~/bin$ ./ozone fs -mkdir /temp1/
2018-08-08 13:50:26 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
hadoop@1a1fa8a11332:~/bin$ ./ozone fs -ls /
2018-08-08 14:09:59 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Found 1 items
drwxrwxrwx - 0 2018-08-08 13:51 /temp1{noformat}
2. create a new key with name 'temp1' at same bucket.
{noformat}
hadoop@1a1fa8a11332:~/bin$ ./ozone oz -putKey root-volume/root-bucket/temp1
-file /etc/passwd
2018-08-08 14:10:34 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
2018-08-08 14:10:35 INFO ConfUtils:41 - raft.rpc.type = GRPC (default)
2018-08-08 14:10:35 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-08 14:10:35 INFO ConfUtils:41 - raft.client.rpc.retryInterval = 300 ms
(default)
2018-08-08 14:10:35 INFO ConfUtils:41 -
raft.client.async.outstanding-requests.max = 100 (default)
2018-08-08 14:10:35 INFO ConfUtils:41 - raft.client.async.scheduler-threads = 3
(default)
2018-08-08 14:10:35 INFO ConfUtils:41 - raft.grpc.flow.control.window = 1MB
(=1048576) (default)
2018-08-08 14:10:35 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
(custom)
2018-08-08 14:10:35 INFO ConfUtils:41 - raft.client.rpc.request.timeout = 3000
ms (default)
Aug 08, 2018 2:10:36 PM
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl detectProxy
WARNING: Failed to construct URI for proxy lookup, proceeding without proxy
java.net.URISyntaxException: Illegal character in hostname at index 13:
https://ozone_datanode_3.ozone_default:9858
at java.net.URI$Parser.fail(URI.java:2848)
at java.net.URI$Parser.parseHostname(URI.java:3387)
at java.net.URI$Parser.parseServer(URI.java:3236)
at java.net.URI$Parser.parseAuthority(URI.java:3155)
at java.net.URI$Parser.parseHierarchical(URI.java:3097)
at java.net.URI$Parser.parse(URI.java:3053)
at java.net.URI.(URI.java:673)
at
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl.detectProxy(ProxyDetectorImpl.java:128)
at
org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl.proxyFor(ProxyDetectorImpl.java:118)
at
org.apache.ratis.shaded.io.grpc.internal.InternalSubchannel.startNewTransport(InternalSubchannel.java:207)
at
org.apache.ratis.shaded.io.grpc.internal.InternalSubchannel.obtainActiveTransport(InternalSubchannel.java:188)
at
org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$SubchannelImpl.requestConnection(ManagedChannelImpl.java:1130)
at
org.apache.ratis.shaded.io.grpc.PickFirstBalancerFactory$PickFirstBalancer.handleResolvedAddressGroups(PickFirstBalancerFactory.java:79)
at
org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$NameResolverListenerImpl$1NamesResolved.run(ManagedChannelImpl.java:1032)
at
org.apache.ratis.shaded.io.grpc.internal.ChannelExecutor.drain(ChannelExecutor.java:73)
at
org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$LbHelperImpl.runSerialized(ManagedChannelImpl.java:1000)
at
org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$NameResolverListenerImpl.onAddresses(ManagedChannelImpl.java:1044)
at
org.apache.ratis.shaded.io.grpc.internal.DnsNameResolver$1.run(DnsNameResolver.java:201)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){noformat}
Observed that there are multiple entries of 'temp1' when ozone fs -ls command
is run. Also . both the entries are considered as file . '/temp1' directory is
not visible anymore.
{noformat}
hadoop@1a1fa8a11332:~/bin$ ./ozone fs -ls /
2018-08-08 14:10:41 WARN NativeCodeLoader:60 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Found 2 items
-rw-rw-rw- 1 963 2018-08-08 14:10 /temp1
-rw-rw-rw- 1 963 2018-08-08 14:10 /temp1{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-403) infoKey shows wrong "createdOn", "modifiedOn" metadata for key
Nilotpal Nandi created HDDS-403:
---
Summary: infoKey shows wrong "createdOn", "modifiedOn" metadata
for key
Key: HDDS-403
URL: https://issues.apache.org/jira/browse/HDDS-403
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Manager
Reporter: Nilotpal Nandi
Fix For: 0.2.1
1. ran putKey command for a file
{noformat}
[root@ctr-e138-1518143905142-459606-01-03 bin]# ./ozone oz -putKey
/test-vol1/test-bucket1/file1 -file /etc/passwd -v
2018-09-05 10:25:11,498 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
Volume Name : test-vol1
Bucket Name : test-bucket1
Key Name : file1
File Hash : 8164cc3d5b05c44b73a6277661aa4645
2018-09-05 10:25:12,377 INFO conf.ConfUtils: raft.rpc.type = GRPC (default)
2018-09-05 10:25:12,390 INFO conf.ConfUtils: raft.grpc.message.size.max =
33554432 (custom)
2018-09-05 10:25:12,402 INFO conf.ConfUtils: raft.client.rpc.retryInterval =
300 ms (default)
2018-09-05 10:25:12,407 INFO conf.ConfUtils:
raft.client.async.outstanding-requests.max = 100 (default)
2018-09-05 10:25:12,407 INFO conf.ConfUtils:
raft.client.async.scheduler-threads = 3 (default)
2018-09-05 10:25:12,518 INFO conf.ConfUtils: raft.grpc.flow.control.window =
1MB (=1048576) (default)
2018-09-05 10:25:12,518 INFO conf.ConfUtils: raft.grpc.message.size.max =
33554432 (custom)
2018-09-05 10:25:12,866 INFO conf.ConfUtils: raft.client.rpc.request.timeout =
3000 ms (default)
2018-09-05 10:25:13,644 INFO conf.ConfUtils: raft.grpc.flow.control.window =
1MB (=1048576) (default)
2018-09-05 10:25:13,644 INFO conf.ConfUtils: raft.grpc.message.size.max =
33554432 (custom)
2018-09-05 10:25:13,645 INFO conf.ConfUtils: raft.client.rpc.request.timeout =
3000 ms (default)
[root@ctr-e138-1518143905142-459606-01-03 bin]# ./ozone oz -getKey
/test-vol1/test-bucket1/file1 -file getkey3
2018-09-05 10:25:22,020 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2018-09-05 10:25:22,778 INFO conf.ConfUtils: raft.rpc.type = GRPC (default)
2018-09-05 10:25:22,790 INFO conf.ConfUtils: raft.grpc.message.size.max =
33554432 (custom)
2018-09-05 10:25:22,800 INFO conf.ConfUtils: raft.client.rpc.retryInterval =
300 ms (default)
2018-09-05 10:25:22,804 INFO conf.ConfUtils:
raft.client.async.outstanding-requests.max = 100 (default)
2018-09-05 10:25:22,805 INFO conf.ConfUtils:
raft.client.async.scheduler-threads = 3 (default)
2018-09-05 10:25:22,890 INFO conf.ConfUtils: raft.grpc.flow.control.window =
1MB (=1048576) (default)
2018-09-05 10:25:22,890 INFO conf.ConfUtils: raft.grpc.message.size.max =
33554432 (custom)
2018-09-05 10:25:23,250 INFO conf.ConfUtils: raft.client.rpc.request.timeout =
3000 ms (default)
2018-09-05 10:25:24,066 INFO conf.ConfUtils: raft.grpc.flow.control.window =
1MB (=1048576) (default)
2018-09-05 10:25:24,067 INFO conf.ConfUtils: raft.grpc.message.size.max =
33554432 (custom)
2018-09-05 10:25:24,067 INFO conf.ConfUtils: raft.client.rpc.request.timeout =
3000 ms (default){noformat}
2. Ran infoKey on that key
{noformat}
[root@ctr-e138-1518143905142-459606-01-03 bin]# ./ozone oz -infoKey
/test-vol1/test-bucket1/file1 -v
2018-09-05 10:54:42,053 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
Volume Name : test-vol1
Bucket Name : test-bucket1
Key Name : file1
{
"version" : 0,
"md5hash" : null,
"createdOn" : "Sat, 14 Dec +114522267 00:51:17 GMT",
"modifiedOn" : "Fri, 09 Jun +50648 04:30:12 GMT",
"size" : 4659,
"keyName" : "file1",
"keyLocations" : [ {
"containerID" : 16,
"localID" : 1536143112267,
"length" : 4659,
"offset" : 0
} ]
}{noformat}
"createdOn" and "modifiedOn" metadata are incorrect.
Here is the current date:
{noformat}
[root@ctr-e138-1518143905142-459606-01-03 bin]# date
Wed Sep 5 10:54:52 UTC 2018{noformat}
Also , the "md5hash" for the key is showing as null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-407) ozone logs are wriiten to ozone.log. instead of ozone.log
Nilotpal Nandi created HDDS-407:
---
Summary: ozone logs are wriiten to ozone.log. instead of
ozone.log
Key: HDDS-407
URL: https://issues.apache.org/jira/browse/HDDS-407
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Reporter: Nilotpal Nandi
Fix For: 0.2.1
Please refer below details
ozone related logs are written to ozone.log.2018-09-05 instead of ozone.log.
Also, please check the timestamps of the logs. The cluster was created
{noformat}
[root@ctr-e138-1518143905142-459606-01-02 logs]# ls -lhart
/root/hadoop_trunk/ozone-0.2.1-SNAPSHOT/logs/
total 968K
drwxr-xr-x 9 root root 4.0K Sep 5 10:04 ..
-rw-r--r-- 1 root root 0 Sep 5 10:04 fairscheduler-statedump.log
-rw-r--r-- 1 root root 17K Sep 5 10:05
hadoop-root-om-ctr-e138-1518143905142-459606-01-02.hwx.site.out.1
-rw-r--r-- 1 root root 16K Sep 5 10:10
hadoop-root-om-ctr-e138-1518143905142-459606-01-02.hwx.site.out
-rw-r--r-- 1 root root 11K Sep 5 10:10
hadoop-root-om-ctr-e138-1518143905142-459606-01-02.hwx.site.log
-rw-r--r-- 1 root root 17K Sep 6 05:42
hadoop-root-datanode-ctr-e138-1518143905142-459606-01-02.hwx.site.out
-rw-r--r-- 1 root root 2.1K Sep 6 13:20 ozone.log
-rw-r--r-- 1 root root 67K Sep 6 13:22
hadoop-root-datanode-ctr-e138-1518143905142-459606-01-02.hwx.site.log
drwxr-xr-x 2 root root 4.0K Sep 6 13:31 .
-rw-r--r-- 1 root root 811K Sep 6 13:39 ozone.log.2018-09-05
[root@ctr-e138-1518143905142-459606-01-02 logs]# date
Thu Sep 6 13:39:47 UTC 2018{noformat}
tail of ozone.log
{noformat}
[root@ctr-e138-1518143905142-459606-01-02 logs]# tail -f ozone.log
2018-09-06 10:51:56,616 [IPC Server handler 13 on 9889] DEBUG
(KeyManagerImpl.java:255) - Key 0file allocated in volume test-vol2 bucket
test-bucket2
2018-09-06 10:52:18,570 [IPC Server handler 9 on 9889] DEBUG
(KeyManagerImpl.java:255) - Key 0file1 allocated in volume test-vol2 bucket
test-bucket2
2018-09-06 10:52:32,256 [IPC Server handler 12 on 9889] DEBUG
(KeyManagerImpl.java:255) - Key 0file2 allocated in volume test-vol2 bucket
test-bucket2
2018-09-06 10:53:11,008 [IPC Server handler 14 on 9889] DEBUG
(KeyManagerImpl.java:255) - Key 0file2 allocated in volume test-vol2 bucket
test-bucket2
2018-09-06 10:53:28,316 [IPC Server handler 10 on 9889] DEBUG
(KeyManagerImpl.java:255) - Key 0file2 allocated in volume test-vol2 bucket
test-bucket2
2018-09-06 10:53:39,509 [IPC Server handler 17 on 9889] DEBUG
(KeyManagerImpl.java:255) - Key 0file3 allocated in volume test-vol2 bucket
test-bucket2
2018-09-06 11:31:02,388 [IPC Server handler 19 on 9889] DEBUG
(KeyManagerImpl.java:255) - Key 2GBFILE allocated in volume test-vol2 bucket
test-bucket2
2018-09-06 11:32:44,269 [IPC Server handler 12 on 9889] DEBUG
(KeyManagerImpl.java:255) - Key 2GBFILE_1 allocated in volume test-vol2 bucket
test-bucket2
2018-09-06 13:17:33,408 [IPC Server handler 16 on 9889] DEBUG
(KeyManagerImpl.java:255) - Key FILEWITHZEROS allocated in volume test-vol2
bucket test-bucket2
2018-09-06 13:20:13,897 [IPC Server handler 15 on 9889] DEBUG
(KeyManagerImpl.java:255) - Key FILEWITHZEROS1 allocated in volume test-vol2
bucket test-bucket2{noformat}
tail of ozone.log.2018-09-05:
{noformat}
root@ctr-e138-1518143905142-459606-01-02 logs]# tail -50
ozone.log.2018-09-05
2018-09-06 13:28:57,866 [BlockDeletingService#8] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending deletion block contained in remaining containers.
2018-09-06 13:29:07,816 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3266
2018-09-06 13:29:13,687 [Datanode ReportManager Thread - 0] DEBUG
(ContainerSet.java:191) - Starting container report iteration.
2018-09-06 13:29:37,816 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3267
2018-09-06 13:29:57,866 [BlockDeletingService#8] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending deletion block contained in remaining containers.
2018-09-06 13:30:07,816 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3268
2018-09-06 13:30:19,186 [Datanode ReportManager Thread - 0] DEBUG
(ContainerSet.java:191) - Starting container report iteration.
2018-09-06 13:30:37,816 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3269
2018-09-06 13:30:57,866 [BlockDeletingService#8] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending deletion block contained in remaining containers.
2018-09-06 13:31:07,816 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3270
[jira] [Created] (HDDS-408) Read (getKey) operation is very slow
Nilotpal Nandi created HDDS-408:
---
Summary: Read (getKey) operation is very slow
Key: HDDS-408
URL: https://issues.apache.org/jira/browse/HDDS-408
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Datanode, Ozone Manager
Reporter: Nilotpal Nandi
Fix For: 0.2.1
PutKey operation for a 2GB file took around ~42 secs .
real 0m41.955s
user 0m19.367s
sys 0m7.480s
For the same Key, getKey operation took around 8min 49 secs.
real 0m41.955s
user 0m19.367s
sys 0m7.480s
Repeated operation multiple times, results were similar.
For read operation , the client waits for a long time. In ozone.log, there is a
long trail of these messages on all datanodes :
{noformat}
2018-09-06 14:10:52,288 [BlockDeletingService#2] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending deletion block contained in remaining containers.
2018-09-06 14:11:02,413 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3350
2018-09-06 14:11:32,414 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3351
2018-09-06 14:11:41,797 [Datanode ReportManager Thread - 0] DEBUG
(ContainerSet.java:191) - Starting container report iteration.
2018-09-06 14:11:52,288 [BlockDeletingService#2] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending deletion block contained in remaining containers.
2018-09-06 14:12:02,414 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3352
2018-09-06 14:12:32,416 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3353
2018-09-06 14:12:48,723 [Datanode ReportManager Thread - 0] DEBUG
(ContainerSet.java:191) - Starting container report iteration.
2018-09-06 14:12:52,289 [BlockDeletingService#2] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending deletion block contained in remaining containers.
2018-09-06 14:13:02,416 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3354
2018-09-06 14:13:32,416 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3355
2018-09-06 14:13:52,289 [BlockDeletingService#2] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending deletion block contained in remaining containers.
2018-09-06 14:14:02,416 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3356
2018-09-06 14:14:31,725 [Datanode ReportManager Thread - 2] DEBUG
(ContainerSet.java:191) - Starting container report iteration.
2018-09-06 14:14:32,416 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3357
2018-09-06 14:14:52,290 [BlockDeletingService#2] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending deletion block contained in remaining containers.
2018-09-06 14:15:02,417 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3358
2018-09-06 14:15:32,418 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3359
2018-09-06 14:15:52,290 [BlockDeletingService#2] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending deletion block contained in remaining containers.
2018-09-06 14:16:02,419 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3360
2018-09-06 14:16:26,897 [Datanode ReportManager Thread - 0] DEBUG
(ContainerSet.java:191) - Starting container report iteration.
2018-09-06 14:16:32,420 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3361
2018-09-06 14:16:52,290 [BlockDeletingService#2] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending deletion block contained in remaining containers.
2018-09-06 14:17:02,420 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3362
2018-09-06 14:17:32,420 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:145) - Executing cycle Number : 3363
2018-09-06 14:17:48,024 [Datanode ReportManager Thread - 0] DEBUG
(ContainerSet.java:191) - Starting container report iteration.
2018-09-06 14:17:52,291 [BlockDeletingService#2] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending de
[jira] [Created] (HDDS-410) ozone scmcli is not working properly
Nilotpal Nandi created HDDS-410:
---
Summary: ozone scmcli is not working properly
Key: HDDS-410
URL: https://issues.apache.org/jira/browse/HDDS-410
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: SCM
Reporter: Nilotpal Nandi
Fix For: 0.2.1
On running ozone scmcli for a container ID, it gives the following output :
{noformat}
[root@ctr-e138-1518143905142-459606-01-02 bin]# ./ozone scmcli list
--start=17
Infinite recursion (StackOverflowError) (through reference chain:
org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSupplier["initializer"]->org.apache.ratis.protocol.RaftId$$Lambda$17/1523553211["arg$1"]->org.apache.ratis.protocol.RaftGroupId["uuidString"]->org.apache.ratis.util.MemoizedSup
[jira] [Resolved] (HDDS-321) ozoneFS put/copyFromLocal command does not work for a directory when the directory contains file(s) as well as subdirectories
[
https://issues.apache.org/jira/browse/HDDS-321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nilotpal Nandi resolved HDDS-321.
-
Resolution: Fixed
> ozoneFS put/copyFromLocal command does not work for a directory when the
> directory contains file(s) as well as subdirectories
> -
>
> Key: HDDS-321
> URL: https://issues.apache.org/jira/browse/HDDS-321
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Nilotpal Nandi
>Assignee: Nilotpal Nandi
>Priority: Blocker
> Fix For: 0.2.1
>
>
> Steps taken :
> -
> # Created a local directory 'TEST_DIR1' which contains directory "SUB_DIR1"
> and a file "test_file1".
> # Ran "./ozone fs -put TEST_DIR1/ /" . The command kept on running ,
> throwing error on console.
> stack trace of the error thrown on the console :
> {noformat}
> 2018-08-02 12:55:46 INFO ConfUtils:41 - raft.grpc.flow.control.window = 1MB
> (=1048576) (default)
> 2018-08-02 12:55:46 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
> (custom)
> 2018-08-02 12:55:46 INFO ConfUtils:41 - raft.client.rpc.request.timeout =
> 3000 ms (default)
> Aug 02, 2018 12:55:46 PM
> org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl detectProxy
> WARNING: Failed to construct URI for proxy lookup, proceeding without proxy
> java.net.URISyntaxException: Illegal character in hostname at index 13:
> https://ozone_datanode_3.ozone_default:9858
> at java.net.URI$Parser.fail(URI.java:2848)
> at java.net.URI$Parser.parseHostname(URI.java:3387)
> at java.net.URI$Parser.parseServer(URI.java:3236)
> at java.net.URI$Parser.parseAuthority(URI.java:3155)
> at java.net.URI$Parser.parseHierarchical(URI.java:3097)
> at java.net.URI$Parser.parse(URI.java:3053)
> at java.net.URI.(URI.java:673)
> at
> org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl.detectProxy(ProxyDetectorImpl.java:128)
> at
> org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl.proxyFor(ProxyDetectorImpl.java:118)
> at
> org.apache.ratis.shaded.io.grpc.internal.InternalSubchannel.startNewTransport(InternalSubchannel.java:207)
> at
> org.apache.ratis.shaded.io.grpc.internal.InternalSubchannel.obtainActiveTransport(InternalSubchannel.java:188)
> at
> org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$SubchannelImpl.requestConnection(ManagedChannelImpl.java:1130)
> at
> org.apache.ratis.shaded.io.grpc.PickFirstBalancerFactory$PickFirstBalancer.handleResolvedAddressGroups(PickFirstBalancerFactory.java:79)
> at
> org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$NameResolverListenerImpl$1NamesResolved.run(ManagedChannelImpl.java:1032)
> at
> org.apache.ratis.shaded.io.grpc.internal.ChannelExecutor.drain(ChannelExecutor.java:73)
> at
> org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$4.get(ManagedChannelImpl.java:403)
> at
> org.apache.ratis.shaded.io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:238)
> at
> org.apache.ratis.shaded.io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1.start(CensusTracingModule.java:386)
> at
> org.apache.ratis.shaded.io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1.start(CensusStatsModule.java:679)
> at
> org.apache.ratis.shaded.io.grpc.stub.ClientCalls.startCall(ClientCalls.java:293)
> at
> org.apache.ratis.shaded.io.grpc.stub.ClientCalls.asyncStreamingRequestCall(ClientCalls.java:283)
> at
> org.apache.ratis.shaded.io.grpc.stub.ClientCalls.asyncBidiStreamingCall(ClientCalls.java:92)
> at
> org.apache.ratis.shaded.proto.grpc.RaftClientProtocolServiceGrpc$RaftClientProtocolServiceStub.append(RaftClientProtocolServiceGrpc.java:208)
> at
> org.apache.ratis.grpc.client.RaftClientProtocolClient.appendWithTimeout(RaftClientProtocolClient.java:139)
> at
> org.apache.ratis.grpc.client.GrpcClientRpc.sendRequest(GrpcClientRpc.java:109)
> at
> org.apache.ratis.grpc.client.GrpcClientRpc.sendRequest(GrpcClientRpc.java:88)
> at
> org.apache.ratis.client.impl.RaftClientImpl.sendRequest(RaftClientImpl.java:302)
> at
> org.apache.ratis.client.impl.RaftClientImpl.sendRequestWithRetry(RaftClientImpl.java:256)
> at org.apache.ratis.client.impl.RaftClientImpl.send(RaftClientImpl.java:192)
> at org.apache.ratis.client.impl.RaftClientImpl.send(RaftClientImpl.java:173)
> at org.apache.ratis.client.RaftClient.send(RaftClient.java:80)
> at
> org.apache.hadoop.hdds.scm.XceiverClientRatis.sendRequest(XceiverClientRatis.java:218)
> at
> org.apache.hadoop.hdds.scm.XceiverClientRatis.sendCommand(XceiverClientRatis.java:235)
> at
> org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.writeChunk(ContainerProtocolCalls.java:219)
> at
> org.apache.hadoop.hdds.scm.storage.ChunkOutputS
[jira] [Created] (HDDS-420) putKey failing with KEY_ALLOCATION_ERROR
Nilotpal Nandi created HDDS-420:
---
Summary: putKey failing with KEY_ALLOCATION_ERROR
Key: HDDS-420
URL: https://issues.apache.org/jira/browse/HDDS-420
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Manager
Reporter: Nilotpal Nandi
Fix For: 0.2.1
Here are the commands run :
{noformat}
[root@ctr-e138-1518143905142-468367-01-02 bin]# ./ozone oz -putKey
/fs-volume/fs-bucket/nn1 -file /etc/passwd
2018-09-09 15:39:31,131 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
Create key failed, error:KEY_ALLOCATION_ERROR
[root@ctr-e138-1518143905142-468367-01-02 bin]#
[root@ctr-e138-1518143905142-468367-01-02 bin]# ./ozone fs -copyFromLocal
/etc/passwd /
2018-09-09 15:40:16,879 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2018-09-09 15:40:23,632 [main] ERROR - Try to allocate more blocks for write
failed, already allocated 0 blocks for this write.
copyFromLocal: Message missing required fields: keyLocation
[root@ctr-e138-1518143905142-468367-01-02 bin]# ./ozone oz -putKey
/fs-volume/fs-bucket/nn2 -file /etc/passwd
2018-09-09 15:44:55,912 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
Create key failed, error:KEY_ALLOCATION_ERROR{noformat}
hadoop version :
---
{noformat}
[root@ctr-e138-1518143905142-468367-01-02 bin]# ./hadoop version
Hadoop 3.2.0-SNAPSHOT
Source code repository git://git.apache.org/hadoop.git -r
bf8a1750e99cfbfa76021ce51b6514c74c06f498
Compiled by root on 2018-09-08T10:22Z
Compiled with protoc 2.5.0
>From source with checksum c5bbb375aed8edabd89c377af83189d
This command was run using
/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/hadoop-common-3.2.0-SNAPSHOT.jar{noformat}
scm log :
---
{noformat}
2018-09-09 15:45:00,907 INFO
org.apache.hadoop.hdds.scm.pipelines.ratis.RatisManagerImpl: Allocating a new
ratis pipeline of size: 3 id: pipelineId=f210716d-ba7b-4adf-91d6-da286e5fd010
2018-09-09 15:45:00,973 INFO org.apache.ratis.conf.ConfUtils: raft.rpc.type =
GRPC (default)
2018-09-09 15:45:01,007 INFO org.apache.ratis.conf.ConfUtils:
raft.grpc.message.size.max = 33554432 (custom)
2018-09-09 15:45:01,011 INFO org.apache.ratis.conf.ConfUtils:
raft.client.rpc.retryInterval = 300 ms (default)
2018-09-09 15:45:01,012 INFO org.apache.ratis.conf.ConfUtils:
raft.client.async.outstanding-requests.max = 100 (default)
2018-09-09 15:45:01,012 INFO org.apache.ratis.conf.ConfUtils:
raft.client.async.scheduler-threads = 3 (default)
2018-09-09 15:45:01,020 INFO org.apache.ratis.conf.ConfUtils:
raft.grpc.flow.control.window = 1MB (=1048576) (default)
2018-09-09 15:45:01,020 INFO org.apache.ratis.conf.ConfUtils:
raft.grpc.message.size.max = 33554432 (custom)
2018-09-09 15:45:01,102 INFO org.apache.ratis.conf.ConfUtils:
raft.client.rpc.request.timeout = 3000 ms (default)
2018-09-09 15:45:01,667 ERROR org.apache.hadoop.hdds.scm.XceiverClientRatis:
Failed to reinitialize
RaftPeer:bfe9c5f2-da9b-4a8f-9013-7540cbbed1c9:172.27.12.96:9858 datanode:
bfe9c5f2-da9b-4a8f-9013-7540cbbed1c9{ip: 172.27.12.96, host:
ctr-e138-1518143905142-468367-01-07.hwx.site}
org.apache.ratis.protocol.GroupMismatchException:
bfe9c5f2-da9b-4a8f-9013-7540cbbed1c9: The group (group-7347726F7570) of
client-409D68EB500F does not match the group (group-2041ABBEE452) of the server
bfe9c5f2-da9b-4a8f-9013-7540cbbed1c9
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.ratis.util.ReflectionUtils.instantiateException(ReflectionUtils.java:222)
at org.apache.ratis.grpc.RaftGrpcUtil.tryUnwrapException(RaftGrpcUtil.java:79)
at org.apache.ratis.grpc.RaftGrpcUtil.unwrapException(RaftGrpcUtil.java:67)
at
org.apache.ratis.grpc.client.RaftClientProtocolClient.blockingCall(RaftClientProtocolClient.java:127)
at
org.apache.ratis.grpc.client.RaftClientProtocolClient.reinitialize(RaftClientProtocolClient.java:102)
at
org.apache.ratis.grpc.client.GrpcClientRpc.sendRequest(GrpcClientRpc.java:77)
at
org.apache.ratis.client.impl.RaftClientImpl.sendRequest(RaftClientImpl.java:302)
at
org.apache.ratis.client.impl.RaftClientImpl.reinitialize(RaftClientImpl.java:216)
at
org.apache.hadoop.hdds.scm.XceiverClientRatis.reinitialize(XceiverClientRatis.java:163)
at
org.apache.hadoop.hdds.scm.XceiverClientRatis.reinitialize(XceiverClientRatis.java:133)
a
[jira] [Created] (HDDS-451) PutKey failed due to error "Rejecting write chunk request. Chunk overwrite without explicit request"
Nilotpal Nandi created HDDS-451:
---
Summary: PutKey failed due to error "Rejecting write chunk
request. Chunk overwrite without explicit request"
Key: HDDS-451
URL: https://issues.apache.org/jira/browse/HDDS-451
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Client
Affects Versions: 0.2.1
Reporter: Nilotpal Nandi
steps taken :
--
# Ran Put Key command to write 50GB data. Put Key client operation failed
after 17 mins.
error seen ozone.log :
{noformat}
2018-09-13 12:11:53,734 [ForkJoinPool.commonPool-worker-20] DEBUG
(ChunkManagerImpl.java:85) - writing
chunk:bd80b58a5eba888200a4832a0f2aafb3_stream_5f3b2505-6964-45c9-a7ad-827388a1e6a0_chunk_1
chunk stage:COMMIT_DATA chunk
file:/tmp/hadoop-root/dfs/data/hdds/de0a9e01-4a12-40e3-b567-51b9bd83248e/current/containerDir0/16/chunks/bd80b58a5eba888200a4832a0f2aafb3_stream_5f3b2505-6964-45c9-a7ad-827388a1e6a0_chunk_1
tmp chunk file
2018-09-13 12:11:56,576 [pool-3-thread-60] DEBUG (ChunkManagerImpl.java:85) -
writing
chunk:bd80b58a5eba888200a4832a0f2aafb3_stream_5f3b2505-6964-45c9-a7ad-827388a1e6a0_chunk_2
chunk stage:WRITE_DATA chunk
file:/tmp/hadoop-root/dfs/data/hdds/de0a9e01-4a12-40e3-b567-51b9bd83248e/current/containerDir0/16/chunks/bd80b58a5eba888200a4832a0f2aafb3_stream_5f3b2505-6964-45c9-a7ad-827388a1e6a0_chunk_2
tmp chunk file
2018-09-13 12:11:56,739 [ForkJoinPool.commonPool-worker-20] DEBUG
(ChunkManagerImpl.java:85) - writing
chunk:bd80b58a5eba888200a4832a0f2aafb3_stream_5f3b2505-6964-45c9-a7ad-827388a1e6a0_chunk_2
chunk stage:COMMIT_DATA chunk
file:/tmp/hadoop-root/dfs/data/hdds/de0a9e01-4a12-40e3-b567-51b9bd83248e/current/containerDir0/16/chunks/bd80b58a5eba888200a4832a0f2aafb3_stream_5f3b2505-6964-45c9-a7ad-827388a1e6a0_chunk_2
tmp chunk file
2018-09-13 12:12:21,410 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:148) - Executing cycle Number : 206
2018-09-13 12:12:51,411 [Datanode State Machine Thread - 0] DEBUG
(DatanodeStateMachine.java:148) - Executing cycle Number : 207
2018-09-13 12:12:53,525 [BlockDeletingService#1] DEBUG
(TopNOrderedContainerDeletionChoosingPolicy.java:79) - Stop looking for next
container, there is no pending deletion block contained in remaining containers.
2018-09-13 12:12:55,048 [Datanode ReportManager Thread - 1] DEBUG
(ContainerSet.java:191) - Starting container report iteration.
2018-09-13 12:13:02,626 [pool-3-thread-1] ERROR (ChunkUtils.java:244) -
Rejecting write chunk request. Chunk overwrite without explicit request.
ChunkInfo{chunkName='bd80b58a5eba888200a4832a0f2aafb3_stream_5f3b2505-6964-45c9-a7ad-827388a1e6a0_chunk_2,
offset=0, len=16777216}
2018-09-13 12:13:03,035 [pool-3-thread-1] INFO (ContainerUtils.java:149) -
Operation: WriteChunk : Trace ID: 54834b29-603d-4ba9-9d68-0885215759d8 :
Message: Rejecting write chunk request. OverWrite flag
required.ChunkInfo{chunkName='bd80b58a5eba888200a4832a0f2aafb3_stream_5f3b2505-6964-45c9-a7ad-827388a1e6a0_chunk_2,
offset=0, len=16777216} : Result: OVERWRITE_FLAG_REQUIRED
2018-09-13 12:13:03,037 [ForkJoinPool.commonPool-worker-11] ERROR
(ChunkUtils.java:244) - Rejecting write chunk request. Chunk overwrite without
explicit request.
ChunkInfo{chunkName='bd80b58a5eba888200a4832a0f2aafb3_stream_5f3b2505-6964-45c9-a7ad-827388a1e6a0_chunk_2,
offset=0, len=16777216}
2018-09-13 12:13:03,037 [ForkJoinPool.commonPool-worker-11] INFO
(ContainerUtils.java:149) - Operation: WriteChunk : Trace ID:
54834b29-603d-4ba9-9d68-0885215759d8 : Message: Rejecting write chunk request.
OverWrite flag
required.ChunkInfo{chunkName='bd80b58a5eba888200a4832a0f2aafb3_stream_5f3b2505-6964-45c9-a7ad-827388a1e6a0_chunk_2,
offset=0, len=16777216} : Result: OVERWRITE_FLAG_REQUIRED
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-458) numberofKeys is 0 for all containers even when keys are present
Nilotpal Nandi created HDDS-458:
---
Summary: numberofKeys is 0 for all containers even when keys are
present
Key: HDDS-458
URL: https://issues.apache.org/jira/browse/HDDS-458
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: SCM Client
Affects Versions: 0.2.1
Reporter: Nilotpal Nandi
numberofKeys field is 0 for all containers even when keys are present
{noformat}
[root@ctr-e138-1518143905142-459606-01-05 bin]# ./ozone scmcli list
--count=40 --start=1 | grep numberOfKeys
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,
"numberOfKeys" : 0,{noformat}
{noformat}
[root@ctr-e138-1518143905142-459606-01-05 bin]# ./ozone oz key list
/fs-volume/fs-bucket/ | grep keyName
2018-09-13 19:10:33,502 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
"keyName" : "15GBFILE"
"keyName" : "15GBFILE1"
"keyName" : "1GB1"
"keyName" : "1GB10"
"keyName" : "1GB11"
"keyName" : "1GB12"
"keyName" : "1GB13"
"keyName" : "1GB14"
"keyName" : "1GB15"
"keyName" : "1GB2"
"keyName" : "1GB3"
"keyName" : "1GB4"
"keyName" : "1GB5"
"keyName" : "1GB6"
"keyName" : "1GB7"
"keyName" : "1GB8"
"keyName" : "1GB9"
"keyName" : "1GBsecond1"
"keyName" : "1GBsecond10"
"keyName" : "1GBsecond11"
"keyName" : "1GBsecond12"
"keyName" : "1GBsecond13"
"keyName" : "1GBsecond14"
"keyName" : "1GBsecond15"
"keyName" : "1GBsecond2"
"keyName" : "1GBsecond3"
"keyName" : "1GBsecond4"
"keyName" : "1GBsecond5"
"keyName" : "1GBsecond6"
"keyName" : "1GBsecond7"
"keyName" : "1GBsecond8"
"keyName" : "1GBsecond9"
"keyName" : "2GBFILE"
"keyName" : "2GBFILE2"
"keyName" : "50GBFILE2"
"keyName" : "passwd1"{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-460) Replication manager failed to import container data
Nilotpal Nandi created HDDS-460:
---
Summary: Replication manager failed to import container data
Key: HDDS-460
URL: https://issues.apache.org/jira/browse/HDDS-460
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: SCM
Affects Versions: 0.2.1
Reporter: Nilotpal Nandi
replication manager is not able to import downloaded container data on datanode
failure.
snippet of ozone.log
-
{noformat}
2018-09-14 09:34:05,249 [grpc-default-executor-139] INFO
(GrpcReplicationClient.java:161) - Container is downloaded to
/tmp/container-copy/container-14.tar.gz
2018-09-14 09:34:05,389 [grpc-default-executor-131] ERROR
(ReplicateContainerCommandHandler.java:164) - Can't import the downloaded
container data id=8
java.io.EOFException
at
org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream.read(GzipCompressorInputStream.java:241)
at
org.apache.commons.compress.archivers.tar.TarBuffer.readBlock(TarBuffer.java:224)
at
org.apache.commons.compress.archivers.tar.TarBuffer.readRecord(TarBuffer.java:195)
at
org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:486)
at
org.apache.commons.compress.archivers.tar.TarArchiveInputStream.skip(TarArchiveInputStream.java:182)
at
org.apache.commons.compress.archivers.tar.TarArchiveInputStream.getNextTarEntry(TarArchiveInputStream.java:220)
at
org.apache.hadoop.ozone.container.keyvalue.TarContainerPacker.unpackContainerDescriptor(TarContainerPacker.java:200)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.ReplicateContainerCommandHandler.importContainer(ReplicateContainerCommandHandler.java:144)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.ReplicateContainerCommandHandler.lambda$handle$0(ReplicateContainerCommandHandler.java:121)
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:656)
at
java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.apache.hadoop.ozone.container.replication.GrpcReplicationClient$StreamDownloader.onCompleted(GrpcReplicationClient.java:160)
at
org.apache.ratis.shaded.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:418)
at
org.apache.ratis.shaded.io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at
org.apache.ratis.shaded.io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at
org.apache.ratis.shaded.io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at
org.apache.ratis.shaded.io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
at
org.apache.ratis.shaded.io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at
org.apache.ratis.shaded.io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at
org.apache.ratis.shaded.io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at
org.apache.ratis.shaded.io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:403)
at
org.apache.ratis.shaded.io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:459)
at
org.apache.ratis.shaded.io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
at
org.apache.ratis.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:546)
at
org.apache.ratis.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:467)
at
org.apache.ratis.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:584)
at
org.apache.ratis.shaded.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at
org.apache.ratis.shaded.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-461) container remains in CLOSING state forever
Nilotpal Nandi created HDDS-461:
---
Summary: container remains in CLOSING state forever
Key: HDDS-461
URL: https://issues.apache.org/jira/browse/HDDS-461
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: SCM
Affects Versions: 0.2.1
Reporter: Nilotpal Nandi
Container id # 13's state is not changing from CLOSING to CLOSED.
{noformat}
[root@ctr-e138-1518143905142-459606-01-02 bin]# ./ozone scmcli info 13
raft.rpc.type = GRPC (default)
raft.grpc.message.size.max = 33554432 (custom)
raft.client.rpc.retryInterval = 300 ms (default)
raft.client.async.outstanding-requests.max = 100 (default)
raft.client.async.scheduler-threads = 3 (default)
raft.grpc.flow.control.window = 1MB (=1048576) (default)
raft.grpc.message.size.max = 33554432 (custom)
raft.client.rpc.request.timeout = 3000 ms (default)
Container id: 13
Container State: OPEN
Container Path:
/tmp/hadoop-root/dfs/data/hdds/de0a9e01-4a12-40e3-b567-51b9bd83248e/current/containerDir0/13/metadata
Container Metadata:
LeaderID: ctr-e138-1518143905142-459606-01-03.hwx.site
Datanodes:
[ctr-e138-1518143905142-459606-01-07.hwx.site,ctr-e138-1518143905142-459606-01-08.hwx.site,ctr-e138-1518143905142-459606-01-03.hwx.site]{noformat}
snippet of scmcli list :
{noformat}
{
"state" : "CLOSING",
"replicationFactor" : "THREE",
"replicationType" : "RATIS",
"allocatedBytes" : 4831838208,
"usedBytes" : 4831838208,
"numberOfKeys" : 0,
"lastUsed" : 4391827471,
"stateEnterTime" : 5435591457,
"owner" : "f8332db1-b8b1-4077-a9ea-097033d074b7",
"containerID" : 13,
"deleteTransactionId" : 0,
"containerOpen" : true
}{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-478) ozone Manager startup message is showing in ozone.log
Nilotpal Nandi created HDDS-478:
---
Summary: ozone Manager startup message is showing in ozone.log
Key: HDDS-478
URL: https://issues.apache.org/jira/browse/HDDS-478
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Manager
Affects Versions: 0.2.1
Reporter: Nilotpal Nandi
Please find the snippet below. Ozone Manager startup message is showing up in
ozone.log :
{noformat}
[root@ctr-e138-1518143905142-468367-01-02 logs]# cat ozone.log.2018-09-16 |
head -20
2018-09-16 05:29:59,638 [main] INFO (LogAdapter.java:51) - STARTUP_MSG:
/
STARTUP_MSG: Starting OzoneManager
STARTUP_MSG: host =
ctr-e138-1518143905142-468367-01-02.hwx.site/172.27.68.129
STARTUP_MSG: args = [-createObjectStore]
STARTUP_MSG: version = 3.2.0-SNAPSHOT
STARTUP_MSG: classpath =
/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/etc/hadoop:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-json-1.19.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jetty-webapp-9.3.19.v20170502.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/netty-3.10.5.Final.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-servlet-1.19.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/token-provider-1.0.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerby-util-1.0.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-lang3-3.7.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/curator-recipes-2.12.0.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-math3-3.1.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/xz-1.0.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-databind-2.9.5.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/guava-11.0.2.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-cli-1.2.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/hadoop-annotations-3.2.0-SNAPSHOT.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-collections-3.2.2.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/json-smart-2.3.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-codec-1.11.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/curator-framework-2.12.0.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/gson-2.2.4.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jetty-server-9.3.19.v20170502.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jetty-security-9.3.19.v20170502.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsp-api-2.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/log4j-1.2.17.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-core-1.19.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/curator-client-2.12.0.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/stax2-api-3.1.4.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jetty-servlet-9.3.19.v20170502.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-common-1.0.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-core-1.0.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-net-3.6.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-logging-1.1.3.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jetty-http-9.3.19.v20170502.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/root/hadoop_trunk/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsch-0.1.54.
[jira] [Created] (HDDS-479) Add more ozone fs tests in the robot integration framework
Nilotpal Nandi created HDDS-479: --- Summary: Add more ozone fs tests in the robot integration framework Key: HDDS-479 URL: https://issues.apache.org/jira/browse/HDDS-479 Project: Hadoop Distributed Data Store Issue Type: Test Reporter: Nilotpal Nandi Assignee: Nilotpal Nandi Fix For: 0.2.1 Currently , we have few number of ozone fs tests in robot integration framework. Need to add more. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-482) NullPointer exception thrown on console when cli operation failed
Nilotpal Nandi created HDDS-482:
---
Summary: NullPointer exception thrown on console when cli
operation failed
Key: HDDS-482
URL: https://issues.apache.org/jira/browse/HDDS-482
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone CLI
Affects Versions: 0.2.1
Reporter: Nilotpal Nandi
Assignee: Nanda kumar
steps taken :
--
# running putKey from cli.
# closed all containers.
# Error thrown on cli -
{noformat}
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.hdds.cli.GenericCli.printError(GenericCli.java:68)
at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:54)
at org.apache.hadoop.ozone.web.ozShell.Shell.main(Shell.java:77)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-493) ozone fs rmdir not working
Nilotpal Nandi created HDDS-493:
---
Summary: ozone fs rmdir not working
Key: HDDS-493
URL: https://issues.apache.org/jira/browse/HDDS-493
Project: Hadoop Distributed Data Store
Issue Type: Bug
Affects Versions: 0.2.1
Reporter: Nilotpal Nandi
ozone fs rmdir not working, also it does not throw any error on console.
{noformat}
[root@ctr-e138-1518143905142-468367-01-02 ~]# ozone fs -ls /TESTDIR3
2018-09-18 07:27:36,275 [main] DEBUG - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2018-09-18 07:27:37,103 [main] DEBUG - Call: public abstract
org.apache.hadoop.ozone.client.OzoneVolume
org.apache.hadoop.ozone.client.protocol.ClientProtocol.getVolumeDetails(java.lang.String)
throws java.io.IOException took 29 ms
2018-09-18 07:27:37,300 [shutdown-hook-0] DEBUG - Call: public abstract void
org.apache.hadoop.ozone.client.protocol.ClientProtocol.close() throws
java.io.IOException took 2 ms
[root@ctr-e138-1518143905142-468367-01-02 ~]#
[root@ctr-e138-1518143905142-468367-01-02 ~]#
[root@ctr-e138-1518143905142-468367-01-02 ~]# ozone fs -rmdir /TESTDIR3
2018-09-18 07:27:51,320 [main] DEBUG - Using
org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2018-09-18 07:27:52,108 [main] DEBUG - Call: public abstract
org.apache.hadoop.ozone.client.OzoneVolume
org.apache.hadoop.ozone.client.protocol.ClientProtocol.getVolumeDetails(java.lang.String)
throws java.io.IOException took 22 ms
2018-09-18 07:27:52,312 [shutdown-hook-0] DEBUG - Call: public abstract void
org.apache.hadoop.ozone.client.protocol.ClientProtocol.close() throws
java.io.IOException took 1 ms
{noformat}
ozone.log
-
{noformat}
2018-09-18 07:27:52,298 [IPC Server handler 2 on 9889] DEBUG
(KeyManagerImpl.java:342) - volume:fs-volume bucket:fs-bucket Key:TESTDIR3 not
found
2018-09-18 07:27:52,298 [IPC Server handler 2 on 9889] ERROR
(KeyManagerImpl.java:349) - Get key failed for volume:fs-volume
bucket:fs-bucket key:TESTDIR3
org.apache.hadoop.ozone.om.exceptions.OMException: Key not found
at org.apache.hadoop.ozone.om.KeyManagerImpl.lookupKey(KeyManagerImpl.java:344)
at org.apache.hadoop.ozone.om.OzoneManager.lookupKey(OzoneManager.java:853)
at
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.lookupKey(OzoneManagerProtocolServerSideTranslatorPB.java:384)
at
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java:35895)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
2018-09-18 07:27:52,305 [IPC Server handler 4 on 9889] DEBUG
(KeyManagerImpl.java:452) - Key TESTDIR3/ deleted from OM DB
2018-09-18 07:27:52,312 [shutdown-hook-0] DEBUG
(OzoneClientInvocationHandler.java:55) - Call: public abstract void
org.apache.hadoop.ozone.client.protocol.ClientProtocol.close() throws
java.io.IOException took 1 ms{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-492) Add more unit tests to ozonefs robot framework
[ https://issues.apache.org/jira/browse/HDDS-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nilotpal Nandi resolved HDDS-492. - Resolution: Duplicate > Add more unit tests to ozonefs robot framework > -- > > Key: HDDS-492 > URL: https://issues.apache.org/jira/browse/HDDS-492 > Project: Hadoop Distributed Data Store > Issue Type: Bug >Reporter: Namit Maheshwari >Assignee: Namit Maheshwari >Priority: Major > > Currently there are only couple of tests inside ozonefs.robot > We should add more unit tests for the same. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-493) ozone fs rmdir not working
[
https://issues.apache.org/jira/browse/HDDS-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nilotpal Nandi resolved HDDS-493.
-
Resolution: Not A Problem
> ozone fs rmdir not working
> --
>
> Key: HDDS-493
> URL: https://issues.apache.org/jira/browse/HDDS-493
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Affects Versions: 0.2.1
>Reporter: Nilotpal Nandi
>Assignee: Nilotpal Nandi
>Priority: Major
>
>
> ozone fs rmdir not working, also it does not throw any error on console.
> {noformat}
>
>
> [root@ctr-e138-1518143905142-468367-01-02 ~]# ozone fs -ls /TESTDIR3
> 2018-09-18 07:27:36,275 [main] DEBUG - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2018-09-18 07:27:37,103 [main] DEBUG - Call: public abstract
> org.apache.hadoop.ozone.client.OzoneVolume
> org.apache.hadoop.ozone.client.protocol.ClientProtocol.getVolumeDetails(java.lang.String)
> throws java.io.IOException took 29 ms
> 2018-09-18 07:27:37,300 [shutdown-hook-0] DEBUG - Call: public abstract void
> org.apache.hadoop.ozone.client.protocol.ClientProtocol.close() throws
> java.io.IOException took 2 ms
> [root@ctr-e138-1518143905142-468367-01-02 ~]#
> [root@ctr-e138-1518143905142-468367-01-02 ~]#
> [root@ctr-e138-1518143905142-468367-01-02 ~]# ozone fs -rmdir /TESTDIR3
> 2018-09-18 07:27:51,320 [main] DEBUG - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2018-09-18 07:27:52,108 [main] DEBUG - Call: public abstract
> org.apache.hadoop.ozone.client.OzoneVolume
> org.apache.hadoop.ozone.client.protocol.ClientProtocol.getVolumeDetails(java.lang.String)
> throws java.io.IOException took 22 ms
> 2018-09-18 07:27:52,312 [shutdown-hook-0] DEBUG - Call: public abstract void
> org.apache.hadoop.ozone.client.protocol.ClientProtocol.close() throws
> java.io.IOException took 1 ms
> {noformat}
>
>
> ozone.log
> -
> {noformat}
> 2018-09-18 07:27:52,298 [IPC Server handler 2 on 9889] DEBUG
> (KeyManagerImpl.java:342) - volume:fs-volume bucket:fs-bucket Key:TESTDIR3
> not found
> 2018-09-18 07:27:52,298 [IPC Server handler 2 on 9889] ERROR
> (KeyManagerImpl.java:349) - Get key failed for volume:fs-volume
> bucket:fs-bucket key:TESTDIR3
> org.apache.hadoop.ozone.om.exceptions.OMException: Key not found
> at
> org.apache.hadoop.ozone.om.KeyManagerImpl.lookupKey(KeyManagerImpl.java:344)
> at org.apache.hadoop.ozone.om.OzoneManager.lookupKey(OzoneManager.java:853)
> at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.lookupKey(OzoneManagerProtocolServerSideTranslatorPB.java:384)
> at
> org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java:35895)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
> 2018-09-18 07:27:52,305 [IPC Server handler 4 on 9889] DEBUG
> (KeyManagerImpl.java:452) - Key TESTDIR3/ deleted from OM DB
> 2018-09-18 07:27:52,312 [shutdown-hook-0] DEBUG
> (OzoneClientInvocationHandler.java:55) - Call: public abstract void
> org.apache.hadoop.ozone.client.protocol.ClientProtocol.close() throws
> java.io.IOException took 1 ms{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-494) client fails to write when all the containers are closed through scmcli
Nilotpal Nandi created HDDS-494: --- Summary: client fails to write when all the containers are closed through scmcli Key: HDDS-494 URL: https://issues.apache.org/jira/browse/HDDS-494 Project: Hadoop Distributed Data Store Issue Type: Bug Affects Versions: 0.2.1 Reporter: Nilotpal Nandi steps taken : -- # running putKey from cli. # closed all containers. # Client fails to write and throws error on cli (related to bug HDDS-482 ) Expectation : --- there should be retry and allocate new containers . -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-543) datanode service start throwing NullPointerException
Nilotpal Nandi created HDDS-543:
---
Summary: datanode service start throwing NullPointerException
Key: HDDS-543
URL: https://issues.apache.org/jira/browse/HDDS-543
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken :
---
# Stopped one of the nodes in 6 node cluster while write operation was
running.
# Tried to start datanode after sometime. Datanode failed to start
ozone version :
-
{noformat}
Source code repository g...@github.com:apache/hadoop.git -r
968082ffa5d9e50ed8538f653c375edd1b8feea5
Compiled by elek on 2018-09-19T20:57Z
Compiled with protoc 2.5.0
>From source with checksum efbdeabb5670d69d9efde85846e4ee98
Using HDDS 0.2.1-alpha
Source code repository g...@github.com:apache/hadoop.git -r
968082ffa5d9e50ed8538f653c375edd1b8feea5
Compiled by elek on 2018-09-19T20:56Z
Compiled with protoc 2.5.0
>From source with checksum 8bf78cff4b73c95d486da5b21053ef
{noformat}
console log :
{noformat}
2018-09-24 09:53:54,913 INFO impl.RaftServerImpl:
9348f4cb-0bd3-41e7-b1c6-c22463a20c09: set configuration 0:
[0ba4abe0-c8d1-4d6c-8d28-ebfa08fb84c9:172.27.18.18:9858,
3d6c8b2c-f1d2-4425-a577-b9ef5dc7c634:172.27.80.23:9858,
9348f4cb-0bd3-41e7-b1c6-c22463a20c09:172.27.23.161:9858], old=null at 0
Sep 24, 2018 9:53:55 AM
com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
2018-09-24 09:53:55,968 [main] INFO - Listening HDDS REST traffic on
/0.0.0.0:9880
2018-09-24 09:53:55,969 [main] INFO - Started plug-in
org.apache.hadoop.ozone.web.OzoneHddsDatanodeService@2fa3be26
2018-09-24 09:53:55,975 [Datanode State Machine Thread - 0] INFO - Attempting
to start container services.
2018-09-24 09:53:56,171 [Datanode State Machine Thread - 0] INFO - Starting
XceiverServerRatis 9348f4cb-0bd3-41e7-b1c6-c22463a20c09 at port 9858
2018-09-24 09:53:56,193 [Datanode State Machine Thread - 0] ERROR - Unable to
start the DatanodeState Machine
java.io.IOException: Premature EOF from inputStream
at org.apache.ratis.util.IOUtils.readFully(IOUtils.java:100)
at org.apache.ratis.server.storage.LogReader.decodeEntry(LogReader.java:250)
at org.apache.ratis.server.storage.LogReader.readEntry(LogReader.java:155)
at
org.apache.ratis.server.storage.LogInputStream.nextEntry(LogInputStream.java:128)
at
org.apache.ratis.server.storage.LogSegment.readSegmentFile(LogSegment.java:110)
at org.apache.ratis.server.storage.LogSegment.loadSegment(LogSegment.java:132)
at
org.apache.ratis.server.storage.RaftLogCache.loadSegment(RaftLogCache.java:110)
at
org.apache.ratis.server.storage.SegmentedRaftLog.loadLogSegments(SegmentedRaftLog.java:155)
at
org.apache.ratis.server.storage.SegmentedRaftLog.open(SegmentedRaftLog.java:123)
at org.apache.ratis.server.impl.ServerState.initLog(ServerState.java:162)
at org.apache.ratis.server.impl.ServerState.(ServerState.java:110)
at org.apache.ratis.server.impl.RaftServerImpl.(RaftServerImpl.java:106)
at
org.apache.ratis.server.impl.RaftServerProxy.lambda$newRaftServerImpl$0(RaftServerProxy.java:191)
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at
java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
2018-09-24 09:53:56,195 [main] ERROR - Exception in HddsDatanodeService.
java.lang.NullPointerException
at
org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.join(DatanodeStateMachine.java:332)
at
org.apache.hadoop.ozone.HddsDatanodeService.join(HddsDatanodeService.java:191)
at
org.apache.hadoop.ozone.HddsDatanodeService.main(HddsDatanodeService.java:250)
2018-09-24 09:53:56,198 INFO util.ExitUtil: Exiting with status 1:
java.lang.NullPointerException
2018-09-24 09:53:56,201 WARN fs.CachingGetSpaceUsed: Thread Interrupted waiting
to refresh disk information: sleep interrupted
2018-09-24 09:53:56,201 [shutdown-hook-0] INFO - SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down HddsDatanodeService at
ctr-e138-1518143905142-468367-01-03.hwx.site/172.27.23.161
{noformat}
datanode.log
--
{noformat}
2018-09-24 09:42:41,305 ERROR org.apache.ratis.server.impl.RaftServerImpl:
9348f4cb-0bd3-41e7-b1c6-c22463a20c09: applyTransaction failed for index:2
proto:(t:2, i:2)SMLOGENTRY, client-DBCE827E62E6, cid=1
2018-09-24 09:42:41,307 ERROR org.apache.ratis.server.impl.StateMachineUpdater:
Terminating with exit
[jira] [Created] (HDDS-545) NullPointerException error thrown while trying to close container
Nilotpal Nandi created HDDS-545:
---
Summary: NullPointerException error thrown while trying to close
container
Key: HDDS-545
URL: https://issues.apache.org/jira/browse/HDDS-545
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
Seen following null pointer in ozone.log while trying to close the container on
receiving SCM container close request'.
ozone version:
--
{noformat}
Source code repository g...@github.com:apache/hadoop.git -r
968082ffa5d9e50ed8538f653c375edd1b8feea5
Compiled by elek on 2018-09-19T20:57Z
Compiled with protoc 2.5.0
>From source with checksum efbdeabb5670d69d9efde85846e4ee98
Using HDDS 0.2.1-alpha
Source code repository g...@github.com:apache/hadoop.git -r
968082ffa5d9e50ed8538f653c375edd1b8feea5
Compiled by elek on 2018-09-19T20:56Z
Compiled with protoc 2.5.0
>From source with checksum 8bf78cff4b73c95d486da5b21053ef
{noformat}
ozone.log
{noformat}
2018-09-24 11:32:55,910 [Thread-2921] DEBUG (XceiverServerRatis.java:401) -
pipeline Action CLOSE on pipeline
pipelineId=eabdcbe2-da3b-41be-a281-f0ea8d4120f7.Reason :
7d1c7be2-7882-4446-be61-be868d2e188a is in candidate state for 1074164ms
2018-09-24 11:32:56,343 [Datanode State Machine Thread - 1] DEBUG
(HeartbeatEndpointTask.java:255) - Received SCM container close request for
container 54
2018-09-24 11:32:56,347 [Datanode State Machine Thread - 1] DEBUG
(HeartbeatEndpointTask.java:255) - Received SCM container close request for
container 42
2018-09-24 11:32:56,347 [Datanode State Machine Thread - 1] DEBUG
(HeartbeatEndpointTask.java:255) - Received SCM container close request for
container 44
2018-09-24 11:32:56,354 [Datanode State Machine Thread - 1] DEBUG
(HeartbeatEndpointTask.java:255) - Received SCM container close request for
container 46
2018-09-24 11:32:56,355 [Datanode State Machine Thread - 1] DEBUG
(HeartbeatEndpointTask.java:255) - Received SCM container close request for
container 48
2018-09-24 11:32:56,357 [Datanode State Machine Thread - 1] DEBUG
(HeartbeatEndpointTask.java:255) - Received SCM container close request for
container 50
2018-09-24 11:32:56,357 [Datanode State Machine Thread - 1] DEBUG
(HeartbeatEndpointTask.java:255) - Received SCM container close request for
container 52
2018-09-24 11:32:56,548 [Command processor thread] DEBUG
(CloseContainerCommandHandler.java:64) - Processing Close Container command.
2018-09-24 11:32:56,636 [Command processor thread] ERROR
(CloseContainerCommandHandler.java:105) - Can't close container 54
java.lang.NullPointerException
at
org.apache.hadoop.ozone.container.ozoneimpl.OzoneContainer.submitContainerRequest(OzoneContainer.java:192)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler.handle(CloseContainerCommandHandler.java:91)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CommandDispatcher.handle(CommandDispatcher.java:93)
at
org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$initCommandHandlerThread$1(DatanodeStateMachine.java:382)
at java.lang.Thread.run(Thread.java:745)
2018-09-24 11:32:56,726 [Command processor thread] DEBUG
(CloseContainerCommandHandler.java:64) - Processing Close Container command.
2018-09-24 11:32:56,728 [Command processor thread] ERROR
(CloseContainerCommandHandler.java:105) - Can't close container 42
java.lang.NullPointerException
at
org.apache.hadoop.ozone.container.ozoneimpl.OzoneContainer.submitContainerRequest(OzoneContainer.java:192)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler.handle(CloseContainerCommandHandler.java:91)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CommandDispatcher.handle(CommandDispatcher.java:93)
at
org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$initCommandHandlerThread$1(DatanodeStateMachine.java:382)
at java.lang.Thread.run(Thread.java:745)
2018-09-24 11:32:56,787 [Command processor thread] DEBUG
(CloseContainerCommandHandler.java:64) - Processing Close Container command.
2018-09-24 11:32:56,814 [Command processor thread] ERROR
(CloseContainerCommandHandler.java:105) - Can't close container 44
java.lang.NullPointerException
at
org.apache.hadoop.ozone.container.ozoneimpl.OzoneContainer.submitContainerRequest(OzoneContainer.java:192)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler.handle(CloseContainerCommandHandler.java:91)
at
org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CommandDispatcher.handle(CommandDispatcher.java:93)
at
org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$initCommandHandlerThread$1(DatanodeStateMachine.java:382)
at java.lang.Thread.run(Thread.java:745)
2
[jira] [Created] (HDDS-553) copyFromLocal subcommand failed
Nilotpal Nandi created HDDS-553:
---
Summary: copyFromLocal subcommand failed
Key: HDDS-553
URL: https://issues.apache.org/jira/browse/HDDS-553
Project: Hadoop Distributed Data Store
Issue Type: Bug
Affects Versions: 0.3.0
Reporter: Nilotpal Nandi
copyFromLocal subcommand is throwing error when run from docker container :
{noformat}
hadoop@bf5c576c7556:~$ ozone fs -copyFromLocal /etc/passwd
o3://bucket1.fstest/testdir/deep/
2018-09-25 18:59:24 ERROR ChunkGroupOutputStream:274 - Try to allocate more
blocks for write failed, already allocated 0 blocks for this write.
copyFromLocal: Allocate block failed, error:INTERNAL_ERROR
{noformat}
ozone version
{noformat}
Source code repository git://git.apache.org/hadoop.git -r
26c94a0fd07dab40fb5e62614ae71d98110a7467
Compiled by nnandi on 2018-09-25T18:30Z
Compiled with protoc 2.5.0
>From source with checksum 1703a7974737c3160f49de7ef4f435b
Using HDDS 0.3.0-SNAPSHOT
Source code repository git://git.apache.org/hadoop.git -r
26c94a0fd07dab40fb5e62614ae71d98110a7467
Compiled by nnandi on 2018-09-25T18:29Z
Compiled with protoc 2.5.0
>From source with checksum 1785735bba1697df364730122de197b
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-553) copyFromLocal subcommand failed
[
https://issues.apache.org/jira/browse/HDDS-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nilotpal Nandi resolved HDDS-553.
-
Resolution: Not A Problem
> copyFromLocal subcommand failed
> ---
>
> Key: HDDS-553
> URL: https://issues.apache.org/jira/browse/HDDS-553
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Affects Versions: 0.3.0
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
>
> copyFromLocal subcommand is throwing error when run from docker container :
>
> {noformat}
> hadoop@bf5c576c7556:~$ ozone fs -copyFromLocal /etc/passwd
> o3://bucket1.fstest/testdir/deep/
> 2018-09-25 18:59:24 ERROR ChunkGroupOutputStream:274 - Try to allocate more
> blocks for write failed, already allocated 0 blocks for this write.
> copyFromLocal: Allocate block failed, error:INTERNAL_ERROR
> {noformat}
>
> ozone version
>
> {noformat}
> Source code repository git://git.apache.org/hadoop.git -r
> 26c94a0fd07dab40fb5e62614ae71d98110a7467
> Compiled by nnandi on 2018-09-25T18:30Z
> Compiled with protoc 2.5.0
> From source with checksum 1703a7974737c3160f49de7ef4f435b
> Using HDDS 0.3.0-SNAPSHOT
> Source code repository git://git.apache.org/hadoop.git -r
> 26c94a0fd07dab40fb5e62614ae71d98110a7467
> Compiled by nnandi on 2018-09-25T18:29Z
> Compiled with protoc 2.5.0
> From source with checksum 1785735bba1697df364730122de197b
> {noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
