[jira] [Updated] (HDDS-847) TestBlockDeletion is failing
[
https://issues.apache.org/jira/browse/HDDS-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lokesh Jain updated HDDS-847:
-
Status: Patch Available (was: Open)
> TestBlockDeletion is failing
>
>
> Key: HDDS-847
> URL: https://issues.apache.org/jira/browse/HDDS-847
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: test
>Reporter: Nanda kumar
>Assignee: Lokesh Jain
>Priority: Major
> Attachments: HDDS-847.001.patch
>
>
> {{TestBlockDeletion}} is failing with the below exception
> {code}
> [ERROR]
> testBlockDeletion(org.apache.hadoop.ozone.container.common.statemachine.commandhandler.TestBlockDeletion)
> Time elapsed: 28.017 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.ozone.container.common.statemachine.commandhandler.TestBlockDeletion.testBlockDeletion(TestBlockDeletion.java:165)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-847) TestBlockDeletion is failing
[
https://issues.apache.org/jira/browse/HDDS-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lokesh Jain updated HDDS-847:
-
Attachment: HDDS-847.001.patch
> TestBlockDeletion is failing
>
>
> Key: HDDS-847
> URL: https://issues.apache.org/jira/browse/HDDS-847
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: test
>Reporter: Nanda kumar
>Assignee: Lokesh Jain
>Priority: Major
> Attachments: HDDS-847.001.patch
>
>
> {{TestBlockDeletion}} is failing with the below exception
> {code}
> [ERROR]
> testBlockDeletion(org.apache.hadoop.ozone.container.common.statemachine.commandhandler.TestBlockDeletion)
> Time elapsed: 28.017 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.ozone.container.common.statemachine.commandhandler.TestBlockDeletion.testBlockDeletion(TestBlockDeletion.java:165)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-718) Introduce new SCM Commands to list and close Pipelines
[ https://issues.apache.org/jira/browse/HDDS-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691324#comment-16691324 ] Mukul Kumar Singh commented on HDDS-718: Thanks for working on this [~ljain]. The patch looks really good to me. Some minor comments. 1) Pipeline.java:191, there are two toString methods, lets keep one of the methods. 2) PipelineStateMap.java:122, lets return a collections here and not construct a new ArrayList. > Introduce new SCM Commands to list and close Pipelines > -- > > Key: HDDS-718 > URL: https://issues.apache.org/jira/browse/HDDS-718 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Nanda kumar >Assignee: Lokesh Jain >Priority: Blocker > Attachments: HDDS-718.001.patch, HDDS-718.002.patch > > > We need to have a tear-down pipeline command in SCM so that an administrator > can close/destroy a pipeline in the cluster. > HDDS-695 brings in the commands in branch ozone-0.3, this Jira is for porting > them to trunk. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-718) Introduce new SCM Commands to list and close Pipelines
[ https://issues.apache.org/jira/browse/HDDS-718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-718: - Status: Patch Available (was: Open) > Introduce new SCM Commands to list and close Pipelines > -- > > Key: HDDS-718 > URL: https://issues.apache.org/jira/browse/HDDS-718 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Nanda kumar >Assignee: Lokesh Jain >Priority: Blocker > Attachments: HDDS-718.001.patch, HDDS-718.002.patch > > > We need to have a tear-down pipeline command in SCM so that an administrator > can close/destroy a pipeline in the cluster. > HDDS-695 brings in the commands in branch ozone-0.3, this Jira is for porting > them to trunk. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-844) Add logic for pipeline teardown after timeout
[ https://issues.apache.org/jira/browse/HDDS-844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691299#comment-16691299 ] Hudson commented on HDDS-844: - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #15458 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15458/]) HDDS-844. Add logic for pipeline teardown after timeout. Contributed by (msingh: rev cfb915f3df43c84d3fb36eb2a430a25946b2ddb7) * (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ScmConfigKeys.java * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineReportHandler.java * (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestSCMPipelineManager.java * (edit) hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/ContainerOperationClient.java * (edit) hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientGrpc.java * (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineClose.java * (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineStateManager.java * (edit) hadoop-hdds/common/src/main/resources/ozone-default.xml * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineStateMap.java * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/StaleNodeHandler.java * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineActionHandler.java * (edit) hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientRatis.java * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java * (add) hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestFreonWithPipelineDestroy.java * (add) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineUtils.java * (edit) hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/TestUtils.java * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java * (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientSpi.java > Add logic for pipeline teardown after timeout > - > > Key: HDDS-844 > URL: https://issues.apache.org/jira/browse/HDDS-844 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.4.0 >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Fix For: 0.4.0 > > Attachments: HDDS-844.001.patch, HDDS-844.002.patch, > HDDS-844.003.patch, HDDS-844.004.patch, HDDS-844.005.patch > > > On receiving pipeline action we close the pipeline and wait for all > containers to get closed. Currently pipeline is destroyed on datanodes only > after all the containers have been closed. There is a possibility for > containers to never get to CLOSED state if there is a two node failure. In > such scenarios the pipeline needs to be destroyed and removed from SCM after > a timeout. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-844) Add logic for pipeline teardown after timeout
[ https://issues.apache.org/jira/browse/HDDS-844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDDS-844: --- Resolution: Fixed Fix Version/s: 0.4.0 Status: Resolved (was: Patch Available) Thanks for the contribution [~ljain]. I have committed this to trunk. > Add logic for pipeline teardown after timeout > - > > Key: HDDS-844 > URL: https://issues.apache.org/jira/browse/HDDS-844 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.4.0 >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Fix For: 0.4.0 > > Attachments: HDDS-844.001.patch, HDDS-844.002.patch, > HDDS-844.003.patch, HDDS-844.004.patch, HDDS-844.005.patch > > > On receiving pipeline action we close the pipeline and wait for all > containers to get closed. Currently pipeline is destroyed on datanodes only > after all the containers have been closed. There is a possibility for > containers to never get to CLOSED state if there is a two node failure. In > such scenarios the pipeline needs to be destroyed and removed from SCM after > a timeout. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14087) RBF : In Router UI NameNode heartbeat printing the negative values
[ https://issues.apache.org/jira/browse/HDFS-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harshakiran Reddy updated HDFS-14087: - Labels: RBF (was: ) > RBF : In Router UI NameNode heartbeat printing the negative values > --- > > Key: HDFS-14087 > URL: https://issues.apache.org/jira/browse/HDFS-14087 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.1.1 >Reporter: Harshakiran Reddy >Assignee: Ranith Sardar >Priority: Major > Labels: RBF > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-14087) RBF : In Router UI NameNode heartbeat printing the negative values
Harshakiran Reddy created HDFS-14087: Summary: RBF : In Router UI NameNode heartbeat printing the negative values Key: HDFS-14087 URL: https://issues.apache.org/jira/browse/HDFS-14087 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.1.1 Reporter: Harshakiran Reddy -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-14087) RBF : In Router UI NameNode heartbeat printing the negative values
[ https://issues.apache.org/jira/browse/HDFS-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ranith Sardar reassigned HDFS-14087: Assignee: Ranith Sardar > RBF : In Router UI NameNode heartbeat printing the negative values > --- > > Key: HDFS-14087 > URL: https://issues.apache.org/jira/browse/HDFS-14087 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.1.1 >Reporter: Harshakiran Reddy >Assignee: Ranith Sardar >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14075) NPE while Edit Logging
[
https://issues.apache.org/jira/browse/HDFS-14075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691283#comment-16691283
]
Ayush Saxena commented on HDFS-14075:
-
Thanx [~elgoiri] for the comment.
{quote}Can we avoid setJournalForTests() and just use
Whitebox.setInternalState()?
{quote}
I guess use of it is deprecated. HDFS-13684 tracks to remove all existing ones
too.
> NPE while Edit Logging
> --
>
> Key: HDFS-14075
> URL: https://issues.apache.org/jira/browse/HDFS-14075
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Critical
> Attachments: HDFS-14075-01.patch, HDFS-14075-02.patch,
> HDFS-14075-03.patch, HDFS-14075-04.patch
>
>
> {noformat}
> 2018-11-10 18:59:38,427 FATAL
> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Exception while edit
> logging: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.doEditTransaction(FSEditLog.java:481)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync$Edit.logEdit(FSEditLogAsync.java:288)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.run(FSEditLogAsync.java:232)
> at java.lang.Thread.run(Thread.java:745)
> 2018-11-10 18:59:38,532 INFO org.apache.hadoop.util.ExitUtil: Exiting with
> status 1: Exception while edit logging: null
> 2018-11-10 18:59:38,552 INFO org.apache.hadoop.hdfs.server.namenode.NameNode:
> SHUTDOWN_MSG:
> {noformat}
> Before NPE Received the following Exception
> {noformat}
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 65110, call
> Call#23241 Retry#0
> org.apache.hadoop.hdfs.server.protocol.NamenodeProtocol.rollEditLog from
>
> java.io.IOException: Unable to start log segment 7964819: too few journals
> successfully started.
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegment(FSEditLog.java:1385)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegmentAndWriteHeaderTxn(FSEditLog.java:1395)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.rollEditLog(FSEditLog.java:1319)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.rollEditLog(FSImage.java:1352)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.rollEditLog(FSNamesystem.java:4669)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.rollEditLog(NameNodeRpcServer.java:1293)
> at
> org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolServerSideTranslatorPB.rollEditLog(NamenodeProtocolServerSideTranslatorPB.java:146)
> at
> org.apache.hadoop.hdfs.protocol.proto.NamenodeProtocolProtos$NamenodeProtocolService$2.callBlockingMethod(NamenodeProtocolProtos.java:12974)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> Caused by: java.io.IOException: starting log segment 7964819 failed for too
> many journals
> at
> org.apache.hadoop.hdfs.server.namenode.JournalSet.mapJournalsAndReportErrors(JournalSet.java:412)
> at
> org.apache.hadoop.hdfs.server.namenode.JournalSet.startLogSegment(JournalSet.java:207)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegment(FSEditLog.java:1383)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDDS-849) fix NPE in TestKeyValueHandler because of audit log write
[
https://issues.apache.org/jira/browse/HDDS-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Chitlangia reassigned HDDS-849:
--
Assignee: Dinesh Chitlangia
> fix NPE in TestKeyValueHandler because of audit log write
> -
>
> Key: HDDS-849
> URL: https://issues.apache.org/jira/browse/HDDS-849
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Datanode
>Affects Versions: 0.4.0
>Reporter: Mukul Kumar Singh
>Assignee: Dinesh Chitlangia
>Priority: Major
> Fix For: 0.4.0
>
>
> TestKeyValueHandler#testCloseInvalidContainer and
> TestKeyValueHandler#testHandlerCommandHandling are failing because of the
> following exception.
> {code}
> [ERROR]
> testCloseInvalidContainer(org.apache.hadoop.ozone.container.keyvalue.TestKeyValueHandler)
> Time elapsed: 0.006 s <<< ERROR!
> java.lang.NullPointerException
> at
> org.apache.hadoop.ozone.audit.AuditLogger.logWriteFailure(AuditLogger.java:64)
> at
> org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.audit(HddsDispatcher.java:433)
> at
> org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.dispatch(HddsDispatcher.java:242)
> at
> org.apache.hadoop.ozone.container.keyvalue.TestKeyValueHandler.testCloseInvalidContainer(TestKeyValueHandler.java:282)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-849) fix NPE in TestKeyValueHandler because of audit log write
Mukul Kumar Singh created HDDS-849:
--
Summary: fix NPE in TestKeyValueHandler because of audit log write
Key: HDDS-849
URL: https://issues.apache.org/jira/browse/HDDS-849
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Datanode
Affects Versions: 0.4.0
Reporter: Mukul Kumar Singh
Fix For: 0.4.0
TestKeyValueHandler#testCloseInvalidContainer and
TestKeyValueHandler#testHandlerCommandHandling are failing because of the
following exception.
{code}
[ERROR]
testCloseInvalidContainer(org.apache.hadoop.ozone.container.keyvalue.TestKeyValueHandler)
Time elapsed: 0.006 s <<< ERROR!
java.lang.NullPointerException
at
org.apache.hadoop.ozone.audit.AuditLogger.logWriteFailure(AuditLogger.java:64)
at
org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.audit(HddsDispatcher.java:433)
at
org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.dispatch(HddsDispatcher.java:242)
at
org.apache.hadoop.ozone.container.keyvalue.TestKeyValueHandler.testCloseInvalidContainer(TestKeyValueHandler.java:282)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14079) RBF: RouterAdmin should have failover concept for router
[
https://issues.apache.org/jira/browse/HDFS-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691273#comment-16691273
]
Surendra Singh Lilhore commented on HDFS-14079:
---
Thansk [~crh] for working on this..
Pls can you upload your patch here..
> RBF: RouterAdmin should have failover concept for router
>
>
> Key: HDFS-14079
> URL: https://issues.apache.org/jira/browse/HDFS-14079
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Affects Versions: 3.1.1
>Reporter: Surendra Singh Lilhore
>Assignee: Surendra Singh Lilhore
>Priority: Major
>
> Currenlty {{RouterAdmin}} connect with only one router for admin operation,
> if the configured router is down then router admin command is failing. It
> should allow to configure all the router admin address.
> {code}
> // Initialize RouterClient
> try {
> String address = getConf().getTrimmed(
> RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_KEY,
> RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_DEFAULT);
> InetSocketAddress routerSocket = NetUtils.createSocketAddr(address);
> client = new RouterClient(routerSocket, getConf());
> } catch (RPC.VersionMismatch v) {
> System.err.println(
> "Version mismatch between client and server... command aborted");
> return exitCode;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14011) RBF: Add more information to HdfsFileStatus for a mount point
[ https://issues.apache.org/jira/browse/HDFS-14011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691263#comment-16691263 ] Surendra Singh Lilhore commented on HDFS-14011: --- [~ajisakaa] I feel the fix is wrong. Pls check the comment in HDFS-14085. "hdfs dfs -ls /" should display the filesystem permission instead of mount table permssion. Please correct me if I am wrong > RBF: Add more information to HdfsFileStatus for a mount point > - > > Key: HDFS-14011 > URL: https://issues.apache.org/jira/browse/HDFS-14011 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > Fix For: HDFS-13891 > > Attachments: HDFS-14011.01.patch, HDFS-14011.02.patch, > HDFS-14011.03.patch > > > RouterClientProtocol#getMountPointStatus does not use information of the > mount point, therefore, 'hdfs dfs -ls' to a directory including mount point > returns the incorrect information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDDS-718) Introduce new SCM Commands to list and close Pipelines
[ https://issues.apache.org/jira/browse/HDDS-718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar reassigned HDDS-718: Assignee: Lokesh Jain (was: Nanda kumar) > Introduce new SCM Commands to list and close Pipelines > -- > > Key: HDDS-718 > URL: https://issues.apache.org/jira/browse/HDDS-718 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Nanda kumar >Assignee: Lokesh Jain >Priority: Blocker > Attachments: HDDS-718.001.patch, HDDS-718.002.patch > > > We need to have a tear-down pipeline command in SCM so that an administrator > can close/destroy a pipeline in the cluster. > HDDS-695 brings in the commands in branch ozone-0.3, this Jira is for porting > them to trunk. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-817) Create SCM metrics for disk from node report
[
https://issues.apache.org/jira/browse/HDDS-817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691254#comment-16691254
]
Yiqun Lin commented on HDDS-817:
Hi [~bharatviswa], almostly looks great to me, some comments for the patch:
The logic for verifyEqualsLong and verifyEqualsInt is completely same. We can
reuse the logic. So I mean we will do a transformation for {{Map nodeCount}}, to the type {{Map}}.
Another point, the logic of verification can be improved. We can remove the
entry when finishing each comparison. Then we verify if the map is empty. So
that we can ensure the actual data is completely same with expected data.
{code:java}
private void verifyEqualsLong(TabularData actualData, Map
expectedData) {
if (actualData == null || expectedData == null) {
fail("Data should not be null.");
}
for (Object obj : actualData.values()) {
...
assertEquals(expectedData.remove(key).longValue(), num);
}
assertTrue(expectedData.isEmpty());
}
{code}
BTW, can you update the description of this JIRA and do removal of following.
Just makes me a little confused to find related change.
{noformat}
Total no of datanodes in cluster (Running, Unhealthy, Failed)
{noformat}
> Create SCM metrics for disk from node report
>
>
> Key: HDDS-817
> URL: https://issues.apache.org/jira/browse/HDDS-817
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Attachments: HDDS-817.00.patch
>
>
> # Disk usage HDD and SSD
> # Total no of datanodes in cluster (Running, Unhealthy, Failed)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-14086) Failure in test_libhdfs_ops
[ https://issues.apache.org/jira/browse/HDFS-14086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HDFS-14086: --- Assignee: (was: Ayush Saxena) > Failure in test_libhdfs_ops > --- > > Key: HDFS-14086 > URL: https://issues.apache.org/jira/browse/HDFS-14086 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Pranay Singh >Priority: Major > > test_libhdfs_ops hdfs_static test was not getting executed, the issue that I > have fixed in HDFS-14083 is > seen because this test program is not getting executed so I had to change the > below file to > execute this test binary as a part of normal run. There are some failures > that are seen when this > test program is run. This jira tracks those failures. > Details of change to enable this test > > hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt > add_libhdfs_test(test_libhdfs_ops hdfs_static) ---> > Failures that are seen when this test is run. > - > Name: file:/tmp/hsperfdata_root, Type: D, Replication: 1, BlockSize: > 33554432, Size: 0, LastMod: Tue Nov 13 18:03:20 2018 > Owner: root, Group: root, Permissions: 493 (rwxr-xr-x) > hdfsGetHosts - SUCCESS! ... > hdfsChown(path=/tmp/testfile.txt, owner=(null), group=users): > FileSystem#setOwner error: > Shell.ExitCodeException: chown: changing group of '/tmp/testfile.txt': > Operation not permitted > ExitCodeException exitCode=1: chown: changing group of '/tmp/testfile.txt': > Operation not permitted > at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008) > at org.apache.hadoop.util.Shell.run(Shell.java:901) > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:1307) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:1289) > at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1350) > at org.apache.hadoop.fs.FileUtil.setOwner(FileUtil.java:1152) > at > org.apache.hadoop.fs.RawLocalFileSystem.setOwner(RawLocalFileSystem.java:851) > at > org.apache.hadoop.fs.ChecksumFileSystem$2.apply(ChecksumFileSystem.java:520) > at > org.apache.hadoop.fs.ChecksumFileSystem$FsOperation.run(ChecksumFileSystem.java:489) > at > org.apache.hadoop.fs.ChecksumFileSystem.setOwner(ChecksumFileSystem.java:523) > hdfsChown: Failed! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-14086) Failure in test_libhdfs_ops
[ https://issues.apache.org/jira/browse/HDFS-14086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HDFS-14086: --- Assignee: Ayush Saxena > Failure in test_libhdfs_ops > --- > > Key: HDFS-14086 > URL: https://issues.apache.org/jira/browse/HDFS-14086 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Pranay Singh >Assignee: Ayush Saxena >Priority: Major > > test_libhdfs_ops hdfs_static test was not getting executed, the issue that I > have fixed in HDFS-14083 is > seen because this test program is not getting executed so I had to change the > below file to > execute this test binary as a part of normal run. There are some failures > that are seen when this > test program is run. This jira tracks those failures. > Details of change to enable this test > > hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt > add_libhdfs_test(test_libhdfs_ops hdfs_static) ---> > Failures that are seen when this test is run. > - > Name: file:/tmp/hsperfdata_root, Type: D, Replication: 1, BlockSize: > 33554432, Size: 0, LastMod: Tue Nov 13 18:03:20 2018 > Owner: root, Group: root, Permissions: 493 (rwxr-xr-x) > hdfsGetHosts - SUCCESS! ... > hdfsChown(path=/tmp/testfile.txt, owner=(null), group=users): > FileSystem#setOwner error: > Shell.ExitCodeException: chown: changing group of '/tmp/testfile.txt': > Operation not permitted > ExitCodeException exitCode=1: chown: changing group of '/tmp/testfile.txt': > Operation not permitted > at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008) > at org.apache.hadoop.util.Shell.run(Shell.java:901) > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:1307) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:1289) > at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1350) > at org.apache.hadoop.fs.FileUtil.setOwner(FileUtil.java:1152) > at > org.apache.hadoop.fs.RawLocalFileSystem.setOwner(RawLocalFileSystem.java:851) > at > org.apache.hadoop.fs.ChecksumFileSystem$2.apply(ChecksumFileSystem.java:520) > at > org.apache.hadoop.fs.ChecksumFileSystem$FsOperation.run(ChecksumFileSystem.java:489) > at > org.apache.hadoop.fs.ChecksumFileSystem.setOwner(ChecksumFileSystem.java:523) > hdfsChown: Failed! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-845) Create a new raftClient instance for every watch request for Ratis
[ https://issues.apache.org/jira/browse/HDDS-845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691201#comment-16691201 ] Mukul Kumar Singh commented on HDDS-845: Thanks for updating the patch [~shashikant]. +1, for v1 patch. > Create a new raftClient instance for every watch request for Ratis > -- > > Key: HDDS-845 > URL: https://issues.apache.org/jira/browse/HDDS-845 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Client >Affects Versions: 0.4.0 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDDS-845.000.patch, HDDS-845.001.patch > > > Currently , watch request go throw sliding window in ratis and hence block as > well as get blocked for other requests submitted before . These are read only > requests and not necessarily require to go throw the sliding window, Until > this gets addressed in Ratis, its better and efficient to create a new raft > Client instance for watch request in XceiverClientRatis. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-844) Add logic for pipeline teardown after timeout
[ https://issues.apache.org/jira/browse/HDDS-844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691200#comment-16691200 ] Mukul Kumar Singh commented on HDDS-844: Thanks for updating the patch [~ljain]. +1, the v5 patch looks good to me. There are some checktyle issues which I will take care of while committing this patch. > Add logic for pipeline teardown after timeout > - > > Key: HDDS-844 > URL: https://issues.apache.org/jira/browse/HDDS-844 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.4.0 >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Attachments: HDDS-844.001.patch, HDDS-844.002.patch, > HDDS-844.003.patch, HDDS-844.004.patch, HDDS-844.005.patch > > > On receiving pipeline action we close the pipeline and wait for all > containers to get closed. Currently pipeline is destroyed on datanodes only > after all the containers have been closed. There is a possibility for > containers to never get to CLOSED state if there is a two node failure. In > such scenarios the pipeline needs to be destroyed and removed from SCM after > a timeout. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-816) Create OM metrics for bucket, volume, keys
[ https://issues.apache.org/jira/browse/HDDS-816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691192#comment-16691192 ] Yiqun Lin commented on HDDS-816: Thanks [~bharatviswa]. LGTM, +1. Let's see others if have comments for this, :) > Create OM metrics for bucket, volume, keys > -- > > Key: HDDS-816 > URL: https://issues.apache.org/jira/browse/HDDS-816 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Manager >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Attachments: HDDS-816.00.patch, HDDS-816.01.patch, HDDS-816.03.patch, > HDDS-816.04.patch, HDDS-816.05.patch, HDDS-816.06.patch, Metrics for number > of volumes, buckets, keys.pdf, Proposed Approach.pdf > > > This Jira is used to create the following metrics in Ozone manager. > # number of volumes > # number of buckets > # number of keys -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13923) Add a configuration to turn on/off observer reads
[
https://issues.apache.org/jira/browse/HDFS-13923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691154#comment-16691154
]
xiangheng commented on HDFS-13923:
--
Hey [~csun],that's a intreasting ideas,I'm studying this question.if you like,
please assign this question to me,i will try to solve this problem,thank you
very much.
> Add a configuration to turn on/off observer reads
> -
>
> Key: HDFS-13923
> URL: https://issues.apache.org/jira/browse/HDFS-13923
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chao Sun
>Priority: Major
>
> In some situations having a config to turn on/off observer reads dynamically
> may be useful. For instance, some applications may be sensitive for data
> freshness and want to always reach directly to active NN. In a more complex
> scenario, services such as Presto may want to apply observer reads for
> different types of queries. In this case, simply change
> {{dfs.client.failover.proxy.provider.}} may not be enough, since
> with FileSystem cache (which is usually turned on) will ignore the change and
> still use the same FileSystem object.
> Here I'm proposing to add a flag in {{HdfsClientConfigKeys}}, such as
> {{dfs.client.observer.reads.enabled}}, that can be used to dynamically turn
> on/off observer reads. The FileSystem cache key should also take account of
> this flag in its {{hashCode}} and {{equals}} impl, so that different
> FileSystem objects will be used depending on the flag.
>
> cc [~shv], [~xkrogen], [~vagarychen], [~zero45] for discussion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDDS-795) RocksDb specific classes leak from DBStore/Table interfaces
[
https://issues.apache.org/jira/browse/HDDS-795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691118#comment-16691118
]
Ajay Kumar edited comment on HDDS-795 at 11/18/18 10:27 PM:
[~elek] thanks for posting the patch. Could you please rebase the patch. v4
looks good, few NITs.
* DBStore
** L109: Rephrase "A init a holder" to "Initialize atomic batch operations." or
something similar?
** L114 missing throws clause?
** L120: typo "de"
* BatchOperation: Should this have api for commit op as well?
* Table
** L51: Rename new put operation to "addToBatch"
* Unused imports in VolumeManagerImpl.
* TestRDBTableStore
** L169: Shall we assert key doesn't exist before commit op?
{{Assert.asserNull(testTable.get(key));}}
** L190: Similarly assert if key exist before we commit the batch.
** Unused imports
was (Author: ajayydv):
[~elek] thanks for posting the patch. Could you please rebase the patch. v4
looks good, few NITs.
* DBStore
** L109: Rephrase "A init a holder" to "Initialize atomic batch operations." or
something similar?
** L114 missing throws clause?
** L120: typo "de"
* BatchOperation: Should this have api for commit op as well?
* Table
** L51: Rename new put operation to "addToBatch"
* Unused imports in VolumeManagerImpl.
* TestRDBTableStore
** L169: Shall we assert key doesn't exist before commit op?
{{Assert.asserNull(testTable.get(key));}}
** L190: Similarly assert if key exist before we commit the batch.
> RocksDb specific classes leak from DBStore/Table interfaces
> ---
>
> Key: HDDS-795
> URL: https://issues.apache.org/jira/browse/HDDS-795
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HDDS-795.001.patch, HDDS-795.002.patch,
> HDDS-795.003.patch, HDDS-795.004.patch
>
>
> org.apache.hadoop.utils.db.RocksDB and Table interfaces provide a
> vendor-independent way to access any key value store.
> The default implementation uses RocksDb but other implementation also could
> be used (for example an InMemory implementation for testing only).
> The current Table interface contains methods which depend on RocksDB specific
> classes. For example:
> {code}
> public interface DBStore extends AutoCloseable {
> //...
> /**
>* Return the Column Family handle. TODO: This leaks an RockDB abstraction
>* into Ozone code, cleanup later.
>*
>* @return ColumnFamilyHandle
>*/
> ColumnFamilyHandle getHandle();
> //...
> {code}
> We need to remove the RocksDB specific classes from the generic interfaces.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-795) RocksDb specific classes leak from DBStore/Table interfaces
[
https://issues.apache.org/jira/browse/HDDS-795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691118#comment-16691118
]
Ajay Kumar commented on HDDS-795:
-
[~elek] thanks for posting the patch. Could you please rebase the patch. v4
looks good, few NITs.
* DBStore
** L109: Rephrase "A init a holder" to "Initialize atomic batch operations." or
something similar?
** L114 missing throws clause?
** L120: typo "de"
* BatchOperation: Should this have api for commit op as well?
* Table
** L51: Rename new put operation to "addToBatch"
* Unused imports in VolumeManagerImpl.
* TestRDBTableStore
** L169: Shall we assert key doesn't exist before commit op?
{{Assert.asserNull(testTable.get(key));}}
** L190: Similarly assert if key exist before we commit the batch.
> RocksDb specific classes leak from DBStore/Table interfaces
> ---
>
> Key: HDDS-795
> URL: https://issues.apache.org/jira/browse/HDDS-795
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HDDS-795.001.patch, HDDS-795.002.patch,
> HDDS-795.003.patch, HDDS-795.004.patch
>
>
> org.apache.hadoop.utils.db.RocksDB and Table interfaces provide a
> vendor-independent way to access any key value store.
> The default implementation uses RocksDb but other implementation also could
> be used (for example an InMemory implementation for testing only).
> The current Table interface contains methods which depend on RocksDB specific
> classes. For example:
> {code}
> public interface DBStore extends AutoCloseable {
> //...
> /**
>* Return the Column Family handle. TODO: This leaks an RockDB abstraction
>* into Ozone code, cleanup later.
>*
>* @return ColumnFamilyHandle
>*/
> ColumnFamilyHandle getHandle();
> //...
> {code}
> We need to remove the RocksDB specific classes from the generic interfaces.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-9) Add GRPC protocol interceptors for Ozone Block Token
[ https://issues.apache.org/jira/browse/HDDS-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690979#comment-16690979 ] Xiaoyu Yao commented on HDDS-9: --- Thanks [~jnp] for the review. The block ID is a string in block token identifier. After HDDS-749, BCSID is already part of BlockID class, when instantiate identifier from BlockToken Manager, we can use the BlockID#toString() to include CID:LID:BCSID. The current container implementation already have logic to reject bcsid greater than container's bcsid without block token. We can also add length as suggested. My concern is that it will restrict the block token usage to stale data even new writes have been committed. The client will need to get new token even it is not expired to read the newly written data, this may add additional load on the OM BlockToken Manager. > Add GRPC protocol interceptors for Ozone Block Token > > > Key: HDDS-9 > URL: https://issues.apache.org/jira/browse/HDDS-9 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task > Components: Security >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Major > Attachments: HDDS-9-HDDS-4.001.patch, HDDS-9-HDDS-4.002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14075) NPE while Edit Logging
[
https://issues.apache.org/jira/browse/HDFS-14075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690976#comment-16690976
]
Hadoop QA commented on HDFS-14075:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 43s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 30s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}144m 20s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
35s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}199m 21s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename |
| | hadoop.hdfs.TestAclsEndToEnd |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean |
| | hadoop.hdfs.server.namenode.TestReconstructStripedBlocks |
| | hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup |
| | hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy |
| | hadoop.hdfs.TestDFSStripedOutputStream |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.mover.TestStorageMover |
| | hadoop.hdfs.TestUnsetAndChangeDirectoryEcPolicy |
| | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| | hadoop.hdfs.server.namenode.TestNamenodeRetryCache |
| | hadoop.hdfs.server.namenode.TestReencryptionWithKMS |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| | hadoop.hdfs.server.datanode.TestDataNodeReconfiguration |
| | hadoop.hdfs.server.namenode.TestNamenodeStorageDirectives |
| | hadoop.hdfs.server.namenode.TestFSImage |
| | hadoop.hdfs.TestHFlush |
| | hadoop.hdfs.TestErasureCodingMultipleRacks |
| | hadoop.hdfs.server.namenode.TestTransferFsImage |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.TestDistributedFileSystemWithECFile |
| | hadoop.hdfs.tools.TestDFSAdminWithHA |
| |
[jira] [Updated] (HDFS-14075) NPE while Edit Logging
[
https://issues.apache.org/jira/browse/HDFS-14075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HDFS-14075:
Attachment: HDFS-14075-04.patch
> NPE while Edit Logging
> --
>
> Key: HDFS-14075
> URL: https://issues.apache.org/jira/browse/HDFS-14075
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Critical
> Attachments: HDFS-14075-01.patch, HDFS-14075-02.patch,
> HDFS-14075-03.patch, HDFS-14075-04.patch
>
>
> {noformat}
> 2018-11-10 18:59:38,427 FATAL
> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Exception while edit
> logging: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.doEditTransaction(FSEditLog.java:481)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync$Edit.logEdit(FSEditLogAsync.java:288)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.run(FSEditLogAsync.java:232)
> at java.lang.Thread.run(Thread.java:745)
> 2018-11-10 18:59:38,532 INFO org.apache.hadoop.util.ExitUtil: Exiting with
> status 1: Exception while edit logging: null
> 2018-11-10 18:59:38,552 INFO org.apache.hadoop.hdfs.server.namenode.NameNode:
> SHUTDOWN_MSG:
> {noformat}
> Before NPE Received the following Exception
> {noformat}
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 65110, call
> Call#23241 Retry#0
> org.apache.hadoop.hdfs.server.protocol.NamenodeProtocol.rollEditLog from
>
> java.io.IOException: Unable to start log segment 7964819: too few journals
> successfully started.
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegment(FSEditLog.java:1385)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegmentAndWriteHeaderTxn(FSEditLog.java:1395)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.rollEditLog(FSEditLog.java:1319)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.rollEditLog(FSImage.java:1352)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.rollEditLog(FSNamesystem.java:4669)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.rollEditLog(NameNodeRpcServer.java:1293)
> at
> org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolServerSideTranslatorPB.rollEditLog(NamenodeProtocolServerSideTranslatorPB.java:146)
> at
> org.apache.hadoop.hdfs.protocol.proto.NamenodeProtocolProtos$NamenodeProtocolService$2.callBlockingMethod(NamenodeProtocolProtos.java:12974)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> Caused by: java.io.IOException: starting log segment 7964819 failed for too
> many journals
> at
> org.apache.hadoop.hdfs.server.namenode.JournalSet.mapJournalsAndReportErrors(JournalSet.java:412)
> at
> org.apache.hadoop.hdfs.server.namenode.JournalSet.startLogSegment(JournalSet.java:207)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegment(FSEditLog.java:1383)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14075) NPE while Edit Logging
[
https://issues.apache.org/jira/browse/HDFS-14075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690911#comment-16690911
]
Hadoop QA commented on HDFS-14075:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} root in trunk failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
16s{color} | {color:red} hadoop-hdfs in trunk failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 3s{color} | {color:orange} The patch fails to run checkstyle in hadoop-hdfs
{color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
6s{color} | {color:red} hadoop-hdfs in trunk failed. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 0m
42s{color} | {color:red} branch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
13s{color} | {color:red} hadoop-hdfs in trunk failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
6s{color} | {color:red} hadoop-hdfs in trunk failed. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
8s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
7s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 7s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 5s{color} | {color:orange} The patch fails to run checkstyle in hadoop-hdfs
{color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
6s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 44 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 600 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
0m 16s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
5s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}175m 19s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
30s{color} | {color:red} The patch generated 17 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}180m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer |
| | hadoop.hdfs.TestAclsEndToEnd |
| | hadoop.hdfs.server.namenode.TestAuditLoggerWithCommands |
| | hadoop.hdfs.TestSeekBug |
| | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup |
| | hadoop.hdfs.TestHDFSServerPorts |
| | hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy |
| | hadoop.hdfs.TestDFSStripedOutputStream |
| | hadoop.hdfs.server.blockmanagement.TestSequentialBlockGroupId |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.TestDFSPermission |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.TestUnsetAndChangeDirectoryEcPolicy |
| | hadoop.hdfs.TestMaintenanceState |
| | hadoop.hdfs.tools.TestViewFSStoragePolicyCommands |
| | hadoop.hdfs.server.namenode.TestNamenodeRetryCache |
| | hadoop.hdfs.server.namenode.TestReencryptionWithKMS |
| |
[jira] [Commented] (HDFS-14064) WEBHDFS: Support Enable/Disable EC Policy
[ https://issues.apache.org/jira/browse/HDFS-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690863#comment-16690863 ] Lokesh Jain commented on HDFS-14064: Sounds good. +1(non-binding). :) > WEBHDFS: Support Enable/Disable EC Policy > - > > Key: HDFS-14064 > URL: https://issues.apache.org/jira/browse/HDFS-14064 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: HDFS-14064-01.patch, HDFS-14064-02.patch, > HDFS-14064-03.patch, HDFS-14064-04.patch, HDFS-14064-04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14075) NPE while Edit Logging
[
https://issues.apache.org/jira/browse/HDFS-14075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HDFS-14075:
Attachment: HDFS-14075-03.patch
> NPE while Edit Logging
> --
>
> Key: HDFS-14075
> URL: https://issues.apache.org/jira/browse/HDFS-14075
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Critical
> Attachments: HDFS-14075-01.patch, HDFS-14075-02.patch,
> HDFS-14075-03.patch
>
>
> {noformat}
> 2018-11-10 18:59:38,427 FATAL
> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Exception while edit
> logging: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.doEditTransaction(FSEditLog.java:481)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync$Edit.logEdit(FSEditLogAsync.java:288)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.run(FSEditLogAsync.java:232)
> at java.lang.Thread.run(Thread.java:745)
> 2018-11-10 18:59:38,532 INFO org.apache.hadoop.util.ExitUtil: Exiting with
> status 1: Exception while edit logging: null
> 2018-11-10 18:59:38,552 INFO org.apache.hadoop.hdfs.server.namenode.NameNode:
> SHUTDOWN_MSG:
> {noformat}
> Before NPE Received the following Exception
> {noformat}
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 65110, call
> Call#23241 Retry#0
> org.apache.hadoop.hdfs.server.protocol.NamenodeProtocol.rollEditLog from
>
> java.io.IOException: Unable to start log segment 7964819: too few journals
> successfully started.
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegment(FSEditLog.java:1385)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegmentAndWriteHeaderTxn(FSEditLog.java:1395)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.rollEditLog(FSEditLog.java:1319)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.rollEditLog(FSImage.java:1352)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.rollEditLog(FSNamesystem.java:4669)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.rollEditLog(NameNodeRpcServer.java:1293)
> at
> org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolServerSideTranslatorPB.rollEditLog(NamenodeProtocolServerSideTranslatorPB.java:146)
> at
> org.apache.hadoop.hdfs.protocol.proto.NamenodeProtocolProtos$NamenodeProtocolService$2.callBlockingMethod(NamenodeProtocolProtos.java:12974)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> Caused by: java.io.IOException: starting log segment 7964819 failed for too
> many journals
> at
> org.apache.hadoop.hdfs.server.namenode.JournalSet.mapJournalsAndReportErrors(JournalSet.java:412)
> at
> org.apache.hadoop.hdfs.server.namenode.JournalSet.startLogSegment(JournalSet.java:207)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegment(FSEditLog.java:1383)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14064) WEBHDFS: Support Enable/Disable EC Policy
[ https://issues.apache.org/jira/browse/HDFS-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690848#comment-16690848 ] Ayush Saxena edited comment on HDFS-14064 at 11/18/18 10:02 AM: Thanx [~ljain] for reviewing. I drove it into a single variable, So that if in near future somebody wants to run this test with some specific other EC policy he can do that by just changing it at one location rather than in all 3 or 4 places.To avoid any goof up during any such activity. :) was (Author: ayushtkn): Thanx [~ljain] for reviewing. I drove it into a single variable, So that if in near future somebody wants to run the this test with some specific other EC policy he can do that by just changing it at one location rather than in all 3 or 4 places.To avoid any goof up during any such activity. :) > WEBHDFS: Support Enable/Disable EC Policy > - > > Key: HDFS-14064 > URL: https://issues.apache.org/jira/browse/HDFS-14064 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: HDFS-14064-01.patch, HDFS-14064-02.patch, > HDFS-14064-03.patch, HDFS-14064-04.patch, HDFS-14064-04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14064) WEBHDFS: Support Enable/Disable EC Policy
[ https://issues.apache.org/jira/browse/HDFS-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690848#comment-16690848 ] Ayush Saxena commented on HDFS-14064: - Thanx [~ljain] for reviewing. I drove it into a single variable, So that if in near future somebody wants to run the this test with some specific other EC policy he can do that by just changing it at one location rather than in all 3 or 4 places.To avoid any goof up during any such activity. :) > WEBHDFS: Support Enable/Disable EC Policy > - > > Key: HDFS-14064 > URL: https://issues.apache.org/jira/browse/HDFS-14064 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: HDFS-14064-01.patch, HDFS-14064-02.patch, > HDFS-14064-03.patch, HDFS-14064-04.patch, HDFS-14064-04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
