[jira] [Commented] (HDDS-4125) Pipeline is not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179375#comment-17179375
]
Glen Geng commented on HDDS-4125:
-
Create a new Jira https://issues.apache.org/jira/browse/HDDS-4128 to track this
RATIS ONE issue.
Leave this Jira to track the RATIS THREE issue.
> Pipeline is not removed when a datanode goes stale
> --
>
> Key: HDDS-4125
> URL: https://issues.apache.org/jira/browse/HDDS-4125
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Nanda kumar
>Assignee: Glen Geng
>Priority: Major
> Labels: pull-request-available
>
> When a node goes stale the pipelines in that node have to be closed and
> removed from {{PipelineManager}}. Currently, the pipeline is only closed and
> left in {{PipelineManager}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-4128) RATIS ONE Pipeline is closed but not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179374#comment-17179374
]
Glen Geng commented on HDDS-4128:
-
FYI [~nanda] [~timmylicheng]
related to https://issues.apache.org/jira/browse/HDDS-4125
Use this Jira to track RATIS ONE issue, use HDDS-4125 to track RATIS TRHEE
issue.
> RATIS ONE Pipeline is closed but not removed when a datanode goes stale
> ---
>
> Key: HDDS-4128
> URL: https://issues.apache.org/jira/browse/HDDS-4128
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Glen Geng
>Assignee: Glen Geng
>Priority: Major
> Labels: pull-request-available
>
> Since the Scheduler in SCMPipelineManager that used to destroyPipeline is
> removed,
> {code:java}
> scheduler.schedule(() -> destroyPipeline(pipeline),
> pipelineDestroyTimeoutInMillis, TimeUnit.MILLISECONDS, LOG,
> String.format("Destroy pipeline failed for pipeline:%s", pipeline));{code}
> meanwhile the PipelineManagerV2Impl::scrubPipeline only handles and remove
> RATIS THREE pipeline,
> {code:java}
> public void scrubPipeline(ReplicationType type, ReplicationFactor factor)
> throws IOException {
> checkLeader();
> if (type != ReplicationType.RATIS || factor != ReplicationFactor.THREE) {
> // Only srub pipeline for RATIS THREE pipeline
> return;
> }
> {code}
>
> RATIS ONE Pipeline is closed but not removed when a datanode goes stale. The
> solution is let StaleNodeHandler immediately remove RATIS ONE Pipeline,
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4128) RATIS ONE Pipeline is closed but not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Glen Geng updated HDDS-4128:
Description:
Since the Scheduler in SCMPipelineManager that used to destroyPipeline is
removed,
{code:java}
scheduler.schedule(() -> destroyPipeline(pipeline),
pipelineDestroyTimeoutInMillis, TimeUnit.MILLISECONDS, LOG,
String.format("Destroy pipeline failed for pipeline:%s", pipeline));{code}
meanwhile the PipelineManagerV2Impl::scrubPipeline only handles and remove
RATIS THREE pipeline,
{code:java}
public void scrubPipeline(ReplicationType type, ReplicationFactor factor)
throws IOException {
checkLeader();
if (type != ReplicationType.RATIS || factor != ReplicationFactor.THREE) {
// Only srub pipeline for RATIS THREE pipeline
return;
}
{code}
RATIS ONE Pipeline is closed but not removed when a datanode goes stale. The
solution is let StaleNodeHandler immediately remove RATIS ONE Pipeline,
was:
RATIS ONE Pipeline is closed but not removed when a datanode goes stale.
Since the Scheduler in SCMPipelineManager that used to destroyPipeline is
removed,
{code:java}
scheduler.schedule(() -> destroyPipeline(pipeline),
pipelineDestroyTimeoutInMillis, TimeUnit.MILLISECONDS, LOG,
String.format("Destroy pipeline failed for pipeline:%s", pipeline));{code}
meanwhile the PipelineManagerV2Impl::scrubPipeline only handles and remove
RATIS THREE pipeline,
{code:java}
public void scrubPipeline(ReplicationType type, ReplicationFactor factor)
throws IOException {
checkLeader();
if (type != ReplicationType.RATIS || factor != ReplicationFactor.THREE) {
// Only srub pipeline for RATIS THREE pipeline
return;
}
{code}
RATIS ONE Pipeline is closed but not removed when a datanode goes stale. The
solution is let StaleNodeHandler immediately remove RATIS ONE Pipeline,
> RATIS ONE Pipeline is closed but not removed when a datanode goes stale
> ---
>
> Key: HDDS-4128
> URL: https://issues.apache.org/jira/browse/HDDS-4128
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Glen Geng
>Assignee: Glen Geng
>Priority: Major
> Labels: pull-request-available
>
> Since the Scheduler in SCMPipelineManager that used to destroyPipeline is
> removed,
> {code:java}
> scheduler.schedule(() -> destroyPipeline(pipeline),
> pipelineDestroyTimeoutInMillis, TimeUnit.MILLISECONDS, LOG,
> String.format("Destroy pipeline failed for pipeline:%s", pipeline));{code}
> meanwhile the PipelineManagerV2Impl::scrubPipeline only handles and remove
> RATIS THREE pipeline,
> {code:java}
> public void scrubPipeline(ReplicationType type, ReplicationFactor factor)
> throws IOException {
> checkLeader();
> if (type != ReplicationType.RATIS || factor != ReplicationFactor.THREE) {
> // Only srub pipeline for RATIS THREE pipeline
> return;
> }
> {code}
>
> RATIS ONE Pipeline is closed but not removed when a datanode goes stale. The
> solution is let StaleNodeHandler immediately remove RATIS ONE Pipeline,
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4128) RATIS ONE Pipeline is closed but not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Glen Geng updated HDDS-4128:
Description:
RATIS ONE Pipeline is closed but not removed when a datanode goes stale.
Since the Scheduler in SCMPipelineManager that used to destroyPipeline is
removed,
{code:java}
scheduler.schedule(() -> destroyPipeline(pipeline),
pipelineDestroyTimeoutInMillis, TimeUnit.MILLISECONDS, LOG,
String.format("Destroy pipeline failed for pipeline:%s", pipeline));{code}
meanwhile the PipelineManagerV2Impl::scrubPipeline only handles and remove
RATIS THREE pipeline,
{code:java}
public void scrubPipeline(ReplicationType type, ReplicationFactor factor)
throws IOException {
checkLeader();
if (type != ReplicationType.RATIS || factor != ReplicationFactor.THREE) {
// Only srub pipeline for RATIS THREE pipeline
return;
}
{code}
RATIS ONE Pipeline is closed but not removed when a datanode goes stale. The
solution is let StaleNodeHandler immediately remove RATIS ONE Pipeline,
> RATIS ONE Pipeline is closed but not removed when a datanode goes stale
> ---
>
> Key: HDDS-4128
> URL: https://issues.apache.org/jira/browse/HDDS-4128
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Glen Geng
>Assignee: Glen Geng
>Priority: Major
> Labels: pull-request-available
>
> RATIS ONE Pipeline is closed but not removed when a datanode goes stale.
>
> Since the Scheduler in SCMPipelineManager that used to destroyPipeline is
> removed,
> {code:java}
> scheduler.schedule(() -> destroyPipeline(pipeline),
> pipelineDestroyTimeoutInMillis, TimeUnit.MILLISECONDS, LOG,
> String.format("Destroy pipeline failed for pipeline:%s", pipeline));{code}
> meanwhile the PipelineManagerV2Impl::scrubPipeline only handles and remove
> RATIS THREE pipeline,
> {code:java}
> public void scrubPipeline(ReplicationType type, ReplicationFactor factor)
> throws IOException {
> checkLeader();
> if (type != ReplicationType.RATIS || factor != ReplicationFactor.THREE) {
> // Only srub pipeline for RATIS THREE pipeline
> return;
> }
> {code}
>
> RATIS ONE Pipeline is closed but not removed when a datanode goes stale. The
> solution is let StaleNodeHandler immediately remove RATIS ONE Pipeline,
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4128) RATIS ONE Pipeline is closed but not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Glen Geng updated HDDS-4128:
Description: (was: When a node goes stale the pipelines in that node
have to be closed and removed from {{PipelineManager}}. Currently, the pipeline
is only closed and left in {{PipelineManager}}.)
> RATIS ONE Pipeline is closed but not removed when a datanode goes stale
> ---
>
> Key: HDDS-4128
> URL: https://issues.apache.org/jira/browse/HDDS-4128
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Glen Geng
>Assignee: Glen Geng
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4128) RATIS ONE Pipeline is closed but not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Glen Geng updated HDDS-4128:
Summary: RATIS ONE Pipeline is closed but not removed when a datanode goes
stale (was: CLONE - Pipeline is not removed when a datanode goes stale)
> RATIS ONE Pipeline is closed but not removed when a datanode goes stale
> ---
>
> Key: HDDS-4128
> URL: https://issues.apache.org/jira/browse/HDDS-4128
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Glen Geng
>Assignee: Glen Geng
>Priority: Major
> Labels: pull-request-available
>
> When a node goes stale the pipelines in that node have to be closed and
> removed from {{PipelineManager}}. Currently, the pipeline is only closed and
> left in {{PipelineManager}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4128) CLONE - Pipeline is not removed when a datanode goes stale
Glen Geng created HDDS-4128:
---
Summary: CLONE - Pipeline is not removed when a datanode goes stale
Key: HDDS-4128
URL: https://issues.apache.org/jira/browse/HDDS-4128
Project: Hadoop Distributed Data Store
Issue Type: Sub-task
Components: SCM HA
Reporter: Glen Geng
Assignee: Glen Geng
When a node goes stale the pipelines in that node have to be closed and removed
from {{PipelineManager}}. Currently, the pipeline is only closed and left in
{{PipelineManager}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-4125) Pipeline is not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179352#comment-17179352
]
Glen Geng commented on HDDS-4125:
-
RATIS THREE pipeline, if stay in closed state, should be removed by
PipelineManagerV2Impl::scrubPipeline(). Since the thread pool is removed, the
scrubber is the only place to remove RATIS THREE pipeline.
I did not encounter the issue for RATIS THREE pipeline, could you tell me how
to reproduce it?
> Pipeline is not removed when a datanode goes stale
> --
>
> Key: HDDS-4125
> URL: https://issues.apache.org/jira/browse/HDDS-4125
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Nanda kumar
>Assignee: Glen Geng
>Priority: Major
> Labels: pull-request-available
>
> When a node goes stale the pipelines in that node have to be closed and
> removed from {{PipelineManager}}. Currently, the pipeline is only closed and
> left in {{PipelineManager}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDDS-4115) CLI command to show current SCM leader and follower status
[ https://issues.apache.org/jira/browse/HDDS-4115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned HDDS-4115: -- Assignee: Rui Wang > CLI command to show current SCM leader and follower status > -- > > Key: HDDS-4115 > URL: https://issues.apache.org/jira/browse/HDDS-4115 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Li Cheng >Assignee: Rui Wang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-4116) SCM CLI command towards certain IP
[ https://issues.apache.org/jira/browse/HDDS-4116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179307#comment-17179307 ] Li Cheng commented on HDDS-4116: [~adoroszlai] The goal of this task is to enable admin cli to send scm related commands to certain SCM with IP address. Right now it's sending command just to SCM service, but with SCM HA, we would have leader SCM and followers. > SCM CLI command towards certain IP > -- > > Key: HDDS-4116 > URL: https://issues.apache.org/jira/browse/HDDS-4116 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Li Cheng >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-3837) Add isLeader check for SCM state updates
[ https://issues.apache.org/jira/browse/HDDS-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Cheng updated HDDS-3837: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Add isLeader check for SCM state updates > > > Key: HDDS-3837 > URL: https://issues.apache.org/jira/browse/HDDS-3837 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Li Cheng >Assignee: Li Cheng >Priority: Major > Labels: pull-request-available > > We only allow leader to make decisions to update map, DB and fire events to DN -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1335: HDDS-4127. Components with web interface should depend on hdds-docs.
bharatviswa504 commented on a change in pull request #1335: URL: https://github.com/apache/hadoop-ozone/pull/1335#discussion_r471813458 ## File path: hadoop-ozone/ozone-manager/pom.xml ## @@ -70,6 +71,12 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd";> bcprov-jdk15on + + org.apache.hadoop + hadoop-hdds-docs Review comment: Repated again, it is already defined in L44 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4119) Improve performance of the BufferPool management of Ozone client
[ https://issues.apache.org/jira/browse/HDDS-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4119: - Labels: pull-request-available (was: ) > Improve performance of the BufferPool management of Ozone client > > > Key: HDDS-4119 > URL: https://issues.apache.org/jira/browse/HDDS-4119 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Marton Elek >Assignee: Marton Elek >Priority: Blocker > Labels: pull-request-available > > Teragen reported to be slow with low number of mappers compared to HDFS. > In my test (one pipeline, 3 yarn nodes) 10 g teragen with HDFS was ~3 mins > but with Ozone it was 6 mins. It could be fixed with using more mappers, but > when I investigated the execution I found a few problems reagrding to the > BufferPool management. > 1. IncrementalChunkBuffer is slow and it might not be required as BufferPool > itself is incremental > 2. For each write operation the bufferPool.allocateBufferIfNeeded is called > which can be a slow operation (positions should be calculated). > 3. There is no explicit support for write(byte) operations > In the flamegraph it's clearly visible that with low number of mappers the > client is busy with buffer operations. After the patch the rpc call and the > checksum calculation give the majority of the time. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] elek opened a new pull request #1336: HDDS-4119. Improve performance of the BufferPool management of Ozone client
elek opened a new pull request #1336: URL: https://github.com/apache/hadoop-ozone/pull/1336 ## What changes were proposed in this pull request? Teragen reported to be slow with low number of mappers compared to HDFS. In my test (one pipeline, 3 yarn nodes) 10 g teragen with HDFS was ~3 mins but with Ozone it was 6 mins. It could be fixed with using more mappers, but when I investigated the execution I found a few problems reagrding to the BufferPool management. 1. IncrementalChunkBuffer is slow and it might not be required as BufferPool itself is incremental 2. For each write operation the bufferPool.allocateBufferIfNeeded is called which can be a slow operation (positions should be calculated). 3. There is no explicit support for write(byte) operations In the [flamegraphs](https://github.com/elek/ozone-notes/tree/master/profiles) it's clearly visible that with low number of mappers the client is busy with buffer operations. After the patch the Rpc call and the checksum calculation give the majority of the time. Overall write performance is improved with at least 30% when minimal number of threads/mappers are used. ## Thanks Special thanks to @lokeshj1703, who helped me find the small mistakes in the original verison of the patch. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-4119 ## How was this patch tested? Teragen 10/100g with 2/30 mappers. (https://github.com/elek/ozone-perf-env/tree/master/teragen-hdfs) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDDS-4125) Pipeline is not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nanda kumar reassigned HDDS-4125:
-
Assignee: Glen Geng (was: Nanda kumar)
> Pipeline is not removed when a datanode goes stale
> --
>
> Key: HDDS-4125
> URL: https://issues.apache.org/jira/browse/HDDS-4125
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Nanda kumar
>Assignee: Glen Geng
>Priority: Major
> Labels: pull-request-available
>
> When a node goes stale the pipelines in that node have to be closed and
> removed from {{PipelineManager}}. Currently, the pipeline is only closed and
> left in {{PipelineManager}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-4125) Pipeline is not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179138#comment-17179138
]
Nanda kumar commented on HDDS-4125:
---
It is not just one node pipeline, we should also handle three node pipelines.
> Pipeline is not removed when a datanode goes stale
> --
>
> Key: HDDS-4125
> URL: https://issues.apache.org/jira/browse/HDDS-4125
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Nanda kumar
>Assignee: Nanda kumar
>Priority: Major
> Labels: pull-request-available
>
> When a node goes stale the pipelines in that node have to be closed and
> removed from {{PipelineManager}}. Currently, the pipeline is only closed and
> left in {{PipelineManager}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] nandakumar131 commented on pull request #1145: HDDS-3895. Implement container related operations in ContainerManagerImpl
nandakumar131 commented on pull request #1145: URL: https://github.com/apache/hadoop-ozone/pull/1145#issuecomment-675004188 It is not just about one node pipeline, we have the same issue in three node pipeline as well. If all the three nodes goes stale (because of restart), we will mark the pipeline as closed and will not remove. The problem is not just there is no instant removal, it is never deleted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4127) Components with web interface should depend on hdds-docs
[
https://issues.apache.org/jira/browse/HDDS-4127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-4127:
-
Labels: pull-request-available (was: )
> Components with web interface should depend on hdds-docs
>
>
> Key: HDDS-4127
> URL: https://issues.apache.org/jira/browse/HDDS-4127
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Marton Elek
>Assignee: Marton Elek
>Priority: Blocker
> Labels: pull-request-available
>
> [~sammichen] reported the problem that the ozone-0.6.0 branch couldn't be
> compiled after changing the version to 1.0.0.
> {code}
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-dependency-plugin:3.0.2:unpack
> (copy-common-html) on project hadoop-hdds-container-service: Unable to
> find/resolve artifact.: Could not find artifact
> org.apache.hadoop:hadoop-hdds-docs:jar:1.0.0 in apache.snapshots.https
> (https://repository.apache.org/content/repositories/snapshots) -> [Help 1]
> [ERROR]
> {code}
> Web components include the compiled docs (in case of hugo is on the path)
> {code}
>
> org.apache.maven.plugins
> maven-dependency-plugin
>
>
> copy-common-html
> prepare-package
>
> unpack
>
>
>
>
> org.apache.hadoop
> hadoop-hdds-server-framework
> ${project.build.outputDirectory}
>
> webapps/static/**/*.*
>
>
> org.apache.hadoop
> hadoop-hdds-docs
>
> ${project.build.outputDirectory}/webapps/ozoneManager
> docs/**/*.*
>
>
> true
>
>
>
>
> {code}
> But the explicit dependency between the container-service and hdds-docs
> accidentally missing. With adding a provided dependency, it can be fixed
> (maven will compile hdds-docs first)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] elek opened a new pull request #1335: HDDS-4127. Components with web interface should depend on hdds-docs.
elek opened a new pull request #1335:
URL: https://github.com/apache/hadoop-ozone/pull/1335
## What changes were proposed in this pull request?
@ChenSammi reported the problem that the ozone-0.6.0 branch couldn't be
compiled after changing the version to 1.0.0.
```
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-dependency-plugin:3.0.2:unpack
(copy-common-html) on project hadoop-hdds-container-service: Unable to
find/resolve artifact.: Could not find artifact
org.apache.hadoop:hadoop-hdds-docs:jar:1.0.0 in apache.snapshots.https
(https://repository.apache.org/content/repositories/snapshots) -> [Help 1]
[ERROR]
```
Web components include the compiled docs (in case of hugo is on the path)
```XML
org.apache.maven.plugins
maven-dependency-plugin
copy-common-html
prepare-package
unpack
org.apache.hadoop
hadoop-hdds-server-framework
${project.build.outputDirectory}
webapps/static/**/*.*
org.apache.hadoop
hadoop-hdds-docs
${project.build.outputDirectory}/webapps/ozoneManager
docs/**/*.*
true
```
But the explicit dependency between the container-service and hdds-docs
accidentally missing. With adding a provided dependency, it can be fixed (maven
will compile hdds-docs first)
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4127
## How was this patch tested?
```
git checkout ozone-0.6.0
find . -name pom.xml -type f | xargs sed -i 's/0.6.0-SNAPSHOT/1.0.0/g'
# apply this patch
mvn clean install -DskipTests
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4127) Components with web interface should depend on hdds-docs
[
https://issues.apache.org/jira/browse/HDDS-4127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Elek updated HDDS-4127:
--
Summary: Components with web interface should depend on hdds-docs (was:
Component with web interface should depend on hdds-docs)
> Components with web interface should depend on hdds-docs
>
>
> Key: HDDS-4127
> URL: https://issues.apache.org/jira/browse/HDDS-4127
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Marton Elek
>Assignee: Marton Elek
>Priority: Blocker
>
> [~sammichen] reported the problem that the ozone-0.6.0 branch couldn't be
> compiled after changing the version to 1.0.0.
> {code}
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-dependency-plugin:3.0.2:unpack
> (copy-common-html) on project hadoop-hdds-container-service: Unable to
> find/resolve artifact.: Could not find artifact
> org.apache.hadoop:hadoop-hdds-docs:jar:1.0.0 in apache.snapshots.https
> (https://repository.apache.org/content/repositories/snapshots) -> [Help 1]
> [ERROR]
> {code}
> Web components include the compiled docs (in case of hugo is on the path)
> {code}
>
> org.apache.maven.plugins
> maven-dependency-plugin
>
>
> copy-common-html
> prepare-package
>
> unpack
>
>
>
>
> org.apache.hadoop
> hadoop-hdds-server-framework
> ${project.build.outputDirectory}
>
> webapps/static/**/*.*
>
>
> org.apache.hadoop
> hadoop-hdds-docs
>
> ${project.build.outputDirectory}/webapps/ozoneManager
> docs/**/*.*
>
>
> true
>
>
>
>
> {code}
> But the explicit dependency between the container-service and hdds-docs
> accidentally missing. With adding a provided dependency, it can be fixed
> (maven will compile hdds-docs first)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4127) Component with web interface should depend on hdds-docs
Marton Elek created HDDS-4127:
-
Summary: Component with web interface should depend on hdds-docs
Key: HDDS-4127
URL: https://issues.apache.org/jira/browse/HDDS-4127
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Marton Elek
Assignee: Marton Elek
[~sammichen] reported the problem that the ozone-0.6.0 branch couldn't be
compiled after changing the version to 1.0.0.
{code}
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-dependency-plugin:3.0.2:unpack
(copy-common-html) on project hadoop-hdds-container-service: Unable to
find/resolve artifact.: Could not find artifact
org.apache.hadoop:hadoop-hdds-docs:jar:1.0.0 in apache.snapshots.https
(https://repository.apache.org/content/repositories/snapshots) -> [Help 1]
[ERROR]
{code}
Web components include the compiled docs (in case of hugo is on the path)
{code}
org.apache.maven.plugins
maven-dependency-plugin
copy-common-html
prepare-package
unpack
org.apache.hadoop
hadoop-hdds-server-framework
${project.build.outputDirectory}
webapps/static/**/*.*
org.apache.hadoop
hadoop-hdds-docs
${project.build.outputDirectory}/webapps/ozoneManager
docs/**/*.*
true
{code}
But the explicit dependency between the container-service and hdds-docs
accidentally missing. With adding a provided dependency, it can be fixed (maven
will compile hdds-docs first)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDDS-4125) Pipeline is not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17178919#comment-17178919
]
Glen Geng edited comment on HDDS-4125 at 8/17/20, 11:32 AM:
Since the thread pool that removes RATIS ONE pipeline is removed, we need
either remove RATIS ONE pipeline immediately or let scrubPipeline() to handle
RATIS ONE Pipeline.
{code:java}
scheduler.schedule(() -> destroyPipeline(pipeline),
pipelineDestroyTimeoutInMillis, TimeUnit.MILLISECONDS, LOG,
String.format("Destroy pipeline failed for pipeline:%s", pipeline));
{code}
was (Author: glengeng):
Since the thread pool that remove RATIS ONE pipeline is removed, we need either
remove the pipeline immediately or let scrubPipeline() to handle RATIS ONE
Pipeline.
{code:java}
scheduler.schedule(() -> destroyPipeline(pipeline),
pipelineDestroyTimeoutInMillis, TimeUnit.MILLISECONDS, LOG,
String.format("Destroy pipeline failed for pipeline:%s", pipeline));
{code}
> Pipeline is not removed when a datanode goes stale
> --
>
> Key: HDDS-4125
> URL: https://issues.apache.org/jira/browse/HDDS-4125
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Nanda kumar
>Assignee: Nanda kumar
>Priority: Major
> Labels: pull-request-available
>
> When a node goes stale the pipelines in that node have to be closed and
> removed from {{PipelineManager}}. Currently, the pipeline is only closed and
> left in {{PipelineManager}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-4125) Pipeline is not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17178919#comment-17178919
]
Glen Geng commented on HDDS-4125:
-
Since the thread pool that remove RATIS ONE pipeline is removed, we need either
remove the pipeline immediately or let scrubPipeline() to handle RATIS ONE
Pipeline.
{code:java}
scheduler.schedule(() -> destroyPipeline(pipeline),
pipelineDestroyTimeoutInMillis, TimeUnit.MILLISECONDS, LOG,
String.format("Destroy pipeline failed for pipeline:%s", pipeline));
{code}
> Pipeline is not removed when a datanode goes stale
> --
>
> Key: HDDS-4125
> URL: https://issues.apache.org/jira/browse/HDDS-4125
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Nanda kumar
>Assignee: Nanda kumar
>Priority: Major
> Labels: pull-request-available
>
> When a node goes stale the pipelines in that node have to be closed and
> removed from {{PipelineManager}}. Currently, the pipeline is only closed and
> left in {{PipelineManager}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-4125) Pipeline is not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17178917#comment-17178917
]
Glen Geng commented on HDDS-4125:
-
Hi Nanda, I encountered this issue as well, and provide a fix. Please have a
look. Thanks!
> Pipeline is not removed when a datanode goes stale
> --
>
> Key: HDDS-4125
> URL: https://issues.apache.org/jira/browse/HDDS-4125
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Nanda kumar
>Assignee: Nanda kumar
>Priority: Major
> Labels: pull-request-available
>
> When a node goes stale the pipelines in that node have to be closed and
> removed from {{PipelineManager}}. Currently, the pipeline is only closed and
> left in {{PipelineManager}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-4125) Pipeline is not removed when a datanode goes stale
[
https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-4125:
-
Labels: pull-request-available (was: )
> Pipeline is not removed when a datanode goes stale
> --
>
> Key: HDDS-4125
> URL: https://issues.apache.org/jira/browse/HDDS-4125
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: SCM HA
>Reporter: Nanda kumar
>Assignee: Nanda kumar
>Priority: Major
> Labels: pull-request-available
>
> When a node goes stale the pipelines in that node have to be closed and
> removed from {{PipelineManager}}. Currently, the pipeline is only closed and
> left in {{PipelineManager}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] GlenGeng opened a new pull request #1334: HDDS-4125: Pipeline is not removed when a datanode goes stale
GlenGeng opened a new pull request #1334: URL: https://github.com/apache/hadoop-ozone/pull/1334 ## What changes were proposed in this pull request? StaleNodeHandler need Immediately remove RATIS ONE Pipeline, since scrubPipeline() only handle RATIS THREE Pipeline. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-4125 ## How was this patch tested? CI This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] timmylicheng commented on pull request #1145: HDDS-3895. Implement container related operations in ContainerManagerImpl
timmylicheng commented on pull request #1145: URL: https://github.com/apache/hadoop-ozone/pull/1145#issuecomment-674816824 @nandakumar131 Ratis One pipeline is not instantly removed in StaleNodeHandler. We will come up with a fix. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-4126) Freon key generator should support >2g files
Marton Elek created HDDS-4126: - Summary: Freon key generator should support >2g files Key: HDDS-4126 URL: https://issues.apache.org/jira/browse/HDDS-4126 Project: Hadoop Distributed Data Store Issue Type: Improvement Reporter: Marton Elek Assignee: Marton Elek fileSize parameter in HadoopFsGenerator is an int which limits it to max 2Gb. But works well with long. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] fapifta commented on pull request #1322: HDDS-3829. Introduce Layout Feature interface in Ozone.
fapifta commented on pull request #1322: URL: https://github.com/apache/hadoop-ozone/pull/1322#issuecomment-674766274 Hi @avijayanhwx, thank you for starting this work, and posting an inital version of the core code for the upgrade support. I have a few general questions and concerns, also added a few comments after a quick review. In HDFS the layoutversion is a monotonically decreasing number, we chose to use monotonically increasing version numbers, I am unsure why HDFS chose the negative numbers, can there be some hidden considerations, we might go through, before committing to the positive layoutversions and monotonic increase? I haven't found one. Also, in HDFS one layoutversion covers one feature, would it be better to do not let two feetures associated with one layoutversion number? What is the benefit of having two features mapped to the same layout version? I don't feel good about it, though I don't have a specific example yet where it can cause trouble. LayoutVersionManager is implemented in a way that it is fully static. I am unsure if this is a good design, I understand the intent that there has to be only one instance of this per component, and seeing it this way it is a reasonable choice to use static a implementation, but it can fire back later when we want to implement tests that involves changing something in this logic, and we can not freely and easily change the behaviour in tests as I fear, also it can introduce invisible interdependencies later that might be hard to detect/factor out. Implementing it in a non-static way does not seem to cause any drawback, even we can be fine with multiple instances for the same component, as the used values will be anyway hardcoded in the real system. What do you think, I would consider making it non-static, as I think it has more possibilities and less limitations later. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] fapifta commented on a change in pull request #1322: HDDS-3829. Introduce Layout Feature interface in Ozone.
fapifta commented on a change in pull request #1322:
URL: https://github.com/apache/hadoop-ozone/pull/1322#discussion_r471346148
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMLayoutFeatureCatalog.java
##
@@ -0,0 +1,71 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.upgrade;
+
+import java.util.Optional;
+
+import org.apache.hadoop.ozone.upgrade.LayoutFeature;
+
+/**
+ * Catalog of Ozone Manager features.
+ */
+public class OMLayoutFeatureCatalog {
Review comment:
If we use an enum to store the list of features in a component, why do
we need the enclosing class here? Is there a future intent hidden here, or it
is just a leftover from an earlier implementation?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] fapifta commented on a change in pull request #1322: HDDS-3829. Introduce Layout Feature interface in Ozone.
fapifta commented on a change in pull request #1322:
URL: https://github.com/apache/hadoop-ozone/pull/1322#discussion_r471332451
##
File path:
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMVersionManager.java
##
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.upgrade;
+
+import static
org.apache.hadoop.ozone.om.upgrade.OMLayoutFeatureCatalog.OMLayoutFeature.CREATE_EC;
+import static
org.apache.hadoop.ozone.om.upgrade.OMLayoutFeatureCatalog.OMLayoutFeature.INITIAL_VERSION;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.io.IOException;
+
+import org.apache.hadoop.ozone.om.OMStorage;
+import org.apache.hadoop.ozone.om.OzoneManager;
+import
org.apache.hadoop.ozone.om.upgrade.OMLayoutFeatureCatalog.OMLayoutFeature;
+import org.apache.hadoop.ozone.upgrade.LayoutFeature;
+import org.junit.Test;
+
+/**
+ * Test OM layout version management.
+ */
+public class TestOMVersionManager {
+
+ @Test
+ public void testOMLayoutVersionManager() throws IOException {
+OMStorage omStorage = mock(OMStorage.class);
+when(omStorage.getLayoutVersion()).thenReturn(0);
+OMVersionManager.init(omStorage);
+assertTrue(OMVersionManager.isAllowed(INITIAL_VERSION));
+assertFalse(OMVersionManager.isAllowed(CREATE_EC));
+assertEquals(0, OMVersionManager.getMetadataLayoutVersion());
+assertTrue(OMVersionManager.needsFinalization());
+OMVersionManager.doFinalize(mock(OzoneManager.class));
Review comment:
Can you please elaborate what will this line do in the test, and why it
has to be here? It is unclear whether it tests anything, or this is just a
leftover from a thought.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] fapifta commented on a change in pull request #1322: HDDS-3829. Introduce Layout Feature interface in Ozone.
fapifta commented on a change in pull request #1322: URL: https://github.com/apache/hadoop-ozone/pull/1322#discussion_r471313928 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/security/package-info.java ## @@ -19,4 +19,4 @@ /** * Package contains classes related to S3 security responses. */ -package org.apache.hadoop.ozone.om.request.s3.security; +package org.apache.hadoop.ozone.om.response.s3.security; Review comment: This might go into a separate pull request and we may commit it way before the branch is commited. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
