[hadoop-ozone] branch master updated: HDDS-3803. [OFS] Add User Guide (#1144)
This is an automated email from the ASF dual-hosted git repository. siyao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git The following commit(s) were added to refs/heads/master by this push: new 1d13b4f HDDS-3803. [OFS] Add User Guide (#1144) 1d13b4f is described below commit 1d13b4fb18d5ceb830380f09b62fa1740c96b5f5 Author: Siyao Meng <50227127+smen...@users.noreply.github.com> AuthorDate: Tue Jun 30 11:47:07 2020 -0700 HDDS-3803. [OFS] Add User Guide (#1144) --- hadoop-hdds/docs/content/design/ofs.md | 143 +++-- 1 file changed, 136 insertions(+), 7 deletions(-) diff --git a/hadoop-hdds/docs/content/design/ofs.md b/hadoop-hdds/docs/content/design/ofs.md index 71e9617..9a2352e 100644 --- a/hadoop-hdds/docs/content/design/ofs.md +++ b/hadoop-hdds/docs/content/design/ofs.md @@ -1,9 +1,9 @@ --- title: Implement new Ozone FileSystem scheme ofs:// summary: A new schema structure for Hadoop compatible file system -date: 2019-12-05 +date: 2020-06-30 jira: HDDS-2665 -status: implementing +status: implemented author: Siyao Meng ---
[hadoop-ozone] branch master updated (3c1d8b6 -> 2fb4a40)
This is an automated email from the ASF dual-hosted git repository. adoroszlai pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 3c1d8b6 HDDS-3246. Include OM hostname info in getserviceroles subcommand of OM CLI (#706) add 2fb4a40 HDDS-2978. Intermittent failure in TestResourceLimitCache (#1146) No new revisions were added by this update. Summary of changes: .../apache/hadoop/hdds/utils/TestResourceLimitCache.java | 14 +- 1 file changed, 9 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (4e6d6c6 -> 3c1d8b6)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 4e6d6c6 HDDS-3876. Display summary of failures as a separate job step (#1131) add 3c1d8b6 HDDS-3246. Include OM hostname info in getserviceroles subcommand of OM CLI (#706) No new revisions were added by this update. Summary of changes: .../ozone/admin/om/GetServiceRolesSubcommand.java | 28 +++-- .../org/apache/hadoop/ozone/admin/om/OMAdmin.java | 29 ++ 2 files changed, 50 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (dd46a55 -> 4e6d6c6)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from dd46a55 HDDS-3161. Block illegal characters when creating keys. (#812) add 4e6d6c6 HDDS-3876. Display summary of failures as a separate job step (#1131) No new revisions were added by this update. Summary of changes: .github/workflows/post-commit.yml | 18 ++ 1 file changed, 18 insertions(+) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (f734182 -> dd46a55)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from f734182 HDDS-3782. Remove podAntiAffinity from datanode-statefulset (#1057) add dd46a55 HDDS-3161. Block illegal characters when creating keys. (#812) No new revisions were added by this update. Summary of changes: .../hadoop/hdds/scm/client/HddsClientUtils.java| 17 + .../java/org/apache/hadoop/ozone/OzoneConsts.java | 17 + .../common/src/main/resources/ozone-default.xml| 10 ++ .../apache/hadoop/ozone/client/rpc/RpcClient.java | 14 ++ .../hadoop/ozone/client/TestHddsClientUtils.java | 22 ++ .../main/java/org/apache/hadoop/ozone/OmUtils.java | 13 + .../org/apache/hadoop/ozone/om/OMConfigKeys.java | 5 + .../ozone/om/request/file/OMFileCreateRequest.java | 13 + .../ozone/om/request/key/OMKeyCommitRequest.java | 13 + .../ozone/om/request/key/OMKeyCreateRequest.java | 9 + .../ozone/om/request/key/OMKeyRenameRequest.java | 10 ++ 11 files changed, 143 insertions(+) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (574763b -> f734182)
This is an automated email from the ASF dual-hosted git repository.
elek pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.
from 574763b HDDS-3770. Improve getPipelines performance (#1066)
add f734182 HDDS-3782. Remove podAntiAffinity from datanode-statefulset
(#1057)
No new revisions were added by this update.
Summary of changes:
.../config.yaml => definitions/onenode.yaml} | 15 ---
.../dist/src/main/k8s/examples/getting-started/Flekszible | 1 +
.../dist/src/main/k8s/examples/ozone-dev/Flekszible | 1 +
3 files changed, 10 insertions(+), 7 deletions(-)
copy
hadoop-ozone/dist/src/main/k8s/definitions/ozone/{transformations/config.yaml
=> definitions/onenode.yaml} (79%)
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated: HDDS-3770. Improve getPipelines performance (#1066)
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
The following commit(s) were added to refs/heads/master by this push:
new 574763b HDDS-3770. Improve getPipelines performance (#1066)
574763b is described below
commit 574763b1cca814a90182a5b820e53dbd0c7de1b8
Author: runzhiwang <51938049+runzhiw...@users.noreply.github.com>
AuthorDate: Tue Jun 30 17:23:30 2020 +0800
HDDS-3770. Improve getPipelines performance (#1066)
---
.../scm/container/common/helpers/ExcludeList.java | 34 +++
.../hdds/scm/container/ContainerManager.java | 3 +-
.../hdds/scm/container/SCMContainerManager.java| 7 ++--
.../hadoop/hdds/scm/pipeline/PipelineStateMap.java | 49 +-
.../hadoop/hdds/scm/block/TestBlockManager.java| 6 ++-
.../TestContainerStateManagerIntegration.java | 10 +++--
6 files changed, 50 insertions(+), 59 deletions(-)
diff --git
a/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/ExcludeList.java
b/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/ExcludeList.java
index dcc3263..f6b111f 100644
---
a/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/ExcludeList.java
+++
b/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/ExcludeList.java
@@ -23,9 +23,9 @@ import org.apache.hadoop.hdds.protocol.proto.HddsProtos;
import org.apache.hadoop.hdds.scm.container.ContainerID;
import org.apache.hadoop.hdds.scm.pipeline.PipelineID;
-import java.util.ArrayList;
-import java.util.List;
import java.util.Collection;
+import java.util.HashSet;
+import java.util.Set;
/**
* This class contains set of dns and containers which ozone client provides
@@ -33,22 +33,22 @@ import java.util.Collection;
*/
public class ExcludeList {
- private final List datanodes;
- private final List containerIds;
- private final List pipelineIds;
+ private final Set datanodes;
+ private final Set containerIds;
+ private final Set pipelineIds;
public ExcludeList() {
-datanodes = new ArrayList<>();
-containerIds = new ArrayList<>();
-pipelineIds = new ArrayList<>();
+datanodes = new HashSet<>();
+containerIds = new HashSet<>();
+pipelineIds = new HashSet<>();
}
- public List getContainerIds() {
+ public Set getContainerIds() {
return containerIds;
}
- public List getDatanodes() {
+ public Set getDatanodes() {
return datanodes;
}
@@ -57,24 +57,18 @@ public class ExcludeList {
}
public void addDatanode(DatanodeDetails dn) {
-if (!datanodes.contains(dn)) {
- datanodes.add(dn);
-}
+datanodes.add(dn);
}
public void addConatinerId(ContainerID containerId) {
-if (!containerIds.contains(containerId)) {
- containerIds.add(containerId);
-}
+containerIds.add(containerId);
}
public void addPipeline(PipelineID pipelineId) {
-if (!pipelineIds.contains(pipelineId)) {
- pipelineIds.add(pipelineId);
-}
+pipelineIds.add(pipelineId);
}
- public List getPipelineIds() {
+ public Set getPipelineIds() {
return pipelineIds;
}
diff --git
a/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManager.java
b/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManager.java
index 43c1ced..0e1c98f 100644
---
a/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManager.java
+++
b/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManager.java
@@ -18,6 +18,7 @@ package org.apache.hadoop.hdds.scm.container;
import java.io.Closeable;
import java.io.IOException;
+import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -180,7 +181,7 @@ public interface ContainerManager extends Closeable {
* @return ContainerInfo for the matching container.
*/
ContainerInfo getMatchingContainer(long size, String owner,
- Pipeline pipeline, List excludedContainerIDS);
+ Pipeline pipeline, Collection excludedContainerIDS);
/**
* Once after report processor handler completes, call this to notify
diff --git
a/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java
b/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java
index 7ac90fc..34177f0 100644
---
a/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java
+++
b/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java
@@ -18,6 +18,7 @@ package org.apache.hadoop.hdds.scm.container;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Collection;
import java
[hadoop-ozone] branch master updated (bf23dcb -> 3479e67)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from bf23dcb HDDS-3868. Implement getTrashRoot and getTrashRoots in o3fs (#1134) add 3479e67 HDDS-3699. Change write chunk failure logging level to ERROR in BlockOutputStream. (#1006) No new revisions were added by this update. Summary of changes: .../org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
