elek commented on a change in pull request #2151:
URL: https://github.com/apache/ozone/pull/2151#discussion_r620096990
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -3813,4 +3809,67 @@ public void setMinMultipartUploadPartSize(int
partSizeForTest) {
this.minMultipartUploadPartSize = partSizeForTest;
}
+ private void initFSOLayout() throws IOException {
+ // TODO: Temporary workaround for OM upgrade path and will be replaced once
+ // upgrade HDDS-3698 story reaches consensus. Instead of cluster level
+ // configuration, OM needs to check this property on every bucket level.
+ String metaLayout = getOMMetadataLayout();
+ boolean omMetadataLayoutPrefix = StringUtils.equalsIgnoreCase(metaLayout,
+ OZONE_OM_METADATA_LAYOUT_PREFIX);
+
+ boolean enableFSPaths = getEnableFileSystemPaths();
+ if (omMetadataLayoutPrefix && !enableFSPaths) {
+ StringBuilder msg = new StringBuilder();
+ msg.append("Invalid Configuration. Failed to start OM in ");
+ msg.append(OZONE_OM_METADATA_LAYOUT_PREFIX);
+ msg.append(" layout format as '");
+ msg.append(OZONE_OM_ENABLE_FILESYSTEM_PATHS);
+ msg.append("' is false!");
+
+ LOG.error(msg.toString());
+ throw new IOException(msg.toString());
+ }
+
+ boolean isBucketFSOptimized = omMetadataLayoutPrefix && enableFSPaths;
Review comment:
```suggestion
boolean isBucketFSOptimized = omMetadataLayoutPrefix;
```
`enableFSPaths` is always true at this point.
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -3813,4 +3809,67 @@ public void setMinMultipartUploadPartSize(int
partSizeForTest) {
this.minMultipartUploadPartSize = partSizeForTest;
}
+ private void initFSOLayout() throws IOException {
+ // TODO: Temporary workaround for OM upgrade path and will be replaced once
+ // upgrade HDDS-3698 story reaches consensus. Instead of cluster level
+ // configuration, OM needs to check this property on every bucket level.
+ String metaLayout = getOMMetadataLayout();
+ boolean omMetadataLayoutPrefix = StringUtils.equalsIgnoreCase(metaLayout,
+ OZONE_OM_METADATA_LAYOUT_PREFIX);
+
+ boolean enableFSPaths = getEnableFileSystemPaths();
+ if (omMetadataLayoutPrefix && !enableFSPaths) {
+ StringBuilder msg = new StringBuilder();
+ msg.append("Invalid Configuration. Failed to start OM in ");
+ msg.append(OZONE_OM_METADATA_LAYOUT_PREFIX);
+ msg.append(" layout format as '");
+ msg.append(OZONE_OM_ENABLE_FILESYSTEM_PATHS);
+ msg.append("' is false!");
+
+ LOG.error(msg.toString());
+ throw new IOException(msg.toString());
Review comment:
Just a question: wouldn't be `IllegalArgumentException` more natural
here (or just `RuntimeException`). I don't see any connection to IO here.
##########
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemWithFSO.java
##########
@@ -56,9 +56,7 @@
public static Collection<Object[]> data() {
return Arrays.asList(
new Object[]{true, true},
- new Object[]{true, false},
Review comment:
This change seems to be independent of this patch (Am I wrong?)
##########
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileInterfacesWithFSO.java
##########
@@ -36,6 +39,11 @@
@RunWith(Parameterized.class)
public class TestOzoneFileInterfacesWithFSO extends TestOzoneFileInterfaces {
+ @Parameterized.Parameters
+ public static Collection<Object[]> data() {
+ return Arrays.asList(new Object[][] {{false, true, true}});
Review comment:
This change seems to be independent of this patch.
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -3813,4 +3809,67 @@ public void setMinMultipartUploadPartSize(int
partSizeForTest) {
this.minMultipartUploadPartSize = partSizeForTest;
}
+ private void initFSOLayout() throws IOException {
+ // TODO: Temporary workaround for OM upgrade path and will be replaced once
+ // upgrade HDDS-3698 story reaches consensus. Instead of cluster level
+ // configuration, OM needs to check this property on every bucket level.
+ String metaLayout = getOMMetadataLayout();
+ boolean omMetadataLayoutPrefix = StringUtils.equalsIgnoreCase(metaLayout,
+ OZONE_OM_METADATA_LAYOUT_PREFIX);
+
+ boolean enableFSPaths = getEnableFileSystemPaths();
+ if (omMetadataLayoutPrefix && !enableFSPaths) {
+ StringBuilder msg = new StringBuilder();
+ msg.append("Invalid Configuration. Failed to start OM in ");
+ msg.append(OZONE_OM_METADATA_LAYOUT_PREFIX);
+ msg.append(" layout format as '");
+ msg.append(OZONE_OM_ENABLE_FILESYSTEM_PATHS);
+ msg.append("' is false!");
+
+ LOG.error(msg.toString());
+ throw new IOException(msg.toString());
+ }
+
+ boolean isBucketFSOptimized = omMetadataLayoutPrefix && enableFSPaths;
+ OzoneManagerRatisUtils.setBucketFSOptimized(isBucketFSOptimized);
+ String status = isBucketFSOptimized ? "enabled" : "disabled";
+ LOG.info("Configured {}={} and {} optimized OM FS operations",
+ OZONE_OM_METADATA_LAYOUT, metaLayout, status);
+ }
+
+ private void validatesBucketLayoutMismatches() throws IOException {
+ String clusterLevelMetaLayout = getOMMetadataLayout();
+
+ TableIterator<String, ? extends Table.KeyValue<String, OmBucketInfo>>
+ iterator = metadataManager.getBucketTable().iterator();
+
+ while (iterator.hasNext()) {
+ Map<String, String> bucketMeta =
iterator.next().getValue().getMetadata();
+ verifyBucketMetaLayout(clusterLevelMetaLayout, bucketMeta);
+ }
+ }
+
+ private void verifyBucketMetaLayout(String clusterLevelMetaLayout,
+ Map<String, String> bucketMetadata) throws IOException {
+ String bucketMetaLayout = bucketMetadata.get(OZONE_OM_METADATA_LAYOUT);
+ if (StringUtils.isBlank(bucketMetaLayout)) {
+ // Defaulting to SIMPLE
+ bucketMetaLayout = OZONE_OM_METADATA_LAYOUT_DEFAULT;
+ }
+ boolean metadataLayoutEnabled =
Review comment:
Very small nit: Strictly speaking this variable name became misleading:
it's more like `supportedMetadataLayout`. As it can be true when the
configuration is `SIMPLE` but bucket is `PREFIX`ed.
##########
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestRootedOzoneFileSystemWithFSO.java
##########
@@ -44,9 +44,7 @@
public static Collection<Object[]> data() {
return Arrays.asList(
new Object[]{true, true},
- new Object[]{true, false},
Review comment:
This change seems to be independent of this patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
