rakeshadr commented on a change in pull request #2151:
URL: https://github.com/apache/ozone/pull/2151#discussion_r620253004
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -3813,4 +3809,67 @@ public void setMinMultipartUploadPartSize(int
partSizeForTest) {
this.minMultipartUploadPartSize = partSizeForTest;
}
+ private void initFSOLayout() throws IOException {
+ // TODO: Temporary workaround for OM upgrade path and will be replaced once
+ // upgrade HDDS-3698 story reaches consensus. Instead of cluster level
+ // configuration, OM needs to check this property on every bucket level.
+ String metaLayout = getOMMetadataLayout();
+ boolean omMetadataLayoutPrefix = StringUtils.equalsIgnoreCase(metaLayout,
+ OZONE_OM_METADATA_LAYOUT_PREFIX);
+
+ boolean enableFSPaths = getEnableFileSystemPaths();
+ if (omMetadataLayoutPrefix && !enableFSPaths) {
+ StringBuilder msg = new StringBuilder();
+ msg.append("Invalid Configuration. Failed to start OM in ");
+ msg.append(OZONE_OM_METADATA_LAYOUT_PREFIX);
+ msg.append(" layout format as '");
+ msg.append(OZONE_OM_ENABLE_FILESYSTEM_PATHS);
+ msg.append("' is false!");
+
+ LOG.error(msg.toString());
+ throw new IOException(msg.toString());
Review comment:
Sure, I will use IllegalArgumentException
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -3813,4 +3809,67 @@ public void setMinMultipartUploadPartSize(int
partSizeForTest) {
this.minMultipartUploadPartSize = partSizeForTest;
}
+ private void initFSOLayout() throws IOException {
+ // TODO: Temporary workaround for OM upgrade path and will be replaced once
+ // upgrade HDDS-3698 story reaches consensus. Instead of cluster level
+ // configuration, OM needs to check this property on every bucket level.
+ String metaLayout = getOMMetadataLayout();
+ boolean omMetadataLayoutPrefix = StringUtils.equalsIgnoreCase(metaLayout,
+ OZONE_OM_METADATA_LAYOUT_PREFIX);
+
+ boolean enableFSPaths = getEnableFileSystemPaths();
+ if (omMetadataLayoutPrefix && !enableFSPaths) {
+ StringBuilder msg = new StringBuilder();
+ msg.append("Invalid Configuration. Failed to start OM in ");
+ msg.append(OZONE_OM_METADATA_LAYOUT_PREFIX);
+ msg.append(" layout format as '");
+ msg.append(OZONE_OM_ENABLE_FILESYSTEM_PATHS);
+ msg.append("' is false!");
+
+ LOG.error(msg.toString());
+ throw new IOException(msg.toString());
+ }
+
+ boolean isBucketFSOptimized = omMetadataLayoutPrefix && enableFSPaths;
+ OzoneManagerRatisUtils.setBucketFSOptimized(isBucketFSOptimized);
+ String status = isBucketFSOptimized ? "enabled" : "disabled";
+ LOG.info("Configured {}={} and {} optimized OM FS operations",
+ OZONE_OM_METADATA_LAYOUT, metaLayout, status);
+ }
+
+ private void validatesBucketLayoutMismatches() throws IOException {
+ String clusterLevelMetaLayout = getOMMetadataLayout();
+
+ TableIterator<String, ? extends Table.KeyValue<String, OmBucketInfo>>
+ iterator = metadataManager.getBucketTable().iterator();
+
+ while (iterator.hasNext()) {
+ Map<String, String> bucketMeta =
iterator.next().getValue().getMetadata();
+ verifyBucketMetaLayout(clusterLevelMetaLayout, bucketMeta);
+ }
+ }
+
+ private void verifyBucketMetaLayout(String clusterLevelMetaLayout,
+ Map<String, String> bucketMetadata) throws IOException {
+ String bucketMetaLayout = bucketMetadata.get(OZONE_OM_METADATA_LAYOUT);
+ if (StringUtils.isBlank(bucketMetaLayout)) {
+ // Defaulting to SIMPLE
+ bucketMetaLayout = OZONE_OM_METADATA_LAYOUT_DEFAULT;
+ }
+ boolean metadataLayoutEnabled =
Review comment:
Agreed. I will update it.
##########
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestRootedOzoneFileSystemWithFSO.java
##########
@@ -44,9 +44,7 @@
public static Collection<Object[]> data() {
return Arrays.asList(
new Object[]{true, true},
- new Object[]{true, false},
Review comment:
Thanks @elek, Its related to the patch. `False` value is setting to
`enabledFileSystemPaths` which is an invalid configuration and throws exception
during OM startup. Thats why, I have removed following two combinations to make
the test happy.
```
new Object[]{false, true},
new Object[]{false, false});
```
##########
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemWithFSO.java
##########
@@ -56,9 +56,7 @@
public static Collection<Object[]> data() {
return Arrays.asList(
new Object[]{true, true},
- new Object[]{true, false},
Review comment:
Thanks @elek, Its related to the patch. `False` value is setting to
`enabledFileSystemPaths` which is an invalid configuration and throws exception
during OM startup. Thats why, I have removed following two combinations to make
the test happy.
```
new Object[]{false, true},
new Object[]{false, false});
```
##########
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileInterfacesWithFSO.java
##########
@@ -36,6 +39,11 @@
@RunWith(Parameterized.class)
public class TestOzoneFileInterfacesWithFSO extends TestOzoneFileInterfaces {
+ @Parameterized.Parameters
+ public static Collection<Object[]> data() {
+ return Arrays.asList(new Object[][] {{false, true, true}});
Review comment:
Thanks @elek, Its related to the patch. `False` value is setting to
`enabledFileSystemPaths` which is an invalid configuration and throws exception
during OM startup. Thats why, I have removed following the combination to make
the test happy.
```
{true, false, false}}
```
Refer code, where its setting enabledFileSystemPaths to false :
[TestOzoneFileInterfaces.java#L101](https://github.com/apache/ozone/blob/master/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileInterfaces.java#L101)
[TestOzoneFileInterfaces.java#L128](https://github.com/apache/ozone/blob/master/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileInterfaces.java#L128)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
