hemantk-12 commented on code in PR #4236:
URL: https://github.com/apache/ozone/pull/4236#discussion_r1096286520
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestKeyManagerImpl.java:
##########
@@ -1351,6 +1353,53 @@ public void testGetFileStatusWithFakeDir() throws
IOException {
Assert.assertTrue(ozoneFileStatus.isFile());
}
+ @Test
+ public void testGetFileStatusWithFakeDirFalsePositive() throws IOException {
+ String dirName = "foo2";
+ String fileName = "bar2";
+ String keyName1 = "foo1";
+ // keyName2 = "foo2/bar2"
+ String keyName2 = dirName + OZONE_URI_DELIMITER + fileName;
+ OzoneFileStatus ozoneFileStatus;
+
+ // create a key "foo1" in bucket1
+ OmKeyArgs keyArgs =
createBuilder(BUCKET_NAME).setKeyName(keyName1).build();
+ OpenKeySession keySession = writeClient.openKey(keyArgs);
+ keyArgs.setLocationInfoList(
+ keySession.getKeyInfo().getLatestVersionLocations().getLocationList());
+ writeClient.commitKey(keyArgs, keySession.getId());
+
+ // create a key "foo2/bar2" in bucket2
+ keyArgs = createBuilder(BUCKET2_NAME).setKeyName(keyName2).build();
+ keySession = writeClient.createFile(keyArgs, true, true);
+ keyArgs.setLocationInfoList(
+ keySession.getKeyInfo().getLatestVersionLocations().getLocationList());
+ writeClient.commitKey(keyArgs, keySession.getId());
Review Comment:
nit: These lines (1373-1377) are repetitive as line from 1366-1370. May be
create a helper function or use existing one if there is any.
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java:
##########
@@ -1178,21 +1178,18 @@ private OmKeyInfo createFakeDirIfShould(String volume,
String bucket,
String keyName, BucketLayout layout) throws IOException {
OmKeyInfo fakeDirKeyInfo = null;
String dirKey = OzoneFSUtils.addTrailingSlashIfNeeded(keyName);
- String fileKeyBytes = metadataManager.getOzoneKey(volume, bucket, keyName);
+ String targetKey = OzoneFSUtils.addTrailingSlashIfNeeded(
+ metadataManager.getOzoneKey(volume, bucket, keyName));
try (TableIterator<String, ? extends Table.KeyValue<String, OmKeyInfo>>
keyTblItr = metadataManager.getKeyTable(layout).iterator()) {
- Table.KeyValue<String, OmKeyInfo> keyValue =
- keyTblItr
- .seek(OzoneFSUtils.addTrailingSlashIfNeeded(fileKeyBytes));
-
- if (keyValue != null) {
- Path fullPath = Paths.get(keyValue.getValue().getKeyName());
- Path subPath = Paths.get(dirKey);
- OmKeyInfo omKeyInfo = keyValue.getValue();
- if (fullPath.startsWith(subPath)) {
- // create fake directory
- fakeDirKeyInfo = createDirectoryKey(omKeyInfo, dirKey);
- }
+ Table.KeyValue<String, OmKeyInfo> keyValue = keyTblItr.seek(targetKey);
+
+ // HDDS-7871: RocksIterator#seek() may position at the key
Review Comment:
Is it because seek may position past target or bloom filter or when item is
not in the table?
As per my understanding it seems like [false positive because of bloom
filter](https://github.com/facebook/rocksdb/wiki/Prefix-Seek). Please correct
me if I'm wrong.
When I was testing it, my test was failure for following scenario.
Items in the table was:
| KeyInfoTable |
|-------------------|
| /vol1/bucket1/key1 |
| /vol1/bucket1/key2 |
| /vol1/bucket1/key3 |
| /vol1/bucket1/key4 |
| /vol1/bucket1/key5 |
Was looking for `/vol2/bucket2/key1` and getting false positive.
May point is if it is because of bloom filter and seek is pointing to wrong
item, we might miss the "fakeDir".
Eg. You have following table
| KeyInfoTable |
|-----------------------|
| /vol1/bucket1/dir1/key1 |
| /vol1/bucket1/dir1/key2 |
| /vol1/bucket1/dir1/key3 |
| /vol1/bucket2/dir1/key1 |
| /vol1/bucket2/dir1/key2 |
and now you are looking for `/vol1/bucket2/dir1` but seek is pointing to `
/vol1/bucket1/dir1/key1`. So it will be a miss all the time now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
