smengcl commented on code in PR #11976:
URL: https://github.com/apache/ozone/pull/11976#discussion_r4534443349


##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMOpenKeysDeleteRequest.java:
##########
@@ -268,6 +271,93 @@ public void testDeleteKeyWithHigherUpdateID(
     assertNotInOpenKeyTable(keysWithSameUpdateID);
   }
 
+  /**
+   * Tests that an open key hsync'ed after it was selected for cleanup is kept
+   * as long as the committed key still refers to it.
+   */
+  @ParameterizedTest
+  @MethodSource("bucketLayouts")
+  public void testDeleteHsyncedKeys(BucketLayout buckLayout) throws Exception {
+    this.bucketLayout = buckLayout;
+    final String volume = UUID.randomUUID().toString();
+    final String bucket = UUID.randomUUID().toString();
+
+    OMRequestTestUtils.addVolumeAndBucketToDB(volume, bucket,
+        omMetadataManager, getBucketLayout());
+
+    final List<Pair<Long, OmKeyInfo>> hsyncedKeys = new ArrayList<>();
+    for (Pair<Long, OmKeyInfo> openKey : makeOpenKeys(volume, bucket, 3)) {
+      final OmKeyInfo keyInfo = openKey.getRight().toBuilder()
+          .setOmKeyLocationInfos(Collections.singletonList(
+              new OmKeyLocationInfoGroup(0L, new ArrayList<>(), false)))
+          .build();
+      hsyncedKeys.add(Pair.of(openKey.getLeft(), hsync(keyInfo, 
openKey.getLeft())));
+    }
+    // a non-zero key size so that the keys have blocks that can be queued for 
deletion
+    addToOpenKeyTableDB(100L, hsyncedKeys);

Review Comment:
   nit: this writes the hsync'ed open keys to the DB with `HSYNC_CLIENT_ID` 
already set, so it tests the skip decision but not quite the race in the JIRA, 
where the DB copy has no `HSYNC_CLIENT_ID` (that is why the scan picked it up) 
and only the table cache has the first hsync.
   
   Would it be worth writing the DB copy without the metadata and putting the 
hsync'ed version in the open key table cache with an updateID below 100? That 
would also cover the cache read in `updateOpenKeyTableCache`. Fine to leave as 
is if you think the current form is enough.



##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java:
##########
@@ -281,13 +281,15 @@ public OMClientResponse 
validateAndUpdateCache(OzoneManager ozoneManager, Execut
         dbOpenKeyToDeleteKey = omMetadataManager.getOpenKey(volumeName, 
bucketName,
             keyName, 
Long.parseLong(keyToDelete.getMetadata().get(OzoneConsts.HSYNC_CLIENT_ID)));
         openKeyToDelete = 
omMetadataManager.getOpenKeyTable(getBucketLayout()).get(dbOpenKeyToDeleteKey);
-        openKeyToDelete = openKeyToDelete.toBuilder()
-            .addMetadata(OzoneConsts.OVERWRITTEN_HSYNC_KEY, "true")
-            .setUpdateID(trxnLogIndex)
-            .build();
-        openKeyToDelete.setModificationTime(Time.now());
-        omMetadataManager.getOpenKeyTable(getBucketLayout()).addCacheEntry(
-            dbOpenKeyToDeleteKey, openKeyToDelete, trxnLogIndex);
+        if (openKeyToDelete != null) {

Review Comment:
   nit: could we log a warning in an `else` branch here (and in 
`OMKeyCommitRequestWithFSO`)?
   
   The delete paths (`OMKeyDeleteRequest`, `OMKeysDeleteRequest` and the FSO 
variants) already log `Potentially inconsistent DB state: open key not found 
with dbOpenKey ...` for the same case. After this fix a missing open key here 
should only come from a cluster that hit the bug before upgrading, where the 
blocks were already queued for deletion, so a log line would help find the 
affected keys.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to