kbuci commented on issue #16591:
URL: https://github.com/apache/hudi/issues/16591#issuecomment-3948098588
@nsivabalan We initially had a short term fix of updating
`getClusteringPlan` to be
```
public static Option<Pair<HoodieInstant, HoodieClusteringPlan>>
getClusteringPlan(HoodieTableMetaClient metaClient, HoodieInstant
pendingReplaceInstant) {
try {
Option<HoodieRequestedReplaceMetadata> requestedReplaceMetadata =
getRequestedReplaceMetadata(metaClient, pendingReplaceInstant);
if (requestedReplaceMetadata.isPresent() &&
WriteOperationType.CLUSTER.name().equals(requestedReplaceMetadata.get().getOperationType()))
{
return Option.of(Pair.of(pendingReplaceInstant,
requestedReplaceMetadata.get().getClusteringPlan()));
}
return Option.empty();
} catch (IOException | HoodieIOException e) {
// Check if requested instant file is still in active timeline
if
(metaClient.reloadActiveTimeline().containsInstant(pendingReplaceInstant)) {
// Since HoodieIOException is not of type IOException, need to
extract the root cause IOException before re-throwing
final IOException rootCause;
if (e instanceof HoodieIOException) {
rootCause = ((HoodieIOException) e).getIOException();
} else {
rootCause = (IOException) e;
}
throw new HoodieIOException("Error reading clustering plan " +
pendingReplaceInstant.getTimestamp(), rootCause);
} else {
LOG.warn(String.format("Error reading requested replace metadata %s
due to no longer being timeline."
+ "This could be due to instant being rolled back",
pendingReplaceInstant));
return Option.empty();
}
}
}
```
but this doesn't seem to work on GCS: if the replacecommit.requested file is
deleted by a concurrent rollback while this function is running then this
function re-throws the
`Caused by: java.io.FileNotFoundException: Item not found: ' . . .
.replacecommit.requested'. Note, it is possible that the live version is still
available but the requested generation is deleted.` as a `HoodieIOException`.
Which implies that the above
`metaClient.reloadActiveTimeline().containsInstant(pendingReplaceInstant)` call
returned `true` despite the fact the file was being concurrently deleted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
