jojochuang commented on code in PR #9399:
URL: https://github.com/apache/ozone/pull/9399#discussion_r2801566909
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java:
##########
@@ -954,24 +954,18 @@ public int getInFlightSnapshotCount() {
return inFlightSnapshotCount.get();
}
- private int getIndexFromToken(final String token) throws IOException {
+ private String validateToken(final String token) throws IOException {
Review Comment:
A valid token is either empty/blank or a string of one or more digits.
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/SnapshotDiffManager.java:
##########
@@ -1256,72 +1272,29 @@ long generateDiffReport(
keyPrefix + codecRegistry.asObject(newKeyName, String.class),
isDirectoryObject ? fsDirTable : fsTable,
isDirectoryObject ? tsDirTable : tsTable);
+ if (!isObjectModified || !Arrays.equals(oldKeyName, newKeyName))
{
+ jobKey = addToReport(jobId, index++,
getDiffReportEntry(RENAME, oldKey, newKey));
+ }
if (isObjectModified) {
// Here, oldKey name is returned as modified. Modified key name
// is based on base snapshot (from snapshot).
- modifyDiffs.add(codecRegistry.asRawData(SnapshotDiffReportOzone
- .getDiffReportEntry(MODIFY, oldKey)));
- }
- if (!isObjectModified || !Arrays.equals(oldKeyName, newKeyName))
{
- renameDiffs.add(codecRegistry.asRawData(
- SnapshotDiffReportOzone.getDiffReportEntry(RENAME, oldKey,
- newKey)));
+ String modifiedJobKey = addToReport(jobId, index++,
getDiffReportEntry(MODIFY, oldKey));
+ if (modifiedJobKey.compareTo(jobKey) > 0) {
Review Comment:
if the modified key is larger than the original job key, update the original
job key, so that later largestJobKey can update to the largest key so far.
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/SnapshotDiffManager.java:
##########
@@ -588,16 +634,12 @@ Pair<List<DiffReportEntry>, String> createPageResponse(
*/
@VisibleForTesting
void checkReportsIntegrity(final SnapshotDiffJob diffJob,
- final int pageStartIdx,
- final int numberOfEntriesInPage)
- throws IOException {
- if ((pageStartIdx >= diffJob.getTotalDiffEntries() &&
- numberOfEntriesInPage != 0) || (pageStartIdx <
- diffJob.getTotalDiffEntries() && numberOfEntriesInPage == 0)) {
- LOG.error("Expected TotalDiffEntries: {} but found " +
- "TotalDiffEntries: {}",
- diffJob.getTotalDiffEntries(),
- pageStartIdx + numberOfEntriesInPage);
+ final String largestPageIndex,
+ boolean lastPage) throws IOException {
+ // For last page check last entry returned if the largest entry key equals
the largest key stored in the job entry.
+ if (lastPage && !diffJob.getLargestEntryKey().equals(largestPageIndex)) {
Review Comment:
LargestEntryKey is used for integrity check.
Without the intermediate tables, there's no longer a simple way to verify
that all expected diff entries were successfully written. The largestEntryKey
serves as a checkpoint for integrity verification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
