peterxcli commented on code in PR #11324:
URL: https://github.com/apache/ozone/pull/11324#discussion_r4121448280
##########
hadoop-hdds/rocksdb-checkpoint-differ/src/test/java/org/apache/ozone/rocksdiff/TestRocksDBCheckpointDiffer.java:
##########
@@ -1065,12 +1046,72 @@ void diffAllSnapshots(RocksDBCheckpointDiffer differ)
assertThat(actualFiles).containsExactlyInAnyOrderElementsOf(expectedFiles);
}
}
- // Guard against getSSTDiffList silently returning nothing for every input.
assertThat(sawNonEmptyDiff)
.as("expected at least one non-empty SST diff across snapshots")
.isTrue();
}
+ private Set<String> allTablesForDiff() {
+ Set<String> tables = new HashSet<>(COLUMN_FAMILIES_TO_TRACK_IN_DAG);
+ tables.add("compactionLogTable");
+ return tables;
+ }
+
+ private List<SstFileInfo> getTrackedSstFilesFromSnapshot(DifferSnapshotInfo
snap) {
+ return snap.getSstFiles(0, allTablesForDiff());
+ }
+
+ private static List<SstFileInfo> requireSstDiffList(
+ Optional<List<SstFileInfo>> diffList,
+ DifferSnapshotInfo src,
+ DifferSnapshotInfo dest) {
+ if (diffList.isPresent()) {
+ return diffList.get();
+ }
+ throw new AssertionError(String.format(
+ "getSSTDiffList returned empty Optional (DAG could not reach all
destination SSTs) "
+ + "from '%s' to '%s'", src.getDbPath(0), dest.getDbPath(0)));
+ }
Review Comment:
this could be one liner. see
https://github.com/apache/ozone/pull/11324/changes#r4121445764
```suggestion
```
##########
hadoop-hdds/rocksdb-checkpoint-differ/src/test/java/org/apache/ozone/rocksdiff/TestRocksDBCheckpointDiffer.java:
##########
@@ -1046,15 +1029,13 @@ void diffAllSnapshots(RocksDBCheckpointDiffer differ)
}
DifferSnapshotVersion srcSnapVersion = new DifferSnapshotVersion(src,
0, tableToLookUp);
DifferSnapshotVersion destSnapVersion = new
DifferSnapshotVersion(snap, 0, tableToLookUp);
- List<SstFileInfo> sstDiffList = differ.getSSTDiffList(srcSnapVersion,
destSnapVersion, null,
- tableToLookUp, true).orElse(Collections.emptyList());
+ List<SstFileInfo> sstDiffList = requireSstDiffList(
+ differ.getSSTDiffList(srcSnapVersion, destSnapVersion, null,
tableToLookUp, true),
+ src, snap);
Review Comment:
```suggestion
List<SstFileInfo> sstDiffList =
differ.getSSTDiffList(srcSnapVersion, destSnapVersion, null, tableToLookUp,
true).orElseThrow();
```
##########
hadoop-hdds/rocksdb-checkpoint-differ/src/test/java/org/apache/ozone/rocksdiff/TestRocksDBCheckpointDiffer.java:
##########
@@ -1046,15 +1029,13 @@ void diffAllSnapshots(RocksDBCheckpointDiffer differ)
}
DifferSnapshotVersion srcSnapVersion = new DifferSnapshotVersion(src,
0, tableToLookUp);
DifferSnapshotVersion destSnapVersion = new
DifferSnapshotVersion(snap, 0, tableToLookUp);
- List<SstFileInfo> sstDiffList = differ.getSSTDiffList(srcSnapVersion,
destSnapVersion, null,
- tableToLookUp, true).orElse(Collections.emptyList());
+ List<SstFileInfo> sstDiffList = requireSstDiffList(
+ differ.getSSTDiffList(srcSnapVersion, destSnapVersion, null,
tableToLookUp, true),
+ src, snap);
Review Comment:
elsewhere
##########
hadoop-hdds/rocksdb-checkpoint-differ/src/test/java/org/apache/ozone/rocksdiff/TestRocksDBCheckpointDiffer.java:
##########
@@ -1065,12 +1046,72 @@ void diffAllSnapshots(RocksDBCheckpointDiffer differ)
assertThat(actualFiles).containsExactlyInAnyOrderElementsOf(expectedFiles);
}
}
- // Guard against getSSTDiffList silently returning nothing for every input.
assertThat(sawNonEmptyDiff)
.as("expected at least one non-empty SST diff across snapshots")
.isTrue();
}
+ private Set<String> allTablesForDiff() {
+ Set<String> tables = new HashSet<>(COLUMN_FAMILIES_TO_TRACK_IN_DAG);
+ tables.add("compactionLogTable");
+ return tables;
+ }
+
+ private List<SstFileInfo> getTrackedSstFilesFromSnapshot(DifferSnapshotInfo
snap) {
+ return snap.getSstFiles(0, allTablesForDiff());
+ }
Review Comment:
inline this too?
##########
hadoop-hdds/rocksdb-checkpoint-differ/src/test/java/org/apache/ozone/rocksdiff/TestRocksDBCheckpointDiffer.java:
##########
@@ -1065,12 +1046,72 @@ void diffAllSnapshots(RocksDBCheckpointDiffer differ)
assertThat(actualFiles).containsExactlyInAnyOrderElementsOf(expectedFiles);
}
}
- // Guard against getSSTDiffList silently returning nothing for every input.
assertThat(sawNonEmptyDiff)
.as("expected at least one non-empty SST diff across snapshots")
.isTrue();
}
+ private Set<String> allTablesForDiff() {
+ Set<String> tables = new HashSet<>(COLUMN_FAMILIES_TO_TRACK_IN_DAG);
+ tables.add("compactionLogTable");
+ return tables;
+ }
+
+ private List<SstFileInfo> getTrackedSstFilesFromSnapshot(DifferSnapshotInfo
snap) {
+ return snap.getSstFiles(0, allTablesForDiff());
+ }
Review Comment:
btw, question while. tracing the code, do you know what is the meaning of
the `version` in `List<SstFileInfo> getSstFiles(int version, Set<String>
tablesToLookup) {`?
```java
public class DifferSnapshotInfo {
...
private final NavigableMap<Integer, List<SstFileInfo>> versionSstFiles;
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
