lilei1128 commented on code in PR #10792:
URL: https://github.com/apache/paimon/pull/10792#discussion_r4374253653
##########
paimon-core/src/main/java/org/apache/paimon/operation/DataEvolutionSplitRead.java:
##########
@@ -1620,10 +1685,72 @@ static long selectedRowCount(DataFileMeta file,
List<Range> rowRanges) {
.sum();
}
- private static long selectedRowCount(
+ @VisibleForTesting
+ static long selectedRowCount(
DataFileMeta file,
@Nullable List<Range> rowRanges,
- @Nullable FileIndexResult fileIndexResult) {
+ @Nullable FileIndexResult fileIndexResult,
+ @Nullable DeletionVector deletionVector,
+ @Nullable Range deletionVectorRange) {
+ if (deletionVector != null && !deletionVector.isEmpty()) {
+ RoaringBitmap32 selection = file.toFileSelection(rowRanges);
+ if (fileIndexResult instanceof BitmapIndexResult) {
+ RoaringBitmap32 indexSelection = ((BitmapIndexResult)
fileIndexResult).get();
+ selection =
+ selection == null
+ ? indexSelection.clone()
+ : RoaringBitmap32.and(selection,
indexSelection);
+ }
+ long rangeStart =
+ deletionVectorRange == null
+ ? file.nonNullFirstRowId()
+ : deletionVectorRange.from;
+ if (selection == null) {
+ if (deletionVectorRange != null
+ &&
deletionVectorRange.equals(file.nonNullRowIdRange())) {
+ return file.rowCount() - deletionVector.getCardinality();
Review Comment:
Fixed by pushing the DV into the reader selection and reused it for the
heuristic: readTarget builds the selection once and carries it in
FileReadTarget, so the sidecar decision and the .row reader share the same
live-rows bitmap.
The pushdown is sidecar-only, so the Parquet path is unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
