wgtmac commented on code in PR #1173:
URL: https://github.com/apache/parquet-mr/pull/1173#discussion_r1381225686
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java:
##########
@@ -1317,6 +1324,27 @@ private int toIntWithCheck(long size) {
return (int)size;
}
+ private void mergeColumnStatistics(Statistics<?> statistics) {
+ if (!currentStatisticsAreValid) {
Review Comment:
From the comment below, I think it serves the same purpose to mark a
statistics as invalid. Merging an empty stats should invalidate aggregate
statistics and ColumnIndexBuilder, IMO.
```
/**
* Returns a boolean specifying if the Statistics object is empty,
* i.e does not contain valid statistics for the page/column yet
* @return true if object is empty, false otherwise
*/
public boolean isEmpty() {
return !hasNonNullValue && !isNumNullsSet();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
