[
https://issues.apache.org/jira/browse/PARQUET-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712875#comment-17712875
]
ASF GitHub Bot commented on PARQUET-2261:
-----------------------------------------
wgtmac commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1168127446
##########
src/main/thrift/parquet.thrift:
##########
@@ -753,6 +820,12 @@ struct ColumnMetaData {
/** Byte offset from beginning of file to Bloom filter data. **/
14: optional i64 bloom_filter_offset;
+ /**
+ * Optional statistics to help estimate total memory when converted to in
memory
+ * representations. The histogram contained on these statistics can also be
useful
+ * in some cases for more fine-grained nullability/list length filter
pushdown.
+ */
+ 15: optional SizeEstimationStatistics size_estimate_statistics;
Review Comment:
```suggestion
16: optional SizeEstimationStatistics size_estimate_statistics;
```
##########
src/main/thrift/parquet.thrift:
##########
@@ -753,6 +820,12 @@ struct ColumnMetaData {
/** Byte offset from beginning of file to Bloom filter data. **/
14: optional i64 bloom_filter_offset;
+ /**
+ * Optional statistics to help estimate total memory when converted to in
memory
+ * representations. The histogram contained on these statistics can also be
useful
+ * in some cases for more fine-grained nullability/list length filter
pushdown.
+ */
+ 15: optional SizeEstimationStatistics size_estimate_statistics;
Review Comment:
This patch probably needs to rebase before merging.
> [Format] Add statistics that reflect decoded size to metadata
> -------------------------------------------------------------
>
> Key: PARQUET-2261
> URL: https://issues.apache.org/jira/browse/PARQUET-2261
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-format
> Reporter: Micah Kornfield
> Assignee: Micah Kornfield
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
