[
https://issues.apache.org/jira/browse/PARQUET-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17777474#comment-17777474
]
ASF GitHub Bot commented on PARQUET-2261:
-----------------------------------------
etseidl commented on PR #1177:
URL: https://github.com/apache/parquet-mr/pull/1177#issuecomment-1771735094
Thanks @wgtmac, this looks great! I'm not sure if this is in scope for this
PR, but it would be nice if the CLI was aware of the changes. Specifically, it
would be great if the `column-index` command could write out the unencoded
sizes and histograms. The former is pretty straightforward, but I'm not
entirely sure what the right approach is for the histograms. Also, `rewrite`
currently ignores the new statistics. For a straight copy, all that's needed is
to add `chunk.getSizeStatistics()` to the arg list of
`ColumnChunkMetadata.get()`
[here](https://github.com/wgtmac/parquet-mr/blob/45e8ee427e8766489e4a84246f452178ceefb75a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java#L1358).
> [Format] Add statistics that reflect decoded size to metadata
> -------------------------------------------------------------
>
> Key: PARQUET-2261
> URL: https://issues.apache.org/jira/browse/PARQUET-2261
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-format
> Reporter: Micah Kornfield
> Assignee: Micah Kornfield
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
