[
https://issues.apache.org/jira/browse/ORC-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170699#comment-16170699
]
ASF GitHub Bot commented on ORC-203:
------------------------------------
Github user sadikovi commented on a diff in the pull request:
https://github.com/apache/orc/pull/169#discussion_r139537518
--- Diff: java/core/src/java/org/apache/orc/impl/ColumnStatisticsImpl.java
---
@@ -608,8 +609,23 @@ public void merge(ColumnStatisticsImpl other) {
OrcProto.StringStatistics.Builder str =
OrcProto.StringStatistics.newBuilder();
if (getNumberOfValues() != 0) {
- str.setMinimum(getMinimum());
- str.setMaximum(getMaximum());
+ String value = getMinimum();
+ if (value != null && value.length() > 1024) {
--- End diff --
I suggest you make it a constant somewhere and reference that.
I would also consider making it configurable with default value of `1024`.
> Modify the StringStatistics to trim minimum and maximum values
> --------------------------------------------------------------
>
> Key: ORC-203
> URL: https://issues.apache.org/jira/browse/ORC-203
> Project: ORC
> Issue Type: Bug
> Reporter: Owen O'Malley
> Assignee: Ajay Yadava
>
> Currently the StringStatistics will record the entire value for minimum or
> maximum. It creates large protobuf objects and serves very little value. I
> think we should trim long strings to 1024 characters and record the fact that
> they were trimmed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
