[jira] [Updated] (FLINK-27655) Implement Avro File statistic collector
[ https://issues.apache.org/jira/browse/FLINK-27655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated FLINK-27655: -- Component/s: Table Store > Implement Avro File statistic collector > --- > > Key: FLINK-27655 > URL: https://issues.apache.org/jira/browse/FLINK-27655 > Project: Flink > Issue Type: Improvement > Components: Table Store >Reporter: Zheng Hu >Priority: Minor > Fix For: table-store-0.2.0 > > > Currently, the flink table store's avro file writer don't provide its File > statistic collector. So we have to use the generic FieldStatsCollector. > In fact, the correct direction is: Making all format writer has their own > FileStatsCollector, so that we can just parse the columnar statistic from the > file tailer, instead of comparing each column max-min when writing the > records into the columnar file. > In this way, I think we can just remove the FileFormatImpl class and > FieldStatsCollector class. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-27655) Implement Avro File statistic collector
[ https://issues.apache.org/jira/browse/FLINK-27655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated FLINK-27655: - Parent: (was: FLINK-27307) Issue Type: Improvement (was: Sub-task) > Implement Avro File statistic collector > --- > > Key: FLINK-27655 > URL: https://issues.apache.org/jira/browse/FLINK-27655 > Project: Flink > Issue Type: Improvement >Reporter: Zheng Hu >Priority: Minor > Fix For: table-store-0.2.0 > > > Currently, the flink table store's avro file writer don't provide its File > statistic collector. So we have to use the generic FieldStatsCollector. > In fact, the correct direction is: Making all format writer has their own > FileStatsCollector, so that we can just parse the columnar statistic from the > file tailer, instead of comparing each column max-min when writing the > records into the columnar file. > In this way, I think we can just remove the FileFormatImpl class and > FieldStatsCollector class. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (FLINK-27655) Implement Avro File statistic collector
[ https://issues.apache.org/jira/browse/FLINK-27655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated FLINK-27655: - Priority: Minor (was: Major) > Implement Avro File statistic collector > --- > > Key: FLINK-27655 > URL: https://issues.apache.org/jira/browse/FLINK-27655 > Project: Flink > Issue Type: Sub-task >Reporter: Zheng Hu >Priority: Minor > Fix For: table-store-0.2.0 > > > Currently, the flink table store's avro file writer don't provide its File > statistic collector. So we have to use the generic FieldStatsCollector. > In fact, the correct direction is: Making all format writer has their own > FileStatsCollector, so that we can just parse the columnar statistic from the > file tailer, instead of comparing each column max-min when writing the > records into the columnar file. > In this way, I think we can just remove the FileFormatImpl class and > FieldStatsCollector class. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (FLINK-27655) Implement Avro File statistic collector
[ https://issues.apache.org/jira/browse/FLINK-27655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated FLINK-27655: - Fix Version/s: table-store-0.2.0 > Implement Avro File statistic collector > --- > > Key: FLINK-27655 > URL: https://issues.apache.org/jira/browse/FLINK-27655 > Project: Flink > Issue Type: Sub-task >Reporter: Zheng Hu >Priority: Major > Fix For: table-store-0.2.0 > > > Currently, the flink table store's avro file writer don't provide its File > statistic collector. So we have to use the generic FieldStatsCollector. > In fact, the correct direction is: Making all format writer has their own > FileStatsCollector, so that we can just parse the columnar statistic from the > file tailer, instead of comparing each column max-min when writing the > records into the columnar file. > In this way, I think we can just remove the FileFormatImpl class and > FieldStatsCollector class. -- This message was sent by Atlassian Jira (v8.20.7#820007)
