@Jason, @Vitalli, Any thoughts on this question, since both you worked on fix of DRILL-4203?
Looking through the code, there is a third case [1], where this flag is set to false when Parquet metadata is cloned (after partition pruning, etc). That means, for the 2nd case where the flag is set to true, if there is pruning happening, the new parquet metadata will see the flag is flipped to false. This does not make sense to me. On Mon, Oct 24, 2016 at 3:10 PM, Jinfeng Ni <[email protected]> wrote: > Hello All, > > DRILL-4203 addressed the date field issue. In the fix, it introduced > a new field in ParquetTableMetadata_v2 : isDateCorrect. I have some > difficulty in understanding the meaning of this field. > > According to [1], this field is set to false, when Drill gets parquet > metadata from parquet footer. This field is set to true in code flow > of [2] and [3], when Drill gets parquet metadata from meta data cache. > > Questions I have: > 1. If the parquet files are generated with Drill after DRILL-4203, > Drill still thinks date field is NOT correct (isDateCorrect = false)? > 2. Why does this filed have nothing to do with "autoCorrection" flag > [4]? If someone turns off autoCorrection, will it have impact on this > "isDateCorrect" flag ? > > Thanks in advance for any input, > > Jinfeng > > > [1] > https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java#L932 > [2] > https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java#L936 > [3] > https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java#L187 > [4] > https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java#L354-L355
