Hi Krzysztof, sorry for the late reply. The community is very busy at the moment with the final two weeks of Flink 1.15.
The parameters you have mentioned are mostly relevant for the internal conversion or representation from Parquet types to Flink's SQL type system. - isUtcTimestamp denotes whether timestamps should be represented as SQL UTC timestamps - batchSize is an internal number of how many rows are put into one vector. Vectors are used internally in Flink SQL for performance reasons to enable faster execution on batches i.e. for Hive we use the following default value [1] - isCaseSensitive is used to map the field/column names from parquet and match them to columns in Flink I have also included @jingsongl...@gmail.com who is more familiar with the parquet format. Best, Fabian [1] https://github.com/apache/flink/blob/d8a031c2b7d7b73fe38a3f894913d3dcaa5a4111/flink-table/flink-table-common/src/main/java/org/apache/flink/table/data/columnar/vector/VectorizedColumnBatch.java#L46 On Mon, Jan 24, 2022 at 4:32 PM Krzysztof Chmielewski <krzysiek.chmielew...@gmail.com> wrote: > > Hi, > I would like to bump this up a little bit. > > The isCaseSensitive is rather clear. If this is false, then column read in > parquet file is case insensitive. > batchSize - how many records we read from the Parquet file before passing it > to the upper classes right? > > Could someone describe what timestamp flab does with some examples? > > Regards, > Krzysztof Chmielewski > > > pon., 10 sty 2022 o 14:59 Krzysztof Chmielewski > <krzysiek.chmielew...@gmail.com> napisał(a): >> >> Hi, >> I would like to ask for some more details regarding three >> ParquetColumnarRowInputFormat contruction parameters. >> >> The parameters are: >> batchSize, >> isUtcTimestamp, >> isCaseSensitive >> >> The parametr names gives some hint about their purpose but there is no >> description in docs (java, flink page). >> >> Could you provide me some information about the batching process and other >> two boolean flags? >> >> Regards, >> Krzysztof Chmielewski
