Re: Reading Parquet file with array of structs cause error

2022-11-16 Thread Jing Ge
day, November 15, 2022 at 3:39 PM >> *To: *Benenson, Michael >> *Cc: *user@flink.apache.org , Deshpande, Omkar < >> omkar_deshpa...@intuit.com>, Vora, Jainik >> *Subject: *Re: Reading Parquet file with array of structs cause error >> >> This email is from an external s

Re: Reading Parquet file with array of structs cause error

2022-11-15 Thread liu ron
ate Jira? > > > > > > *From: *Jing Ge > *Date: *Tuesday, November 15, 2022 at 3:39 PM > *To: *Benenson, Michael > *Cc: *user@flink.apache.org , Deshpande, Omkar < > omkar_deshpa...@intuit.com>, Vora, Jainik > *Subject: *Re: Reading Parquet file with arra

Re: Reading Parquet file with array of structs cause error

2022-11-15 Thread Benenson, Michael via user
, Vora, Jainik Subject: Re: Reading Parquet file with array of structs cause error This email is from an external sender. Hi Michael, Currently, ParquetColumnarRowInputFormat does not support schemas with nested columns. If your parquet file stores Avro records. You might want to try e.g.

Re: Reading Parquet file with array of structs cause error

2022-11-15 Thread Jing Ge
Hi Michael, Currently, ParquetColumnarRowInputFormat does not support schemas with nested columns. If your parquet file stores Avro records. You might want to try e.g. Avro Generic record[1]. [1] https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/connectors/datastream/formats/parquet

Reading Parquet file with array of structs cause error

2022-11-15 Thread Benenson, Michael via user
Hi, folks I’m using flink 1.16.0, and I would like to read Parquet file (attached), that has schema [1]. I could read this file with Spark, but when I try to read it with Flink 1.16.0 (program attached) using schema [2] I got IndexOutOfBoundsException [3] My code, and parquet file are attache