If you can reproduce the issue with Spark 2.0.2 I'd suggest opening a JIRA.
On Fri, Nov 4, 2016 at 5:11 PM, Sam Goodwin wrote:
> I have a table with a few columns, some of which are arrays. Since
> upgrading from Spark 1.6 to Spark 2.0.1, the array fields are always null
> when reading in a Data
I have a table with a few columns, some of which are arrays. Since
upgrading from Spark 1.6 to Spark 2.0.1, the array fields are always null
when reading in a DataFrame.
When writing the Parquet files, the schema of the column is specified as
StructField("packageIds",ArrayType(StringType))
The s