any help please
On Tue, Aug 7, 2018 at 1:49 PM, Pranav Agrawal
wrote:
> I am hitting issue,
> https://issues.cloudera.org/browse/DISTRO-800 (related to
> https://issues.apache.org/jira/browse/HIVE-11625)
>
> I am unable to write empty array of types int or string (
I am hitting issue,
https://issues.cloudera.org/browse/DISTRO-800 (related to
https://issues.apache.org/jira/browse/HIVE-11625)
I am unable to write empty array of types int or string (array of size 0)
into parquet, please assist or suggest workaround for the same.
spark version: 2.2.1
AWS EMR:
rray stuff try to explode that columns
> into simple types.
>
> Jorge Machado
>
>
>
>
>
>
> On 4 Jun 2018, at 11:09, Pranav Agrawal wrote:
>
> I am ordering the columns before doing union, so I think that should not
> be an issue,
>
>
>
>
>
>
>
>
union with a dataframe without the arrays types. Could be
> something strange there like ordering or so.
>
> Jorge Machado
>
>
>
>
>
> On 4 Jun 2018, at 10:17, Pranav Agrawal wrote:
>
> schema is exactly the same, not sure why it is failing though.
>
> roo
e one element of the array per line
> 3) flatten the struct
>
> At each step use printSchema() to double check if the types are as you
> expect them to be, and if they are the same for both datasets.
>
> Best regards,
> Alessandro
>
> On 2 June 2018 at 19:48, Pranav Agrawal wr
can't get around this error when performing union of two datasets
(ds1.union(ds2)) having complex data type (struct, list),
*18/06/02 15:12:00 INFO ApplicationMaster: Final app status: FAILED,
exitCode: 15, (reason: User class threw exception:
org.apache.spark.sql.AnalysisException: Union can
can't get around this error when performing union of two datasets having
complex data type (struct, list),
*18/06/02 15:12:00 INFO ApplicationMaster: Final app status: FAILED,
exitCode: 15, (reason: User class threw exception:
org.apache.spark.sql.AnalysisException: Union can only be performed
