Re: Dataset schema incompatibility bug when reading column partitioned data

2019-04-13 Thread Felix Cheung
I kinda agree it is confusing when a parameter is not used... From: Ryan Blue Sent: Thursday, April 11, 2019 11:07:25 AM To: Bruce Robbins Cc: Dávid Szakállas; Spark Dev List Subject: Re: Dataset schema incompatibility bug when reading column partitioned data

Re: Dataset schema incompatibility bug when reading column partitioned data

2019-04-11 Thread Ryan Blue
;> fix? >> >> Thanks, >> David >> >> Begin forwarded message: >> >> *From: *Dávid Szakállas >> *Subject: **Dataset schema incompatibility bug when reading column >> partitioned data* >> *Date: *2019. March 29. 14:15:27 CET &

Re: Dataset schema incompatibility bug when reading column partitioned data

2019-04-11 Thread Bruce Robbins
id Szakállas > *Subject: **Dataset schema incompatibility bug when reading column > partitioned data* > *Date: *2019. March 29. 14:15:27 CET > *To: *u...@spark.apache.org > > We observed the following bug on Spark 2.4.0: > > scala> > spark.createDataset(Seq((1,2))).write.p

Fwd: Dataset schema incompatibility bug when reading column partitioned data

2019-04-11 Thread Dávid Szakállas
+dev for more visibility. Is this a known issue? Is there a plan for a fix? Thanks, David > Begin forwarded message: > > From: Dávid Szakállas > Subject: Dataset schema incompatibility bug when reading column partitioned > data > Date: 2019. March 29. 14:15:27 CET > To: