Dataset -- Schema for type scala.collection.Set[scala.Int] is not supported

Is this not supported? I found this diff and wonder if this is a bug or am I doing something wrong? see below = import scala.collection.Set case class A(ps: Set[Int], x: Int) val az = Seq(A(Set(1, 2), 1), A(Set(2), 2)) az.toDS

Re: Spark SQL reads all leaf directories on a partitioned Hive table

also need others as well using soft link ls -l cd $SPARK_HOME/conf hive-site.xml -> ${HIVE_HOME/conf/hive-site.xml core-site.xml -> ${HADOOP_HOME}/etc/hadoop/core-site.xml hdfs-site.xml -> ${HADOOP_HOME}/etc/hadoop/hdfs-site.xml Dr Mich Talebzadeh LinkedIn *

Fwd: Spark SQL reads all leaf directories on a partitioned Hive table

-- Forwarded message - From: Hao Ren Date: Thu, Aug 8, 2019 at 4:15 PM Subject: Re: Spark SQL reads all leaf directories on a partitioned Hive table To: Gourav Sengupta Hi Gourva, I am using enableHiveSupport. The table was not created by Spark. The table already exists in

Re: Spark SQL reads all leaf directories on a partitioned Hive table

Hi, Just out of curiosity did you start the SPARK session using enableHiveSupport() ? Or are you creating the table using SPARK? Regards, Gourav On Wed, Aug 7, 2019 at 3:28 PM Hao Ren wrote: > Hi, > I am using Spark SQL 2.3.3 to read a hive table which is partitioned by > day, hour,