[ https://issues.apache.org/jira/browse/SPARK-26663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17308648#comment-17308648 ]
Surendar Thina G commented on SPARK-26663: ------------------------------------------ A similar issue is documented here, that is specific to ORC files SPARK-28098. > Cannot query a Hive table with subdirectories > --------------------------------------------- > > Key: SPARK-26663 > URL: https://issues.apache.org/jira/browse/SPARK-26663 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.4.0 > Reporter: Aäron > Priority: Major > > Hello, > > I want to report the following issue (my first one :) ) > When I create a table in Hive based on a union all then Spark 2.4 is unable > to query this table. > To reproduce: > *Hive 1.2.1* > {code:java} > hive> creat table a(id int); > insert into a values(1); > hive> creat table b(id int); > insert into b values(2); > hive> create table c(id int) as select id from a union all select id from b; > {code} > > *Spark 2.3.1* > > {code:java} > scala> spark.table("c").show > +---+ > | id| > +---+ > | 1| > | 2| > +---+ > scala> spark.table("c").count > res5: Long = 2 > {code} > > *Spark 2.4.0* > {code:java} > scala> spark.table("c").show > 19/01/18 17:00:49 WARN HiveMetastoreCatalog: Unable to infer schema for table > perftest_be.c from file format ORC (inference mode: INFER_AND_SAVE). Using > metastore schema. > +---+ > | id| > +---+ > +---+ > scala> spark.table("c").count > res3: Long = 0 > {code} > I did not find an existing issue for this. Might be important to investigate. > > +Extra info:+ Spark 2.3.1 and 2.4.0 use the same spark-defaults.conf. > > Kind regards. > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org