Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21893#discussion_r206234291 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -437,6 +437,8 @@ private[hive] class HiveClientImpl( throw new AnalysisException("Hive index table is not supported.") }, schema = schema, + hasMultiFormatPartitions = + shim.getAllPartitions(client, h).map(_.getInputFormatClass).distinct.size > 1, --- End diff -- Fetching the metadata of all the partitions will slow down all the queries when the number of partitions is huge. We will face a huge perf regression.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org