Hi Nicolas,
thanks a ton for your kind response, I will surely try this out.
Regards,
Gourav Sengupta
On Sun, Aug 29, 2021 at 11:01 PM Nicolas Paris
wrote:
> as a workaround turn off pruning :
>
> spark.sql.hive.metastorePartitionPruning false
> spark.sql.hive.convertMetastoreParquet false
>
Sessional Greetings ,
We're doing tpc-ds query tests using Spark 3.0.2 on kubernetes with data
on HDFS and we're observing delays in query execution time when compared to
Spark 3.0.1 on same environment. We've observed that some stages fail, but
looks like it is taking some time to realise
as a workaround turn off pruning :
spark.sql.hive.metastorePartitionPruning false
spark.sql.hive.convertMetastoreParquet false
see
https://github.com/awslabs/aws-glue-data-catalog-client-for-apache-hive-metastore/issues/45
On Tue Aug 24, 2021 at 9:18 AM CEST, Gourav Sengupta wrote:
> Hi,
>
> I
