CBO not working for Parquet Files

2018-08-24 Thread rajat mishra
Hi All, I have an external table in spark whose underlying data files are in parquet format. The table is partitioned. When I try to computed the statistics for a query where partition column is in where clause, the statistics returned contains only the sizeInBytes and not the no of rows count.

CBO not predicting cardinality on partition columns for Parquet tables

2018-08-21 Thread rajat mishra
Hi All, I have an external table in spark whose underlying data files are in parquet format. The table is partitioned. When I try to computed the statistics for a query where partition column is in where clause, the statistics returned contains only the sizeInBytes and not the no of rows count.