Yeah, Spark SQL Parquet support need to do some metadata discovery when
firstly importing a folder containing Parquet files, and discovered
metadata is cached.
Cheng
On 7/17/15 1:56 PM, shsh...@tsmc.com wrote:
Hi all,
our scenario is to generate lots of folders containinig parquet file and
Hi all,
our scenario is to generate lots of folders containinig parquet file and
then uses add partition to add these folder locations to a hive table;
when trying to read the hive table using Spark,
following logs would show up and took a lot of time on reading them;
but this won't happen after
