Re: Explain Plan for Parquet data is taking a lot of timre

2017-03-05 Thread Padma Penumarthy
Basically, it is rebuilding the metadata cache (reading all the parquet files that are there), since it thinks something is changed in the directory. During planning, we check if metadata cache file needs to be updated by checking modification time of directory against modification time of met

RE: Explain Plan for Parquet data is taking a lot of timre

2017-03-05 Thread Jeena Vinod
Hi Kunal, Thanks for the response. Attaching the log with DEBUG enabled for the mentioned loggers. I had to trim the log for the query, since this mailer allows max 1MB. >From the log files, the below step seems to be taking the most time. Since >refresh metadata on this path is already done,

Re: Minimise query plan time for dfs plugin for local file system on tsv file

2017-03-05 Thread PROJJWAL SAHA
The files are copied to the same location in all the nodes of the cluster. And all the nodes have equal access to the files. The files are not located to a single shared file system. On Fri, Mar 3, 2017 at 7:12 PM, John Omernik wrote: > Can you help me understand what "local to the cluster" mean