Also, how many files? What format? Being so slow is an anomaly.
On Thu, Jun 23, 2016 at 11:15 AM, Khurram Faraaz <[email protected]> wrote: > Can you please share the query plan for that long running query here ? > > On Thu, Jun 23, 2016 at 11:40 PM, Tanmay Solanki < > [email protected]> wrote: > > > I am trying to run a query on Apache drill to simply count the number of > > rows in a table stored in parquet format in S3. I am running this on a 20 > > node r3.8xlarge EC2 instance cluster and I have my direct memory set to > > 80GB, heap memory set to 32GB and set the > > planner.memory.max_memory_per_node to a very high value. However, > counting > > the rows in this table takes around 7662 seconds, or around 2 hours, for > > drill to finish the query on a 9.93TB, 56 billion rows, and 174 column > > dataset.It seems like, from the logs and the web console that query > > planning itself is taking near 99% of the time and actual query execution > > is almost taking no time. I ran the same query on PrestoDB of a similar > > setup (20 node r3.8xlarge) and found that it completed in 137 seconds or > > just over 2 minutes. Is there someting wrong with my configuration of > drill > > possibly or is this what is expected for drill. > > >
