The files are copied to the same location in all the nodes of the cluster. And all the nodes have equal access to the files. The files are not located to a single shared file system.
On Fri, Mar 3, 2017 at 7:12 PM, John Omernik <j...@omernik.com> wrote: > Can you help me understand what "local to the cluster" means in the context > of a 5 node cluster? In the plan, the files are all file:// Are the files > replicated to each node? is it a common shared filesystem? Do all 5 nodes > have equal access to the 10 files? I wonder if using a local FS in a > distributed cluster is having some effect on the planning... > > On Fri, Mar 3, 2017 at 6:08 AM, PROJJWAL SAHA <proj.s...@gmail.com> wrote: > > > I did not change the default values used by drill. > > Are you talking of changing planner.memory_limit > > and planner.memory.max_query_memory_per_node ? > > If there are any other debug work that I can do, pls suggest > > > > Regards > > > > On Fri, Mar 3, 2017 at 5:14 PM, Nitin Pawar <nitinpawar...@gmail.com> > > wrote: > > > > > how much memory have you set for planner ? > > > > > > On Fri, Mar 3, 2017 at 5:06 PM, PROJJWAL SAHA <proj.s...@gmail.com> > > wrote: > > > > > > > Hello all, > > > > > > > > I am quering select * from dfs.xxx where yyy (filter condition) > > > > > > > > I am using dfs storage plugin that comes out of the box from drill > on a > > > > 1GB file, local to the drill cluster. > > > > The 1GB file is split into 10 files of 100 MB each. > > > > As expected I see 11 minor and 2 major fagments. > > > > The drill cluster is 5 nodes cluster with 4 cores, 32 GB each. > > > > > > > > One observation is that the query plan time is more than 30 seconds. > I > > > ran > > > > the explain plan query to validate this. > > > > The query execution time is 2 secs. > > > > total time taken is 32secs > > > > > > > > I wanted to understand how can i minimise the query plan time. > > > Suggestions > > > > ? > > > > Is the time taken described above expected ? > > > > Attached is result from explain plan query > > > > > > > > Regards, > > > > Projjwal > > > > > > > > > > > > > > > > > -- > > > Nitin Pawar > > > > > >
