Since the accounting changed (more accurate), the termination condition for the sort operator will be different than before. In fact, this likely will be sooner since our accounting is much larger than previously (since we correctly consider the entire allocation rather than simply the used allocation).
Hakim, Steven and I were discussing the need to update the ExternalSort operator to use the new allocator functionality to better manage its memory envelope. Would you be interested in working on this since you seem to be working with that code the most? Basically, it used to be that there was no way the sort operator would be able to correctly detect a memory condition and so it jumped through a bunch of hoops to try to figure out the termination condition.With the transfer accounting in place, this code can be greatly simplified to just use the current operator memory allocation. -- Jacques Nadeau CTO and Co-Founder, Dremio On Wed, Dec 30, 2015 at 10:48 AM, rahul challapalli < challapallira...@gmail.com> wrote: > I installed the latest master and ran this query. So > planner.memory.max_query_memory_per_node should have been the default > value. I switched back to 1.4.0 branch and this query completed > successfully. > > On Wed, Dec 30, 2015 at 10:37 AM, Abdel Hakim Deneche < > adene...@maprtech.com > > wrote: > > > Rahul, > > > > How much memory was assigned to the sort operator ( > > planner.memory.max_query_memory_per_node) ? > > > > On Wed, Dec 30, 2015 at 9:54 AM, rahul challapalli < > > challapallira...@gmail.com> wrote: > > > > > I am seeing an OOM error while executing a simple CTAS query. I raised > > > DRILL-4324 for this. The query mentioned in the JIRA used to complete > > > successfully without any issue prior to 1.5. Any idea what could have > > > caused the regression? > > > > > > - Rahul > > > > > > > > > > > -- > > > > Abdelhakim Deneche > > > > Software Engineer > > > > <http://www.mapr.com/> > > > > > > Now Available - Free Hadoop On-Demand Training > > < > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > >
