Attendees: Parth, Aman, Hanumath, John O, Jason, Lau Sennels, Vitalii, Arina, Gautam, Jinfeng, Paul, Subbu Srinivasan, Zelaine
1) John indicated he's running into a problem where Drill hangs and becomes unresponsive. Parth indicated he suspects Drill is hitting some type of memory limit. In general, Drill should catch these conditions and throw an exception. But there may be cases where Drill isn't doing a good enough job. In those situations, the last resort is to restart the offending Drillbit. Parth has suggested to John that he collects logs so we can further diagnose the problem. We then went into a discussion about whether Drill could be made more resilient once this situation is hit. E.g., avoid sending queries to the offending Drillbits. Jason indicated that we should avoid getting into this state in the first place and fix these bugs. Everyone agreed, but better handling this error state would also be good. 2) We will be doing a 1.7 dev preview release. Aman will be the release manager. He will email the dev list with a proposed list of fixes to include and a timeline. 3) Jason walked through his new operator unit test framework. The motivation for this is to be able to test operators in isolation without a SQL statement and large data sets. This came about as a result of trying to fix issues with spill to disk in sort. With the framework, you can define batches in JSON where you can better control the size of the batch. I.e., make the batches small so you can trigger spill to disk with a small input. PhysicalOpUnitTestBase is the base class. Jason has implemented a set of basic tests for most operators, derived from this class.
