Re: Discuss about Drill's schedule policy

2017-08-23 Thread Paul Rogers
Hi Weijie, Thanks for the link. I’d seen this project a bit earlier, along with Apollo [1]. Sparrow is quite interesting, but is designed to place tasks (processes) on available nodes. This is not quite what Drill does: Drill launches multiple waves of “fragments” to all nodes across the

[jira] [Created] (DRILL-5739) Query reads all files after issuing REFRESH TABLE METADATA command

2017-08-23 Thread Divya (JIRA)
Divya created DRILL-5739: Summary: Query reads all files after issuing REFRESH TABLE METADATA command Key: DRILL-5739 URL: https://issues.apache.org/jira/browse/DRILL-5739 Project: Apache Drill

[jira] [Created] (DRILL-5738) Drill query takes 10+ minutes before start executing, excessive Hive metastore queries

2017-08-23 Thread kevin zou (JIRA)
kevin zou created DRILL-5738: Summary: Drill query takes 10+ minutes before start executing, excessive Hive metastore queries Key: DRILL-5738 URL: https://issues.apache.org/jira/browse/DRILL-5738

[GitHub] drill issue #920: DRILL-5737: Hash Agg uses more than the allocated memory u...

2017-08-23 Thread sohami
Github user sohami commented on the issue: https://github.com/apache/drill/pull/920 @Ben-Zvi - Please help to review this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134778939 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/expr/TestSchemaPathMaterialization.java --- @@ -93,4 +93,23 @@ public void

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134788919 --- Diff: exec/vector/src/main/java/org/apache/drill/exec/vector/accessor/TupleAccessor.java --- @@ -48,9 +48,21 @@ MaterializedField

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134539810 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java --- @@ -359,30 +361,109 @@ public Mutator(OperatorExecContext

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134686740 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java --- @@ -359,30 +361,109 @@ public Mutator(OperatorExecContext

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134688368 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java --- @@ -293,7 +294,7 @@ private

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134764401 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/SchemaPathUtil.java --- @@ -0,0 +1,59 @@ +/* +* Licensed to the

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134782398 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/record/TestMaterializedField.java --- @@ -84,4 +89,22 @@ public void testClone() {

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134787569 --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSetSchema.java --- @@ -83,7 +85,13 @@ private void updateStructure(int index,

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134540459 --- Diff: contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java --- @@ -147,10 +147,10 @@ public Boolean

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134788014 --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSetSchema.java --- @@ -94,12 +102,20 @@ private void updateStructure(int index,

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134795741 --- Diff: logical/src/main/java/org/apache/drill/common/expression/SchemaPath.java --- @@ -115,6 +112,33 @@ public static SchemaPath create(NamePart

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

2017-08-23 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r134689741 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -362,16 +363,16 @@ protected boolean

[GitHub] drill issue #910: DRILL-5726: Support Impersonation without authentication f...

2017-08-23 Thread arina-ielchiieva
Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/910 @sohami PR is updated, information about new approach is added in Jira. Please review when possible. --- If your project is set up for it, you can reply to this email and have your reply

Re: Discuss about Drill's schedule policy

2017-08-23 Thread weijie tong
@paul have you noticed the Sparrow project ( https://github.com/radlab/sparrow ) and related paper mentioned in the github . Sparrow is a non-central ,low latency scheduler . This seems meet Drill's demand. I think we can first abstract a scheduler interface like what Spark does , then we can

[GitHub] drill issue #889: DRILL-5691: enhance scalar sub queries checking for the ca...

2017-08-23 Thread arina-ielchiieva
Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/889 +1, LGMT. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] drill issue #889: DRILL-5691: enhance scalar sub queries checking for the ca...

2017-08-23 Thread weijietong
Github user weijietong commented on the issue: https://github.com/apache/drill/pull/889 @arina-ielchiieva thanks for the advice, has corrected that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does