[jira] [Updated] (DRILL-4657) Rank() will return wrong results if a frame of data is too big (more than 2 batches)

[Deneche A. Hakim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-4657:

Assignee: Aman Sinha  (was: Deneche A. Hakim)

> Rank() will return wrong results if a frame of data is too big (more than 2 
> batches)
> 
>
> Key: DRILL-4657
> URL: https://issues.apache.org/jira/browse/DRILL-4657
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Relational Operators
>Affects Versions: 1.3.0
>Reporter: Deneche A. Hakim
>Assignee: Aman Sinha
>Priority: Critical
> Fix For: 1.7.0
>
>
> When you run a query with RANK, and one particular frame is too long to fit 
> in 2 batches of data, you will get wrong result.
> I was able to reproduce the issue in a unit test, thanks to the fact that we 
> can control the size of the batches processed by the window operator. I will 
> post a fix soon along with the unit test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4657) Rank() will return wrong results if a frame of data is too big (more than 2 batches)

[Deneche A. Hakim (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271783#comment-15271783
 ] 

Deneche A. Hakim commented on DRILL-4657:
-

[~amansinha100] can you please review ? Thanks

> Rank() will return wrong results if a frame of data is too big (more than 2 
> batches)
> 
>
> Key: DRILL-4657
> URL: https://issues.apache.org/jira/browse/DRILL-4657
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Relational Operators
>Affects Versions: 1.3.0
>Reporter: Deneche A. Hakim
>Assignee: Aman Sinha
>Priority: Critical
> Fix For: 1.7.0
>
>
> When you run a query with RANK, and one particular frame is too long to fit 
> in 2 batches of data, you will get wrong result.
> I was able to reproduce the issue in a unit test, thanks to the fact that we 
> can control the size of the batches processed by the window operator. I will 
> post a fix soon along with the unit test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4657) Rank() will return wrong results if a frame of data is too big (more than 2 batches)

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271681#comment-15271681
 ] 

ASF GitHub Bot commented on DRILL-4657:
---

GitHub user adeneche opened a pull request:

https://github.com/apache/drill/pull/499

DRILL-4657: Rank() will return wrong results if a frame of data is to…

…o big (more than 2 batches)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/adeneche/incubator-drill DRILL-4657

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/499.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #499


commit bd8b8edfa6f9b919c29edc52c1ca2a2d147372ba
Author: adeneche 
Date:   2016-05-04T19:08:36Z

DRILL-4657: Rank() will return wrong results if a frame of data is too big 
(more than 2 batches)




> Rank() will return wrong results if a frame of data is too big (more than 2 
> batches)
> 
>
> Key: DRILL-4657
> URL: https://issues.apache.org/jira/browse/DRILL-4657
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Relational Operators
>Affects Versions: 1.3.0
>Reporter: Deneche A. Hakim
>Assignee: Deneche A. Hakim
>Priority: Critical
> Fix For: 1.7.0
>
>
> When you run a query with RANK, and one particular frame is too long to fit 
> in 2 batches of data, you will get wrong result.
> I was able to reproduce the issue in a unit test, thanks to the fact that we 
> can control the size of the batches processed by the window operator. I will 
> post a fix soon along with the unit test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-4657) Rank() will return wrong results if a frame of data is too big (more than 2 batches)

[Deneche A. Hakim (JIRA)]

Deneche A. Hakim created DRILL-4657:
---

 Summary: Rank() will return wrong results if a frame of data is 
too big (more than 2 batches)
 Key: DRILL-4657
 URL: https://issues.apache.org/jira/browse/DRILL-4657
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Relational Operators
Affects Versions: 1.3.0
Reporter: Deneche A. Hakim
Assignee: Deneche A. Hakim
Priority: Critical
 Fix For: 1.7.0


When you run a query with RANK, and one particular frame is too long to fit in 
2 batches of data, you will get wrong result.

I was able to reproduce the issue in a unit test, thanks to the fact that we 
can control the size of the batches processed by the window operator. I will 
post a fix soon along with the unit test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4510) IllegalStateException: Failure while reading vector. Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.dr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271603#comment-15271603
 ] 

ASF GitHub Bot commented on DRILL-4510:
---

GitHub user hsuanyi opened a pull request:

https://github.com/apache/drill/pull/498

DRILL-4510: Revert DRILL-4476 since UnionAllRecordBatch cannot determ…

…ine if a given record batch is from an empty file or not, just according 
to the number of rows

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hsuanyi/incubator-drill DRILL-4510

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/498.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #498


commit acb00f3a545e2db578aeb2dc14b0180858dd9a5f
Author: hsuanyi 
Date:   2016-05-04T22:56:40Z

DRILL-4510: Revert DRILL-4476 since UnionAllRecordBatch cannot determine if 
a given record batch is from an empty file or not, just according to the number 
of rows




> IllegalStateException: Failure while reading vector.  Expected vector class 
> of org.apache.drill.exec.vector.NullableIntVector but was holding vector 
> class org.apache.drill.exec.vector.NullableVarCharVector
> -
>
> Key: DRILL-4510
> URL: https://issues.apache.org/jira/browse/DRILL-4510
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Data Types
>Reporter: Chun Chang
>Assignee: Sean Hsuan-Yi Chu
>Priority: Critical
>
> Hit the following regression running advanced automation. Regression happened 
> between commit b979bebe83d7017880b0763adcbf8eb80acfcee8 and 
> 1f23b89623c72808f2ee866cec9b4b8a48929d68
> {noformat}
> Execution Failures:
> /root/drillAutomation/framework-master/framework/resources/Advanced/tpcds/tpcds_sf100/original/query66.sql
> Query: 
> -- start query 66 in stream 0 using template query66.tpl 
> SELECT w_warehouse_name, 
>w_warehouse_sq_ft, 
>w_city, 
>w_county, 
>w_state, 
>w_country, 
>ship_carriers, 
>year1,
>Sum(jan_sales) AS jan_sales, 
>Sum(feb_sales) AS feb_sales, 
>Sum(mar_sales) AS mar_sales, 
>Sum(apr_sales) AS apr_sales, 
>Sum(may_sales) AS may_sales, 
>Sum(jun_sales) AS jun_sales, 
>Sum(jul_sales) AS jul_sales, 
>Sum(aug_sales) AS aug_sales, 
>Sum(sep_sales) AS sep_sales, 
>Sum(oct_sales) AS oct_sales, 
>Sum(nov_sales) AS nov_sales, 
>Sum(dec_sales) AS dec_sales, 
>Sum(jan_sales / w_warehouse_sq_ft) AS jan_sales_per_sq_foot, 
>Sum(feb_sales / w_warehouse_sq_ft) AS feb_sales_per_sq_foot, 
>Sum(mar_sales / w_warehouse_sq_ft) AS mar_sales_per_sq_foot, 
>Sum(apr_sales / w_warehouse_sq_ft) AS apr_sales_per_sq_foot, 
>Sum(may_sales / w_warehouse_sq_ft) AS may_sales_per_sq_foot, 
>Sum(jun_sales / w_warehouse_sq_ft) AS jun_sales_per_sq_foot, 
>Sum(jul_sales / w_warehouse_sq_ft) AS jul_sales_per_sq_foot, 
>Sum(aug_sales / w_warehouse_sq_ft) AS aug_sales_per_sq_foot, 
>Sum(sep_sales / w_warehouse_sq_ft) AS sep_sales_per_sq_foot, 
>Sum(oct_sales / w_warehouse_sq_ft) AS oct_sales_per_sq_foot, 
>Sum(nov_sales / w_warehouse_sq_ft) AS nov_sales_per_sq_foot, 
>Sum(dec_sales / w_warehouse_sq_ft) AS dec_sales_per_sq_foot, 
>Sum(jan_net)   AS jan_net, 
>Sum(feb_net)   AS feb_net, 
>Sum(mar_net)   AS mar_net, 
>Sum(apr_net)   AS apr_net, 
>Sum(may_net)   AS may_net, 
>Sum(jun_net)   AS jun_net, 
>Sum(jul_net)   AS jul_net, 
>Sum(aug_net)   AS aug_net, 
>Sum(sep_net)   AS sep_net, 
>Sum(oct_net)   AS 

[jira] [Updated] (DRILL-4656) The inferred nullability of "is null" can at times be incorrect

[Sean Hsuan-Yi Chu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Hsuan-Yi Chu updated DRILL-4656:
-
Description: 
In SQL, the nullability of "is null" should always be non-nullable, regardless 
of the input type.

Because of the short-cut here [1], the nullability is true. That shortcut is to 
avoid the following failure in WHERE clause:
{code}
WHERE booleanFunc(ANY type)
{code}


[1] 
https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/TypeInferenceUtils.java#L250

  was:
In SQL, the nullability of "is null" should always be non-nullable, regardless 
of the input type.

Because of the short-cut here [1], the nullability is true. That shortcut is to 
avoid the following failure in WHERE clause:
{code}
WHERE booleanFunc(ANY)
{code}

Thus, instead of terminating the logic some early, Drill should have kept going 
down till function resolution.

[1] 
https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/TypeInferenceUtils.java#L250


> The inferred nullability of "is null" can at times be incorrect
> ---
>
> Key: DRILL-4656
> URL: https://issues.apache.org/jira/browse/DRILL-4656
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Reporter: Sean Hsuan-Yi Chu
>
> In SQL, the nullability of "is null" should always be non-nullable, 
> regardless of the input type.
> Because of the short-cut here [1], the nullability is true. That shortcut is 
> to avoid the following failure in WHERE clause:
> {code}
> WHERE booleanFunc(ANY type)
> {code}
> [1] 
> https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/TypeInferenceUtils.java#L250



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4642) Let RexBuilder.ensureType() mechanism take place during Rex conversion.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271472#comment-15271472
 ] 

ASF GitHub Bot commented on DRILL-4642:
---

Github user hsuanyi commented on a diff in the pull request:

https://github.com/apache/drill/pull/489#discussion_r62118608
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/TypeInferenceUtils.java
 ---
@@ -734,6 +772,41 @@ public static FunctionCall 
convertSqlOperatorBindingToFunctionCall(final SqlOper
   }
 
   /**
+   * Based on whether decimal type is supported or not, this method 
creates an ExplicitOperatorBinding which interprets
+   * the type of decimal literals accordingly.
+   */
+  public static SqlOperatorBinding convertDecimalLiteralToDouble(final 
SqlOperatorBinding sqlOperatorBinding, final boolean isDecimalSupported) {
+final List types = Lists.newArrayList();
+for(int i = 0; i < sqlOperatorBinding.getOperandCount(); ++i) {
+  final RelDataType relDataType;
+  if(isDecimalLiteral(sqlOperatorBinding, i) && !isDecimalSupported) {
+relDataType = createCalciteTypeWithNullability(
+sqlOperatorBinding.getTypeFactory(),
+SqlTypeName.DOUBLE,
+sqlOperatorBinding.getOperandType(i).isNullable());
+  } else {
+relDataType = sqlOperatorBinding.getOperandType(i);
+  }
+  types.add(relDataType);
+}
+return new 
ExplicitOperatorBinding(sqlOperatorBinding.getTypeFactory(), 
sqlOperatorBinding.getOperator(), types);
--- End diff --

addressed.


> Let RexBuilder.ensureType() mechanism take place during Rex conversion.
> ---
>
> Key: DRILL-4642
> URL: https://issues.apache.org/jira/browse/DRILL-4642
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.7.0
>
>
> In DRILL-4372, the logic of ensuring same type is removed since, in some case 
> such as below, undesirable cast function will be added and cause failure.
> {code}
> SELECT * 
> FROM T 
> WHERE (cast(col1 as timestamp)  - to_timestamp(col2,'-MM-dd HH:mm:ss') < 
> interval 'X XX:XX:XX' day to second)
> {code}
> The fundamental reason for this behavior roots in Drill-Calcite [1], where 
> SqlNode WHERE is expanded to a new object but is not passed into validation 
> step.
> [1] 
> https://github.com/mapr/incubator-calcite/blob/DrillCalcite1.4.0-mapr-1.4.0/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L3362



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4642) Let RexBuilder.ensureType() mechanism take place during Rex conversion.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271461#comment-15271461
 ] 

ASF GitHub Bot commented on DRILL-4642:
---

Github user hsuanyi commented on a diff in the pull request:

https://github.com/apache/drill/pull/489#discussion_r62117749
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/TestFunctionsWithTypeExpoQueries.java
 ---
@@ -719,4 +727,73 @@ public void testWindowSumConstant() throws Exception {
 final String[] excludedPlan = {};
 PlanTestBase.testPlanMatchingPatterns(query, expectedPlan, 
excludedPlan);
   }
+
+  @Test // DRILL-4552
+  public void testDecimalPlusWhenDecimalEnabled() throws Exception {
+final String query = "select cast('99' as decimal(9,0)) + cast('99' as 
decimal(9,0)) as col \n" +
+"from cp.`tpch/region.parquet` \n" +
+"limit 0";
+
+try {
+  final TypeProtos.MajorType majorTypeDouble = 
TypeProtos.MajorType.newBuilder()
+  .setMinorType(TypeProtos.MinorType.FLOAT8)
+  .setMode(TypeProtos.DataMode.REQUIRED)
+  .build();
+
+  final List> 
expectedSchemaDouble = Lists.newArrayList();
+  expectedSchemaDouble.add(Pair.of(SchemaPath.getSimplePath("col"), 
majorTypeDouble));
+
+  testBuilder()
--- End diff --

Addressed


> Let RexBuilder.ensureType() mechanism take place during Rex conversion.
> ---
>
> Key: DRILL-4642
> URL: https://issues.apache.org/jira/browse/DRILL-4642
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.7.0
>
>
> In DRILL-4372, the logic of ensuring same type is removed since, in some case 
> such as below, undesirable cast function will be added and cause failure.
> {code}
> SELECT * 
> FROM T 
> WHERE (cast(col1 as timestamp)  - to_timestamp(col2,'-MM-dd HH:mm:ss') < 
> interval 'X XX:XX:XX' day to second)
> {code}
> The fundamental reason for this behavior roots in Drill-Calcite [1], where 
> SqlNode WHERE is expanded to a new object but is not passed into validation 
> step.
> [1] 
> https://github.com/mapr/incubator-calcite/blob/DrillCalcite1.4.0-mapr-1.4.0/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L3362



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-4656) The inferred nullability of "is null" can at times be incorrect

[Sean Hsuan-Yi Chu (JIRA)]

Sean Hsuan-Yi Chu created DRILL-4656:


 Summary: The inferred nullability of "is null" can at times be 
incorrect
 Key: DRILL-4656
 URL: https://issues.apache.org/jira/browse/DRILL-4656
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning & Optimization
Reporter: Sean Hsuan-Yi Chu


In SQL, the nullability of "is null" should always be non-nullable, regardless 
of the input type.

Because of the short-cut here [1], the nullability is true. That shortcut is to 
avoid the following failure in WHERE clause:
{code}
WHERE booleanFunc(ANY)
{code}

Thus, instead of terminating the logic some early, Drill should have kept going 
down till function resolution.

[1] 
https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/TypeInferenceUtils.java#L250



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4642) Let RexBuilder.ensureType() mechanism take place during Rex conversion.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271140#comment-15271140
 ] 

ASF GitHub Bot commented on DRILL-4642:
---

Github user jinfengni commented on a diff in the pull request:

https://github.com/apache/drill/pull/489#discussion_r62089298
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/TestFunctionsWithTypeExpoQueries.java
 ---
@@ -719,4 +727,73 @@ public void testWindowSumConstant() throws Exception {
 final String[] excludedPlan = {};
 PlanTestBase.testPlanMatchingPatterns(query, expectedPlan, 
excludedPlan);
   }
+
+  @Test // DRILL-4552
+  public void testDecimalPlusWhenDecimalEnabled() throws Exception {
+final String query = "select cast('99' as decimal(9,0)) + cast('99' as 
decimal(9,0)) as col \n" +
+"from cp.`tpch/region.parquet` \n" +
+"limit 0";
+
+try {
+  final TypeProtos.MajorType majorTypeDouble = 
TypeProtos.MajorType.newBuilder()
+  .setMinorType(TypeProtos.MinorType.FLOAT8)
+  .setMode(TypeProtos.DataMode.REQUIRED)
+  .build();
+
+  final List> 
expectedSchemaDouble = Lists.newArrayList();
+  expectedSchemaDouble.add(Pair.of(SchemaPath.getSimplePath("col"), 
majorTypeDouble));
+
+  testBuilder()
--- End diff --

We had better to explicitly set the decimal option to false, before the 
test on line 746. For now, the option defaults to be false. But what if it's 
changed to true some day?




> Let RexBuilder.ensureType() mechanism take place during Rex conversion.
> ---
>
> Key: DRILL-4642
> URL: https://issues.apache.org/jira/browse/DRILL-4642
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.7.0
>
>
> In DRILL-4372, the logic of ensuring same type is removed since, in some case 
> such as below, undesirable cast function will be added and cause failure.
> {code}
> SELECT * 
> FROM T 
> WHERE (cast(col1 as timestamp)  - to_timestamp(col2,'-MM-dd HH:mm:ss') < 
> interval 'X XX:XX:XX' day to second)
> {code}
> The fundamental reason for this behavior roots in Drill-Calcite [1], where 
> SqlNode WHERE is expanded to a new object but is not passed into validation 
> step.
> [1] 
> https://github.com/mapr/incubator-calcite/blob/DrillCalcite1.4.0-mapr-1.4.0/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L3362



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4642) Let RexBuilder.ensureType() mechanism take place during Rex conversion.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271100#comment-15271100
 ] 

ASF GitHub Bot commented on DRILL-4642:
---

Github user jinfengni commented on a diff in the pull request:

https://github.com/apache/drill/pull/489#discussion_r62085424
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/TestFunctionsWithTypeExpoQueries.java
 ---
@@ -213,7 +221,7 @@ public void tesIsNull() throws Exception {
 List> expectedSchema = 
Lists.newArrayList();
 TypeProtos.MajorType majorType = TypeProtos.MajorType.newBuilder()
 .setMinorType(TypeProtos.MinorType.BIT)
-.setMode(TypeProtos.DataMode.REQUIRED)
+.setMode(TypeProtos.DataMode.OPTIONAL)
--- End diff --

why do you have to change the expected result for this query? Should  
"r_name is null" be a required type?  


> Let RexBuilder.ensureType() mechanism take place during Rex conversion.
> ---
>
> Key: DRILL-4642
> URL: https://issues.apache.org/jira/browse/DRILL-4642
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.7.0
>
>
> In DRILL-4372, the logic of ensuring same type is removed since, in some case 
> such as below, undesirable cast function will be added and cause failure.
> {code}
> SELECT * 
> FROM T 
> WHERE (cast(col1 as timestamp)  - to_timestamp(col2,'-MM-dd HH:mm:ss') < 
> interval 'X XX:XX:XX' day to second)
> {code}
> The fundamental reason for this behavior roots in Drill-Calcite [1], where 
> SqlNode WHERE is expanded to a new object but is not passed into validation 
> step.
> [1] 
> https://github.com/mapr/incubator-calcite/blob/DrillCalcite1.4.0-mapr-1.4.0/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L3362



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4642) Let RexBuilder.ensureType() mechanism take place during Rex conversion.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271089#comment-15271089
 ] 

ASF GitHub Bot commented on DRILL-4642:
---

Github user jinfengni commented on a diff in the pull request:

https://github.com/apache/drill/pull/489#discussion_r62084962
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/TypeInferenceUtils.java
 ---
@@ -734,6 +772,41 @@ public static FunctionCall 
convertSqlOperatorBindingToFunctionCall(final SqlOper
   }
 
   /**
+   * Based on whether decimal type is supported or not, this method 
creates an ExplicitOperatorBinding which interprets
+   * the type of decimal literals accordingly.
+   */
+  public static SqlOperatorBinding convertDecimalLiteralToDouble(final 
SqlOperatorBinding sqlOperatorBinding, final boolean isDecimalSupported) {
+final List types = Lists.newArrayList();
+for(int i = 0; i < sqlOperatorBinding.getOperandCount(); ++i) {
+  final RelDataType relDataType;
+  if(isDecimalLiteral(sqlOperatorBinding, i) && !isDecimalSupported) {
+relDataType = createCalciteTypeWithNullability(
+sqlOperatorBinding.getTypeFactory(),
+SqlTypeName.DOUBLE,
+sqlOperatorBinding.getOperandType(i).isNullable());
+  } else {
+relDataType = sqlOperatorBinding.getOperandType(i);
+  }
+  types.add(relDataType);
+}
+return new 
ExplicitOperatorBinding(sqlOperatorBinding.getTypeFactory(), 
sqlOperatorBinding.getOperator(), types);
--- End diff --

Here we do not have to return a new OperatorBinding. In case there is no 
Decimal -> Double replacement, why can't we return the old binding directly? 


> Let RexBuilder.ensureType() mechanism take place during Rex conversion.
> ---
>
> Key: DRILL-4642
> URL: https://issues.apache.org/jira/browse/DRILL-4642
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.7.0
>
>
> In DRILL-4372, the logic of ensuring same type is removed since, in some case 
> such as below, undesirable cast function will be added and cause failure.
> {code}
> SELECT * 
> FROM T 
> WHERE (cast(col1 as timestamp)  - to_timestamp(col2,'-MM-dd HH:mm:ss') < 
> interval 'X XX:XX:XX' day to second)
> {code}
> The fundamental reason for this behavior roots in Drill-Calcite [1], where 
> SqlNode WHERE is expanded to a new object but is not passed into validation 
> step.
> [1] 
> https://github.com/mapr/incubator-calcite/blob/DrillCalcite1.4.0-mapr-1.4.0/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L3362



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4476) Enhance Union-All operator for dealing with empty left input or empty both inputs

[Sean Hsuan-Yi Chu (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271035#comment-15271035
 ] 

Sean Hsuan-Yi Chu commented on DRILL-4476:
--

The logic in this patch can cause problems where Union-all operator makes wrong 
judgement regarding whether a record-batch is coming from an empty file/table.

To be more specific, this patch uses the number of rows to determine whether 
the record-batch from the {color:red} left-side {color} was produced by an 
empty file/table.
(Actually, this logic was applied to the {color:red} right-side {color} of 
Union-all in DRILL-2288 patch.)

However, the number of rows is not sufficient to make a correct judgement. For 
instance, a record-batch which had passed by a "false filter" / "limit 0" will 
carry 0 row.

For another example (DRILL-4510), when multiple Union-alls are running in 
parallel, one of the Union-all might just happen to not receive 0 row (due to 
data partition). 

Then, this Union-all will infer an output schema which is different from others.

Unless the record-batch can somehow capture the information regarding its 
source (whether this record-batch is from an empty file), Union-all cannot be 
make a correct judgement. 

Thus, I think we have to back out this patch. And do not start to support empty 
files until there is a way to carry the empty file information with the 
record-batch (possibly by having a new type to represent non-existent column?). 

> Enhance Union-All operator for dealing with empty left input or empty both 
> inputs
> -
>
> Key: DRILL-4476
> URL: https://issues.apache.org/jira/browse/DRILL-4476
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Relational Operators
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.7.0
>
>
> Union-All operator does not deal with the situation where left side comes 
> from empty source.
> Due to DRILL-2288's enhancement for empty sources, Union-All operator now can 
> be allowed to support this scenario.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4653) Malformed JSON should not stop the entire query from progressing

[Deneche A. Hakim (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271026#comment-15271026
 ] 

Deneche A. Hakim commented on DRILL-4653:
-

You may also be interested into the following JIRA: DRILL-3764

> Malformed JSON should not stop the entire query from progressing
> 
>
> Key: DRILL-4653
> URL: https://issues.apache.org/jira/browse/DRILL-4653
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Storage - JSON
>Affects Versions: 1.6.0
>Reporter: subbu srinivasan
>
> Currently Drill query terminates upon first encounter of a invalid JSON line.
> Drill has to continue progressing after ignoring the bad records. Something 
> similar to a setting of (ignore.malformed.json) would help.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4654) Expose New System Metrics

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271022#comment-15271022
 ] 

ASF GitHub Bot commented on DRILL-4654:
---

Github user jaltekruse commented on the pull request:

https://github.com/apache/drill/pull/495#issuecomment-216937718
  
We currently use multiple drillbits in the same JVM process for our unit 
tests, won't this cause them to conflict with one another? I don't know if 
production use cases try to run multiple Drillbits in a single JVM, but even if 
this isn't a use case, a stateful API like the metrics registry seems like the 
wrong thing to move more towards static shared API. While I know it is 
currently static, I think the idea with these methods you were deprecating is 
that we would try to move to not be accessing anything static.


> Expose New System Metrics
> -
>
> Key: DRILL-4654
> URL: https://issues.apache.org/jira/browse/DRILL-4654
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Sudheesh Katkam
>Assignee: Sudheesh Katkam
> Fix For: 1.8.0
>
>
> + Add more metrics to the DrillMetrics registry (exposed through web UI and 
> jconsole, through JMX): pending queries, running queries, completed queries, 
> current memory usage (root allocator)
> + Clean up and document metric registration API
> + Deprecate getMetrics() method in contextual objects; use 
> DrillMetrics.getRegistry() directly
> + Make JMX reporting and log reporting configurable through system properties 
> (since config file is not meant to be used in common module)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4132) Ability to submit simple type of physical plan directly to EndPoint DrillBit for execution

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270999#comment-15270999
 ] 

ASF GitHub Bot commented on DRILL-4132:
---

Github user amansinha100 commented on the pull request:

https://github.com/apache/drill/pull/368#issuecomment-216933980
  
My review comments were previously addressed, so +1 assuming tests pass.  


> Ability to submit simple type of physical plan directly to EndPoint DrillBit 
> for execution
> --
>
> Key: DRILL-4132
> URL: https://issues.apache.org/jira/browse/DRILL-4132
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Execution - Flow, Execution - RPC, Query Planning & 
> Optimization
>Reporter: Yuliya Feldman
>Assignee: Yuliya Feldman
>
> Today Drill Query execution is optimistic and stateful (at least due to data 
> exchanges) - if any of the stages of query execution fails whole query fails. 
> If query is just simple scan, filter push down and project where no data 
> exchange happens between DrillBits there is no need to fail whole query when 
> one DrillBit fails, as minor fragments running on that DrillBit can be rerun 
> on the other DrillBit. There are probably multiple ways to achieve this. This 
> JIRA is to open discussion on: 
> 1. agreement that we need to support above use case 
> 2. means of achieving it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (DRILL-4476) Enhance Union-All operator for dealing with empty left input or empty both inputs

[Sean Hsuan-Yi Chu (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270943#comment-15270943
 ] 

Sean Hsuan-Yi Chu edited comment on DRILL-4476 at 5/4/16 4:30 PM:
--

The change in UnionAllRecordBatch in DRILL-2288 is related to this JIRA.


was (Author: seanhychu):
The change in UnionAllRecordBatch is related.

> Enhance Union-All operator for dealing with empty left input or empty both 
> inputs
> -
>
> Key: DRILL-4476
> URL: https://issues.apache.org/jira/browse/DRILL-4476
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Relational Operators
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.7.0
>
>
> Union-All operator does not deal with the situation where left side comes 
> from empty source.
> Due to DRILL-2288's enhancement for empty sources, Union-All operator now can 
> be allowed to support this scenario.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4476) Enhance Union-All operator for dealing with empty left input or empty both inputs

[Sean Hsuan-Yi Chu (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270943#comment-15270943
 ] 

Sean Hsuan-Yi Chu commented on DRILL-4476:
--

The change in UnionAllRecordBatch is related.

> Enhance Union-All operator for dealing with empty left input or empty both 
> inputs
> -
>
> Key: DRILL-4476
> URL: https://issues.apache.org/jira/browse/DRILL-4476
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Relational Operators
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.7.0
>
>
> Union-All operator does not deal with the situation where left side comes 
> from empty source.
> Due to DRILL-2288's enhancement for empty sources, Union-All operator now can 
> be allowed to support this scenario.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4525) Query with BETWEEN clause on Date and Timestamp values fails with Validation Error

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270941#comment-15270941
 ] 

ASF GitHub Bot commented on DRILL-4525:
---

Github user hsuanyi closed the pull request at:

https://github.com/apache/drill/pull/439


> Query with BETWEEN clause on Date and Timestamp values fails with Validation 
> Error
> --
>
> Key: DRILL-4525
> URL: https://issues.apache.org/jira/browse/DRILL-4525
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Reporter: Abhishek Girish
>Assignee: Sean Hsuan-Yi Chu
>Priority: Critical
>
> Query: (simplified variant of TPC-DS Query37)
> {code}
> SELECT
>*
> FROM   
>date_dim
> WHERE   
>d_date BETWEEN Cast('1999-03-06' AS DATE) AND  (
>   Cast('1999-03-06' AS DATE) + INTERVAL '60' day)
> LIMIT 10;
> {code}
> Error:
> {code}
> Error: VALIDATION ERROR: From line 6, column 8 to line 7, column 64: Cannot 
> apply 'BETWEEN ASYMMETRIC' to arguments of type ' BETWEEN ASYMMETRIC 
>  AND '. Supported form(s): ' BETWEEN 
>  AND '
> SQL Query null
> [Error Id: 223fb37c-f561-4a37-9283-871dc6f4d6d0 on abhi2:31010] 
> (state=,code=0)
> {code}
> This is a regression from 1.6.0. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4525) Query with BETWEEN clause on Date and Timestamp values fails with Validation Error

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270940#comment-15270940
 ] 

ASF GitHub Bot commented on DRILL-4525:
---

Github user hsuanyi commented on the pull request:

https://github.com/apache/drill/pull/439#issuecomment-216920059
  
merge this PR with https://github.com/apache/drill/pull/489. So close this 
one.


> Query with BETWEEN clause on Date and Timestamp values fails with Validation 
> Error
> --
>
> Key: DRILL-4525
> URL: https://issues.apache.org/jira/browse/DRILL-4525
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Reporter: Abhishek Girish
>Assignee: Sean Hsuan-Yi Chu
>Priority: Critical
>
> Query: (simplified variant of TPC-DS Query37)
> {code}
> SELECT
>*
> FROM   
>date_dim
> WHERE   
>d_date BETWEEN Cast('1999-03-06' AS DATE) AND  (
>   Cast('1999-03-06' AS DATE) + INTERVAL '60' day)
> LIMIT 10;
> {code}
> Error:
> {code}
> Error: VALIDATION ERROR: From line 6, column 8 to line 7, column 64: Cannot 
> apply 'BETWEEN ASYMMETRIC' to arguments of type ' BETWEEN ASYMMETRIC 
>  AND '. Supported form(s): ' BETWEEN 
>  AND '
> SQL Query null
> [Error Id: 223fb37c-f561-4a37-9283-871dc6f4d6d0 on abhi2:31010] 
> (state=,code=0)
> {code}
> This is a regression from 1.6.0. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3823) Exception when grouping by case expression and columns array

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270935#comment-15270935
 ] 

ASF GitHub Bot commented on DRILL-3823:
---

Github user hsuanyi commented on the pull request:

https://github.com/apache/drill/pull/490#issuecomment-216919724
  
merge this PR with https://github.com/apache/drill/pull/489. So close this 
one.


> Exception when grouping by case expression and columns array
> 
>
> Key: DRILL-3823
> URL: https://issues.apache.org/jira/browse/DRILL-3823
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Affects Versions: 1.1.0, 1.2.0
>Reporter: Victoria Markman
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.7.0
>
> Attachments: t1.csv
>
>
> In drill version 1.2.0:
> {code}
> 0: jdbc:drill:schema=dfs> select 
> . . . . . . . . . . . . > count(*),
> . . . . . . . . . . . . > case when columns[0] = '' then cast(null as 
> integer) else cast(columns[0] as integer) end
> . . . . . . . . . . . . > from
> . . . . . . . . . . . . > `t1.csv`
> . . . . . . . . . . . . > group by
> . . . . . . . . . . . . > case when columns[0] = '' then cast(null as 
> integer) else cast(columns[0] as integer) end;
> Error: SYSTEM ERROR: UnsupportedOperationException: class 
> org.apache.calcite.sql.SqlLiteral: NULL
> [Error Id: e4b37748-e81a-41c9-bfbc-e8f60cb43977 on atsqa4-133.qa.lab:31010] 
> (state=,code=0)
> {code}
> drillbit.log
> {code}
> 2015-09-23 00:00:13,947 [29fe1472-1af8-ba17-4184-3bc38da1d372:foreman] ERROR 
> o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: 
> UnsupportedOperationException: class org.apache.calcite.sql.SqlLiteral: NULL
> [Error Id: 46bce8a5-048a-4caa-bcfc-9b2326cfd831 on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
> UnsupportedOperationException: class org.apache.calcite.sql.SqlLiteral: NULL
> [Error Id: 46bce8a5-048a-4caa-bcfc-9b2326cfd831 on atsqa4-133.qa.lab:31010]
> at 
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:524)
>  ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:742)
>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:841)
>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:786)
>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.common.EventProcessor.sendEvent(EventProcessor.java:73) 
> [drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.exec.work.foreman.Foreman$StateSwitch.moveToState(Foreman.java:788)
>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:894) 
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:255) 
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_71]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected 
> exception during fragment initialization: Internal error: while converting 
> CASE WHEN `t1.csv`.`columns`[0] = '' THEN CAST(NULL AS INTEGER) ELSE 
> CAST(`t1.csv`.`columns`[0] AS INTEGER)
>  END
> ... 4 common frames omitted
> Caused by: java.lang.AssertionError: Internal error: while converting CASE 
> WHEN `t1.csv`.`columns`[0] = '' THEN CAST(NULL AS INTEGER) ELSE 
> CAST(`t1.csv`.`columns`[0] AS INTEGER) END
> at org.apache.calcite.util.Util.newInternal(Util.java:792) 
> ~[calcite-core-1.4.0-drill-r2.jar:1.4.0-drill-r2]
> at 
> org.apache.calcite.sql2rel.ReflectiveConvertletTable$1.convertCall(ReflectiveConvertletTable.java:96)
>  ~[calcite-core-1.4.0-drill-r2.jar:1.4.0-drill-r2]
> at 
> org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:60)
>  ~[calcite-core-1.4.0-drill-r2.jar:1.4.0-drill-r2]
> at 
> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4162)
>  ~[calcite-core-1.4.0-drill-r2.jar:1.4.0-drill-r2]
> at 
> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3595)
>  ~[calcite-core-1.4.0-drill-r2.jar:1.4.0-drill-r2]

[jira] [Commented] (DRILL-3823) Exception when grouping by case expression and columns array

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270936#comment-15270936
 ] 

ASF GitHub Bot commented on DRILL-3823:
---

Github user hsuanyi closed the pull request at:

https://github.com/apache/drill/pull/490


> Exception when grouping by case expression and columns array
> 
>
> Key: DRILL-3823
> URL: https://issues.apache.org/jira/browse/DRILL-3823
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Affects Versions: 1.1.0, 1.2.0
>Reporter: Victoria Markman
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.7.0
>
> Attachments: t1.csv
>
>
> In drill version 1.2.0:
> {code}
> 0: jdbc:drill:schema=dfs> select 
> . . . . . . . . . . . . > count(*),
> . . . . . . . . . . . . > case when columns[0] = '' then cast(null as 
> integer) else cast(columns[0] as integer) end
> . . . . . . . . . . . . > from
> . . . . . . . . . . . . > `t1.csv`
> . . . . . . . . . . . . > group by
> . . . . . . . . . . . . > case when columns[0] = '' then cast(null as 
> integer) else cast(columns[0] as integer) end;
> Error: SYSTEM ERROR: UnsupportedOperationException: class 
> org.apache.calcite.sql.SqlLiteral: NULL
> [Error Id: e4b37748-e81a-41c9-bfbc-e8f60cb43977 on atsqa4-133.qa.lab:31010] 
> (state=,code=0)
> {code}
> drillbit.log
> {code}
> 2015-09-23 00:00:13,947 [29fe1472-1af8-ba17-4184-3bc38da1d372:foreman] ERROR 
> o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: 
> UnsupportedOperationException: class org.apache.calcite.sql.SqlLiteral: NULL
> [Error Id: 46bce8a5-048a-4caa-bcfc-9b2326cfd831 on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
> UnsupportedOperationException: class org.apache.calcite.sql.SqlLiteral: NULL
> [Error Id: 46bce8a5-048a-4caa-bcfc-9b2326cfd831 on atsqa4-133.qa.lab:31010]
> at 
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:524)
>  ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:742)
>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:841)
>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:786)
>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.common.EventProcessor.sendEvent(EventProcessor.java:73) 
> [drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.exec.work.foreman.Foreman$StateSwitch.moveToState(Foreman.java:788)
>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:894) 
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:255) 
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_71]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected 
> exception during fragment initialization: Internal error: while converting 
> CASE WHEN `t1.csv`.`columns`[0] = '' THEN CAST(NULL AS INTEGER) ELSE 
> CAST(`t1.csv`.`columns`[0] AS INTEGER)
>  END
> ... 4 common frames omitted
> Caused by: java.lang.AssertionError: Internal error: while converting CASE 
> WHEN `t1.csv`.`columns`[0] = '' THEN CAST(NULL AS INTEGER) ELSE 
> CAST(`t1.csv`.`columns`[0] AS INTEGER) END
> at org.apache.calcite.util.Util.newInternal(Util.java:792) 
> ~[calcite-core-1.4.0-drill-r2.jar:1.4.0-drill-r2]
> at 
> org.apache.calcite.sql2rel.ReflectiveConvertletTable$1.convertCall(ReflectiveConvertletTable.java:96)
>  ~[calcite-core-1.4.0-drill-r2.jar:1.4.0-drill-r2]
> at 
> org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:60)
>  ~[calcite-core-1.4.0-drill-r2.jar:1.4.0-drill-r2]
> at 
> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4162)
>  ~[calcite-core-1.4.0-drill-r2.jar:1.4.0-drill-r2]
> at 
> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3595)
>  ~[calcite-core-1.4.0-drill-r2.jar:1.4.0-drill-r2]
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:130) 
> 

[jira] [Commented] (DRILL-4510) IllegalStateException: Failure while reading vector. Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.dr

[Sean Hsuan-Yi Chu (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270929#comment-15270929
 ] 

Sean Hsuan-Yi Chu commented on DRILL-4510:
--

This issue is caused by the patch of DRILL-4476. The details are written in 
that JIRA.

> IllegalStateException: Failure while reading vector.  Expected vector class 
> of org.apache.drill.exec.vector.NullableIntVector but was holding vector 
> class org.apache.drill.exec.vector.NullableVarCharVector
> -
>
> Key: DRILL-4510
> URL: https://issues.apache.org/jira/browse/DRILL-4510
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Data Types
>Reporter: Chun Chang
>Assignee: Sean Hsuan-Yi Chu
>Priority: Critical
>
> Hit the following regression running advanced automation. Regression happened 
> between commit b979bebe83d7017880b0763adcbf8eb80acfcee8 and 
> 1f23b89623c72808f2ee866cec9b4b8a48929d68
> {noformat}
> Execution Failures:
> /root/drillAutomation/framework-master/framework/resources/Advanced/tpcds/tpcds_sf100/original/query66.sql
> Query: 
> -- start query 66 in stream 0 using template query66.tpl 
> SELECT w_warehouse_name, 
>w_warehouse_sq_ft, 
>w_city, 
>w_county, 
>w_state, 
>w_country, 
>ship_carriers, 
>year1,
>Sum(jan_sales) AS jan_sales, 
>Sum(feb_sales) AS feb_sales, 
>Sum(mar_sales) AS mar_sales, 
>Sum(apr_sales) AS apr_sales, 
>Sum(may_sales) AS may_sales, 
>Sum(jun_sales) AS jun_sales, 
>Sum(jul_sales) AS jul_sales, 
>Sum(aug_sales) AS aug_sales, 
>Sum(sep_sales) AS sep_sales, 
>Sum(oct_sales) AS oct_sales, 
>Sum(nov_sales) AS nov_sales, 
>Sum(dec_sales) AS dec_sales, 
>Sum(jan_sales / w_warehouse_sq_ft) AS jan_sales_per_sq_foot, 
>Sum(feb_sales / w_warehouse_sq_ft) AS feb_sales_per_sq_foot, 
>Sum(mar_sales / w_warehouse_sq_ft) AS mar_sales_per_sq_foot, 
>Sum(apr_sales / w_warehouse_sq_ft) AS apr_sales_per_sq_foot, 
>Sum(may_sales / w_warehouse_sq_ft) AS may_sales_per_sq_foot, 
>Sum(jun_sales / w_warehouse_sq_ft) AS jun_sales_per_sq_foot, 
>Sum(jul_sales / w_warehouse_sq_ft) AS jul_sales_per_sq_foot, 
>Sum(aug_sales / w_warehouse_sq_ft) AS aug_sales_per_sq_foot, 
>Sum(sep_sales / w_warehouse_sq_ft) AS sep_sales_per_sq_foot, 
>Sum(oct_sales / w_warehouse_sq_ft) AS oct_sales_per_sq_foot, 
>Sum(nov_sales / w_warehouse_sq_ft) AS nov_sales_per_sq_foot, 
>Sum(dec_sales / w_warehouse_sq_ft) AS dec_sales_per_sq_foot, 
>Sum(jan_net)   AS jan_net, 
>Sum(feb_net)   AS feb_net, 
>Sum(mar_net)   AS mar_net, 
>Sum(apr_net)   AS apr_net, 
>Sum(may_net)   AS may_net, 
>Sum(jun_net)   AS jun_net, 
>Sum(jul_net)   AS jul_net, 
>Sum(aug_net)   AS aug_net, 
>Sum(sep_net)   AS sep_net, 
>Sum(oct_net)   AS oct_net, 
>Sum(nov_net)   AS nov_net, 
>Sum(dec_net)   AS dec_net 
> FROM   (SELECT w_warehouse_name, 
>w_warehouse_sq_ft, 
>w_city, 
>w_county, 
>w_state, 
>w_country, 
>'ZOUROS' 
>|| ',' 
>|| 'ZHOU' AS ship_carriers, 
>d_yearAS year1, 
>Sum(CASE 
>  WHEN d_moy = 1 THEN ws_ext_sales_price * ws_quantity 
>  ELSE 0 
>END)  AS jan_sales, 
>Sum(CASE 
>  WHEN d_moy = 2 THEN ws_ext_sales_price * ws_quantity 
>  ELSE 0 
>END)  AS feb_sales, 
>Sum(CASE 
>  WHEN d_moy = 3 THEN 

[jira] [Updated] (DRILL-4602) Avro files dont work if the union format is ["some-type", "null"]

[Christian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian updated DRILL-4602:
-
Description: 
An avro file generated by a different system (e.g. Spark) can have a slightly 
different union format, that is not understood by drill. For example 
["some-type", "null"] will cause an error when [ "null", "some-type"] still 
works. 



  was:
An avro files generated by a different system (e.g. Spark) can have a slightly 
different union format, that is not understood by drill. For example 
["some-type", "null"] will cause an error when [ "null", "some-type"] still 
works. 




> Avro files dont work if the union format is ["some-type", "null"]
> -
>
> Key: DRILL-4602
> URL: https://issues.apache.org/jira/browse/DRILL-4602
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Avro
>Affects Versions: 1.6.0
>Reporter: Christian
>  Labels: easyfix, patch
> Fix For: 1.7.0
>
> Attachments: DRILL-4602.patch
>
>
> An avro file generated by a different system (e.g. Spark) can have a slightly 
> different union format, that is not understood by drill. For example 
> ["some-type", "null"] will cause an error when [ "null", "some-type"] still 
> works. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-4602) Avro files dont work if the union format is ["some-type", "null"]

[Christian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian updated DRILL-4602:
-
Labels: easyfix patch  (was: )

> Avro files dont work if the union format is ["some-type", "null"]
> -
>
> Key: DRILL-4602
> URL: https://issues.apache.org/jira/browse/DRILL-4602
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Avro
>Affects Versions: 1.6.0
>Reporter: Christian
>  Labels: easyfix, patch
> Fix For: 1.7.0
>
> Attachments: DRILL-4602.patch
>
>
> An avro files generated by a different system (e.g. Spark) can have a 
> slightly different union format, that is not understood by drill. For example 
> ["some-type", "null"] will cause an error when [ "null", "some-type"] still 
> works. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-4602) Avro files dont work if the union format is ["some-type", "null"]

[Christian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian updated DRILL-4602:
-
Attachment: DRILL-4602.patch

Attaching a further improvement to the patch for Avro union types in Select 
statements. 

> Avro files dont work if the union format is ["some-type", "null"]
> -
>
> Key: DRILL-4602
> URL: https://issues.apache.org/jira/browse/DRILL-4602
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Avro
>Affects Versions: 1.6.0
>Reporter: Christian
> Fix For: 1.7.0
>
> Attachments: DRILL-4602.patch
>
>
> An avro files generated by a different system (e.g. Spark) can have a 
> slightly different union format, that is not understood by drill. For example 
> ["some-type", "null"] will cause an error when [ "null", "some-type"] still 
> works. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-4602) Avro files dont work if the union format is ["some-type", "null"]

[Christian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian updated DRILL-4602:
-
Attachment: (was: 0001-Fixing-avro-union-types.patch)

> Avro files dont work if the union format is ["some-type", "null"]
> -
>
> Key: DRILL-4602
> URL: https://issues.apache.org/jira/browse/DRILL-4602
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Avro
>Affects Versions: 1.6.0
>Reporter: Christian
> Fix For: 1.7.0
>
>
> An avro files generated by a different system (e.g. Spark) can have a 
> slightly different union format, that is not understood by drill. For example 
> ["some-type", "null"] will cause an error when [ "null", "some-type"] still 
> works. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-3894) Directory functions (MaxDir, MinDir ..) should have optional filename parameter

[Vitalii Diravka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitalii Diravka updated DRILL-3894:
---
Labels: doc-impacting  (was: )

> Directory functions (MaxDir, MinDir ..) should have optional filename 
> parameter
> ---
>
> Key: DRILL-3894
> URL: https://issues.apache.org/jira/browse/DRILL-3894
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Functions - Drill
>Affects Versions: 1.2.0
>Reporter: Neeraja
>Assignee: Vitalii Diravka
>  Labels: doc-impacting
> Fix For: 1.7.0
>
>
> https://drill.apache.org/docs/query-directory-functions/
> The directory functions documented above should provide ability to have 
> second parameter(file name) as optional.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-4584) JDBC/ODBC Client IP in Drill audit logs

[Vitalii Diravka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitalii Diravka updated DRILL-4584:
---
Labels: doc-impacting  (was: documentation)

> JDBC/ODBC Client IP in Drill audit logs
> ---
>
> Key: DRILL-4584
> URL: https://issues.apache.org/jira/browse/DRILL-4584
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Client - JDBC, Client - ODBC
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Minor
>  Labels: doc-impacting
> Fix For: 1.7.0
>
>
> Currently Drill audit logs - sqlline_queries.json and drillbit_queries.json 
> provide information about client username who fired the query . It will be 
> good to also have the client IP from where the query was fired .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-4571) Add link to local Drill logs from the web UI

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-4571:

Labels: doc-impacting  (was: documentation)

> Add link to local Drill logs from the web UI
> 
>
> Key: DRILL-4571
> URL: https://issues.apache.org/jira/browse/DRILL-4571
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.6.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>  Labels: doc-impacting
> Fix For: 1.7.0
>
> Attachments: display_log.JPG, log_list.JPG
>
>
> Now we have link to the profile from the web UI.
> It will be handy for the users to have the link to local logs as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-4584) JDBC/ODBC Client IP in Drill audit logs

[Vitalii Diravka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitalii Diravka updated DRILL-4584:
---
Labels: documentation  (was: )

> JDBC/ODBC Client IP in Drill audit logs
> ---
>
> Key: DRILL-4584
> URL: https://issues.apache.org/jira/browse/DRILL-4584
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Client - JDBC, Client - ODBC
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Minor
>  Labels: documentation
> Fix For: 1.7.0
>
>
> Currently Drill audit logs - sqlline_queries.json and drillbit_queries.json 
> provide information about client username who fired the query . It will be 
> good to also have the client IP from where the query was fired .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-4571) Add link to local Drill logs from the web UI

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-4571:

Labels: documentation  (was: )

> Add link to local Drill logs from the web UI
> 
>
> Key: DRILL-4571
> URL: https://issues.apache.org/jira/browse/DRILL-4571
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.6.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>  Labels: documentation
> Fix For: 1.7.0
>
> Attachments: display_log.JPG, log_list.JPG
>
>
> Now we have link to the profile from the web UI.
> It will be handy for the users to have the link to local logs as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-4655) Custom Authentication in Window

[Sanjiv Kumar (JIRA)]

Sanjiv Kumar created DRILL-4655:
---

 Summary: Custom Authentication in Window
 Key: DRILL-4655
 URL: https://issues.apache.org/jira/browse/DRILL-4655
 Project: Apache Drill
  Issue Type: Bug
Affects Versions: 1.6.0
Reporter: Sanjiv Kumar


Hello
I am using Drill in Window 8.1 with latest version (1.6). 

I am trying to create custom authentication in in window operating system  
following this java file:-
  
https://drill.apache.org/docs/configuring-user-authentication/#implementing-and-configuring-a-custom-authenticator.

While compiling this java file in window i got error:- 

cannot find symbol 
   @UserAuthenticatorTemplate(type = "myCustomAuthenticatorType")
^
  symbol: class UserAuthenticatorTemplate
.\org\apache\drill\common\config\DrillConfig.java:44: error: cannot find symbol
public class DrillConfig extends NestedConfig {
 ^
  symbol: class NestedConfig
.\myorg\dept\drill\security\MyCustomDrillUserAuthenticatorImpl.java:36: error: c
cannot find symbol
public void authenticate(String userName, String password) throws UserAuthen
ticationException {
  ^
  symbol:   class UserAuthenticationException
  location: class MyCustomDrillUserAuthenticatorImpl
.\com\google\common\collect\ImmutableList.java:24: error: cannot find symbol
import static com.google.common.collect.RegularImmutableList.EMPTY;
   ^
  symbol:   class RegularImmutableList
  location: package com.google.common.collect
.\com\google\common\collect\ImmutableList.java:24: error: static import only fro
m classes and interfaces
import static com.google.common.collect.RegularImmutableList.EMPTY;
^
.\com\google\common\collect\ImmutableList.java:54: error: cannot find symbol
public abstract class ImmutableList extends ImmutableCollection
   ^
  symbol: class ImmutableCollection
.\com\typesafe\config\Config.java:166: error: cannot find symbol
public interface Config extends ConfigMergeable {
^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:179: error: cannot find symbol
public void trace(Marker marker, String format, Object arg);
  ^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:192: error: cannot find symbol
public void trace(Marker marker, String format, Object arg1, Object arg2);
  ^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:204: error: cannot find symbol
public void trace(Marker marker, String format, Object... argArray);
   ^
  
   symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:320: error: cannot find symbol
public void debug(Marker marker, String format, Object arg1, Object arg2);
  ^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:331: error: cannot find symbol
public void debug(Marker marker, String format, Object... arguments);
  ^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:341: error: cannot find symbol
public void debug(Marker marker, String msg, Throwable t);
  ^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:415: error: cannot find symbol
public boolean isInfoEnabled(Marker marker);
 ^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:423: error: cannot find symbol
public void info(Marker marker, String msg);
 ^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:433: error: cannot find symbol
public void info(Marker marker, String format, Object arg);
 ^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:445: error: cannot find symbol
public void info(Marker marker, String format, Object arg1, Object arg2);
 ^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:456: error: cannot find symbol
public void info(Marker marker, String format, Object... arguments);
 ^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:466: error: cannot find symbol
public void info(Marker marker, String msg, Throwable t);
 ^
  symbol:   class Marker
  location: interface Logger
.\org\slf4j\Logger.java:541: error: cannot find symbol
public boolean isWarnEnabled(Marker marker);
 ^
  
 symbol:   method value()
  location: @interface GwtIncompatible

[jira] [Resolved] (DRILL-4584) JDBC/ODBC Client IP in Drill audit logs

[Vitalii Diravka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitalii Diravka resolved DRILL-4584.

Resolution: Done

Implemented in 
[2d9f9ab|https://github.com/apache/drill/commit/2d9f9abb4c47d08f8462599c8d6076a61a1708fe].

> JDBC/ODBC Client IP in Drill audit logs
> ---
>
> Key: DRILL-4584
> URL: https://issues.apache.org/jira/browse/DRILL-4584
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Client - JDBC, Client - ODBC
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Minor
> Fix For: 1.7.0
>
>
> Currently Drill audit logs - sqlline_queries.json and drillbit_queries.json 
> provide information about client username who fired the query . It will be 
> good to also have the client IP from where the query was fired .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (DRILL-3894) Directory functions (MaxDir, MinDir ..) should have optional filename parameter

[Vitalii Diravka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitalii Diravka resolved DRILL-3894.

   Resolution: Implemented
Fix Version/s: 1.7.0

Implemented in 
[a6a85ab|https://github.com/apache/drill/commit/a6a85ab66360cac81ab4777cec20292470ac483d].

> Directory functions (MaxDir, MinDir ..) should have optional filename 
> parameter
> ---
>
> Key: DRILL-3894
> URL: https://issues.apache.org/jira/browse/DRILL-3894
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Functions - Drill
>Affects Versions: 1.2.0
>Reporter: Neeraja
>Assignee: Vitalii Diravka
> Fix For: 1.7.0
>
>
> https://drill.apache.org/docs/query-directory-functions/
> The directory functions documented above should provide ability to have 
> second parameter(file name) as optional.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-2100) Drill not deleting spooling files

[Vitalii Diravka (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270384#comment-15270384
 ] 

Vitalii Diravka commented on DRILL-2100:


[~jaltekruse] Not certainly in that way. 
Spill directories will delete immediately after success, failed or canceled 
query due to fs.delete() in the close() method of ExternalSortBatch.java (also 
spill directories will be deleted from deleteOnExit set). It can work without 
using fs.deleteOnExit(). 
What about fs.deleteOnExit() I use it to delete temporary spill folders for the 
case when drillbit process is killed.

> Drill not deleting spooling files
> -
>
> Key: DRILL-2100
> URL: https://issues.apache.org/jira/browse/DRILL-2100
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Relational Operators
>Affects Versions: 0.8.0
>Reporter: Abhishek Girish
>Assignee: Deneche A. Hakim
> Fix For: 1.7.0
>
>
> Currently, after forcing queries to use an external sort by switching off 
> hash join/agg causes spill-to-disk files accumulating. 
> This causes issues with disk space availability when the spill is configured 
> to be on the local file system (/tmp/drill). Also not optimal when configured 
> to use DFS (custom). 
> Drill must clean up all temporary files created after a query completes or 
> after a drillbit restart. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-4571) Add link to local Drill logs from the web UI

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-4571:

Affects Version/s: 1.6.0
Fix Version/s: 1.7.0

> Add link to local Drill logs from the web UI
> 
>
> Key: DRILL-4571
> URL: https://issues.apache.org/jira/browse/DRILL-4571
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.6.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.7.0
>
> Attachments: display_log.JPG, log_list.JPG
>
>
> Now we have link to the profile from the web UI.
> It will be handy for the users to have the link to local logs as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4571) Add link to local Drill logs from the web UI

[Arina Ielchiieva (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270332#comment-15270332
 ] 

Arina Ielchiieva commented on DRILL-4571:
-

Fixed in 1a89a7f.

> Add link to local Drill logs from the web UI
> 
>
> Key: DRILL-4571
> URL: https://issues.apache.org/jira/browse/DRILL-4571
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.6.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.7.0
>
> Attachments: display_log.JPG, log_list.JPG
>
>
> Now we have link to the profile from the web UI.
> It will be handy for the users to have the link to local logs as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (DRILL-4571) Add link to local Drill logs from the web UI

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva resolved DRILL-4571.
-
Resolution: Fixed

> Add link to local Drill logs from the web UI
> 
>
> Key: DRILL-4571
> URL: https://issues.apache.org/jira/browse/DRILL-4571
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.6.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.7.0
>
> Attachments: display_log.JPG, log_list.JPG
>
>
> Now we have link to the profile from the web UI.
> It will be handy for the users to have the link to local logs as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4654) Expose New System Metrics

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270172#comment-15270172
 ] 

ASF GitHub Bot commented on DRILL-4654:
---

GitHub user sudheeshkatkam opened a pull request:

https://github.com/apache/drill/pull/495

DRILL-4654: Add new metrics to the MetricRegistry

+ New metrics: running queries, pending queries, completed queries,
  used memory (root allocator)
+ Borrow SystemPropertyUtil class from Netty
+ Configure DrillMetrics params through system properties
+ Deprecate getMetrics method in contextual objects
+ Rename "current" to "used" for RPC allocator current memory usage to
  follow convention

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sudheeshkatkam/drill DRILL-4654

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/495.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #495


commit 1fc205bebaf3c1f20df85996ba49cf98b60e4c9e
Author: Sudheesh Katkam 
Date:   2016-05-04T06:10:47Z

DRILL-4654: Add new metrics to the MetricRegistry

+ New metrics: running queries, pending queries, completed queries,
  used memory (root allocator)
+ Borrow SystemPropertyUtil class from Netty
+ Configure DrillMetrics params through system properties
+ Deprecate getMetrics method in contextual objects
+ Rename "current" to "used" for RPC allocator current memory usage to
  follow convention




> Expose New System Metrics
> -
>
> Key: DRILL-4654
> URL: https://issues.apache.org/jira/browse/DRILL-4654
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Sudheesh Katkam
>Assignee: Sudheesh Katkam
> Fix For: 1.8.0
>
>
> + Add more metrics to the DrillMetrics registry (exposed through web UI and 
> jconsole, through JMX): pending queries, running queries, completed queries, 
> current memory usage (root allocator)
> + Clean up and document metric registration API
> + Deprecate getMetrics() method in contextual objects; use 
> DrillMetrics.getRegistry() directly
> + Make JMX reporting and log reporting configurable through system properties 
> (since config file is not meant to be used in common module)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-4654) Expose New System Metrics

[Sudheesh Katkam (JIRA)]

Sudheesh Katkam created DRILL-4654:
--

 Summary: Expose New System Metrics
 Key: DRILL-4654
 URL: https://issues.apache.org/jira/browse/DRILL-4654
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Sudheesh Katkam
Assignee: Sudheesh Katkam
 Fix For: 1.8.0


+ Add more metrics to the DrillMetrics registry (exposed through web UI and 
jconsole, through JMX): pending queries, running queries, completed queries, 
current memory usage (root allocator)
+ Clean up and document metric registration API
+ Deprecate getMetrics() method in contextual objects; use 
DrillMetrics.getRegistry() directly
+ Make JMX reporting and log reporting configurable through system properties 
(since config file is not meant to be used in common module)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)