[jira] [Commented] (DRILL-6185) Error is displaying while accessing query profiles via the Web-UI

2018-02-26 Thread Parth Chandra (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378137#comment-16378137
 ] 

Parth Chandra commented on DRILL-6185:
--

[~kkhatua] are you looking into this?

> Error is displaying while accessing query profiles via the Web-UI
> -
>
> Key: DRILL-6185
> URL: https://issues.apache.org/jira/browse/DRILL-6185
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Anton Gozhiy
>Assignee: Kunal Khatua
>Priority: Blocker
> Fix For: 1.13.0
>
>
> *Steps:*
>  # Execute the following query:
> {code:sql}
> show schemas;
> {code}
> # On the Web-UI, go to the Profiles tab
> # Open the profile for the query you executed
> *Expected result:* You can access to the profile entry
> *Actual result:* Error is displayed:
> {code:json}
> {
>   "errorMessage" : "1"
> }
> {code}
> *Note:* This error doesn't happen with every query. For example, "select * 
> from system.version" can be accessed without error, while "show tables", "use 
> dfs", "alter sessions" etc end with this error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6125) PartitionSenderRootExec can leak memory because close method is not synchronized

2018-02-26 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-6125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377840#comment-16377840
 ] 

ASF GitHub Bot commented on DRILL-6125:
---

Github user vrozov commented on a diff in the pull request:

https://github.com/apache/drill/pull/1105#discussion_r170782805
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java
 ---
@@ -62,7 +62,7 @@
   private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(PartitionSenderRootExec.class);
   private RecordBatch incoming;
   private HashPartitionSender operator;
-  private PartitionerDecorator partitioner;
+  private volatile PartitionerDecorator partitioner;
--- End diff --

Consider using `AtomicReference` instead of `volatile` 


> PartitionSenderRootExec can leak memory because close method is not 
> synchronized
> 
>
> Key: DRILL-6125
> URL: https://issues.apache.org/jira/browse/DRILL-6125
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Minor
> Fix For: 1.13.0
>
>
> PartitionSenderRootExec creates a PartitionerDecorator and saves it in the 
> *partitioner* field. The creation of the partitioner happens in the 
> createPartitioner method. This method get's called by the main fragment 
> thread. The partitioner field is accessed by the fragment thread during 
> normal execution but it can also be accessed by the receivingFragmentFinished 
> method which is a callback executed by the event processor thread. Because 
> multiple threads can access the partitioner field synchronization is done on 
> creation and on when receivingFragmentFinished. However, the close method can 
> also be called by the event processor thread, and the close method does not 
> synchronize before accessing the partitioner field. Since synchronization is 
> not done the event processor thread may have an old reference to the 
> partitioner when a query cancellation is done. Since it has an old reference 
> the current partitioner can may not be cleared and a memory leak may occur.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (DRILL-6187) Exception in RPC communication between DataClient/ControlClient and respective servers when bit-to-bit security is on

2018-02-26 Thread Sorabh Hamirwasia (JIRA) 
Sorabh Hamirwasia created DRILL-6187:


 Summary: Exception in RPC communication between 
DataClient/ControlClient and respective servers when bit-to-bit security is on
 Key: DRILL-6187
 URL: https://issues.apache.org/jira/browse/DRILL-6187
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - RPC, Security
Reporter: Sorabh Hamirwasia
Assignee: Sorabh Hamirwasia
 Fix For: 1.13.0


 
{color:#00}Below is the summary of issue: {color}
 
{color:#00}*Scenario:*{color}
{color:#00}It seems like first sendRecordBatch was sent to Foreman which 
initiated the Authentication handshake. But before initiating handshake for 
auth we establish a connection and store that in a registry. Now if in parallel 
there is another recordBatch (by a different minor fragment running on same 
Drillbit) to be sent then that will see the connection available in registry 
and will initiate the send. Before the authentication is completed this second 
request reached foreman and it throws below exception saying RPC type 3 message 
is not allowed and closes the connection. This also fails the authentication 
handshake which was in progress.{color}{color:#00} Here the logs with 
details:{color}
{color:#00} {color}
{color:#00}*Forman received the SASL_START message from another 
node:*{color}
{color:#00}*_2018-02-21 18:43:30,759 [_*{color}{color:#00}_BitServer-4] 
TRACE o.a.d.e.r.s.ServerAuthenticationHandler - Received SASL message 
SASL_START from /10.10.100.161:35482_{color}
{color:#00} {color}
{color:#00}*Then around same time it received another message from client 
of Rpc Type 3 which is for SendRecordBatch and fails since handshake is not 
completed yet.*{color}
{color:#00} {color}
{color:#00}*_2018-02-21 18:43:30,762_*{color}{color:#00} _[BitServer-4] 
ERROR o.a.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication.  
Connection: /10.10.100.162:31012 <--> /__10.10.100.161:35482_ _(data server).  
Closing connection._{color}
{color:#00}_io.netty.handler.codec.DecoderException: 
org.apache.drill.exec.rpc.RpcException: Request of type 3 is not allowed 
without authentication. Client on /__10.10.100.161:35482_ _must authenticate 
before making requests. Connection dropped. [Details: Encryption: enabled , 
MaxWrappedSize: 65536 , WrapSizeLimit: 0]_{color}
{color:#00} {color}
{color:#00}*Then client receives an channel closed exception:*{color}
{color:#00} {color}
{color:#00}*2018-02-21 18:43:30,764 [*{color}{color:#00}BitClient-4] 
WARN  o.a.d.exec.rpc.RpcExceptionHandler - Exception occurred with closed 
channel.  Connection: /_10.10.100.161:35482_ <--> _10.10.100.162:31012_ (data 
client){color}
{color:#00} {color}
{color:#00}*and due to this it's initial command for authentication also 
fails. Since there is channel closed exception above I will think that 
triggered the failure of authentication request as well.*{color}
{color:#00} {color}
{color:#00}_Caused by: org.apache.drill.exec.rpc.RpcException: Command 
failed while establishing connection.  Failure type AUTHENTICATION._{color}
{color:#00}        _at 
org.apache.drill.exec.rpc.RpcException.mapException(RpcException.java:67) 
~[drill-rpc-1.12.0-mapr.jar:1.12.0-mapr]_{color}
{color:#00}        _at 
org.apache.drill.exec.rpc.ListeningCommand.connectionFailed(ListeningCommand.java:66)
 ~[drill-rpc-1.12.0-mapr.jar:1.12.0-mapr]_{color}
{color:#00}        _at 
org.apache.drill.exec.rpc.data.DataTunnel$SendBatchAsyncListen.connectionFailed(DataTunnel.java:166)
 ~[drill-java-exec-1.12.0-mapr.jar:1.12.0-mapr]_{color}
{color:#00}        _at 
org.apache.drill.exec.rpc.data.DataClient$AuthenticationCommand.connectionSucceeded(DataClient.java:203)
 ~[drill-java-exec-1.12.0-mapr.jar:1.12.0-mapr]_{color}
{color:#00}        _at 
org.apache.drill.exec.rpc.data.DataClient$AuthenticationCommand.connectionSucceeded(DataClient.java:147)
 ~[drill-java-exec-1.12.0-mapr.jar:1.12.0-mapr]_{color}
{color:#00}        _at 
org.apache.drill.exec.rpc.ReconnectingConnection$ConnectionListeningFuture.waitAndRun(ReconnectingConnection.java:122)
 ~[drill-rpc-1.12.0-mapr.jar:1.12.0-mapr]_{color}
{color:#00}        _at 
org.apache.drill.exec.rpc.ReconnectingConnection.runCommand(ReconnectingConnection.java:83)
 ~[drill-rpc-1.12.0-mapr.jar:1.12.0-mapr]_{color}
{color:#00}        _at 
org.apache.drill.exec.rpc.data.DataTunnel._{color}{color:#00}*_sendRecordBatch_*{color}{color:#00}_(DataTunnel.java:84)
 ~[drill-java-exec-1.12.0-mapr.jar:1.12.0-mapr]_{color}
{color:#00}        _at 
org.apache.drill.exec.ops.AccountingDataTunnel.sendRecordBatch(AccountingDataTunnel.java:45)
 ~[drill-java-exec-1.12.0-mapr.jar:1.12.0-mapr]_{color}
{color:#00}        _at

[jira] [Commented] (DRILL-6186) Document support for delegationUID client side ODBC/JDBC property in open source JDBC driver

2018-02-26 Thread Robert Hou (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-6186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377438#comment-16377438
 ] 

Robert Hou commented on DRILL-6186:
---

The open-source documentation calls it impersonation.  You can find 
documentation here:

 

https://drill.apache.org/docs/configuring-inbound-impersonation/

> Document support for delegationUID client side ODBC/JDBC property in open 
> source JDBC driver
> 
>
> Key: DRILL-6186
> URL: https://issues.apache.org/jira/browse/DRILL-6186
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - JDBC
>Affects Versions: 1.12.0
>Reporter: Veera Naranammalpuram
>Priority: Major
>  Labels: security
>
> There is no documentation around the "delegationUID' property in the open 
> source documentation. We at MapR ask our customers to use this property as 
> one form of impersonation. Because sqlline ships with the open source JDBC 
> driver, if users want to use delegationUID from sqlline because they use it 
> from ODBC/JDBC/ BI tools, they should be able to but there's no documentation 
> on drill.apache.org on how to do so. There is documentation on ODBC driver 
> but not on JDBC. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (DRILL-6186) Document support for delegationUID client side ODBC/JDBC property in open source JDBC driver

2018-02-26 Thread Veera Naranammalpuram (JIRA) 
Veera Naranammalpuram created DRILL-6186:


 Summary: Document support for delegationUID client side ODBC/JDBC 
property in open source JDBC driver
 Key: DRILL-6186
 URL: https://issues.apache.org/jira/browse/DRILL-6186
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - JDBC
Affects Versions: 1.12.0
Reporter: Veera Naranammalpuram


There is no documentation around the "delegationUID' property in the open 
source documentation. We at MapR ask our customers to use this property as one 
form of impersonation. Because sqlline ships with the open source JDBC driver, 
if users want to use delegationUID from sqlline because they use it from 
ODBC/JDBC/ BI tools, they should be able to but there's no documentation on 
drill.apache.org on how to do so. There is documentation on ODBC driver but not 
on JDBC. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (DRILL-5978) Updating of Apache and MapR Hive libraries to 2.3.2 and 2.1.1-mapr-1710 versions respectively

2018-02-26 Thread Vitalii Diravka (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitalii Diravka updated DRILL-5978:
---
Description: 
Currently Drill uses [Hive version 1.2.1 
libraries|https://github.com/apache/drill/blob/master/pom.xml#L53] to perform 
queries on Hive. This version of library can be used for Hive1.x versions and 
Hive2.x versions too, but some features of Hive2.x are broken (for example 
using of ORC transactional tables). To fix that it will be good to update 
drill-hive library version to 2.1 or newer. 
 Tasks which should be done:
 - resolving dependency conflicts;
 - investigating backward compatibility of newer drill-hive library with older 
Hive versions (1.x);
 - updating drill-hive version for 
[MapR|https://github.com/apache/drill/blob/master/pom.xml#L1777] profile too.



Starting from this commit:
 * Drill Hive client is updated to 2.3.2 and 2.1.1-mapr-1710 versions for 
default and MapR profiles respectively;
 * Drill supports of querying Hive transactional ORC bucketed tables 
[https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions].
 Note: Updated Drill Hive client preserves backward compatibility with older 
1.2.1 Hive server/metastore version.

  was:
Currently Drill uses [Hive version 1.2.1 
libraries|https://github.com/apache/drill/blob/master/pom.xml#L53] to perform 
queries on Hive. This version of library can be used for Hive1.x versions and 
Hive2.x versions too, but some features of Hive2.x are broken (for example 
using of ORC transactional tables). To fix that it will be good to update 
drill-hive library version to 2.1 or newer. 
Tasks which should be done:
- resolving dependency conflicts;
- investigating backward compatibility of newer drill-hive library with older 
Hive versions (1.x);
- updating drill-hive version for 
[MapR|https://github.com/apache/drill/blob/master/pom.xml#L1777] profile too.

Strarting from this commit Drill Hive client is updated to supports Hive 
transactional tables 
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions


> Updating of Apache and MapR Hive libraries to 2.3.2 and 2.1.1-mapr-1710 
> versions respectively
> -
>
> Key: DRILL-5978
> URL: https://issues.apache.org/jira/browse/DRILL-5978
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Storage - Hive
>Affects Versions: 1.11.0
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Major
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> Currently Drill uses [Hive version 1.2.1 
> libraries|https://github.com/apache/drill/blob/master/pom.xml#L53] to perform 
> queries on Hive. This version of library can be used for Hive1.x versions and 
> Hive2.x versions too, but some features of Hive2.x are broken (for example 
> using of ORC transactional tables). To fix that it will be good to update 
> drill-hive library version to 2.1 or newer. 
>  Tasks which should be done:
>  - resolving dependency conflicts;
>  - investigating backward compatibility of newer drill-hive library with 
> older Hive versions (1.x);
>  - updating drill-hive version for 
> [MapR|https://github.com/apache/drill/blob/master/pom.xml#L1777] profile too.
> Starting from this commit:
>  * Drill Hive client is updated to 2.3.2 and 2.1.1-mapr-1710 versions for 
> default and MapR profiles respectively;
>  * Drill supports of querying Hive transactional ORC bucketed tables 
> [https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions].
>  Note: Updated Drill Hive client preserves backward compatibility with older 
> 1.2.1 Hive server/metastore version.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (DRILL-5978) Updating of Apache and MapR Hive libraries to 2.3.2 and 2.1.1-mapr-1710 versions respectively

2018-02-26 Thread Vitalii Diravka (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitalii Diravka updated DRILL-5978:
---
Description: 
Currently Drill uses [Hive version 1.2.1 
libraries|https://github.com/apache/drill/blob/master/pom.xml#L53] to perform 
queries on Hive. This version of library can be used for Hive1.x versions and 
Hive2.x versions too, but some features of Hive2.x are broken (for example 
using of ORC transactional tables). To fix that it will be good to update 
drill-hive library version to 2.1 or newer. 
Tasks which should be done:
- resolving dependency conflicts;
- investigating backward compatibility of newer drill-hive library with older 
Hive versions (1.x);
- updating drill-hive version for 
[MapR|https://github.com/apache/drill/blob/master/pom.xml#L1777] profile too.

Strarting from this commit Drill Hive client is updated to supports Hive 
transactional tables 
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions

  was:
Currently Drill uses [Hive version 1.2.1 
libraries|https://github.com/apache/drill/blob/master/pom.xml#L53] to perform 
queries on Hive. This version of library can be used for Hive1.x versions and 
Hive2.x versions too, but some features of Hive2.x are broken (for example 
using of ORC transactional tables). To fix that it will be good to update 
drill-hive library version to 2.1 or newer. 
Tasks which should be done:
- resolving dependency conflicts;
- investigating backward compatibility of newer drill-hive library with older 
Hive versions (1.x);
- updating drill-hive version for 
[MapR|https://github.com/apache/drill/blob/master/pom.xml#L1777] profile too.


> Updating of Apache and MapR Hive libraries to 2.3.2 and 2.1.1-mapr-1710 
> versions respectively
> -
>
> Key: DRILL-5978
> URL: https://issues.apache.org/jira/browse/DRILL-5978
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Storage - Hive
>Affects Versions: 1.11.0
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Major
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> Currently Drill uses [Hive version 1.2.1 
> libraries|https://github.com/apache/drill/blob/master/pom.xml#L53] to perform 
> queries on Hive. This version of library can be used for Hive1.x versions and 
> Hive2.x versions too, but some features of Hive2.x are broken (for example 
> using of ORC transactional tables). To fix that it will be good to update 
> drill-hive library version to 2.1 or newer. 
> Tasks which should be done:
> - resolving dependency conflicts;
> - investigating backward compatibility of newer drill-hive library with older 
> Hive versions (1.x);
> - updating drill-hive version for 
> [MapR|https://github.com/apache/drill/blob/master/pom.xml#L1777] profile too.
> Strarting from this commit Drill Hive client is updated to supports Hive 
> transactional tables 
> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (DRILL-5978) Updating of Apache and MapR Hive libraries to 2.3.2 and 2.1.1-mapr-1710 versions respectively

2018-02-26 Thread Vitalii Diravka (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitalii Diravka updated DRILL-5978:
---
Summary: Updating of Apache and MapR Hive libraries to 2.3.2 and 
2.1.1-mapr-1710 versions respectively  (was: Updating of Apache and MapR Hive 
libraries to 2.3.2 and 2.1.2-mapr-1710 versions respectively)

> Updating of Apache and MapR Hive libraries to 2.3.2 and 2.1.1-mapr-1710 
> versions respectively
> -
>
> Key: DRILL-5978
> URL: https://issues.apache.org/jira/browse/DRILL-5978
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Storage - Hive
>Affects Versions: 1.11.0
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Major
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> Currently Drill uses [Hive version 1.2.1 
> libraries|https://github.com/apache/drill/blob/master/pom.xml#L53] to perform 
> queries on Hive. This version of library can be used for Hive1.x versions and 
> Hive2.x versions too, but some features of Hive2.x are broken (for example 
> using of ORC transactional tables). To fix that it will be good to update 
> drill-hive library version to 2.1 or newer. 
> Tasks which should be done:
> - resolving dependency conflicts;
> - investigating backward compatibility of newer drill-hive library with older 
> Hive versions (1.x);
> - updating drill-hive version for 
> [MapR|https://github.com/apache/drill/blob/master/pom.xml#L1777] profile too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

2018-02-26 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377058#comment-16377058
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170633434
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/stat/ParquetPredicates.java
 ---
@@ -66,6 +66,27 @@ public ParquetBooleanPredicate(String name, 
List args, Expres
 }
   }
 
+  public static abstract class ParquetIsPredicate extends 
LogicalExpressionBase implements ParquetFilterPredicate {
--- End diff --

`ParquetPredicates` class becomes too overloaded. Please split it into two: 
`ParquetComparisionPredicates` and `ParquetIsPredicate`.


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

2018-02-26 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377059#comment-16377059
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170630734
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestParquetFilterPushDownForDateTimeCasts.java
 ---
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.parquet;
+
+import org.apache.drill.PlanTestBase;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import static org.junit.Assert.assertEquals;
+
+public class TestParquetFilterPushDownForDateTimeCasts extends 
PlanTestBase {
+
+  private static final String TABLE_NAME = "dateTimeCasts";
+
+  @BeforeClass
+  public static void init() throws Exception {
+test("use dfs.tmp");
+test("create table `%s/p1` as\n" +
+"select timestamp '2017-01-01 00:00:00' as col_timestamp, date 
'2017-01-01' as col_date, time '00:00:00' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-02 00:00:00' as col_timestamp, date 
'2017-01-02' as col_date, time '00:00:00' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-02 21:01:15' as col_timestamp, date 
'2017-01-02' as col_date, time '21:01:15' as col_time from (values(1))", 
TABLE_NAME);
+
+test("create table `%s/p2` as\n" +
+"select timestamp '2017-01-03 08:50:00' as col_timestamp, date 
'2017-01-03' as col_date, time '08:50:00' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-04 15:25:00' as col_timestamp, date 
'2017-01-04' as col_date, time '15:25:00' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-04 22:14:29' as col_timestamp, date 
'2017-01-04' as col_date, time '22:14:29' as col_time from (values(1))", 
TABLE_NAME);
+
+test("create table `%s/p3` as\n" +
+"select timestamp '2017-01-05 05:46:11' as col_timestamp, date 
'2017-01-05' as col_date, time '05:46:11' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-06 06:17:59' as col_timestamp, date 
'2017-01-06' as col_date, time '06:17:59' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-06 06:17:59' as col_timestamp, date 
'2017-01-06' as col_date, time '06:17:59' as col_time from (values(1)) union 
all\n" +
+"select cast(null as timestamp) as col_timestamp, cast(null as 
date) as col_date, cast(null as time) as col_time from (values(1))", 
TABLE_NAME);
+  }
+
+  @AfterClass
+  public static void tearDown() throws Exception {
+test("drop table if exists `%s`", TABLE_NAME);
+  }
+
+  @Test
+  public void testCastTimestampVarchar() throws Exception {
+testParquetFilterPushDown("col_timestamp = '2017-01-05 05:46:11'", 1, 
1);
+testParquetFilterPushDown("col_timestamp = cast('2017-01-05 05:46:11' 
as varchar)", 1, 1);
+testParquetFilterPushDown("col_timestamp = cast('2017-01-05 05:46:11' 
as timestamp)", 1, 1);
+testParquetFilterPushDown("col_timestamp > '2017-01-02 00:00:00'", 7, 
3);
+testParquetFilterPushDown("col_timestamp between '2017-01-03 21:01:15' 
and '2017-01-06 05:46:11'", 3, 2);
+testParquetFilterPushDown("col_timestamp between '2017-01-03' and 
'2017-01-06'", 4, 2);
+  }
+
+  @Test
+  public void testCastTimestampDate() throws Exception {
+testParquetFilterPushDown("col_timestamp = date '2017-01-02'", 1, 1);
+testParquetFilterPushDown("col_timestamp = cast(date '2017-01-02' as 
timestamp)", 1, 1);
+testParquetFilterPushDown("col_timestamp > date '2017-01-02'", 7, 3);
+testParquetFilterPushDown("col_timestamp between date '2017-01-03' and 
date '2017-01-06'", 4,

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

2018-02-26 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377057#comment-16377057
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170629893
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFilterBuilder.java
 ---
@@ -261,6 +273,37 @@ private LogicalExpression 
handleCompareFunction(FunctionHolderExpression functio
 }
   }
 
+  private LogicalExpression handleIsFunction(FunctionHolderExpression 
functionHolderExpression, Set value) {
+String funcName;
+
+if (functionHolderExpression.getHolder() instanceof 
DrillSimpleFuncHolder) {
+  funcName = ((DrillSimpleFuncHolder) 
functionHolderExpression.getHolder()).getRegisteredNames()[0];
+} else {
+  logger.warn("Can not cast {} to DrillSimpleFuncHolder. Parquet 
filter pushdown can not handle function.",
+  functionHolderExpression.getHolder());
+  return null;
+}
+LogicalExpression arg = functionHolderExpression.args.get(0);
+
+switch (funcName) {
--- End diff --

Please use proper formatting for switch case.


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

2018-02-26 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377056#comment-16377056
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170630466
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestParquetFilterPushDown.java
 ---
@@ -388,6 +388,45 @@ public void testTimeStampPredicate() throws Exception {
 testParquetFilterPD(query3, 49, 2, false);
   }
 
+  @Test
+  public void testBooleanPredicate() throws Exception {
+// Table blnTbl was created by CTAS in drill 1.12.0 and consist of 4 
files withe the next data:
--- End diff --

Can we generate files on the fly rather then storing them? This will help 
to prevent any issues, for example, if files generation will change in the next 
version.


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

2018-02-26 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377013#comment-16377013
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

GitHub user KulykRoman opened a pull request:

https://github.com/apache/drill/pull/1131

DRILL-6174: Parquet filter pushdown improvements.

Added support IS [NOT] NULL/TRUE/FALSE operator for the parquet filter 
pushdown.
Added timestamp/date/time implicit/explicit casts.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/KulykRoman/drill DRILL-6174

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1131.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1131


commit c77fe23f7742325f534f45df816d80230fe45e75
Author: Roman Kulyk 
Date:   2018-02-15T15:37:43Z

DRILL-6174: Parquet filter pushdown improvements.

Added support IS [NOT] NULL/TRUE/FALSE operator for the parquet filter 
pushdown.
Added timestamp/date/time implicit/explicit casts.




> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (DRILL-6185) Error is displaying while accessing query profiles via the Web-UI

2018-02-26 Thread Arina Ielchiieva (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-6185:

Priority: Blocker  (was: Major)

> Error is displaying while accessing query profiles via the Web-UI
> -
>
> Key: DRILL-6185
> URL: https://issues.apache.org/jira/browse/DRILL-6185
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Anton Gozhiy
>Assignee: Kunal Khatua
>Priority: Blocker
> Fix For: 1.13.0
>
>
> *Steps:*
>  # Execute the following query:
> {code:sql}
> show schemas;
> {code}
> # On the Web-UI, go to the Profiles tab
> # Open the profile for the query you executed
> *Expected result:* You can access to the profile entry
> *Actual result:* Error is displayed:
> {code:json}
> {
>   "errorMessage" : "1"
> }
> {code}
> *Note:* This error doesn't happen with every query. For example, "select * 
> from system.version" can be accessed without error, while "show tables", "use 
> dfs", "alter sessions" etc end with this error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6185) Error is displaying while accessing query profiles via the Web-UI

2018-02-26 Thread Arina Ielchiieva (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376845#comment-16376845
 ] 

Arina Ielchiieva commented on DRILL-6185:
-

This is a regression after DRILL-6140. Change in which assume that query plan 
always exists but this is not true for special operators (show, alter, create 
etc) or queries with invalid syntax.

> Error is displaying while accessing query profiles via the Web-UI
> -
>
> Key: DRILL-6185
> URL: https://issues.apache.org/jira/browse/DRILL-6185
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Anton Gozhiy
>Assignee: Kunal Khatua
>Priority: Blocker
> Fix For: 1.13.0
>
>
> *Steps:*
>  # Execute the following query:
> {code:sql}
> show schemas;
> {code}
> # On the Web-UI, go to the Profiles tab
> # Open the profile for the query you executed
> *Expected result:* You can access to the profile entry
> *Actual result:* Error is displayed:
> {code:json}
> {
>   "errorMessage" : "1"
> }
> {code}
> *Note:* This error doesn't happen with every query. For example, "select * 
> from system.version" can be accessed without error, while "show tables", "use 
> dfs", "alter sessions" etc end with this error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (DRILL-6185) Error is displaying while accessing query profiles via the Web-UI

2018-02-26 Thread Arina Ielchiieva (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376845#comment-16376845
 ] 

Arina Ielchiieva edited comment on DRILL-6185 at 2/26/18 1:25 PM:
--

This is a regression after DRILL-6140. Change in which assumes that query plan 
always exists but this is not true for special operators (show, alter, create 
etc) or queries with invalid syntax.


was (Author: arina):
This is a regression after DRILL-6140. Change in which assume that query plan 
always exists but this is not true for special operators (show, alter, create 
etc) or queries with invalid syntax.

> Error is displaying while accessing query profiles via the Web-UI
> -
>
> Key: DRILL-6185
> URL: https://issues.apache.org/jira/browse/DRILL-6185
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Anton Gozhiy
>Assignee: Kunal Khatua
>Priority: Blocker
> Fix For: 1.13.0
>
>
> *Steps:*
>  # Execute the following query:
> {code:sql}
> show schemas;
> {code}
> # On the Web-UI, go to the Profiles tab
> # Open the profile for the query you executed
> *Expected result:* You can access to the profile entry
> *Actual result:* Error is displayed:
> {code:json}
> {
>   "errorMessage" : "1"
> }
> {code}
> *Note:* This error doesn't happen with every query. For example, "select * 
> from system.version" can be accessed without error, while "show tables", "use 
> dfs", "alter sessions" etc end with this error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (DRILL-6185) Error is displaying while accessing query profiles via the Web-UI

2018-02-26 Thread Arina Ielchiieva (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva reassigned DRILL-6185:
---

Assignee: Kunal Khatua

> Error is displaying while accessing query profiles via the Web-UI
> -
>
> Key: DRILL-6185
> URL: https://issues.apache.org/jira/browse/DRILL-6185
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Anton Gozhiy
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.13.0
>
>
> *Steps:*
>  # Execute the following query:
> {code:sql}
> show schemas;
> {code}
> # On the Web-UI, go to the Profiles tab
> # Open the profile for the query you executed
> *Expected result:* You can access to the profile entry
> *Actual result:* Error is displayed:
> {code:json}
> {
>   "errorMessage" : "1"
> }
> {code}
> *Note:* This error doesn't happen with every query. For example, "select * 
> from system.version" can be accessed without error, while "show tables", "use 
> dfs", "alter sessions" etc end with this error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (DRILL-6185) Error is displaying while accessing query profiles via the Web-UI

2018-02-26 Thread Arina Ielchiieva (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-6185:

Fix Version/s: 1.13.0

> Error is displaying while accessing query profiles via the Web-UI
> -
>
> Key: DRILL-6185
> URL: https://issues.apache.org/jira/browse/DRILL-6185
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Anton Gozhiy
>Assignee: Kunal Khatua
>Priority: Blocker
> Fix For: 1.13.0
>
>
> *Steps:*
>  # Execute the following query:
> {code:sql}
> show schemas;
> {code}
> # On the Web-UI, go to the Profiles tab
> # Open the profile for the query you executed
> *Expected result:* You can access to the profile entry
> *Actual result:* Error is displayed:
> {code:json}
> {
>   "errorMessage" : "1"
> }
> {code}
> *Note:* This error doesn't happen with every query. For example, "select * 
> from system.version" can be accessed without error, while "show tables", "use 
> dfs", "alter sessions" etc end with this error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (DRILL-6185) Error is displaying while accessing query profiles via the Web-UI

2018-02-26 Thread Anton Gozhiy (JIRA) 
Anton Gozhiy created DRILL-6185:
---

 Summary: Error is displaying while accessing query profiles via 
the Web-UI
 Key: DRILL-6185
 URL: https://issues.apache.org/jira/browse/DRILL-6185
 Project: Apache Drill
  Issue Type: Bug
Affects Versions: 1.13.0
Reporter: Anton Gozhiy


*Steps:*
 # Execute the following query:

{code:sql}
show schemas;
{code}

# On the Web-UI, go to the Profiles tab
# Open the profile for the query you executed

*Expected result:* You can access to the profile entry

*Actual result:* Error is displayed:

{code:json}
{
  "errorMessage" : "1"
}
{code}

*Note:* This error doesn't happen with every query. For example, "select * from 
system.version" can be accessed without error, while "show tables", "use dfs", 
"alter sessions" etc end with this error.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)