[jira] [Assigned] (DRILL-5868) Support SQL syntax highlighting and formatting in the Edit Query textbox

[Kunal Khatua (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Khatua reassigned DRILL-5868:
---

Assignee: Kunal Khatua

> Support SQL syntax highlighting and formatting in the Edit Query textbox 
> -
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Web Server
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Fix For: Future
>
>
> It would be nice to have the Query Editor support syntax highlighting.
> An autocomplete would be even better as new functions are introduced in Drill



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5993) Allow Copier to Copy a Record and Append to the End of an Outgoing Batch

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271921#comment-16271921
 ] 

ASF GitHub Bot commented on DRILL-5993:
---

Github user ilooner commented on the issue:

https://github.com/apache/drill/pull/1057
  
Please let me know if I should make any more changes.


> Allow Copier to Copy a Record and Append to the End of an Outgoing Batch
> 
>
> Key: DRILL-5993
> URL: https://issues.apache.org/jira/browse/DRILL-5993
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently the copier can only copy record from an incoming batch to the 
> beginning of an outgoing batch. We need to be able to copy a record and 
> append it to the end of the outgoing batch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5993) Allow Copier to Copy a Record and Append to the End of an Outgoing Batch

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271919#comment-16271919
 ] 

ASF GitHub Bot commented on DRILL-5993:
---

Github user ilooner commented on the issue:

https://github.com/apache/drill/pull/1057
  
@paul-rogers since HashJoin does not need to support Selection Vectors 
maybe we can postpone adding the corresponding appendRow methods if and when 
they are needed. I suspect by the time anyone needs those methods we will have 
already migrated over to the new batch framework.


> Allow Copier to Copy a Record and Append to the End of an Outgoing Batch
> 
>
> Key: DRILL-5993
> URL: https://issues.apache.org/jira/browse/DRILL-5993
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently the copier can only copy record from an incoming batch to the 
> beginning of an outgoing batch. We need to be able to copy a record and 
> append it to the end of the outgoing batch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5993) Allow Copier to Copy a Record and Append to the End of an Outgoing Batch

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271912#comment-16271912
 ] 

ASF GitHub Bot commented on DRILL-5993:
---

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/1057#discussion_r153958856
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/record/TestVectorContainer.java
 ---
@@ -124,4 +132,52 @@ public void testContainerMerge() {
 leftIndirect.clear();
 right.clear();
   }
+
+  @Test
+  public void testAppendRow()
+  {
+MaterializedField colA = MaterializedField.create("colA", 
Types.required(TypeProtos.MinorType.INT));
+MaterializedField colB = MaterializedField.create("colB", 
Types.required(TypeProtos.MinorType.INT));
--- End diff --

Added more interesting types. Currently RowSet classes don't support the 
Map data type. Paul asked me to look into adding support for this a while ago 
for DRILL-5870 . I'll update the test framework to support that in the next PR.


> Allow Copier to Copy a Record and Append to the End of an Outgoing Batch
> 
>
> Key: DRILL-5993
> URL: https://issues.apache.org/jira/browse/DRILL-5993
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently the copier can only copy record from an incoming batch to the 
> beginning of an outgoing batch. We need to be able to copy a record and 
> append it to the end of the outgoing batch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-3640) Drill JDBC driver support Statement.setQueryTimeout(int)

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-3640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271773#comment-16271773
 ] 

ASF GitHub Bot commented on DRILL-3640:
---

Github user kkhatua commented on the issue:

https://github.com/apache/drill/pull/1024
  
@laurentgo 
I've added server-triggered timeout tests and made other changes as well, 
but they require support for 
[DRILL-5973](https://issues.apache.org/jira/browse/DRILL-5973) . I tested this 
commit (#1024 ) as a cherry pick on top of that PR's commit (#1055) and I was 
able to simulate the server-induced timeout.
Will need a +1 for that PR before I can enable the tests here.
For now, I've marked these tests as `@ignore` to ensure that the remaining 
tests pass and the feature works as intended. 

Can you review them both (this and #1055 )?


> Drill JDBC driver support Statement.setQueryTimeout(int)
> 
>
> Key: DRILL-3640
> URL: https://issues.apache.org/jira/browse/DRILL-3640
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Client - JDBC
>Affects Versions: 1.2.0
>Reporter: Chun Chang
>Assignee: Kunal Khatua
> Fix For: 1.13.0
>
>
> It would be nice if we have this implemented. Run away queries can be 
> automatically canceled by setting the timeout. 
> java.sql.SQLFeatureNotSupportedException: Setting network timeout is not 
> supported.
>   at 
> org.apache.drill.jdbc.impl.DrillStatementImpl.setQueryTimeout(DrillStatementImpl.java:152)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-6002) Avoid memory copy from direct buffer to heap while spilling to local disk

[Vlad Rozov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vlad Rozov updated DRILL-6002:
--
Reviewer: Paul Rogers

> Avoid memory copy from direct buffer to heap while spilling to local disk
> -
>
> Key: DRILL-6002
> URL: https://issues.apache.org/jira/browse/DRILL-6002
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>
> When spilling to a local disk or to any file system that supports 
> WritableByteChannel it is preferable to avoid copy from off-heap to java heap 
> as WritableByteChannel can work directly with the off-heap memory.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-6002) Avoid memory copy from direct buffer to heap while spilling to local disk

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271768#comment-16271768
 ] 

ASF GitHub Bot commented on DRILL-6002:
---

GitHub user vrozov opened a pull request:

https://github.com/apache/drill/pull/1058

DRILL-6002: Avoid memory copy from direct buffer to heap while spilling to 
local disk

@paul-rogers Please review

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vrozov/drill DRILL-6002

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1058.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1058


commit 8e9124de681d3a8cd70bf0bb243460cb78dcb295
Author: Vlad Rozov 
Date:   2017-11-22T22:06:13Z

DRILL-6002: Avoid memory copy from direct buffer to heap while spilling to 
local disk




> Avoid memory copy from direct buffer to heap while spilling to local disk
> -
>
> Key: DRILL-6002
> URL: https://issues.apache.org/jira/browse/DRILL-6002
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>
> When spilling to a local disk or to any file system that supports 
> WritableByteChannel it is preferable to avoid copy from off-heap to java heap 
> as WritableByteChannel can work directly with the off-heap memory.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-6002) Avoid memory copy from direct buffer to heap while spilling to local disk

[Vlad Rozov (JIRA)]

Vlad Rozov created DRILL-6002:
-

 Summary: Avoid memory copy from direct buffer to heap while 
spilling to local disk
 Key: DRILL-6002
 URL: https://issues.apache.org/jira/browse/DRILL-6002
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Vlad Rozov
Assignee: Vlad Rozov


When spilling to a local disk or to any file system that supports 
WritableByteChannel it is preferable to avoid copy from off-heap to java heap 
as WritableByteChannel can work directly with the off-heap memory.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5993) Allow Copier to Copy a Record and Append to the End of an Outgoing Batch

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271670#comment-16271670
 ] 

ASF GitHub Bot commented on DRILL-5993:
---

Github user Ben-Zvi commented on a diff in the pull request:

https://github.com/apache/drill/pull/1057#discussion_r153934729
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java 
---
@@ -353,6 +353,23 @@ public int getRecordCount() {
 
   public boolean hasRecordCount() { return recordCount != -1; }
 
+  /**
+   * This works with non-hyper {@link VectorContainer}s which have no 
selection vectors.
+   * Appends a row taken from a source {@link VectorContainer} to this 
{@link VectorContainer}.
+   * @param srcContainer The {@link VectorContainer} to copy a row from.
+   * @param srcIndex The index of the row to copy from the source {@link 
VectorContainer}.
+   */
+  public void appendRow(VectorContainer srcContainer, int srcIndex) {
+for (int vectorIndex = 0; vectorIndex < wrappers.size(); 
vectorIndex++) {
+  ValueVector destVector = wrappers.get(vectorIndex).getValueVector();
+  ValueVector srcVector = 
srcContainer.wrappers.get(vectorIndex).getValueVector();
+
+  destVector.copyEntry(recordCount, srcVector, srcIndex);
+}
+
+recordCount++;
--- End diff --

 The immediate need for appendRow() is to distribute rows from a single 
incoming batch into multiple other batches (for the Hash Join internal 
partitioning), based on the hash value of the key columns at each row. This 
would not work well with the second suggestion (vectorizing - column 1, column 
2, etc.)  


> Allow Copier to Copy a Record and Append to the End of an Outgoing Batch
> 
>
> Key: DRILL-5993
> URL: https://issues.apache.org/jira/browse/DRILL-5993
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently the copier can only copy record from an incoming batch to the 
> beginning of an outgoing batch. We need to be able to copy a record and 
> append it to the end of the outgoing batch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5993) Allow Copier to Copy a Record and Append to the End of an Outgoing Batch

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271671#comment-16271671
 ] 

ASF GitHub Bot commented on DRILL-5993:
---

Github user Ben-Zvi commented on a diff in the pull request:

https://github.com/apache/drill/pull/1057#discussion_r153935071
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/record/TestVectorContainer.java
 ---
@@ -124,4 +132,52 @@ public void testContainerMerge() {
 leftIndirect.clear();
 right.clear();
   }
+
+  @Test
+  public void testAppendRow()
+  {
+MaterializedField colA = MaterializedField.create("colA", 
Types.required(TypeProtos.MinorType.INT));
+MaterializedField colB = MaterializedField.create("colB", 
Types.required(TypeProtos.MinorType.INT));
--- End diff --

 Maybe add some "interesting" datatypes ?  Testing integers only may miss 
some issue. 



> Allow Copier to Copy a Record and Append to the End of an Outgoing Batch
> 
>
> Key: DRILL-5993
> URL: https://issues.apache.org/jira/browse/DRILL-5993
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently the copier can only copy record from an incoming batch to the 
> beginning of an outgoing batch. We need to be able to copy a record and 
> append it to the end of the outgoing batch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-6001) Deprecate using assertions (-ea) to enable direct memory allocation tracing.

[Vlad Rozov (JIRA)]

Vlad Rozov created DRILL-6001:
-

 Summary: Deprecate using assertions (-ea) to enable direct memory 
allocation tracing.
 Key: DRILL-6001
 URL: https://issues.apache.org/jira/browse/DRILL-6001
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Vlad Rozov
Assignee: Vlad Rozov
Priority: Minor


Drill uses assertion (-ea) to enable memory allocation tracing. Most of the 
time assertions are enabled/disabled globally (for all packages) by using "-ea" 
java command line option and it leads to excessive CPU and heap utilization. It 
will be better to limit the impact of assertion enabled to the java "assert" 
statement as expected by a majority of Java developers and use a separate 
property (that already exists) to enable/disable direct memory allocation 
tracing/debugging.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5993) Allow Copier to Copy a Record and Append to the End of an Outgoing Batch

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271600#comment-16271600
 ] 

ASF GitHub Bot commented on DRILL-5993:
---

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1057#discussion_r153926390
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java 
---
@@ -353,6 +353,23 @@ public int getRecordCount() {
 
   public boolean hasRecordCount() { return recordCount != -1; }
 
+  /**
+   * This works with non-hyper {@link VectorContainer}s which have no 
selection vectors.
+   * Appends a row taken from a source {@link VectorContainer} to this 
{@link VectorContainer}.
+   * @param srcContainer The {@link VectorContainer} to copy a row from.
+   * @param srcIndex The index of the row to copy from the source {@link 
VectorContainer}.
+   */
+  public void appendRow(VectorContainer srcContainer, int srcIndex) {
+for (int vectorIndex = 0; vectorIndex < wrappers.size(); 
vectorIndex++) {
+  ValueVector destVector = wrappers.get(vectorIndex).getValueVector();
+  ValueVector srcVector = 
srcContainer.wrappers.get(vectorIndex).getValueVector();
+
+  destVector.copyEntry(recordCount, srcVector, srcIndex);
+}
+
+recordCount++;
--- End diff --

This is OK for a row-by-row copy. But, you'll get better performance if you 
optimize for the entire batch. Because you have no SV4, the source and dest 
batches are the same so the vectors can be preloaded into an array of vectors 
to avoid the vector wrapper lookup per column.

Plus, if the code is written per batch, you can go a step further: 
vectorize the operation. Copy all values for column 1, then all for column 2, 
and so on. (In this case, you only get each vector once, so sticking with the 
wrappers is fine.) By vectorizing, you may get the vectorized cache-locality 
benefit that Drill promises from its operations. Worth a try to see if you get 
any speed-up.


> Allow Copier to Copy a Record and Append to the End of an Outgoing Batch
> 
>
> Key: DRILL-5993
> URL: https://issues.apache.org/jira/browse/DRILL-5993
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently the copier can only copy record from an incoming batch to the 
> beginning of an outgoing batch. We need to be able to copy a record and 
> append it to the end of the outgoing batch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5993) Allow Copier to Copy a Record and Append to the End of an Outgoing Batch

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271587#comment-16271587
 ] 

ASF GitHub Bot commented on DRILL-5993:
---

Github user paul-rogers commented on the issue:

https://github.com/apache/drill/pull/1057
  
To answer the two questions:

1. The copier is used in multiple locations, some of which include 
selection vectors. Sort uses a copier to merge rows coming from multiple sorted 
batches. The SVR compresses out SVs. A filter will produce an SV2 which the SVR 
removes. An in-memory sort produces an SV4. But, because of the ways plans are 
generated, the hash join will never see a batch with an SV. (An SVR will be 
inserted, if needed, to remove the SV.)

2. We never write a batch using an SV. The SV is always a source 
indirection. Because we do indirection on the source side (and vectors are 
append only), there can be no SV on the destination side.

Note also that the {{VectorContainer}} class, despite it's API, knows 
nothing about SVs. The SV is tacked on separately by the {{RecordBatch}}. (This 
is a less-than-ideal design, but it is how things work at present.) FWIW, the 
test-oriented {{RowSet}} abstractions came about as wrappers around both the 
{{VectorContainer}} and SV to provide a unified view.

Because of how we do SVs, you'll need three copy methods: one for no SV, 
one for an SV2 and another for an SV4.

In the fullness of time, the new "column reader" and "column writer" 
abstractions will hide all this stuff, but it will take time before those tools 
come online.


> Allow Copier to Copy a Record and Append to the End of an Outgoing Batch
> 
>
> Key: DRILL-5993
> URL: https://issues.apache.org/jira/browse/DRILL-5993
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently the copier can only copy record from an incoming batch to the 
> beginning of an outgoing batch. We need to be able to copy a record and 
> append it to the end of the outgoing batch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5702) Jdbc Driver Class not found

[Holger Kiel (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271584#comment-16271584
 ] 

Holger Kiel commented on DRILL-5702:


Same problem with 1.12.0rc0 binary package:
http://home.apache.org/~arina/drill/releases/1.12.0/rc0/

Where as when compiling from latested repository, the jdbc-driver binary is 
fine.

> Jdbc Driver Class not found
> ---
>
> Key: DRILL-5702
> URL: https://issues.apache.org/jira/browse/DRILL-5702
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - JDBC
>Affects Versions: 1.11.0
>Reporter: Holger Kiel
>Priority: Critical
>
> Cannot connect to drill cluster after upgrade to new Jar 
> drill-jdbc-all-1.11.0.jar. When replacing Jar file with older release 
> drill-jdbc-all-1.10.0.jar, connection works again. Tested with various client 
> applications:
> {code}
> java.lang.RuntimeException: java.lang.RuntimeException: 
> java.lang.ClassNotFoundException: Class 
> ${package.namespace.prefix}org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback
>  not found
> at 
> oadd.org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2227)
> at oadd.org.apache.hadoop.security.Groups.(Groups.java:80)
> at oadd.org.apache.hadoop.security.Groups.(Groups.java:74)
> at 
> oadd.org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:303)
> at 
> oadd.org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:283)
> at 
> oadd.org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:311)
> at 
> oadd.org.apache.drill.exec.rpc.security.plain.PlainFactory.createAndLoginUser(PlainFactory.java:63)
> at 
> oadd.org.apache.drill.exec.rpc.user.UserClient.authenticate(UserClient.java:244)
> at 
> oadd.org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:171)
> at 
> oadd.org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> oadd.org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:158)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> oadd.org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at java.sql.DriverManager.getConnection(DriverManager.java:664)
> at java.sql.DriverManager.getConnection(DriverManager.java:208)
> {code}
> Workaround is using the old driver version.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5993) Allow Copier to Copy a Record and Append to the End of an Outgoing Batch

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271512#comment-16271512
 ] 

ASF GitHub Bot commented on DRILL-5993:
---

Github user ilooner commented on the issue:

https://github.com/apache/drill/pull/1057
  
@Ben-Zvi @paul-rogers 


> Allow Copier to Copy a Record and Append to the End of an Outgoing Batch
> 
>
> Key: DRILL-5993
> URL: https://issues.apache.org/jira/browse/DRILL-5993
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently the copier can only copy record from an incoming batch to the 
> beginning of an outgoing batch. We need to be able to copy a record and 
> append it to the end of the outgoing batch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5993) Allow Copier to Copy a Record and Append to the End of an Outgoing Batch

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271511#comment-16271511
 ] 

ASF GitHub Bot commented on DRILL-5993:
---

GitHub user ilooner opened a pull request:

https://github.com/apache/drill/pull/1057

DRILL-5993 Append Row Method For VectorContainer (WIP)

## Motivation

HashJoin requires a method that can take a row from a VectorContainer and 
append it to a destination VectorContainer. This is a WIP and this PR is mainly 
opened to improve my understanding.

## Implementation

This is an initial implementation that works with simple VectorContainers 
that are not hyper batches and do not have selection vectors. It is also 
assumed that the user called **SchemaUtil.coerceContainer** on the source 
VectorContainer before using the newly added **appendRow** method.

## Questions

- Do we have to worry about selection vectors in the source container?
- Do we have to think about support hyper batches in the destination 
container?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ilooner/drill DRILL-5993

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1057.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1057


commit ee43d6808562a1ff60c17fa7622b8358b63c7276
Author: Timothy Farkas 
Date:   2017-11-29T20:38:41Z

 - Initial Implementation of append row for a vector container




> Allow Copier to Copy a Record and Append to the End of an Outgoing Batch
> 
>
> Key: DRILL-5993
> URL: https://issues.apache.org/jira/browse/DRILL-5993
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently the copier can only copy record from an incoming batch to the 
> beginning of an outgoing batch. We need to be able to copy a record and 
> append it to the end of the outgoing batch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5971) Fix INT64, INT32 logical types in complex parquet reader

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271510#comment-16271510
 ] 

ASF GitHub Bot commented on DRILL-5971:
---

Github user parthchandra commented on a diff in the pull request:

https://github.com/apache/drill/pull/1049#discussion_r153909807
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReaderFactory.java
 ---
@@ -138,6 +138,8 @@
 return new 
ParquetFixedWidthDictionaryReaders.DictionaryBigIntReader(recordReader, 
allocateSize, descriptor, columnChunkMetaData, fixedLength, (BigIntVector) v, 
schemaElement);
--- End diff --

OK I'll try to add these. BTW, I realized that the test files that I added 
for the unit tests are not annotated, so I'll need to fix those as well!


> Fix INT64, INT32 logical types in complex parquet reader
> 
>
> Key: DRILL-5971
> URL: https://issues.apache.org/jira/browse/DRILL-5971
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Parth Chandra
>Assignee: Parth Chandra
> Fix For: 1.13.0
>
>
> The 'complex' Parquet reader does not recognize the Parquet logical types 
> INT64, and INT32. 
> Should be a simple change to add these logical types.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-6000) Graceful Shutdown Unit Tests Should Not Be Run On Travis

[Timothy Farkas (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Timothy Farkas updated DRILL-6000:
--
Reviewer: Arina Ielchiieva

> Graceful Shutdown Unit Tests Should Not Be Run On Travis
> 
>
> Key: DRILL-6000
> URL: https://issues.apache.org/jira/browse/DRILL-6000
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently these tests fail on Travis. But since they are heavy weight tests 
> which test a corner case, they should not be run as part of the smoke tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-6000) Graceful Shutdown Unit Tests Should Not Be Run On Travis

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271490#comment-16271490
 ] 

ASF GitHub Bot commented on DRILL-6000:
---

GitHub user ilooner opened a pull request:

https://github.com/apache/drill/pull/1056

DRILL-6000: Categorized graceful shutdown unit tests as SlowTests

Graceful shutdown unit tests were failing on Travis, and should not be run 
as part of the SmokeTests

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ilooner/drill DRILL-6000

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1056.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1056


commit 5ecaade4b5cdc91a3153a4e2394cfdd993eeb5cf
Author: Timothy Farkas 
Date:   2017-11-29T18:52:29Z

DRILL-6000: Categorized graceful shutdown unit tests as SlowTests




> Graceful Shutdown Unit Tests Should Not Be Run On Travis
> 
>
> Key: DRILL-6000
> URL: https://issues.apache.org/jira/browse/DRILL-6000
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently these tests fail on Travis. But since they are heavy weight tests 
> which test a corner case, they should not be run as part of the smoke tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-6000) Graceful Shutdown Unit Tests Should Not Be Run On Travis

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271491#comment-16271491
 ] 

ASF GitHub Bot commented on DRILL-6000:
---

Github user ilooner commented on the issue:

https://github.com/apache/drill/pull/1056
  
@arina-ielchiieva 


> Graceful Shutdown Unit Tests Should Not Be Run On Travis
> 
>
> Key: DRILL-6000
> URL: https://issues.apache.org/jira/browse/DRILL-6000
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently these tests fail on Travis. But since they are heavy weight tests 
> which test a corner case, they should not be run as part of the smoke tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5973) Support injection of time-bound pauses in server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271433#comment-16271433
 ] 

ASF GitHub Bot commented on DRILL-5973:
---

Github user kkhatua commented on the issue:

https://github.com/apache/drill/pull/1055
  
@laurentgo / @parthchandra 
Please review this. It is the basis for unit tests in DRILL-3640


> Support injection of time-bound pauses in server
> 
>
> Key: DRILL-5973
> URL: https://issues.apache.org/jira/browse/DRILL-5973
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
> Fix For: 1.13.0
>
>
> While working on DRILL-3640 , when creating a unit test for a server-induced 
> timeout, the injecting a pause leaves the JUnit framework's DrillClient 
> without a handle to the query on the server. This is because we injected the 
> pause to occur before the server could send back a query ID, so the 
> DrillClient has no way to unpause the server.
> The workaround to support this unit test is to allow for injecting pauses 
> with a defined time-bound, after which the server would resume.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5973) Support injection of time-bound pauses in server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271431#comment-16271431
 ] 

ASF GitHub Bot commented on DRILL-5973:
---

GitHub user kkhatua opened a pull request:

https://github.com/apache/drill/pull/1055

DRILL-5973 : Support injection of time-bound pauses in server

Support pause injections in the test framework that are time-bound, to 
allow for testing high latency scenarios.
e.g. delayed server response to the Drill client allows for test a 
server-induced timeout
This would allow for testing of DRILL-3640 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kkhatua/drill DRILL-5973

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1055.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1055


commit 62e6f721183d648797d5329e94b277cd5722bba6
Author: Kunal Khatua 
Date:   2017-11-29T19:00:11Z

DRILL-5973 : Support injection of time-bound pauses in server

Support pause injections in the test framework that are time-bound, to 
allow for testing high latency scenarios.
e.g. delayed server response to the Drill client allows for test a 
server-induced timeout




> Support injection of time-bound pauses in server
> 
>
> Key: DRILL-5973
> URL: https://issues.apache.org/jira/browse/DRILL-5973
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
> Fix For: 1.13.0
>
>
> While working on DRILL-3640 , when creating a unit test for a server-induced 
> timeout, the injecting a pause leaves the JUnit framework's DrillClient 
> without a handle to the query on the server. This is because we injected the 
> pause to occur before the server could send back a query ID, so the 
> DrillClient has no way to unpause the server.
> The workaround to support this unit test is to allow for injecting pauses 
> with a defined time-bound, after which the server would resume.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5973) Support injection of time-bound pauses in server

[Kunal Khatua (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Khatua updated DRILL-5973:

Summary: Support injection of time-bound pauses in server  (was: Support 
injections of a time-bound pause after which the server resumes)

> Support injection of time-bound pauses in server
> 
>
> Key: DRILL-5973
> URL: https://issues.apache.org/jira/browse/DRILL-5973
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
> Fix For: 1.13.0
>
>
> While working on DRILL-3640 , when creating a unit test for a server-induced 
> timeout, the injecting a pause leaves the JUnit framework's DrillClient 
> without a handle to the query on the server. This is because we injected the 
> pause to occur before the server could send back a query ID, so the 
> DrillClient has no way to unpause the server.
> The workaround to support this unit test is to allow for injecting pauses 
> with a defined time-bound, after which the server would resume.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-6000) Graceful Shutdown Unit Tests Should Not Be Run On Travis

[Timothy Farkas (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Timothy Farkas updated DRILL-6000:
--
Description: Currently these tests fail on Travis. But since they are heavy 
weight tests which test a corner case, they should not be run as part of the 
smoke tests.

> Graceful Shutdown Unit Tests Should Not Be Run On Travis
> 
>
> Key: DRILL-6000
> URL: https://issues.apache.org/jira/browse/DRILL-6000
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>
> Currently these tests fail on Travis. But since they are heavy weight tests 
> which test a corner case, they should not be run as part of the smoke tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-6000) Graceful Shutdown Unit Tests Should Not Be Run On Travis

[Timothy Farkas (JIRA)]

Timothy Farkas created DRILL-6000:
-

 Summary: Graceful Shutdown Unit Tests Should Not Be Run On Travis
 Key: DRILL-6000
 URL: https://issues.apache.org/jira/browse/DRILL-6000
 Project: Apache Drill
  Issue Type: Bug
Reporter: Timothy Farkas
Assignee: Timothy Farkas






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-5999) Add support for LATERAL join

[Aman Sinha (JIRA)]

Aman Sinha created DRILL-5999:
-

 Summary: Add support for LATERAL join
 Key: DRILL-5999
 URL: https://issues.apache.org/jira/browse/DRILL-5999
 Project: Apache Drill
  Issue Type: New Feature
  Components: Query Planning & Optimization
Affects Versions: 1.11.0
Reporter: Aman Sinha


The LATERAL keyword in SQL standard can precede a sub-SELECT FROM item. This 
allows the sub-SELECT to refer to columns of FROM items that appear before it 
in the FROM list. (Without LATERAL, each sub-SELECT is evaluated independently 
and so cannot cross-reference any other FROM item.)  

Calcite supports the LATERAL syntax.  In Drill, we should add support for it in 
the planning and execution phase.  

The main motivation of supporting it is it makes it more expressive and 
performant to handling complex types such as arrays and maps.  For instance, 
suppose you have a customer table which contains 1 row per customer containing 
customer-id, name and an array of Orders corresponding to each customer.   
Suppose you want to find out for each customer what is the average order 
amount.  This could be expressed as follows using SQL standard LATERAL and 
UNNEST syntax:
{noformat}
SELECT customer_name FROM customers c 
   LATERAL (SELECT AVG(order_amount) FROM UNNEST(c.orders));
{noformat}

The subquery may contain other operations such as filtering etc which operate 
on the output of the  un-nested c.orders array.  The UNNEST operation is 
supported in Drill today using FLATTEN operator.  More details of the use cases 
for LATERAL is available from existing product documentations .. e.g see [1].   

[1] https://www.postgresql.org/docs/9.4/static/queries-table-expressions.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5989) Run Smoke Tests On Travis

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270609#comment-16270609
 ] 

ASF GitHub Bot commented on DRILL-5989:
---

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/1053


> Run Smoke Tests On Travis
> -
>
> Key: DRILL-5989
> URL: https://issues.apache.org/jira/browse/DRILL-5989
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>  Labels: ready-to-commit
> Fix For: 1.12.0
>
>
> Currently the smoke tests don't run on travis. We should update the 
> travis.yml to cache maven dependencies and run the smoke tets.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-4286) Have an ability to put server in quiescent mode of operation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270607#comment-16270607
 ] 

ASF GitHub Bot commented on DRILL-4286:
---

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/921


> Have an ability to put server in quiescent mode of operation
> 
>
> Key: DRILL-4286
> URL: https://issues.apache.org/jira/browse/DRILL-4286
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Execution - Flow
>Affects Versions: 1.11.0
>Reporter: Victoria Markman
>Assignee: Venkata Jyothsna Donapati
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.12.0
>
> Attachments: consoleText.txt, consoleText_after.txt, 
> consoleText_before.txt
>
>
> I think drill will benefit from mode of operation that is called "quiescent" 
> in some databases. 
> From IBM Informix server documentation:
> {code}
> Change gracefully from online to quiescent mode
> Take the database server gracefully from online mode to quiescent mode to 
> restrict access to the database server without interrupting current 
> processing. After you perform this task, the database server sets a flag that 
> prevents new sessions from gaining access to the database server. The current 
> sessions are allowed to finish processing. After you initiate the mode 
> change, it cannot be canceled. During the mode change from online to 
> quiescent, the database server is considered to be in Shutdown mode.
> {code}
> This is different from shutdown, when processes are terminated. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5972) Slow performance for query on INFORMATION_SCHEMA.TABLE

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270608#comment-16270608
 ] 

ASF GitHub Bot commented on DRILL-5972:
---

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/1038


> Slow performance for query on INFORMATION_SCHEMA.TABLE
> --
>
> Key: DRILL-5972
> URL: https://issues.apache.org/jira/browse/DRILL-5972
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Information Schema
>Affects Versions: 1.11.0
>Reporter: Padma Penumarthy
>Assignee: Padma Penumarthy
>  Labels: ready-to-commit
> Fix For: 1.12.0
>
>
> A query like the following on INFORMATION_SCHEMA takes a long time to 
> execute. 
> select TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME, TABLE_TYPE from 
> INFORMATION_SCHEMA.`TABLES` WHERE TABLE_NAME LIKE '%' AND ( TABLE_SCHEMA = 
> 'hive.default' ) ORDER BY TABLE_TYPE, TABLE_CATALOG, TABLE_SCHEMA, 
> TABLE_NAME; 
> Reason being we fetch table information for all schemas instead of just 
> 'hive.default' schema.
> If we  change the predicate like this, it executes very fast.
> select TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME, TABLE_TYPE from 
> INFORMATION_SCHEMA.`TABLES` WHERE  ( TABLE_SCHEMA = 'hive.default' ) AND 
> TABLE_NAME LIKE '%'  ORDER BY TABLE_TYPE, TABLE_CATALOG, TABLE_SCHEMA, 
> TABLE_NAME; 
> The difference is in the order in which we evaluate the expressions in the 
> predicate.
> In the first case,  we first evaluate TABLE_NAME LIKE '%' and decide that it 
> is inconclusive (since we do not know the schema). So, we go get all tables 
> for all the schemas.
> In the second case, we first evaluate  TABLE_SCHEMA = 'hive.default' and 
> decide that we need to fetch only tables for that schema.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5964) Do not allow queries to access paths outside the current workspace root

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270606#comment-16270606
 ] 

ASF GitHub Bot commented on DRILL-5964:
---

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/1050


> Do not allow queries to access paths outside the current workspace root
> ---
>
> Key: DRILL-5964
> URL: https://issues.apache.org/jira/browse/DRILL-5964
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Parth Chandra
>Assignee: Parth Chandra
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.12.0
>
>
> Workspace definitions in the dfs plugin are intended to provide a convenient 
> shortcut to long directory paths. However, some users may wish to disallow 
> access to paths outside the root of the workspace, possibly to prevent 
> accidental access. Note that this is a convenience option and not a 
> substitute for permissions on the file system.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5971) Fix INT64, INT32 logical types in complex parquet reader

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270549#comment-16270549
 ] 

ASF GitHub Bot commented on DRILL-5971:
---

Github user vvysotskyi commented on a diff in the pull request:

https://github.com/apache/drill/pull/1049#discussion_r153739725
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReaderFactory.java
 ---
@@ -138,6 +138,8 @@
 return new 
ParquetFixedWidthDictionaryReaders.DictionaryBigIntReader(recordReader, 
allocateSize, descriptor, columnChunkMetaData, fixedLength, (BigIntVector) v, 
schemaElement);
--- End diff --

`DATE` logical type also encoded as the `INT32` physical type [1], so could 
you please also add its support?

[1] 
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#date


> Fix INT64, INT32 logical types in complex parquet reader
> 
>
> Key: DRILL-5971
> URL: https://issues.apache.org/jira/browse/DRILL-5971
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Parth Chandra
>Assignee: Parth Chandra
> Fix For: 1.13.0
>
>
> The 'complex' Parquet reader does not recognize the Parquet logical types 
> INT64, and INT32. 
> Should be a simple change to add these logical types.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5971) Fix INT64, INT32 logical types in complex parquet reader

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270548#comment-16270548
 ] 

ASF GitHub Bot commented on DRILL-5971:
---

Github user vvysotskyi commented on a diff in the pull request:

https://github.com/apache/drill/pull/1049#discussion_r153735994
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReaderFactory.java
 ---
@@ -138,6 +138,8 @@
 return new 
ParquetFixedWidthDictionaryReaders.DictionaryBigIntReader(recordReader, 
allocateSize, descriptor, columnChunkMetaData, fixedLength, (BigIntVector) v, 
schemaElement);
--- End diff --

This comment had to be placed in line 
[117](https://github.com/apache/drill/pull/1049/files?diff=unified#diff-4a7ec07122bfb16e4ff696af256f56dcR117),
 but I could not add it there. 
Should we also handle the case when `columnChunkMetaData.getType()` type is 
`INT32` and `convertedType` is `INT_32`? 


> Fix INT64, INT32 logical types in complex parquet reader
> 
>
> Key: DRILL-5971
> URL: https://issues.apache.org/jira/browse/DRILL-5971
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Parth Chandra
>Assignee: Parth Chandra
> Fix For: 1.13.0
>
>
> The 'complex' Parquet reader does not recognize the Parquet logical types 
> INT64, and INT32. 
> Should be a simple change to add these logical types.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5972) Slow performance for query on INFORMATION_SCHEMA.TABLE

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5972:

Fix Version/s: (was: 1.13.0)
   1.12.0

> Slow performance for query on INFORMATION_SCHEMA.TABLE
> --
>
> Key: DRILL-5972
> URL: https://issues.apache.org/jira/browse/DRILL-5972
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Information Schema
>Affects Versions: 1.11.0
>Reporter: Padma Penumarthy
>Assignee: Padma Penumarthy
>  Labels: ready-to-commit
> Fix For: 1.12.0
>
>
> A query like the following on INFORMATION_SCHEMA takes a long time to 
> execute. 
> select TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME, TABLE_TYPE from 
> INFORMATION_SCHEMA.`TABLES` WHERE TABLE_NAME LIKE '%' AND ( TABLE_SCHEMA = 
> 'hive.default' ) ORDER BY TABLE_TYPE, TABLE_CATALOG, TABLE_SCHEMA, 
> TABLE_NAME; 
> Reason being we fetch table information for all schemas instead of just 
> 'hive.default' schema.
> If we  change the predicate like this, it executes very fast.
> select TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME, TABLE_TYPE from 
> INFORMATION_SCHEMA.`TABLES` WHERE  ( TABLE_SCHEMA = 'hive.default' ) AND 
> TABLE_NAME LIKE '%'  ORDER BY TABLE_TYPE, TABLE_CATALOG, TABLE_SCHEMA, 
> TABLE_NAME; 
> The difference is in the order in which we evaluate the expressions in the 
> predicate.
> In the first case,  we first evaluate TABLE_NAME LIKE '%' and decide that it 
> is inconclusive (since we do not know the schema). So, we go get all tables 
> for all the schemas.
> In the second case, we first evaluate  TABLE_SCHEMA = 'hive.default' and 
> decide that we need to fetch only tables for that schema.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)