[jira] [Commented] (DRILL-4246) New allocator causing a flatten regression test to fail with IllegalStateException
[ https://issues.apache.org/jira/browse/DRILL-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097550#comment-15097550 ] ASF GitHub Bot commented on DRILL-4246: --- GitHub user jacques-n opened a pull request: https://github.com/apache/drill/pull/323 DRILL-4246: Fix Allocator concurrency bug and improve error detection - Rename the internal DrillBuf field to udle to better express its purpose. - Rename AllocatorManager to AllocationManager to better express its purpose. - Address situation where dangling ledger could be transferred into while it was being released released by protecting association and release inside the AllocationManager. - Add allocator assertions to ensure allocator operations are done while the allocator is open. - Simplify AllocationManager locking model. - Exclude HDFS reference to netty-all - Improve debugging messages for allocators (and fix debug message bugs) Tests pending but evaluation shows previously frequently failing flatten tests to consistently pass. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jacques-n/drill DRILL-4246 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/323.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #323 commit 4b193d71a9f3e06aadf59c66832bcbcb4b060c92 Author: Jacques Nadeau Date: 2016-01-11T04:08:03Z DRILL-4246: Fix Allocator concurrency bug and improve error detection - Rename the internal DrillBuf field to udle to better express its purpose. - Rename AllocatorManager to AllocationManager to better express its purpose. - Address situation where dangling ledger could be transferred into while it was being released released by protecting association and release inside the AllocationManager. - Add allocator assertions to ensure allocator operations are done while the allocator is open. - Simplify AllocationManager locking model. - Exclude HDFS reference to netty-all - Improve debugging messages for allocators (and fix debug message bugs) > New allocator causing a flatten regression test to fail with > IllegalStateException > -- > > Key: DRILL-4246 > URL: https://issues.apache.org/jira/browse/DRILL-4246 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Relational Operators >Affects Versions: 1.5.0 >Reporter: Deneche A. Hakim >Assignee: Jacques Nadeau >Priority: Blocker > > We are seeing the following error in the test cluster: > {noformat} > /framework/resources/Functional/flatten_operators/10rows/filter3.q > Query: > select uid, flatten(events) from `data.json` where uid > 1 > Failed with exception > java.sql.SQLException: SYSTEM ERROR: IllegalStateException: Unaccounted for > outstanding allocation (851968) > Allocator(op:0:0:0:Screen) 100/851968/1941504/100 > (res/actual/peak/limit) > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4271) Drill query does not return all results from HBase when the Hbase contains more than 6 columns
[ https://issues.apache.org/jira/browse/DRILL-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumiko Yada updated DRILL-4271: --- Description: Hbase `test1` contains 1 column family & 5 columns, 1000 rows Hbase `test2` contains 1 column family & 6 columns, 1000 rows Query: select count ( * ) from `hbase`.`test1` Result: 1000 rows Query: select count ( * ) from `hbase`.`test2` Result: 6724 rows Note: Correct row count returned with the hbase table with 1 column family 5 or less columns was: Hbase `test1` contains 1 column family & 5 columns, 1000 rows Hbase `test2` contains 1 column family & 6 columns, 1000 rows Query: select count ( * ) from `hbase`.`test1` Result: 1000 rows Query: select count ( * ) from `hbase`.`test1` Result: 6724 rows Note: Correct row count returned with the hbase table with 1 column family 5 or less columns > Drill query does not return all results from HBase when the Hbase contains > more than 6 columns > -- > > Key: DRILL-4271 > URL: https://issues.apache.org/jira/browse/DRILL-4271 > Project: Apache Drill > Issue Type: Bug > Environment: Drill 1.4, HBase 1.0.0, CDH 5.4 >Reporter: Kumiko Yada >Priority: Critical > > Hbase `test1` contains 1 column family & 5 columns, 1000 rows > Hbase `test2` contains 1 column family & 6 columns, 1000 rows > Query: select count ( * ) from `hbase`.`test1` > Result: 1000 rows > Query: select count ( * ) from `hbase`.`test2` > Result: 6724 rows > Note: > Correct row count returned with the hbase table with 1 column family 5 or > less columns -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4271) Drill query does not return all results from HBase when the Hbase contains more than 6 columns
[ https://issues.apache.org/jira/browse/DRILL-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumiko Yada updated DRILL-4271: --- Summary: Drill query does not return all results from HBase when the Hbase contains more than 6 columns (was: Drill query does not return all results from HBase when the Hbase contains more than 6 column) > Drill query does not return all results from HBase when the Hbase contains > more than 6 columns > -- > > Key: DRILL-4271 > URL: https://issues.apache.org/jira/browse/DRILL-4271 > Project: Apache Drill > Issue Type: Bug > Environment: Drill 1.4, HBase 1.0.0, CDH 5.4 >Reporter: Kumiko Yada >Priority: Critical > > Hbase `test1` contains 1 column family & 5 columns, 1000 rows > Hbase `test2` contains 1 column family & 6 columns, 1000 rows > Query: select count ( * ) from `hbase`.`test1` > Result: 1000 rows > Query: select count ( * ) from `hbase`.`test1` > Result: 6724 rows > Note: > Correct row count returned with the hbase table with 1 column family 5 or > less columns -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4271) Drill query does not return all results from HBase when the Hbase contains more than 6 column
[ https://issues.apache.org/jira/browse/DRILL-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumiko Yada updated DRILL-4271: --- Description: Hbase `test1` contains 1 column family & 5 columns, 1000 rows Hbase `test2` contains 1 column family & 6 columns, 1000 rows Query: select count ( * ) from `hbase`.`test1` Result: 1000 rows Query: select count ( * ) from `hbase`.`test1` Result: 6724 rows Note: Correct row count returned with the hbase table with 1 column family 5 or less columns was: Hbase `test1` contains 1 column family & 5 columns, 1000 rows Hbase `test2` contains 1 column family & 6 columns, 1000 rows Query: select count (*) from `hbase`.`test1` Result: 1000 rows Query: select count (*) from `hbase`.`test1` Result: 6724 rows Note: Correct row count returned with the hbase table with 1 column family 5 or less columns > Drill query does not return all results from HBase when the Hbase contains > more than 6 column > - > > Key: DRILL-4271 > URL: https://issues.apache.org/jira/browse/DRILL-4271 > Project: Apache Drill > Issue Type: Bug > Environment: Drill 1.4, HBase 1.0.0, CDH 5.4 >Reporter: Kumiko Yada >Priority: Critical > > Hbase `test1` contains 1 column family & 5 columns, 1000 rows > Hbase `test2` contains 1 column family & 6 columns, 1000 rows > Query: select count ( * ) from `hbase`.`test1` > Result: 1000 rows > Query: select count ( * ) from `hbase`.`test1` > Result: 6724 rows > Note: > Correct row count returned with the hbase table with 1 column family 5 or > less columns -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4271) Drill query does not return all results from HBase when the Hbase contains more than 6 column
Kumiko Yada created DRILL-4271: -- Summary: Drill query does not return all results from HBase when the Hbase contains more than 6 column Key: DRILL-4271 URL: https://issues.apache.org/jira/browse/DRILL-4271 Project: Apache Drill Issue Type: Bug Environment: Drill 1.4, HBase 1.0.0, CDH 5.4 Reporter: Kumiko Yada Priority: Critical Hbase `test1` contains 1 column family & 5 columns, 1000 rows Hbase `test2` contains 1 column family & 6 columns, 1000 rows Query: select count(*) from `hbase`.`test1` Result: 1000 rows Query: select count(*) from `hbase`.`test1` Result: 6724 rows Note: Correct row count returned with the hbase table with 1 column family 5 or less columns -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4271) Drill query does not return all results from HBase when the Hbase contains more than 6 column
[ https://issues.apache.org/jira/browse/DRILL-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumiko Yada updated DRILL-4271: --- Description: Hbase `test1` contains 1 column family & 5 columns, 1000 rows Hbase `test2` contains 1 column family & 6 columns, 1000 rows Query: select count (*) from `hbase`.`test1` Result: 1000 rows Query: select count (*) from `hbase`.`test1` Result: 6724 rows Note: Correct row count returned with the hbase table with 1 column family 5 or less columns was: Hbase `test1` contains 1 column family & 5 columns, 1000 rows Hbase `test2` contains 1 column family & 6 columns, 1000 rows Query: select count(*) from `hbase`.`test1` Result: 1000 rows Query: select count(*) from `hbase`.`test1` Result: 6724 rows Note: Correct row count returned with the hbase table with 1 column family 5 or less columns > Drill query does not return all results from HBase when the Hbase contains > more than 6 column > - > > Key: DRILL-4271 > URL: https://issues.apache.org/jira/browse/DRILL-4271 > Project: Apache Drill > Issue Type: Bug > Environment: Drill 1.4, HBase 1.0.0, CDH 5.4 >Reporter: Kumiko Yada >Priority: Critical > > Hbase `test1` contains 1 column family & 5 columns, 1000 rows > Hbase `test2` contains 1 column family & 6 columns, 1000 rows > Query: select count (*) from `hbase`.`test1` > Result: 1000 rows > Query: select count (*) from `hbase`.`test1` > Result: 6724 rows > Note: > Correct row count returned with the hbase table with 1 column family 5 or > less columns -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4270) Create a separate WindowFramer that supports the FRAME clause
[ https://issues.apache.org/jira/browse/DRILL-4270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097266#comment-15097266 ] ASF GitHub Bot commented on DRILL-4270: --- Github user adeneche commented on the pull request: https://github.com/apache/drill/pull/322#issuecomment-171470081 The purpose of this PR is to separate the functions that support the FRAME clause in CustomFrameTemplate from the ones that don't in DefaultFrameTemplate. Work still needs to be done to make the refactoring complete (like making sure code is not duplicated between the templates), but I decided to leave it after I make the necessary changes to support the FRAME clause. @amansinha100 can you please review ? > Create a separate WindowFramer that supports the FRAME clause > - > > Key: DRILL-4270 > URL: https://issues.apache.org/jira/browse/DRILL-4270 > Project: Apache Drill > Issue Type: Sub-task > Components: Execution - Relational Operators >Affects Versions: 1.5.0 >Reporter: Deneche A. Hakim >Assignee: Aman Sinha > Fix For: Future > > > Currently most the window functions logic is handled by DefaultFrameTemplate. > Create a separate CustomFrameTemplate that handles the FRAME clause, this > should make the code in both classes more focused and will make it easier for > us to add support for the FRAME clause. > Aggregations, FIRST_VALUE and LAST_VALUE will be handled by > CustomFrameTemplate, and all remaining window functions (Ranking, ROW_NUMBER, > LEAD and LAG) will be handled by DefaultFrameTemplate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4270) Create a separate WindowFramer that supports the FRAME clause
[ https://issues.apache.org/jira/browse/DRILL-4270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deneche A. Hakim updated DRILL-4270: Assignee: Aman Sinha (was: Deneche A. Hakim) > Create a separate WindowFramer that supports the FRAME clause > - > > Key: DRILL-4270 > URL: https://issues.apache.org/jira/browse/DRILL-4270 > Project: Apache Drill > Issue Type: Sub-task > Components: Execution - Relational Operators >Affects Versions: 1.5.0 >Reporter: Deneche A. Hakim >Assignee: Aman Sinha > Fix For: Future > > > Currently most the window functions logic is handled by DefaultFrameTemplate. > Create a separate CustomFrameTemplate that handles the FRAME clause, this > should make the code in both classes more focused and will make it easier for > us to add support for the FRAME clause. > Aggregations, FIRST_VALUE and LAST_VALUE will be handled by > CustomFrameTemplate, and all remaining window functions (Ranking, ROW_NUMBER, > LEAD and LAG) will be handled by DefaultFrameTemplate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4270) Create a separate WindowFramer that supports the FRAME clause
[ https://issues.apache.org/jira/browse/DRILL-4270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097015#comment-15097015 ] ASF GitHub Bot commented on DRILL-4270: --- GitHub user adeneche opened a pull request: https://github.com/apache/drill/pull/322 DRILL-4270: Create a separate WindowFramer that supports the FRAME cl… …ause separate DefaultFrameTemplate into 2 implementations: one that supports custom frames (aggregations, first_value, last_value) and one that doesn't You can merge this pull request into a Git repository by running: $ git pull https://github.com/adeneche/incubator-drill DRILL-4270 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/322.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #322 commit 8bcde155d23550d6733f4dcb18868bdc3fb3b119 Author: adeneche Date: 2016-01-11T22:47:32Z DRILL-4270: Create a separate WindowFramer that supports the FRAME clause separate DefaultFrameTemplate into 2 implementations: one that supports custom frames (aggregations, first_value, last_value) and one that doesn't > Create a separate WindowFramer that supports the FRAME clause > - > > Key: DRILL-4270 > URL: https://issues.apache.org/jira/browse/DRILL-4270 > Project: Apache Drill > Issue Type: Sub-task > Components: Execution - Relational Operators >Affects Versions: 1.5.0 >Reporter: Deneche A. Hakim >Assignee: Deneche A. Hakim > Fix For: Future > > > Currently most the window functions logic is handled by DefaultFrameTemplate. > Create a separate CustomFrameTemplate that handles the FRAME clause, this > should make the code in both classes more focused and will make it easier for > us to add support for the FRAME clause. > Aggregations, FIRST_VALUE and LAST_VALUE will be handled by > CustomFrameTemplate, and all remaining window functions (Ranking, ROW_NUMBER, > LEAD and LAG) will be handled by DefaultFrameTemplate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4047) Select with options
[ https://issues.apache.org/jira/browse/DRILL-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096789#comment-15096789 ] Julien Le Dem commented on DRILL-4047: -- You can refer to unit tests for more examples: https://github.com/apache/drill/blob/1.4.0/exec/java-exec/src/test/java/org/apache/drill/TestSelectWithOption.java > Select with options > --- > > Key: DRILL-4047 > URL: https://issues.apache.org/jira/browse/DRILL-4047 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Relational Operators >Reporter: Julien Le Dem >Assignee: Julien Le Dem > Fix For: 1.4.0 > > > Add a mechanism to pass parameters down to the StoragePlugin when writing a > Select statement. > Some discussion here: > http://mail-archives.apache.org/mod_mbox/drill-dev/201510.mbox/%3CCAO%2Bvc4AcGK3%2B3QYvQV1-xPPdpG3Tc%2BfG%3D0xDGEUPrhd6ktHv5Q%40mail.gmail.com%3E > http://mail-archives.apache.org/mod_mbox/drill-dev/201511.mbox/%3ccao+vc4clzylvjevisfjqtcyxb-zsmfy4bqrm-jhbidwzgqf...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4047) Select with options
[ https://issues.apache.org/jira/browse/DRILL-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096786#comment-15096786 ] Julien Le Dem commented on DRILL-4047: -- Here is how it works: Context: You can query files in a hadoop file system in the following way: {noformat} select columns from dfs.`mytable/file.csv`; {noformat} The format plugin is selected based on file extension or header according to the configuration. This feature enables providing the format plugin and its config in the SQL query instead as follows: {noformat} select columns from table(dfs.`mytable/file.csv`(type => 'text', fieldDelimiter => ',')) {noformat} The table(...) syntax tells the SQL parser that we're passing parameters to the table. The type parameter is required and selects the Format plugin as defined by its Config class (as would be done when deserializing from json) The other parameters are defined by the fields of the same class. (excluding "extensions" that does not apply in that case) For example in the case of Text: https://github.com/apache/drill/blob/1.4.0/exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/TextFormatPlugin.java#L135 {noformat} @JsonTypeName("text") @JsonInclude(Include.NON_DEFAULT) public static class TextFormatConfig implements FormatPluginConfig { public List extensions = ImmutableList.of(); public String lineDelimiter = "\n"; public char fieldDelimiter = '\n'; public char quote = '"'; public char escape = '"'; public char comment = '#'; public boolean skipFirstLine = false; public boolean extractHeader = false; {noformat} > Select with options > --- > > Key: DRILL-4047 > URL: https://issues.apache.org/jira/browse/DRILL-4047 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Relational Operators >Reporter: Julien Le Dem >Assignee: Julien Le Dem > Fix For: 1.4.0 > > > Add a mechanism to pass parameters down to the StoragePlugin when writing a > Select statement. > Some discussion here: > http://mail-archives.apache.org/mod_mbox/drill-dev/201510.mbox/%3CCAO%2Bvc4AcGK3%2B3QYvQV1-xPPdpG3Tc%2BfG%3D0xDGEUPrhd6ktHv5Q%40mail.gmail.com%3E > http://mail-archives.apache.org/mod_mbox/drill-dev/201511.mbox/%3ccao+vc4clzylvjevisfjqtcyxb-zsmfy4bqrm-jhbidwzgqf...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4270) Create a separate WindowFramer that supports the FRAME clause
Deneche A. Hakim created DRILL-4270: --- Summary: Create a separate WindowFramer that supports the FRAME clause Key: DRILL-4270 URL: https://issues.apache.org/jira/browse/DRILL-4270 Project: Apache Drill Issue Type: Sub-task Components: Execution - Relational Operators Affects Versions: 1.5.0 Reporter: Deneche A. Hakim Assignee: Deneche A. Hakim Currently most the window functions logic is handled by DefaultFrameTemplate. Create a separate CustomFrameTemplate that handles the FRAME clause, this should make the code in both classes more focused and will make it easier for us to add support for the FRAME clause. Aggregations, FIRST_VALUE and LAST_VALUE will be handled by CustomFrameTemplate, and all remaining window functions (Ranking, ROW_NUMBER, LEAD and LAG) will be handled by DefaultFrameTemplate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4269) Using -(-1999) in a filter gives NPE
Rahul Challapalli created DRILL-4269: Summary: Using -(-1999) in a filter gives NPE Key: DRILL-4269 URL: https://issues.apache.org/jira/browse/DRILL-4269 Project: Apache Drill Issue Type: Bug Components: Execution - Flow, Query Planning & Optimization Affects Versions: 1.4.0 Reporter: Rahul Challapalli The below query give rise to an NPE {code} select l_orderkey, l_partkey, l_quantity, l_shipdate, l_shipinstruct, dir0, dir1 from cp.`tpch/lineitem.parquet` where l_orderkey = -(-1993); Error: SYSTEM ERROR: NullPointerException {code} Error from the logs {code} 2016-01-13 17:59:27,074 [29696f81-35fc-a422-f9e2-a08d5c001192:foreman] ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: NullPointerException [Error Id: c6001655-6e29-4b4c-9c1a-0c9b0c1152c3 on qa-node191.qa.lab:31010] org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: NullPointerException [Error Id: c6001655-6e29-4b4c-9c1a-0c9b0c1152c3 on qa-node191.qa.lab:31010] at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534) ~[drill-common-1.4.0.jar:1.4.0] at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:742) [drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:841) [drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:786) [drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.common.EventProcessor.sendEvent(EventProcessor.java:73) [drill-common-1.4.0.jar:1.4.0] at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.moveToState(Foreman.java:788) [drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:894) [drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:255) [drill-java-exec-1.4.0.jar:1.4.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71] Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Internal error: Error while applying rule DrillReduceExpressionsRule(Filter), args [rel#702:LogicalFilter.NONE.ANY([]).[](input=rel#701:Subset#3.NONE.ANY([]).[],condition==($0, -(-1993)))] ... 4 common frames omitted Caused by: java.lang.AssertionError: Internal error: Error while applying rule DrillReduceExpressionsRule(Filter), args [rel#702:LogicalFilter.NONE.ANY([]).[](input=rel#701:Subset#3.NONE.ANY([]).[],condition==($0, -(-1993)))] at org.apache.calcite.util.Util.newInternal(Util.java:792) ~[calcite-core-1.4.0-drill-1.4.0-mapr-r1.jar:1.4.0-drill-1.4.0-mapr-r1] at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:251) ~[calcite-core-1.4.0-drill-1.4.0-mapr-r1.jar:1.4.0-drill-1.4.0-mapr-r1] at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:808) ~[calcite-core-1.4.0-drill-1.4.0-mapr-r1.jar:1.4.0-drill-1.4.0-mapr-r1] at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303) ~[calcite-core-1.4.0-drill-1.4.0-mapr-r1.jar:1.4.0-drill-1.4.0-mapr-r1] at org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:313) ~[calcite-core-1.4.0-drill-1.4.0-mapr-r1.jar:1.4.0-drill-1.4.0-mapr-r1] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.doLogicalPlanning(DefaultSqlHandler.java:562) ~[drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:230) ~[drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:263) ~[drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:172) ~[drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:218) ~[drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:909) [drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:244) [drill-java-exec-1.4.0.jar:1.4.0] ... 3 common frames omitted Caused by: java.lang.NullPointerException: null at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:131) ~[drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill
[jira] [Commented] (DRILL-4239) Update documentation to reflect 64bit requirement to run Drill on Windows.
[ https://issues.apache.org/jira/browse/DRILL-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096483#comment-15096483 ] Andries Engelbrecht commented on DRILL-4239: Kristine Hahn the issue here relates only to actually running Drill on a 32bit Windows machine, which is a poor platform choice with likely minimal user adoption (compared to 64bit Windows). However the ODBC drivers is a different topic as it for client systems, the 32bit and 64bit ODBC drivers works on a 64bit Windows machine as certain client software may need a 32bit ODBC driver. > Update documentation to reflect 64bit requirement to run Drill on Windows. > --- > > Key: DRILL-4239 > URL: https://issues.apache.org/jira/browse/DRILL-4239 > Project: Apache Drill > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.3.0, 1.4.0 >Reporter: Peder Jakobsen >Assignee: Kristine Hahn > Labels: newbie > Fix For: 1.5.0 > > > Winutils.exe has been compiled to run on the 64 bit version of windows. For > this reason, some part of the documentation that suggest that Drill can run > on 32bit Windows must be fixed. Furthermore, although few user run 64 bit > windows these days, it would be helpful to make this requirement more > explicit. In particular, rapid installation of Windows on VirtualBox will > often result in 32 bit version being installed by default, since it's the > preselected default during installation process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (DRILL-4239) Update documentation to reflect 64bit requirement to run Drill on Windows.
[ https://issues.apache.org/jira/browse/DRILL-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn closed DRILL-4239. > Update documentation to reflect 64bit requirement to run Drill on Windows. > --- > > Key: DRILL-4239 > URL: https://issues.apache.org/jira/browse/DRILL-4239 > Project: Apache Drill > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.3.0, 1.4.0 >Reporter: Peder Jakobsen >Assignee: Kristine Hahn > Labels: newbie > Fix For: 1.5.0 > > > Winutils.exe has been compiled to run on the 64 bit version of windows. For > this reason, some part of the documentation that suggest that Drill can run > on 32bit Windows must be fixed. Furthermore, although few user run 64 bit > windows these days, it would be helpful to make this requirement more > explicit. In particular, rapid installation of Windows on VirtualBox will > often result in 32 bit version being installed by default, since it's the > preselected default during installation process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-4239) Update documentation to reflect 64bit requirement to run Drill on Windows.
[ https://issues.apache.org/jira/browse/DRILL-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn resolved DRILL-4239. -- Resolution: Fixed Revised [install doc|https://drill.apache.org/docs/installing-drill-on-windows/] to say: {quote} Currently, Drill supports 64-bit Windows only. {quote} Revised ODBC doc accordingly. The 64-bit requirement for the machine is stated. A note says the 32- or 64-bit driver is supported on a 64-bit machine. https://drill.apache.org/docs/installing-the-driver-on-windows/ > Update documentation to reflect 64bit requirement to run Drill on Windows. > --- > > Key: DRILL-4239 > URL: https://issues.apache.org/jira/browse/DRILL-4239 > Project: Apache Drill > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.3.0, 1.4.0 >Reporter: Peder Jakobsen >Assignee: Kristine Hahn > Labels: newbie > Fix For: 1.5.0 > > > Winutils.exe has been compiled to run on the 64 bit version of windows. For > this reason, some part of the documentation that suggest that Drill can run > on 32bit Windows must be fixed. Furthermore, although few user run 64 bit > windows these days, it would be helpful to make this requirement more > explicit. In particular, rapid installation of Windows on VirtualBox will > often result in 32 bit version being installed by default, since it's the > preselected default during installation process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4239) Update documentation to reflect 64bit requirement to run Drill on Windows.
[ https://issues.apache.org/jira/browse/DRILL-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn updated DRILL-4239: - Fix Version/s: 1.5.0 > Update documentation to reflect 64bit requirement to run Drill on Windows. > --- > > Key: DRILL-4239 > URL: https://issues.apache.org/jira/browse/DRILL-4239 > Project: Apache Drill > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.3.0, 1.4.0 >Reporter: Peder Jakobsen >Assignee: Kristine Hahn > Labels: newbie > Fix For: 1.5.0 > > > Winutils.exe has been compiled to run on the 64 bit version of windows. For > this reason, some part of the documentation that suggest that Drill can run > on 32bit Windows must be fixed. Furthermore, although few user run 64 bit > windows these days, it would be helpful to make this requirement more > explicit. In particular, rapid installation of Windows on VirtualBox will > often result in 32 bit version being installed by default, since it's the > preselected default during installation process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2912) Exception is not propagated correctly in case when directory contains mix of file types
[ https://issues.apache.org/jira/browse/DRILL-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095934#comment-15095934 ] Khurram Faraaz commented on DRILL-2912: --- There is a related assertion that is seen when assertions are enabled on MapR Drill 1.4.0 git.commit.id=556d9c5d Can someone take a look into this ? {noformat} Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Internal error: Error while applying rule DrillPushProjIntoScan, args [rel#803367:LogicalProject.NONE.ANY([]).[](input=rel#803366:Subset#0.ENUMERABLE.ANY([]).[],$f0=0), rel#803358:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[dfs, /drill/testdata/metadata_caching/lineitem_deletecache/])] ... 4 common frames omitted Caused by: java.lang.AssertionError: Internal error: Error while applying rule DrillPushProjIntoScan, args [rel#803367:LogicalProject.NONE.ANY([]).[](input=rel#803366:Subset#0.ENUMERABLE.ANY([]).[],$f0=0), rel#803358:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[dfs, /drill/testdata/metadata_caching/lineitem_deletecache/])] at org.apache.calcite.util.Util.newInternal(Util.java:792) ~[calcite-core-1.4.0-drill-1.4.0-mapr-r1.jar:1.4.0-drill-1.4.0-mapr-r1] at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:251) ~[calcite-core-1.4.0-drill-1.4.0-mapr-r1.jar:1.4.0-drill-1.4.0-mapr-r1] at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:808) ~[calcite-core-1.4.0-drill-1.4.0-mapr-r1.jar:1.4.0-drill-1.4.0-mapr-r1] at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303) ~[calcite-core-1.4.0-drill-1.4.0-mapr-r1.jar:1.4.0-drill-1.4.0-mapr-r1] at org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:313) ~[calcite-core-1.4.0-drill-1.4.0-mapr-r1.jar:1.4.0-drill-1.4.0-mapr-r1] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.doLogicalPlanning(DefaultSqlHandler.java:562) ~[drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:230) ~[drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:263) ~[drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:172) ~[drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:218) ~[drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:909) [drill-java-exec-1.4.0.jar:1.4.0] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:244) [drill-java-exec-1.4.0.jar:1.4.0] ... 3 common frames omitted {noformat} > Exception is not propagated correctly in case when directory contains mix of > file types > --- > > Key: DRILL-2912 > URL: https://issues.apache.org/jira/browse/DRILL-2912 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow, Storage - JSON, Storage - Parquet >Reporter: Victoria Markman > Fix For: Future > > > While trying to read from directory that has a mix of parquet and json files > I ran into an exception: > {code} > 0: jdbc:drill:schema=dfs> select max(dir0) from bigtable; > Query failed: SYSTEM ERROR: Unexpected exception during fragment > initialization: Internal error: Error while applying rule > DrillPushProjIntoScan, args > [rel#4207:LogicalProject.NONE.ANY([]).[](input=rel#4206:Subset#0.ENUMERABLE.ANY([]).[],dir0=$1), > rel#4198:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[dfs, test, > bigtable])] > [72d7f7ee-3045-44d9-b13c-1d03bea4e22c on atsqa4-133.qa.lab:31010] > Error: exception while executing query: Failure while executing query. > (state=,code=0) > {code} > The real problem is that directory contains 2 parquet and one json files: > {code} > [Wed Apr 29 14:50:58 > root@/mapr/vmarkman.cluster.com/test/bigtable/F114/2014-03-27 ] # pwd > /mapr/vmarkman.cluster.com/test/bigtable/F114/2014-03-27 > [Wed Apr 29 14:51:06 > root@/mapr/vmarkman.cluster.com/test/bigtable/F114/2014-03-27 ] # ls -ltr > total 2 > -rwxr-xr-x 1 root root 483 Apr 16 16:05 0_0_0.parquet > -rwxr-xr-x 1 root root 483 Apr 17 13:06 > 214c279334946e65-7e32c56eed93cbc2_1965630551_data.0.parq > -rw-r--r-- 1 root root 17 Apr 23 15:24 t1.json > {code} > drillbit.log > {code} > [72d7f7ee-3045-44d9-b13c-1d03bea4e22c on atsqa4-133.qa.lab:31010] > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: Unexpected > exception during fragment initialization: Internal error: Error while