[jira] [Created] (DRILL-4239) Update documentation to reflect 64bit requirement to run Drill on Windows.
Peder Jakobsen created DRILL-4239: - Summary: Update documentation to reflect 64bit requirement to run Drill on Windows. Key: DRILL-4239 URL: https://issues.apache.org/jira/browse/DRILL-4239 Project: Apache Drill Issue Type: Improvement Components: Documentation Affects Versions: 1.3.0, 1.4.0 Reporter: Peder Jakobsen Winutils.exe has been compiled to run on the 64 bit version of windows. For this reason, some part of the documentation that suggest that Drill can run on 32bit Windows must be fixed. Furthermore, although few user run 64 bit windows these days, it would be helpful to make this requirement more explicit. In particular, rapid installation of Windows on VirtualBox will often result in 32 bit version being installed by default, since it's the preselected default during installation process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4240) Provide 32bit version of Winutils as default
Peder Jakobsen created DRILL-4240: - Summary: Provide 32bit version of Winutils as default Key: DRILL-4240 URL: https://issues.apache.org/jira/browse/DRILL-4240 Project: Apache Drill Issue Type: Wish Affects Versions: 1.3.0, 1.4.0 Reporter: Peder Jakobsen Priority: Minor This *might* make it possible to run Drill on all versions of Windows 7 etc. Would also make updating of documentation moot. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4241) Add Experimental Kudu plugin
Jacques Nadeau created DRILL-4241: - Summary: Add Experimental Kudu plugin Key: DRILL-4241 URL: https://issues.apache.org/jira/browse/DRILL-4241 Project: Apache Drill Issue Type: New Feature Components: Storage - Other Reporter: Jacques Nadeau Assignee: Jacques Nadeau Fix For: 1.5.0 Merge the work done here into Drill master so others can utilize the plugin: https://github.com/dremio/drill-storage-kudu -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4241) Add Experimental Kudu plugin
[ https://issues.apache.org/jira/browse/DRILL-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080563#comment-15080563 ] ASF GitHub Bot commented on DRILL-4241: --- GitHub user jacques-n opened a pull request: https://github.com/apache/drill/pull/314 DRILL-4241: Add Kudu reader Implements an experimental Kudu reader & writer. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jacques-n/drill DRILL-4241 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/314.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #314 commit bc9da49f93b90507c94a6eab3f83187b44d1d213 Author: Jacques Nadeau Date: 2015-11-19T02:45:56Z DRILL-4241: initial commit commit dc9c0d097ab7cfd1b722a9065509a5167155a3b1 Author: Todd Lipcon Date: 2015-11-19T22:15:58Z DRILL-4241: Add pushdown of column projections commit 6fe5b352de522670b77985334d7b1af0f15c7467 Author: Jacques Nadeau Date: 2015-11-19T22:21:14Z DRILL-4241: Add table metadata and DROP table support commit 756b5dfb5e2de3b5c45b5943451d7e73ab0b65e6 Author: Todd Lipcon Date: 2015-11-19T23:00:00Z DRILL-4241: Improve record reader and type mappings commit 4765e2d499510a844e24b33782b0402d6ea640d7 Author: Steven Phillips Date: 2015-11-20T00:00:40Z DRILL-4241: Add parallelization and assignment commit 4e8517898ed966f08ac8c0b1b20f7a9b08d2aae3 Author: Todd Lipcon Date: 2015-11-20T02:56:52Z DRILL-4241: Various Fixes - ReaderFix scanning of tables with >4096 rows - Fixing the build, make RAT and checkstyle happy. - Fix last batch of query not to NPE commit a7ff45079379dda2fb49bff71e1b12e4e37c7dda Author: Todd Lipcon Date: 2015-11-20T00:15:36Z DRILL-4241: Rewrite RecordReader to support NULLs and be less Java-like commit 5f31e7a0a6e57db4d71214b52413b2e5937f545a Author: Amit Hadke Date: 2015-11-20T00:16:17Z DRILL-4241: Create table with multiple tablets in kudu test. commit 48a988a4ef7cd85dfd8dbf8f348b8fc9c9185ce7 Author: Todd Lipcon Date: 2015-11-20T00:19:45Z DRILL-4241: Timestamps should divide by 1000 in reader commit 77921147bcc2d9ebd6dac3241109c90337440868 Author: Steven Phillips Date: 2015-11-20T00:54:41Z DRILL-4241: Add wait stats and estimated row count in RecordReader commit 64c8e99a05f69af6d6118531d2d9081e843d1b0f Author: Jacques Nadeau Date: 2015-11-20T02:20:03Z DRILL-4241: Add Single Tablet Writer - Also move to a test bootstrap - Update to the latest kudu and Drill - Add plugin to Drill distribution - Checkstyle and directory cleanup > Add Experimental Kudu plugin > > > Key: DRILL-4241 > URL: https://issues.apache.org/jira/browse/DRILL-4241 > Project: Apache Drill > Issue Type: New Feature > Components: Storage - Other >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau > Fix For: 1.5.0 > > > Merge the work done here into Drill master so others can utilize the plugin: > https://github.com/dremio/drill-storage-kudu -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4241) Add Experimental Kudu plugin
[ https://issues.apache.org/jira/browse/DRILL-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated DRILL-4241: -- Assignee: Jason Altekruse (was: Jacques Nadeau) > Add Experimental Kudu plugin > > > Key: DRILL-4241 > URL: https://issues.apache.org/jira/browse/DRILL-4241 > Project: Apache Drill > Issue Type: New Feature > Components: Storage - Other >Reporter: Jacques Nadeau >Assignee: Jason Altekruse > Fix For: 1.5.0 > > > Merge the work done here into Drill master so others can utilize the plugin: > https://github.com/dremio/drill-storage-kudu -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4241) Add Experimental Kudu plugin
[ https://issues.apache.org/jira/browse/DRILL-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080565#comment-15080565 ] Jacques Nadeau commented on DRILL-4241: --- [~jaltekruse], can you please review? > Add Experimental Kudu plugin > > > Key: DRILL-4241 > URL: https://issues.apache.org/jira/browse/DRILL-4241 > Project: Apache Drill > Issue Type: New Feature > Components: Storage - Other >Reporter: Jacques Nadeau >Assignee: Jason Altekruse > Fix For: 1.5.0 > > > Merge the work done here into Drill master so others can utilize the plugin: > https://github.com/dremio/drill-storage-kudu -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4242) Update Mongo storage layer
[ https://issues.apache.org/jira/browse/DRILL-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated DRILL-4242: -- Fix Version/s: 1.5.0 > Update Mongo storage layer > -- > > Key: DRILL-4242 > URL: https://issues.apache.org/jira/browse/DRILL-4242 > Project: Apache Drill > Issue Type: New Feature >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau > Fix For: 1.5.0 > > > Depends on the latest Mongo Java driver (3.2.0) > Uses MongoDatabase rather than the deprecated DB class > Uses Document and Bson types rather than the legacy DBObject type > From https://github.com/apache/drill/pull/304 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4242) Update Mongo storage layer
Jacques Nadeau created DRILL-4242: - Summary: Update Mongo storage layer Key: DRILL-4242 URL: https://issues.apache.org/jira/browse/DRILL-4242 Project: Apache Drill Issue Type: New Feature Reporter: Jacques Nadeau Assignee: Jacques Nadeau Depends on the latest Mongo Java driver (3.2.0) Uses MongoDatabase rather than the deprecated DB class Uses Document and Bson types rather than the legacy DBObject type >From https://github.com/apache/drill/pull/304 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-3478) Bson Record Reader for Mongo storage plugin
[ https://issues.apache.org/jira/browse/DRILL-3478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved DRILL-3478. --- Resolution: Fixed Resolved in 5c7a992d2a271ab38c2cdb79f145b5f9d59592b7 > Bson Record Reader for Mongo storage plugin > --- > > Key: DRILL-3478 > URL: https://issues.apache.org/jira/browse/DRILL-3478 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - MongoDB >Reporter: B Anil Kumar >Assignee: B Anil Kumar > Fix For: 1.5.0 > > Attachments: 0001-DRILL-3478_1-Review-comments-fixes.patch, > Test_queries_with_review_comment_fixes, drill_bson_sqlline_test_2015_1 > > > Improve the mongo query performance. > We are considering the suggestions provided by [~dragoncurve] and [~hgunes] > in drill mailing chain. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3478) Bson Record Reader for Mongo storage plugin
[ https://issues.apache.org/jira/browse/DRILL-3478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated DRILL-3478: -- Fix Version/s: (was: Future) 1.5.0 > Bson Record Reader for Mongo storage plugin > --- > > Key: DRILL-3478 > URL: https://issues.apache.org/jira/browse/DRILL-3478 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - MongoDB >Reporter: B Anil Kumar >Assignee: B Anil Kumar > Fix For: 1.5.0 > > Attachments: 0001-DRILL-3478_1-Review-comments-fixes.patch, > Test_queries_with_review_comment_fixes, drill_bson_sqlline_test_2015_1 > > > Improve the mongo query performance. > We are considering the suggestions provided by [~dragoncurve] and [~hgunes] > in drill mailing chain. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4242) Update Mongo storage layer
[ https://issues.apache.org/jira/browse/DRILL-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated DRILL-4242: -- Component/s: Storage - MongoDB > Update Mongo storage layer > -- > > Key: DRILL-4242 > URL: https://issues.apache.org/jira/browse/DRILL-4242 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - MongoDB >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau > Fix For: 1.5.0 > > > Depends on the latest Mongo Java driver (3.2.0) > Uses MongoDatabase rather than the deprecated DB class > Uses Document and Bson types rather than the legacy DBObject type > From https://github.com/apache/drill/pull/304 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-4242) Update Mongo storage layer
[ https://issues.apache.org/jira/browse/DRILL-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved DRILL-4242. --- Resolution: Fixed Fixed in e4372f224a4b474494388356355a53808092a67a > Update Mongo storage layer > -- > > Key: DRILL-4242 > URL: https://issues.apache.org/jira/browse/DRILL-4242 > Project: Apache Drill > Issue Type: New Feature > Components: Storage - MongoDB >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau > Fix For: 1.5.0 > > > Depends on the latest Mongo Java driver (3.2.0) > Uses MongoDatabase rather than the deprecated DB class > Uses Document and Bson types rather than the legacy DBObject type > From https://github.com/apache/drill/pull/304 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4242) Update Mongo storage layer
[ https://issues.apache.org/jira/browse/DRILL-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated DRILL-4242: -- Issue Type: Improvement (was: New Feature) > Update Mongo storage layer > -- > > Key: DRILL-4242 > URL: https://issues.apache.org/jira/browse/DRILL-4242 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - MongoDB >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau > Fix For: 1.5.0 > > > Depends on the latest Mongo Java driver (3.2.0) > Uses MongoDatabase rather than the deprecated DB class > Uses Document and Bson types rather than the legacy DBObject type > From https://github.com/apache/drill/pull/304 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4241) Add Experimental Kudu plugin
[ https://issues.apache.org/jira/browse/DRILL-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080726#comment-15080726 ] ASF GitHub Bot commented on DRILL-4241: --- Github user StevenMPhillips commented on the pull request: https://github.com/apache/drill/pull/314#issuecomment-168588400 +1 As long as this doesn't break the build, I say we go ahead and merge it. We can add tests and fixes later. > Add Experimental Kudu plugin > > > Key: DRILL-4241 > URL: https://issues.apache.org/jira/browse/DRILL-4241 > Project: Apache Drill > Issue Type: New Feature > Components: Storage - Other >Reporter: Jacques Nadeau >Assignee: Jason Altekruse > Fix For: 1.5.0 > > > Merge the work done here into Drill master so others can utilize the plugin: > https://github.com/dremio/drill-storage-kudu -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4190) TPCDS queries are running out of memory when hash join is disabled
[
https://issues.apache.org/jira/browse/DRILL-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080735#comment-15080735
]
ASF GitHub Bot commented on DRILL-4190:
---
GitHub user amithadke opened a pull request:
https://github.com/apache/drill/pull/315
DRILL-4190 Don't hold on to batches from left side of merge join.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/amithadke/drill DRILL-4190
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/315.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #315
commit a8ebc410a27c270c543c6d5a2c3b699ddcc4e38e
Author: Amit Hadke
Date: 2016-01-04T06:08:21Z
DRILL-4190 Don't hold on to batches from left side of merge join.
> TPCDS queries are running out of memory when hash join is disabled
> --
>
> Key: DRILL-4190
> URL: https://issues.apache.org/jira/browse/DRILL-4190
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.3.0, 1.4.0, 1.5.0
>Reporter: Victoria Markman
>Assignee: amit hadke
>Priority: Blocker
> Attachments: 2990f5f8-ec64-1223-c1d8-97dd7e601cee.sys.drill,
> exception.log, query3.sql
>
>
> TPCDS queries with the latest 1.4.0 release when hash join is disabled:
> 22 queries fail with out of memory
> 2 wrong results (I did not validate the nature of wrong result yet)
> Only query97.sql is a legitimate failure: we don't support full outer join
> with the merge join.
> It is important to understand what has changed between 1.2.0 and 1.4.0 that
> made these tests not runnable with the same configuration.
> Same tests with the same drill configuration pass in 1.2.0 release.
> (I hope I did not make a mistake somewhere in my cluster setup :))
> {code}
> 0: jdbc:drill:schema=dfs> select * from sys.version;
> +-+---+-++--++
> | version | commit_id |
> commit_message|commit_time
> | build_email | build_time |
> +-+---+-++--++
> | 1.4.0-SNAPSHOT | b9068117177c3b47025f52c00f67938e0c3e4732 | DRILL-4165
> Add a precondition for size of merge join record batch. | 08.12.2015 @
> 01:25:34 UTC | Unknown | 08.12.2015 @ 03:36:25 UTC |
> +-+---+-++--++
> 1 row selected (2.211 seconds)
> Execution Failures:
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query50.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query33.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query74.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query68.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query34.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query21.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query46.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query91.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query59.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query3.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query66.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query84.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query97.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query19.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query96.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query43.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/original/query15.sql
> /root/drill-tests-new/framework/resources/Advanced/tpcds/tpcds_sf100/o
[jira] [Closed] (DRILL-4044) NPE in partition pruning test on (JDK8 + Drill 1.3)
[
https://issues.apache.org/jira/browse/DRILL-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Khurram Faraaz closed DRILL-4044.
-
> NPE in partition pruning test on (JDK8 + Drill 1.3)
> ---
>
> Key: DRILL-4044
> URL: https://issues.apache.org/jira/browse/DRILL-4044
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Affects Versions: 1.3.0
> Environment: 4 node cluster CentOS
>Reporter: Khurram Faraaz
>Assignee: Deneche A. Hakim
> Fix For: 1.5.0
>
>
> NPE reported in drillbit.log for parquet partition pruning test with Drill
> 1.3 and JDK 8
> Failing query is from test file
> Functional/partition_pruning/hive/parquet/dynamic_int_partition/data/parquetSelectPartOrMultipleWithDirIN.q
> {code}
> select l_orderkey, l_partkey, l_quantity, cast(l_shipdate as date)
> l_shipdate, l_shipinstruct, `year` from
> hive.dynamic_partitions.lineitem_parquet_partitioned_hive where (`year` IN
> (1993) and l_orderkey>29600) or (`year` IN (1994) and l_orderkey>29700);
> {code}
> From test output file -
> dynamicPartitionDirectoryHive-IntPartitionData_parquetSelectPartOrMultipleWithDirIN.output_Fri_Nov_06_00:44:16_UTC_2015
> {code}
> 0 SYSTEM ERROR: NullPointerException
> Fragment 0:0
> [Error Id: c6f424ce-10b9-48e1-8783-b0dd281b6fc3 on centos-01.qa.lab:31010]
> {code}
> From the drillbit.log
> {code}
> 2015-11-06 00:44:17,222 [29c4081f-628a-fda7-05c5-a70aa9aa148b:foreman] INFO
> o.a.d.e.p.l.partition.PruneScanRule - Pruned 7 partitions down to 2
> 2015-11-06 00:44:17,256 [29c4081f-628a-fda7-05c5-a70aa9aa148b:foreman] INFO
> o.a.d.e.p.l.partition.PruneScanRule - No partitions were eligible for pruning
> 2015-11-06 00:44:17,323 [29c4081f-628a-fda7-05c5-a70aa9aa148b:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c4081f-628a-fda7-05c5-a70aa9aa148b:0:0: State change requested
> AWAITING_ALLOCATION --> FAILED
> 2015-11-06 00:44:17,323 [29c4081f-628a-fda7-05c5-a70aa9aa148b:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c4081f-628a-fda7-05c5-a70aa9aa148b:0:0: State change requested FAILED -->
> FINISHED
> 2015-11-06 00:44:17,324 [29c4081f-628a-fda7-05c5-a70aa9aa148b:frag:0:0] ERROR
> o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: NullPointerException
> Fragment 0:0
> [Error Id: c6f424ce-10b9-48e1-8783-b0dd281b6fc3 on centos-01.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> NullPointerException
> Fragment 0:0
> [Error Id: c6f424ce-10b9-48e1-8783-b0dd281b6fc3 on centos-01.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
> [drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
> [drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
> [drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_65]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_65]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2590) Union all query returns results and Exception
[
https://issues.apache.org/jira/browse/DRILL-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080762#comment-15080762
]
Khurram Faraaz commented on DRILL-2590:
---
Union all query still shows same behavior as before. Query returns some results
and then we see a SchemaChangeException. Question here is should the query only
return SchemaChangeException and not return "some" of the results ?
{noformat}
0: jdbc:drill:schema=dfs.tmp> alter session set
`store.json.all_text_mode`=false;
+---++
| ok | summary |
+---++
| true | store.json.all_text_mode updated. |
+---++
1 row selected (0.279 seconds)
0: jdbc:drill:schema=dfs.tmp> select key from `intData.json` union all select
key from `booldata.json`;
++
|key |
++
| -291344610 |
| 10331709374297784 |
| -1184860307|
| 67862516816374816 |
| -1921037123|
...
| 1585644903 |
| 20460160889932564 |
| 341564214 |
| 18953296325892812 |
Error: SYSTEM ERROR: SchemaChangeException: Failure while trying to materialize
incoming schema. Errors:
Error in expression at index -1. Error: Missing function implementation:
[castBIGINT(BIT-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--..
Fragment 0:0
[Error Id: 44466004-11fa-41bd-8b12-d58166c5d172 on centos-02.qa.lab:31010]
(state=,code=0)
{noformat}
Stack trace from drillbit.log for the above problem.
{noformat}
2016-01-04 06:35:53,738 [2975ed36-550b-bdaa-29d5-f2fbdb0a227d:frag:0:0] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: SchemaChangeException:
Failure while trying to materialize incoming schema. Errors:
Error in expression at index -1. Error: Missing function implementation:
[castBIGINT(BIT-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--..
Fragment 0:0
[Error Id: 44466004-11fa-41bd-8b12-d58166c5d172 on centos-02.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
SchemaChangeException: Failure while trying to materialize incoming schema.
Errors:
Error in expression at index -1. Error: Missing function implementation:
[castBIGINT(BIT-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--..
Fragment 0:0
[Error Id: 44466004-11fa-41bd-8b12-d58166c5d172 on centos-02.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[drill-common-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.4.0.jar:1.4.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: org.apache.drill.exec.exception.SchemaChangeException: Failure while
trying to materialize incoming schema. Errors: Error in expression at index
-1. Error: Missing function implementation: [castBIGINT(BIT-OPTIONAL)]. Full
expression: --UNKNOWN EXPRESSION--..
at
org.apache.drill.exec.physical.impl.union.UnionAllRecordBatch.doWork(UnionAllRecordBatch.java:228)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.physical.impl.union.UnionAllRecordBatch.innerNext(UnionAllRecordBatch.java:116)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.
[jira] [Commented] (DRILL-2590) Union all query returns results and Exception
[
https://issues.apache.org/jira/browse/DRILL-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080770#comment-15080770
]
Sean Hsuan-Yi Chu commented on DRILL-2590:
--
The behavior is correct.
Drill follows a pipeline model. It can not see the schema change until there is
one.
> Union all query returns results and Exception
> -
>
> Key: DRILL-2590
> URL: https://issues.apache.org/jira/browse/DRILL-2590
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.9.0
> Environment: {code}
> 0: jdbc:drill:> select * from sys.version;
> +++-+-++
> | commit_id | commit_message | commit_time | build_email | build_time |
> +++-+-++
> | 9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580: Exit early from
> HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53 EDT | Unknown
> | 26.03.2015 @ 16:53:21 EDT |
> +++-+-++
> 1 row selected (0.109 seconds)
> {code}
>Reporter: Khurram Faraaz
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 0.9.0
>
> Attachments: DRILL-2590.1.patch
>
>
> Union all over JSON data file (int data and boolean data) results in
> Exception. Query returns some results and also returns Exception.
> {code}
> 0: jdbc:drill:> select key from `intData.json` union all select key from
> `booldata.json`;
> Query failed: Query stopped., Failure while trying to materialize incoming
> schema. Errors:
>
> Error in expression at index -1. Error: Missing function implementation:
> [castBIGINT(BIT-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--.. [
> ebb1ddb9-789e-4c69-ba6f-ca44cf67d67c on centos-04.qa.lab:31010 ]
> ++
> |key |
> ++
> | -291344610 |
> | 10331709374297784 |
> | -1184860307 |
> | 67862516816374816 |
> | -1921037123 |
> | 11968068521452206 |
> | 1023332588 |
> | 89090303541569808 |
> | 1780205110 |
> | 5730049513235057 |
> | -127065493 |
> | 45860081566575456 |
> | -299030658 |
> | 869008363115253 |
> | -2108348330 |
> | 78779878523955616 |
> | -709349595 |
> | 77042673985263184 |
> ...
> | 341564214 |
> | 18953296325892812 |
> ++
> |key |
> ++
> | 0 |
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing
> query.
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)
> {code}
> {code}
> 2015-03-27 00:02:26,715 [2aeb61ed-1770-79c8-e0a2-b94f408e7b5b:frag:0:0] ERROR
> o.a.drill.exec.work.foreman.Foreman - Error
> e47f6df7-678f-4f1a-b144-2c120b331e95: RemoteRpcException: Failure while
> running fragment., Failure while trying to materialize incoming schema.
> Errors:
> Error in expression at index -1. Error: Missing function implementation:
> [castBIGINT(BIT-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--.. [
> 3f0a4e99-fe80-4187-a17e-32aa7914285c on centos-04.qa.lab:31010 ]
> [ 3f0a4e99-fe80-4187-a17e-32aa7914285c on centos-04.qa.lab:31010 ]
> org.apache.drill.exec.rpc.RemoteRpcException: Failure while running
> fragment., Failure while trying to materialize incoming schema. Errors:
> Error in expression at index -1. Error: Missing function implementation:
> [castBIGINT(BIT-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--.. [
> 3f0a4e99-fe80-4187-a17e-32aa7914285c on centos-04.qa.lab:31010 ]
> [ 3f0a4e99-fe80-4187-a17e-32aa7914285c on centos-04.qa.lab:31010 ]
> at
> org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:163)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:281)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:114)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:110)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.int
[jira] [Closed] (DRILL-2590) Union all query returns results and Exception
[
https://issues.apache.org/jira/browse/DRILL-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Khurram Faraaz closed DRILL-2590.
-
> Union all query returns results and Exception
> -
>
> Key: DRILL-2590
> URL: https://issues.apache.org/jira/browse/DRILL-2590
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.9.0
> Environment: {code}
> 0: jdbc:drill:> select * from sys.version;
> +++-+-++
> | commit_id | commit_message | commit_time | build_email | build_time |
> +++-+-++
> | 9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580: Exit early from
> HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53 EDT | Unknown
> | 26.03.2015 @ 16:53:21 EDT |
> +++-+-++
> 1 row selected (0.109 seconds)
> {code}
>Reporter: Khurram Faraaz
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 0.9.0
>
> Attachments: DRILL-2590.1.patch
>
>
> Union all over JSON data file (int data and boolean data) results in
> Exception. Query returns some results and also returns Exception.
> {code}
> 0: jdbc:drill:> select key from `intData.json` union all select key from
> `booldata.json`;
> Query failed: Query stopped., Failure while trying to materialize incoming
> schema. Errors:
>
> Error in expression at index -1. Error: Missing function implementation:
> [castBIGINT(BIT-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--.. [
> ebb1ddb9-789e-4c69-ba6f-ca44cf67d67c on centos-04.qa.lab:31010 ]
> ++
> |key |
> ++
> | -291344610 |
> | 10331709374297784 |
> | -1184860307 |
> | 67862516816374816 |
> | -1921037123 |
> | 11968068521452206 |
> | 1023332588 |
> | 89090303541569808 |
> | 1780205110 |
> | 5730049513235057 |
> | -127065493 |
> | 45860081566575456 |
> | -299030658 |
> | 869008363115253 |
> | -2108348330 |
> | 78779878523955616 |
> | -709349595 |
> | 77042673985263184 |
> ...
> | 341564214 |
> | 18953296325892812 |
> ++
> |key |
> ++
> | 0 |
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing
> query.
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)
> {code}
> {code}
> 2015-03-27 00:02:26,715 [2aeb61ed-1770-79c8-e0a2-b94f408e7b5b:frag:0:0] ERROR
> o.a.drill.exec.work.foreman.Foreman - Error
> e47f6df7-678f-4f1a-b144-2c120b331e95: RemoteRpcException: Failure while
> running fragment., Failure while trying to materialize incoming schema.
> Errors:
> Error in expression at index -1. Error: Missing function implementation:
> [castBIGINT(BIT-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--.. [
> 3f0a4e99-fe80-4187-a17e-32aa7914285c on centos-04.qa.lab:31010 ]
> [ 3f0a4e99-fe80-4187-a17e-32aa7914285c on centos-04.qa.lab:31010 ]
> org.apache.drill.exec.rpc.RemoteRpcException: Failure while running
> fragment., Failure while trying to materialize incoming schema. Errors:
> Error in expression at index -1. Error: Missing function implementation:
> [castBIGINT(BIT-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--.. [
> 3f0a4e99-fe80-4187-a17e-32aa7914285c on centos-04.qa.lab:31010 ]
> [ 3f0a4e99-fe80-4187-a17e-32aa7914285c on centos-04.qa.lab:31010 ]
> at
> org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:163)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:281)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:114)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:110)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.internalFail(FragmentExecutor.java:230)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(Fragm
