[jira] [Commented] (DRILL-5796) Filter pruning for multi rowgroup parquet file
[ https://issues.apache.org/jira/browse/DRILL-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737743#comment-16737743 ] Robert Hou commented on DRILL-5796: --- Found our documentation on this, and the default limit is 10K rowgroups. Which means we are limited to 10K files. > Filter pruning for multi rowgroup parquet file > -- > > Key: DRILL-5796 > URL: https://issues.apache.org/jira/browse/DRILL-5796 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Parquet >Reporter: Damien Profeta >Assignee: Jean-Blas IMBERT >Priority: Major > Labels: ready-to-commit > Fix For: 1.15.0 > > > Today, filter pruning use the file name as the partitioning key. This means > you can remove a partition only if the whole file is for the same partition. > With parquet, you can prune the filter if the rowgroup make a partition of > your dataset as the unit of work if the rowgroup not the file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6949) Query fails with "UNSUPPORTED_OPERATION ERROR: Hash-Join can not partition the inner data any further" when Semi join is enabled
[
https://issues.apache.org/jira/browse/DRILL-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua updated DRILL-6949:
Fix Version/s: 1.16.0
> Query fails with "UNSUPPORTED_OPERATION ERROR: Hash-Join can not partition
> the inner data any further" when Semi join is enabled
>
>
> Key: DRILL-6949
> URL: https://issues.apache.org/jira/browse/DRILL-6949
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning Optimization
>Affects Versions: 1.15.0
>Reporter: Abhishek Ravi
>Priority: Major
> Fix For: 1.16.0
>
> Attachments: 23cc1240-74ff-a0c0-8cd5-938fc136e4e2.sys.drill,
> 23cc1369-0812-63ce-1861-872636571437.sys.drill
>
>
> Following query fails when with *Error: UNSUPPORTED_OPERATION ERROR:
> Hash-Join can not partition the inner data any further (probably due to too
> many join-key duplicates)* on TPC-H SF100 data.
> {code:sql}
> set `exec.hashjoin.enable.runtime_filter` = true;
> set `exec.hashjoin.runtime_filter.max.waiting.time` = 1;
> set `planner.enable_broadcast_join` = false;
> select
> count(*)
> from
> lineitem l1
> where
> l1.l_discount IN (
> select
> distinct(cast(l2.l_discount as double))
> from
> lineitem l2);
> reset `exec.hashjoin.enable.runtime_filter`;
> reset `exec.hashjoin.runtime_filter.max.waiting.time`;
> reset `planner.enable_broadcast_join`;
> {code}
> The subquery contains *distinct* keyword and hence there should not be
> duplicate values.
> I suspect that the failure is caused by semijoin because the query succeeds
> when semijoin is disabled explicitly.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6957) Parquet rowgroup filtering can have incorrect file count
Robert Hou created DRILL-6957:
-
Summary: Parquet rowgroup filtering can have incorrect file count
Key: DRILL-6957
URL: https://issues.apache.org/jira/browse/DRILL-6957
Project: Apache Drill
Issue Type: Bug
Reporter: Robert Hou
Assignee: Jean-Blas IMBERT
If a query accesses all the files, the Scan operator indicates that one file is
accessed. The number of rowgroups is correct.
Here is an example query:
{noformat}
select count(*) from
dfs.`/custdata/tudata/fact/vintage/snapshot_period_id=20151231/comp_id=120`
where cur_tot_bal_amt < 100
{noformat}
Here is the plan:
{noformat}
Screen : rowType = RecordType(BIGINT EXPR$0): rowcount = 1.0, cumulative cost =
{9.8376721446E9 rows, 4.35668337906E10 cpu, 2.810763469E9 io, 4096.0 network,
0.0 memory}, id = 4477
00-01 Project(EXPR$0=[$0]) : rowType = RecordType(BIGINT EXPR$0): rowcount
= 1.0, cumulative cost = {9.8376721445E9 rows, 4.35668337905E10 cpu,
2.810763469E9 io, 4096.0 network, 0.0 memory}, id = 4476
00-02StreamAgg(group=[{}], EXPR$0=[$SUM0($0)]) : rowType =
RecordType(BIGINT EXPR$0): rowcount = 1.0, cumulative cost = {9.8376721435E9
rows, 4.35668337895E10 cpu, 2.810763469E9 io, 4096.0 network, 0.0 memory}, id =
4475
00-03 UnionExchange : rowType = RecordType(BIGINT EXPR$0): rowcount =
1.0, cumulative cost = {9.8376721425E9 rows, 4.35668337775E10 cpu,
2.810763469E9 io, 4096.0 network, 0.0 memory}, id = 4474
01-01StreamAgg(group=[{}], EXPR$0=[COUNT()]) : rowType =
RecordType(BIGINT EXPR$0): rowcount = 1.0, cumulative cost = {9.8376721415E9
rows, 4.35668337695E10 cpu, 2.810763469E9 io, 0.0 network, 0.0 memory}, id =
4473
01-02 Project($f0=[0]) : rowType = RecordType(INTEGER $f0):
rowcount = 1.4053817345E9, cumulative cost = {8.432290407E9 rows,
2.67022529555E10 cpu, 2.810763469E9 io, 0.0 network, 0.0 memory}, id = 4472
01-03SelectionVectorRemover : rowType = RecordType(ANY
cur_tot_bal_amt): rowcount = 1.4053817345E9, cumulative cost = {7.0269086725E9
rows, 2.10807260175E10 cpu, 2.810763469E9 io, 0.0 network, 0.0 memory}, id =
4471
01-04 Filter(condition=[($0, 100)]) : rowType =
RecordType(ANY cur_tot_bal_amt): rowcount = 1.4053817345E9, cumulative cost =
{5.621526938E9 rows, 1.9675344283E10 cpu, 2.810763469E9 io, 0.0 network, 0.0
memory}, id = 4470
01-05Scan(table=[[dfs,
/custdata/tudata/fact/vintage/snapshot_period_id=20151231/comp_id=120]],
groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath
[path=maprfs:///custdata/tudata/fact/vintage/snapshot_period_id=20151231/comp_id=120]],
selectionRoot=maprfs:/custdata/tudata/fact/vintage/snapshot_period_id=20151231/comp_id=120,
numFiles=1, numRowGroups=1007, usedMetadataFile=false,
columns=[`cur_tot_bal_amt`]]]) : rowType = RecordType(ANY cur_tot_bal_amt):
rowcount = 2.810763469E9, cumulative cost = {2.810763469E9 rows, 2.810763469E9
cpu, 2.810763469E9 io, 0.0 network, 0.0 memory}, id = 4469
{noformat}
numFiles is set to 1 when it should be set to 21.
All the files are in one directory. If I add a level of directories (i.e. a
directory with multiple directories, each with files), then I get the correct
file count.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5796) Filter pruning for multi rowgroup parquet file
[ https://issues.apache.org/jira/browse/DRILL-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737694#comment-16737694 ] Robert Hou commented on DRILL-5796: --- It looks like pushdown is performed if there are up to 10K rowgroups. If there are more than 10K rowgroups, I cannot tell if pushdown is being performed. The explain plan suggests it is not being performed. > Filter pruning for multi rowgroup parquet file > -- > > Key: DRILL-5796 > URL: https://issues.apache.org/jira/browse/DRILL-5796 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Parquet >Reporter: Damien Profeta >Assignee: Jean-Blas IMBERT >Priority: Major > Labels: ready-to-commit > Fix For: 1.15.0 > > > Today, filter pruning use the file name as the partitioning key. This means > you can remove a partition only if the whole file is for the same partition. > With parquet, you can prune the filter if the rowgroup make a partition of > your dataset as the unit of work if the rowgroup not the file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6880) Hash-Join: Many null keys on the build side form a long linked chain in the Hash Table
[
https://issues.apache.org/jira/browse/DRILL-6880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pritesh Maker updated DRILL-6880:
-
Reviewer: Aman Sinha
> Hash-Join: Many null keys on the build side form a long linked chain in the
> Hash Table
> --
>
> Key: DRILL-6880
> URL: https://issues.apache.org/jira/browse/DRILL-6880
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
>Affects Versions: 1.14.0
>Reporter: Boaz Ben-Zvi
>Assignee: Boaz Ben-Zvi
>Priority: Critical
> Fix For: 1.16.0
>
>
> When building the Hash Table for the Hash-Join, each new key is matched with
> an existing key (same bucket) by calling the generated method
> `isKeyMatchInternalBuild`, which compares the two. However when both keys are
> null, the method returns *false* (meaning not-equal; i.e. it is a new key),
> thus the new key is added into the list following the old key. When a third
> null key is found, it would be matched with the prior two, and added as well.
> Etc etc ...
> This way many null values would perform checks at order N^2 / 2.
> _Suggested improvement_: The generated code should return a third result,
> meaning "two null keys". Then in case of Inner or Left joins all the
> duplicate nulls can be discarded.
> Below is a simple example, note the time difference between non-null and the
> all-nulls tables (also instrumentation showed that for nulls, the method
> above was called 1249975000 times!!)
> {code:java}
> 0: jdbc:drill:zk=local> use dfs.tmp;
> 0: jdbc:drill:zk=local> create table testNull as (select cast(null as int)
> mycol from
> dfs.`/data/test128M.tbl` limit 5);
> 0: jdbc:drill:zk=local> create table test1 as (select cast(1 as int) mycol1
> from
> dfs.`/data/test128M.tbl` limit 6);
> 0: jdbc:drill:zk=local> create table test2 as (select cast(2 as int) mycol2
> from dfs.`/data/test128M.tbl` limit 5);
> 0: jdbc:drill:zk=local> select count(*) from test1 join test2 on test1.mycol1
> = test2.mycol2;
> +-+
> | EXPR$0 |
> +-+
> | 0 |
> +-+
> 1 row selected (0.443 seconds)
> 0: jdbc:drill:zk=local> select count(*) from test1 join testNull on
> test1.mycol1 = testNull.mycol;
> +-+
> | EXPR$0 |
> +-+
> | 0 |
> +-+
> 1 row selected (140.098 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6956) Maintain a single entry for Drill Version in the pom file
[ https://issues.apache.org/jira/browse/DRILL-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khatua updated DRILL-6956: Reviewer: Vitalii Diravka > Maintain a single entry for Drill Version in the pom file > - > > Key: DRILL-6956 > URL: https://issues.apache.org/jira/browse/DRILL-6956 > Project: Apache Drill > Issue Type: Improvement > Components: Tools, Build Test >Affects Versions: 1.15.0 >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Major > Fix For: 1.16.0 > > > Currently, updating the version information for a Drill release involves > updating 30+ pom files. > The right way would be to use the Multi Module Setup for Maven CI. > https://maven.apache.org/maven-ci-friendly.html#Multi_Module_Setup -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6956) Maintain a single entry for Drill Version in the pom file
Kunal Khatua created DRILL-6956: --- Summary: Maintain a single entry for Drill Version in the pom file Key: DRILL-6956 URL: https://issues.apache.org/jira/browse/DRILL-6956 Project: Apache Drill Issue Type: Improvement Components: Tools, Build Test Affects Versions: 1.15.0 Reporter: Kunal Khatua Assignee: Kunal Khatua Fix For: 1.16.0 Currently, updating the version information for a Drill release involves updating 30+ pom files. The right way would be to use the Multi Module Setup for Maven CI. https://maven.apache.org/maven-ci-friendly.html#Multi_Module_Setup -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6894) CTAS and CTTAS are not working on S3 storage when cache is disabled
[
https://issues.apache.org/jira/browse/DRILL-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737522#comment-16737522
]
Denys Ordynskiy commented on DRILL-6894:
Successfully tested on S3 for CTAS and CTTAS
> CTAS and CTTAS are not working on S3 storage when cache is disabled
> ---
>
> Key: DRILL-6894
> URL: https://issues.apache.org/jira/browse/DRILL-6894
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.16.0
>
> Attachments: CTAS_drillbit.log, CTAS_drillbit_queries.json,
> CTAS_sqlline.log, CTTAS_drillbit.log, CTTAS_drillbit_queries.json,
> CTTAS_sqlline.log, s3.json
>
>
> When S3 storage plugin option "fs.s3a.impl.disable.cache" is true in the
> config section and Drill is restarted,
> CTAS and CTTAS queries throwing error in Sqlline:
> *create temporary table s3.tmp.`cttastblwithcache2` as select * from
> cp.`employee.json`;*
> {color:#d04437}Error: SYSTEM ERROR: SdkClientException: Unable to load AWS
> credentials from any provider in the chain
> Fragment 0:0
> Please, refer to logs for more information.
> [Error Id: 8e386b68-d4fb-4cc6-ba1c-fb41ae0cc9ed on maprhost:31010]
> (state=,code=0){color}
> *create table s3.tmp.`ctastblwithcache` as select * from cp.`employee.json`;*
> {color:#d04437}Error: SYSTEM ERROR: SdkClientException: Unable to load AWS
> credentials from any provider in the chain
> Fragment 0:0
> Please, refer to logs for more information.
> [Error Id: 4346d300-44be-4f17-90b6-4f3a0db0a148 on maprhost:31010]
> (state=,code=0){color}
> Logs and my storage plugin are in the attachments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6931) Drill "SHOW FILES" command duplicates empty S3 folders as subfolders
[ https://issues.apache.org/jira/browse/DRILL-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737491#comment-16737491 ] Denys Ordynskiy commented on DRILL-6931: Successfully tested on Drill Explorer and Sqlline for storages: - MapRFS - S3 - FileSystem > Drill "SHOW FILES" command duplicates empty S3 folders as subfolders > > > Key: DRILL-6931 > URL: https://issues.apache.org/jira/browse/DRILL-6931 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.14.0 >Reporter: Denys Ordynskiy >Assignee: Arina Ielchiieva >Priority: Major > Labels: ready-to-commit > Fix For: 1.16.0 > > > *Description:* > If folder was created by pressing "+ Create folder" button on the S3 > management console, > Drill "SHOW FILES" command showing in this folder another subfolder with the > same name. > But this subfolder doesn't exist. > *Example:* > - login to the S3 management console; > - open your bucket `some_bucket`; > - create an empty folder `my_test_folder` by pressing "+ Create folder" > button; > - run Drill and open sqlline client; > - exec query "show files in s3.tmp.`my_test_folder`;" > *Actual result:* > |name|isDirectory|isFile|length|owner|group|permissions|accessTime|modificationTime| > |my_test_folder|true|false|0| | |rwxrwxrwx|1970-01-01 03:00:00.0|1970-01-01 > 03:00:00.0| > 1 row selected (1.318 seconds) > *Expected result:* > an empty result set. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6954) ADDRESS_COUNT doesn't work anymore
[
https://issues.apache.org/jira/browse/DRILL-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Volodymyr Vysotskyi updated DRILL-6954:
---
Labels: ready-to-commit (was: )
> ADDRESS_COUNT doesn't work anymore
> --
>
> Key: DRILL-6954
> URL: https://issues.apache.org/jira/browse/DRILL-6954
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.15.0
>Reporter: benj
>Assignee: Arina Ielchiieva
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.16.0
>
>
> The networking function ADDRESS_COUNT doesn't work anymore in DRILL *1.15.0*
> In *1.14.0* it's OK :
> {code:java}
> SELECT address_count('192.168.100.1/24');
> +-+
> | EXPR$0 |
> +-+
> | 254 |
> +-+{code}
> But in *1.15.0* it's NOK
> {code:java}
> SELECT address_count('192.168.100.1/24');
> Exception in thread "drill-executor-1" java.lang.NoSuchMethodError:
> org.apache.commons.net.util.SubnetUtils$SubnetInfo.getAddressCountLong()J
> at
> org.apache.drill.exec.udfs.NetworkFunctions$AddressCountFunction.eval(NetworkFunctions.java:87)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluateFunction(InterpreterEvaluator.java:129)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitFunctionHolderExpression(InterpreterEvaluator.java:334)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitFunctionHolderExpression(InterpreterEvaluator.java:194)
> at
> org.apache.drill.common.expression.FunctionHolderExpression.accept(FunctionHolderExpression.java:53)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluateConstantExpr(InterpreterEvaluator.java:69)
> at
> org.apache.drill.exec.planner.logical.DrillConstExecutor.reduce(DrillConstExecutor.java:151)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressionsInternal(ReduceExpressionsRule.java:620)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressions(ReduceExpressionsRule.java:540)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:288)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:648)
> at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:339)
> ...
> {code}
>
> Note that other Networking function seems to work well.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6944) UnsupportedOperationException thrown for view over MapR-DB binary table
[
https://issues.apache.org/jira/browse/DRILL-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Guzenko updated DRILL-6944:
Description:
1. Create MapR-DB binary table and put some data using HBase shell:
{code:none}
hbase shell
create '/tmp/bintable','name','address'
put '/tmp/bintable','100','name:first_name','john'
put '/tmp/bintable','100','name:last_name','doe'
put '/tmp/bintable','100','address:city','Newark'
put '/tmp/bintable','100','address:state','nj'
scan '/tmp/bintable'
{code}
2. Drill config: ensure that dfs storage plugin has "connection": "maprfs:///"
and contains format:
{code:java}
"maprdb": {
"type": "maprdb",
"allTextMode": true,
"enablePushdown": false,
"disableCountOptimization": true
}
{code}
3. Check that table can be selected from Drill :
{code:java}
select * from dfs.`/tmp/bintable`;
{code}
4. Create Drill view
{code:java}
create view dfs.tmp.`testview` as select * from dfs.`/tmp/bintable`;
{code}
5. Query the view results into exception:
{code:java}
0: jdbc:drill:> select * from dfs.tmp.`testview`;
Error: SYSTEM ERROR: UnsupportedOperationException: Unable to convert cast
expression CastExpression [input=`address`, type=minor_type: MAP
mode: REQUIRED
] into string.
Please, refer to logs for more information.
[Error Id: 109acd00-7456-4a74-8a17-485f8999000f on node1.cluster.com:31010]
(state=,code=0)
{code}
*UPDATE*
This issue may be reproduced also when avro files with map columns are queried
using Drill. Appropriate test was added to PR commit.
was:
1. Create MapR-DB binary table and put some data using HBase shell:
{code:none}
hbase shell
create '/tmp/bintable','name','address'
put '/tmp/bintable','100','name:first_name','john'
put '/tmp/bintable','100','name:last_name','doe'
put '/tmp/bintable','100','address:city','Newark'
put '/tmp/bintable','100','address:state','nj'
scan '/tmp/bintable'
{code}
2. Drill config: ensure that dfs storage plugin has "connection": "maprfs:///"
and contains format:
{code:java}
"maprdb": {
"type": "maprdb",
"allTextMode": true,
"enablePushdown": false,
"disableCountOptimization": true
}
{code}
3. Check that table can be selected from Drill :
{code:java}
select * from dfs.`/tmp/bintable`;
{code}
4. Create Drill view
{code:java}
create view dfs.tmp.`testview` as select * from dfs.`/tmp/bintable`;
{code}
5. Query the view results into exception:
{code:java}
0: jdbc:drill:> select * from dfs.tmp.`testview`;
Error: SYSTEM ERROR: UnsupportedOperationException: Unable to convert cast
expression CastExpression [input=`address`, type=minor_type: MAP
mode: REQUIRED
] into string.
Please, refer to logs for more information.
[Error Id: 109acd00-7456-4a74-8a17-485f8999000f on node1.cluster.com:31010]
(state=,code=0)
{code}
> UnsupportedOperationException thrown for view over MapR-DB binary table
> ---
>
> Key: DRILL-6944
> URL: https://issues.apache.org/jira/browse/DRILL-6944
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types, Storage - MapRDB
>Affects Versions: 1.15.0
>Reporter: Igor Guzenko
>Assignee: Igor Guzenko
>Priority: Major
> Fix For: 1.16.0
>
>
> 1. Create MapR-DB binary table and put some data using HBase shell:
> {code:none}
> hbase shell
> create '/tmp/bintable','name','address'
> put '/tmp/bintable','100','name:first_name','john'
> put '/tmp/bintable','100','name:last_name','doe'
> put '/tmp/bintable','100','address:city','Newark'
> put '/tmp/bintable','100','address:state','nj'
> scan '/tmp/bintable'
> {code}
> 2. Drill config: ensure that dfs storage plugin has "connection":
> "maprfs:///" and contains format:
> {code:java}
> "maprdb": {
> "type": "maprdb",
> "allTextMode": true,
> "enablePushdown": false,
> "disableCountOptimization": true
> }
> {code}
> 3. Check that table can be selected from Drill :
> {code:java}
> select * from dfs.`/tmp/bintable`;
> {code}
> 4. Create Drill view
> {code:java}
> create view dfs.tmp.`testview` as select * from dfs.`/tmp/bintable`;
> {code}
> 5. Query the view results into exception:
> {code:java}
> 0: jdbc:drill:> select * from dfs.tmp.`testview`;
> Error: SYSTEM ERROR: UnsupportedOperationException: Unable to convert cast
> expression CastExpression [input=`address`, type=minor_type: MAP
> mode: REQUIRED
> ] into string.
> Please, refer to logs for more information.
> [Error Id: 109acd00-7456-4a74-8a17-485f8999000f on node1.cluster.com:31010]
> (state=,code=0)
> {code}
> *UPDATE*
> This issue may be reproduced also when avro files with map columns are
> queried using Drill. Appropriate test was added to PR commit.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6955) storage-jdbc unit tests improvements
Volodymyr Vysotskyi created DRILL-6955: -- Summary: storage-jdbc unit tests improvements Key: DRILL-6955 URL: https://issues.apache.org/jira/browse/DRILL-6955 Project: Apache Drill Issue Type: Task Components: Tools, Build Test Affects Versions: 1.15.0 Reporter: Volodymyr Vysotskyi Assignee: Volodymyr Vysotskyi Fix For: 1.16.0 Currently, for unit tests in storage-jdbc module used inmemdb-maven-plugin, jcabi-mysql-maven-plugin, sql-maven-plugin, and other maven plugins. So databases are started and tables are populated after the concrete maven goal is executed. It makes the debugging process more complex since tests cannot be executed directly from IDE. Another problem is that most of these plugins weren't released for a long time. The goal of this Jira to remove these plugins usage and rework tests to provide a way to run them from IDE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6928) exec.query.return_result_set_for_ddl does not affect Web-UI query results
[
https://issues.apache.org/jira/browse/DRILL-6928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6928:
Reviewer: Arina Ielchiieva
> exec.query.return_result_set_for_ddl does not affect Web-UI query results
> -
>
> Key: DRILL-6928
> URL: https://issues.apache.org/jira/browse/DRILL-6928
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Bohdan Kazydub
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.16.0
>
>
> For the case when {{exec.query.return_result_set_for_ddl}} is set to
> {{false}} at the system level, it should affect also results returned by
> Web-UI, but independently of the value of this option, the result is always
> displayed for non-"select" queries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6928) exec.query.return_result_set_for_ddl does not affect Web-UI query results
[
https://issues.apache.org/jira/browse/DRILL-6928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6928:
Labels: ready-to-commit (was: )
> exec.query.return_result_set_for_ddl does not affect Web-UI query results
> -
>
> Key: DRILL-6928
> URL: https://issues.apache.org/jira/browse/DRILL-6928
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Bohdan Kazydub
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.16.0
>
>
> For the case when {{exec.query.return_result_set_for_ddl}} is set to
> {{false}} at the system level, it should affect also results returned by
> Web-UI, but independently of the value of this option, the result is always
> displayed for non-"select" queries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6954) ADDRESS_COUNT doesn't work anymore
[
https://issues.apache.org/jira/browse/DRILL-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6954:
Reviewer: Volodymyr Vysotskyi
> ADDRESS_COUNT doesn't work anymore
> --
>
> Key: DRILL-6954
> URL: https://issues.apache.org/jira/browse/DRILL-6954
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.15.0
>Reporter: benj
>Assignee: Arina Ielchiieva
>Priority: Major
> Fix For: 1.16.0
>
>
> The networking function ADDRESS_COUNT doesn't work anymore in DRILL *1.15.0*
> In *1.14.0* it's OK :
> {code:java}
> SELECT address_count('192.168.100.1/24');
> +-+
> | EXPR$0 |
> +-+
> | 254 |
> +-+{code}
> But in *1.15.0* it's NOK
> {code:java}
> SELECT address_count('192.168.100.1/24');
> Exception in thread "drill-executor-1" java.lang.NoSuchMethodError:
> org.apache.commons.net.util.SubnetUtils$SubnetInfo.getAddressCountLong()J
> at
> org.apache.drill.exec.udfs.NetworkFunctions$AddressCountFunction.eval(NetworkFunctions.java:87)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluateFunction(InterpreterEvaluator.java:129)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitFunctionHolderExpression(InterpreterEvaluator.java:334)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitFunctionHolderExpression(InterpreterEvaluator.java:194)
> at
> org.apache.drill.common.expression.FunctionHolderExpression.accept(FunctionHolderExpression.java:53)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluateConstantExpr(InterpreterEvaluator.java:69)
> at
> org.apache.drill.exec.planner.logical.DrillConstExecutor.reduce(DrillConstExecutor.java:151)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressionsInternal(ReduceExpressionsRule.java:620)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressions(ReduceExpressionsRule.java:540)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:288)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:648)
> at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:339)
> ...
> {code}
>
> Note that other Networking function seems to work well.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (DRILL-6954) ADDRESS_COUNT doesn't work anymore
[
https://issues.apache.org/jira/browse/DRILL-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva reassigned DRILL-6954:
---
Assignee: Arina Ielchiieva
> ADDRESS_COUNT doesn't work anymore
> --
>
> Key: DRILL-6954
> URL: https://issues.apache.org/jira/browse/DRILL-6954
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.15.0
>Reporter: benj
>Assignee: Arina Ielchiieva
>Priority: Major
> Fix For: 1.16.0
>
>
> The networking function ADDRESS_COUNT doesn't work anymore in DRILL *1.15.0*
> In *1.14.0* it's OK :
> {code:java}
> SELECT address_count('192.168.100.1/24');
> +-+
> | EXPR$0 |
> +-+
> | 254 |
> +-+{code}
> But in *1.15.0* it's NOK
> {code:java}
> SELECT address_count('192.168.100.1/24');
> Exception in thread "drill-executor-1" java.lang.NoSuchMethodError:
> org.apache.commons.net.util.SubnetUtils$SubnetInfo.getAddressCountLong()J
> at
> org.apache.drill.exec.udfs.NetworkFunctions$AddressCountFunction.eval(NetworkFunctions.java:87)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluateFunction(InterpreterEvaluator.java:129)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitFunctionHolderExpression(InterpreterEvaluator.java:334)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitFunctionHolderExpression(InterpreterEvaluator.java:194)
> at
> org.apache.drill.common.expression.FunctionHolderExpression.accept(FunctionHolderExpression.java:53)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluateConstantExpr(InterpreterEvaluator.java:69)
> at
> org.apache.drill.exec.planner.logical.DrillConstExecutor.reduce(DrillConstExecutor.java:151)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressionsInternal(ReduceExpressionsRule.java:620)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressions(ReduceExpressionsRule.java:540)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:288)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:648)
> at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:339)
> ...
> {code}
>
> Note that other Networking function seems to work well.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6954) ADDRESS_COUNT doesn't work anymore
[
https://issues.apache.org/jira/browse/DRILL-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6954:
Fix Version/s: 1.16.0
> ADDRESS_COUNT doesn't work anymore
> --
>
> Key: DRILL-6954
> URL: https://issues.apache.org/jira/browse/DRILL-6954
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.15.0
>Reporter: benj
>Priority: Major
> Fix For: 1.16.0
>
>
> The networking function ADDRESS_COUNT doesn't work anymore in DRILL *1.15.0*
> In *1.14.0* it's OK :
> {code:java}
> SELECT address_count('192.168.100.1/24');
> +-+
> | EXPR$0 |
> +-+
> | 254 |
> +-+{code}
> But in *1.15.0* it's NOK
> {code:java}
> SELECT address_count('192.168.100.1/24');
> Exception in thread "drill-executor-1" java.lang.NoSuchMethodError:
> org.apache.commons.net.util.SubnetUtils$SubnetInfo.getAddressCountLong()J
> at
> org.apache.drill.exec.udfs.NetworkFunctions$AddressCountFunction.eval(NetworkFunctions.java:87)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluateFunction(InterpreterEvaluator.java:129)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitFunctionHolderExpression(InterpreterEvaluator.java:334)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitFunctionHolderExpression(InterpreterEvaluator.java:194)
> at
> org.apache.drill.common.expression.FunctionHolderExpression.accept(FunctionHolderExpression.java:53)
> at
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluateConstantExpr(InterpreterEvaluator.java:69)
> at
> org.apache.drill.exec.planner.logical.DrillConstExecutor.reduce(DrillConstExecutor.java:151)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressionsInternal(ReduceExpressionsRule.java:620)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressions(ReduceExpressionsRule.java:540)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:288)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:648)
> at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:339)
> ...
> {code}
>
> Note that other Networking function seems to work well.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6954) ADDRESS_COUNT doesn't work anymore
benj created DRILL-6954:
---
Summary: ADDRESS_COUNT doesn't work anymore
Key: DRILL-6954
URL: https://issues.apache.org/jira/browse/DRILL-6954
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 1.15.0
Reporter: benj
The networking function ADDRESS_COUNT doesn't work anymore in DRILL *1.15.0*
In *1.14.0* it's OK :
{code:java}
SELECT address_count('192.168.100.1/24');
+-+
| EXPR$0 |
+-+
| 254 |
+-+{code}
But in *1.15.0* it's NOK
{code:java}
SELECT address_count('192.168.100.1/24');
Exception in thread "drill-executor-1" java.lang.NoSuchMethodError:
org.apache.commons.net.util.SubnetUtils$SubnetInfo.getAddressCountLong()J
at
org.apache.drill.exec.udfs.NetworkFunctions$AddressCountFunction.eval(NetworkFunctions.java:87)
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluateFunction(InterpreterEvaluator.java:129)
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitFunctionHolderExpression(InterpreterEvaluator.java:334)
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitFunctionHolderExpression(InterpreterEvaluator.java:194)
at
org.apache.drill.common.expression.FunctionHolderExpression.accept(FunctionHolderExpression.java:53)
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluateConstantExpr(InterpreterEvaluator.java:69)
at
org.apache.drill.exec.planner.logical.DrillConstExecutor.reduce(DrillConstExecutor.java:151)
at
org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressionsInternal(ReduceExpressionsRule.java:620)
at
org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressions(ReduceExpressionsRule.java:540)
at
org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:288)
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:648)
at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:339)
...
{code}
Note that other Networking function seems to work well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6922) QUERY-level options are shown on Profiles tab
[ https://issues.apache.org/jira/browse/DRILL-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736943#comment-16736943 ] Arina Ielchiieva commented on DRILL-6922: - [~bbevens] done. > QUERY-level options are shown on Profiles tab > - > > Key: DRILL-6922 > URL: https://issues.apache.org/jira/browse/DRILL-6922 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.15.0 >Reporter: Bohdan Kazydub >Assignee: Arina Ielchiieva >Priority: Blocker > Labels: doc-complete, ready-to-commit > Fix For: 1.15.0 > > > Option `exec.return_result_set_for_ddl` is shown on Web UI's Profiles even > when it was not set explicitly. The issue is because the option is being set > on query level internally. > *For documentation* > Option `exec.return_result_set_for_ddl` was renamed to > `exec.query.return_result_set_for_ddl`. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6922) QUERY-level options are shown on Profiles tab
[ https://issues.apache.org/jira/browse/DRILL-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6922: Labels: doc-complete ready-to-commit (was: doc-impacting ready-to-commit) > QUERY-level options are shown on Profiles tab > - > > Key: DRILL-6922 > URL: https://issues.apache.org/jira/browse/DRILL-6922 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.15.0 >Reporter: Bohdan Kazydub >Assignee: Arina Ielchiieva >Priority: Blocker > Labels: doc-complete, ready-to-commit > Fix For: 1.15.0 > > > Option `exec.return_result_set_for_ddl` is shown on Web UI's Profiles even > when it was not set explicitly. The issue is because the option is being set > on query level internally. > *For documentation* > Option `exec.return_result_set_for_ddl` was renamed to > `exec.query.return_result_set_for_ddl`. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (DRILL-6922) QUERY-level options are shown on Profiles tab
[ https://issues.apache.org/jira/browse/DRILL-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reopened DRILL-6922: - > QUERY-level options are shown on Profiles tab > - > > Key: DRILL-6922 > URL: https://issues.apache.org/jira/browse/DRILL-6922 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.15.0 >Reporter: Bohdan Kazydub >Assignee: Arina Ielchiieva >Priority: Blocker > Labels: doc-impacting, ready-to-commit > Fix For: 1.15.0 > > > Option `exec.return_result_set_for_ddl` is shown on Web UI's Profiles even > when it was not set explicitly. The issue is because the option is being set > on query level internally. > *For documentation* > Option `exec.return_result_set_for_ddl` was renamed to > `exec.query.return_result_set_for_ddl`. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DRILL-6922) QUERY-level options are shown on Profiles tab
[ https://issues.apache.org/jira/browse/DRILL-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva closed DRILL-6922. --- Resolution: Fixed > QUERY-level options are shown on Profiles tab > - > > Key: DRILL-6922 > URL: https://issues.apache.org/jira/browse/DRILL-6922 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.15.0 >Reporter: Bohdan Kazydub >Assignee: Arina Ielchiieva >Priority: Blocker > Labels: doc-complete, ready-to-commit > Fix For: 1.15.0 > > > Option `exec.return_result_set_for_ddl` is shown on Web UI's Profiles even > when it was not set explicitly. The issue is because the option is being set > on query level internally. > *For documentation* > Option `exec.return_result_set_for_ddl` was renamed to > `exec.query.return_result_set_for_ddl`. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6928) exec.query.return_result_set_for_ddl does not affect Web-UI query results
[
https://issues.apache.org/jira/browse/DRILL-6928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736904#comment-16736904
]
Bohdan Kazydub commented on DRILL-6928:
---
[~arina], my bad. I thought I did mention that it works for JDBC connection
only in the description. Now I see that it is not included. The description can
be updated in the scope of the issue.
The behaviour could be supported for REST for consistence, but not sure if it
is mandatory as there are no prerequisites for doing so (yet).
> exec.query.return_result_set_for_ddl does not affect Web-UI query results
> -
>
> Key: DRILL-6928
> URL: https://issues.apache.org/jira/browse/DRILL-6928
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> For the case when {{exec.query.return_result_set_for_ddl}} is set to
> {{false}} at the system level, it should affect also results returned by
> Web-UI, but independently of the value of this option, the result is always
> displayed for non-"select" queries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
