Error with union all and a hint
Hi I have a problem with executing a UNION ALL statment with added hints. I need the hints to be able to use the index. For example: explain SELECT /*+ INDEX(table1 INDEX_TABLE1_COL) */ col1, col2 FROM table1 WHERE col1='123' UNION ALL SELECT /*+ INDEX(table2 INDEX_TABLE2_COL) */ col1, col2 FROM table2 WHERE col1='123'; I get an error: java.lang.NullPointerException at java.util.ArrayList.addAll(ArrayList.java:559) at org.apache.phoenix.iterate.UnionResultIterators.(UnionResultIterators.java:51) at org.apache.phoenix.execute.UnionPlan.iterator(UnionPlan.java:131) at org.apache.phoenix.execute.UnionPlan.iterator(UnionPlan.java:127) at org.apache.phoenix.execute.UnionPlan.getExplainPlan(UnionPlan.java:150) at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableExplainStatement.compilePlan(PhoenixStatement.java:424) at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableExplainStatement.compilePlan(PhoenixStatement.java:403) at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:246) at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:241) at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:240) at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1250) at sqlline.Commands.execute(Commands.java:822) at sqlline.Commands.sql(Commands.java:732) at sqlline.SqlLine.dispatch(SqlLine.java:808) at sqlline.SqlLine.begin(SqlLine.java:681) at sqlline.SqlLine.start(SqlLine.java:398) at sqlline.SqlLine.main(SqlLine.java:292) I am using phoenix-4.4.0-HBase-1.1, the tables and indexes are as follows. I will greatly appriciate any help. Regards, Gasper CREATE TABLE table1 ( id BIGINT, col1 VARCHAR, col2 VARCHAR, CONSTRAINT pk PRIMARY KEY (id)) IMMUTABLE_ROWS=true; CREATE TABLE table2 ( id BIGINT, col1 VARCHAR, col2 VARCHAR, CONSTRAINT pk PRIMARY KEY (id)) IMMUTABLE_ROWS=true; CREATE INDEX INDEX_TABLE1_COL ON table1 (col1); CREATE INDEX INDEX_TABLE2_COL ON table2 (col1);
RE: Error with union all and a hint
I have created the jira PHOENIX-2295<https://issues.apache.org/jira/browse/PHOENIX-2295>. Running the query without the hint, executes the statment with a full table scan: 0: jdbc:phoenix:localhost> explain SELECT col1, col2 FROM table1 WHERE col1='123'; +--+ | PLAN | +--+ | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER TABLE1 | | SERVER FILTER BY COL1 = '123'| +--+ Running the query wit the hint: 0: jdbc:phoenix:localhost> explain SELECT /*+ INDEX(table1 INDEX_TABLE1_COL) */ col1, col2 FROM table1 WHERE col1='123'; +--+ | PLAN | +--+ | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER TABLE1 | | SKIP-SCAN-JOIN TABLE 0 | | CLIENT 1-CHUNK PARALLEL 1-WAY RANGE SCAN OVER INDEX_TABLE1_COL ['123'] | | SERVER FILTER BY FIRST KEY ONLY | | DYNAMIC SERVER FILTER BY "TABLE1.ID" IN ($72.$74) | +--+ Regards, Gasper From: Alicia Shu [mailto:a...@hortonworks.com] Sent: Tuesday, September 29, 2015 9:24 AM To: user@phoenix.apache.org Subject: Re: Error with union all and a hint By default if there is an index defined for a table, the index will be used if the plan with the index is better. Please file a Jira for the NPE. Thanks, Alicia From: Gašper Metelko mailto:gasper.mete...@salviol.com>> Reply-To: "user@phoenix.apache.org<mailto:user@phoenix.apache.org>" mailto:user@phoenix.apache.org>> Date: Friday, September 25, 2015 at 6:52 AM To: "user@phoenix.apache.org<mailto:user@phoenix.apache.org>" mailto:user@phoenix.apache.org>> Subject: Error with union all and a hint Hi I have a problem with executing a UNION ALL statment with added hints. I need the hints to be able to use the index. For example: explain SELECT /*+ INDEX(table1 INDEX_TABLE1_COL) */ col1, col2 FROM table1 WHERE col1='123' UNION ALL SELECT /*+ INDEX(table2 INDEX_TABLE2_COL) */ col1, col2 FROM table2 WHERE col1='123'; I get an error: java.lang.NullPointerException at java.util.ArrayList.addAll(ArrayList.java:559) at org.apache.phoenix.iterate.UnionResultIterators.(UnionResultIterators.java:51) at org.apache.phoenix.execute.UnionPlan.iterator(UnionPlan.java:131) at org.apache.phoenix.execute.UnionPlan.iterator(UnionPlan.java:127) at org.apache.phoenix.execute.UnionPlan.getExplainPlan(UnionPlan.java:150) at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableExplainStatement.compilePlan(PhoenixStatement.java:424) at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableExplainStatement.compilePlan(PhoenixStatement.java:403) at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:246) at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:241) at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:240) at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1250) at sqlline.Commands.execute(Commands.java:822) at sqlline.Commands.sql(Commands.java:732) at sqlline.SqlLine.dispatch(SqlLine.java:808) at sqlline.SqlLine.begin(SqlLine.java:681) at sqlline.SqlLine.start(SqlLine.java:398) at sqlline.SqlLine.main(SqlLine.java:292) I am using phoenix-4.4.0-HBase-1.1, the tables and indexes are as follows. I will greatly appriciate any help. Regards, Gasper CREATE TABLE table1 ( id BIGINT, col1 VARCHAR, col2 VARCHAR, CONSTRAINT pk PRIMARY KEY (id)) IMMUTABLE_ROWS=true; CREATE TABLE table2 ( id BIGINT, col1 VARCHAR, col2 VARCHAR, CONSTRAINT pk PRIMARY KEY (id)) IMMUTABLE_ROWS=true; CREATE INDEX INDEX_TABLE1_COL ON table1 (col1); CREATE INDEX INDEX_TABLE2_COL ON table2 (col1);
Multiple hints in phoenix join
Hi With using phoenix-4.8.1-HBase-1.1-bin I have problems doing a join with hints. Doing a select with a join on two tables when trying to use multiple hints phoenix does include them in the execution plan: Should be: CLIENT 1-CHUNK 0 ROWS 0 BYTES PARALLEL 1-WAY ROUND ROBIN RANGE SCAN OVER C12_SIDR_CDR_20160820_INDEX ['293400115278890'] ) For example: SELECT /*+ INDEX(SIDR_CDR_20160820 c12_SIDR_CDR_20160820_INDEX), USE_SORT_MERGE_JOIN */ crm.c5, crm.c6,crm.c7, crm.c10, crm.c11, cdr.c2, cdr.c4, cdr.c7, cdr.c8, cdr.c9, cdr.c10, cdr.c11, cdr.c12, cdr.c5, cdr.c6, cdr.c19, cdr.c20, cdr.c15, cdr.c16, cdr.c13, cdr.c5 FROM SIDR_CDR_20160820 cdr, SIDR_CRM crm WHERE ((cdr.c8 IN (1, 130, 29, 3, 2, 131, 31, 132, 30, 133, 136, 137, 135, 134, 150)) AND (cdr.c12 = '293400115278890')) AND (cdr.c2 <= TO_DATE('20.08.2016 23:59:59', 'dd.MM. HH:mm:ss')) AND crm.c1 = '0' || SUBSTR(cdr.c5, 4) AND cdr.c2 >= crm.c2 AND (cdr.c2 < crm.c3 OR crm.c3 IS NULL); ++ | SORT-MERGE-JOIN (INNER) TABLES | | CLIENT 794-CHUNK 41607713 ROWS 20709402545 BYTES PARALLEL 794-WAY FULL SCAN OVER SIDR_CDR_20160820 | | SERVER FILTER BY (C8 IN (1,2,3,29,30,31,130,131,132,133,134,135,136,137,150) AND C12 = '293400115278890' AND C2 <= TIMESTAMP '2016-08-20 23:59:59.000' | | SERVER SORTED BY [('0' || SUBSTR(CDR.C5, 4, null))] | | CLIENT MERGE SORT | | AND | | CLIENT 35-CHUNK 1864177 ROWS 865075995 BYTES PARALLEL 1-WAY FULL SCAN OVER SIDR_CRM | | CLIENT FILTER BY (CDR.C2 >= CRM.C2 AND (CDR.C2 < CRM.C3 OR CRM.C3 IS NULL)) | ++ When further investigating this, phoenix I think has two errors: 1. This is a query without a join only selecting from SIDR_CDR_20160820 cdr. I am not able to mix a real table name (for hints) and synonime (for select), the second query has the correct execution plan explain SELECT /*+ INDEX(SIDR_CDR_20160820 c12_SIDR_CDR_20160820_INDEX)*/ cdr.c2, cdr.c4, cdr.c7, cdr.c8, cdr.c9, cdr.c10, cdr.c11, cdr.c12, cdr.c5, cdr.c6, cdr.c19, cdr.c20, cdr.c15, cdr.c16, cdr.c13, cdr.c5 FROM SIDR_CDR_20160820 cdr WHERE ((cdr.c8 IN (1, 130, 29, 3, 2, 131, 31, 132, 30, 133, 136, 137, 135, 134, 150)) AND (cdr.c12 = '293400115278890')) AND (cdr.c2 <= TO_DATE('20.08.2016 23:59:59', 'dd.MM. HH:mm:ss')); ++ | CLIENT 794-CHUNK 41607713 ROWS 20709402545 BYTES PARALLEL 1-WAY ROUND ROBIN FULL SCAN OVER SIDR_CDR_20160820 | | SERVER FILTER BY ((C8 IN (1,2,3,29,30,31,130,131,132,133,134,135,136,137,150) AND C12 = '293400115278890') AND C2 <= TIMESTAMP '2016-08-20 23:59:59.000') | ++ A correct execution plan with the same query only with a changed hint /*+ INDEX(cdr c12_SIDR_CDR_20160820_INDEX)*/ 0: jdbc:phoenix:lxdr1> explain SELECT /*+ INDEX(cdr c12_SIDR_CDR_20160820_INDEX)*/ cdr.c2, cdr.c4, cdr.c7, cdr.c8, cdr.c9, cdr.c10, cdr.c11, cdr.c12, cdr.c5, cdr.c6, cdr.c19, cdr.c20, cdr.c15, cdr.c16, cdr.c13, cdr.c5 FROM SIDR_CDR_20160820 cdr WHERE ((cdr.c8 IN (1, 130, 29, 3, 2, 131, 31, 132, 30, 133, 136, 137, 135, 134, 150)) AND (cdr.c12 = '293400115278890')) AND (cdr.c2 <= TO_DATE('20.08.2016 23:59:59', 'dd.MM. HH:mm:ss')); +---+ | PLAN | +---+ | CLIENT 794-CHUNK 4160