Error with union all and a hint

2015-09-25 Thread Gašper Metelko 
Hi

I have a problem with executing a UNION ALL statment with added hints. I need 
the hints to be able to use the index. For example:

explain SELECT /*+ INDEX(table1 INDEX_TABLE1_COL) */ col1, col2 FROM table1 
WHERE col1='123' UNION ALL SELECT /*+ INDEX(table2 INDEX_TABLE2_COL) */ col1, 
col2 FROM table2 WHERE col1='123';

I get an error:
java.lang.NullPointerException
at java.util.ArrayList.addAll(ArrayList.java:559)
at 
org.apache.phoenix.iterate.UnionResultIterators.(UnionResultIterators.java:51)
at org.apache.phoenix.execute.UnionPlan.iterator(UnionPlan.java:131)
at org.apache.phoenix.execute.UnionPlan.iterator(UnionPlan.java:127)
at 
org.apache.phoenix.execute.UnionPlan.getExplainPlan(UnionPlan.java:150)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableExplainStatement.compilePlan(PhoenixStatement.java:424)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableExplainStatement.compilePlan(PhoenixStatement.java:403)
at 
org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:246)
at 
org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:241)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at 
org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:240)
at 
org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1250)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)

I am using phoenix-4.4.0-HBase-1.1, the tables and indexes are as follows. I 
will greatly appriciate any help.

Regards, Gasper

CREATE TABLE table1 (
  id BIGINT,
  col1 VARCHAR,
  col2 VARCHAR,
  CONSTRAINT pk PRIMARY KEY (id)) IMMUTABLE_ROWS=true;

CREATE TABLE table2 (
  id BIGINT,
  col1 VARCHAR,
  col2 VARCHAR,
  CONSTRAINT pk PRIMARY KEY (id)) IMMUTABLE_ROWS=true;

CREATE INDEX INDEX_TABLE1_COL ON table1 (col1);

CREATE INDEX INDEX_TABLE2_COL ON table2 (col1);

RE: Error with union all and a hint

2015-09-29 Thread Gašper Metelko 
I have created the jira 
PHOENIX-2295<https://issues.apache.org/jira/browse/PHOENIX-2295>.

Running the query without the hint, executes the statment with a full table 
scan:

0: jdbc:phoenix:localhost> explain SELECT  col1, col2 FROM table1 WHERE 
col1='123';
+--+
|   PLAN   |
+--+
| CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER TABLE1 |
| SERVER FILTER BY COL1 = '123'|
+--+

Running the query wit the hint:

0: jdbc:phoenix:localhost> explain SELECT /*+ INDEX(table1 INDEX_TABLE1_COL) */ 
col1, col2 FROM table1 WHERE col1='123';
+--+
|   PLAN   |
+--+
| CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER TABLE1 |
| SKIP-SCAN-JOIN TABLE 0   |
| CLIENT 1-CHUNK PARALLEL 1-WAY RANGE SCAN OVER INDEX_TABLE1_COL 
['123'] |
| SERVER FILTER BY FIRST KEY ONLY |
| DYNAMIC SERVER FILTER BY "TABLE1.ID" IN ($72.$74) |
+--+

Regards, Gasper

From: Alicia Shu [mailto:a...@hortonworks.com]
Sent: Tuesday, September 29, 2015 9:24 AM
To: user@phoenix.apache.org
Subject: Re: Error with union all and a hint

By default if there is an index defined for a table, the index will be used if 
the plan with the index is better. Please file a Jira for the NPE.

Thanks,
Alicia

From: Gašper Metelko 
mailto:gasper.mete...@salviol.com>>
Reply-To: "user@phoenix.apache.org<mailto:user@phoenix.apache.org>" 
mailto:user@phoenix.apache.org>>
Date: Friday, September 25, 2015 at 6:52 AM
To: "user@phoenix.apache.org<mailto:user@phoenix.apache.org>" 
mailto:user@phoenix.apache.org>>
Subject: Error with union all and a hint

Hi

I have a problem with executing a UNION ALL statment with added hints. I need 
the hints to be able to use the index. For example:

explain SELECT /*+ INDEX(table1 INDEX_TABLE1_COL) */ col1, col2 FROM table1 
WHERE col1='123' UNION ALL SELECT /*+ INDEX(table2 INDEX_TABLE2_COL) */ col1, 
col2 FROM table2 WHERE col1='123';

I get an error:
java.lang.NullPointerException
at java.util.ArrayList.addAll(ArrayList.java:559)
at 
org.apache.phoenix.iterate.UnionResultIterators.(UnionResultIterators.java:51)
at org.apache.phoenix.execute.UnionPlan.iterator(UnionPlan.java:131)
at org.apache.phoenix.execute.UnionPlan.iterator(UnionPlan.java:127)
at 
org.apache.phoenix.execute.UnionPlan.getExplainPlan(UnionPlan.java:150)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableExplainStatement.compilePlan(PhoenixStatement.java:424)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableExplainStatement.compilePlan(PhoenixStatement.java:403)
at 
org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:246)
at 
org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:241)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at 
org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:240)
at 
org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1250)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)

I am using phoenix-4.4.0-HBase-1.1, the tables and indexes are as follows. I 
will greatly appriciate any help.

Regards, Gasper

CREATE TABLE table1 (
  id BIGINT,
  col1 VARCHAR,
  col2 VARCHAR,
  CONSTRAINT pk PRIMARY KEY (id)) IMMUTABLE_ROWS=true;

CREATE TABLE table2 (
  id BIGINT,
  col1 VARCHAR,
  col2 VARCHAR,
  CONSTRAINT pk PRIMARY KEY (id)) IMMUTABLE_ROWS=true;

CREATE INDEX INDEX_TABLE1_COL ON table1 (col1);

CREATE INDEX INDEX_TABLE2_COL ON table2 (col1);

Multiple hints in phoenix join

2016-11-14 Thread Gašper Metelko 
Hi

With using phoenix-4.8.1-HBase-1.1-bin I have problems doing a join with hints.

Doing a select with a join on two tables when trying to use multiple hints 
phoenix does include them in the execution plan:

Should be: CLIENT 1-CHUNK 0 ROWS 0 BYTES PARALLEL 1-WAY ROUND ROBIN RANGE SCAN 
OVER C12_SIDR_CDR_20160820_INDEX ['293400115278890'] )

For example:
SELECT /*+ INDEX(SIDR_CDR_20160820 c12_SIDR_CDR_20160820_INDEX), 
USE_SORT_MERGE_JOIN */
crm.c5, crm.c6,crm.c7, crm.c10, crm.c11, cdr.c2, cdr.c4, cdr.c7, cdr.c8, 
cdr.c9, cdr.c10, cdr.c11, cdr.c12, cdr.c5, cdr.c6, cdr.c19, cdr.c20, cdr.c15, 
cdr.c16, cdr.c13, cdr.c5
FROM
SIDR_CDR_20160820 cdr,
SIDR_CRM crm
WHERE
((cdr.c8 IN (1, 130, 29, 3, 2, 131, 31, 132, 30, 133, 136, 137, 135, 134, 
150))
AND (cdr.c12 = '293400115278890'))
AND (cdr.c2 <= TO_DATE('20.08.2016 23:59:59', 'dd.MM. HH:mm:ss'))
AND crm.c1 = '0' || SUBSTR(cdr.c5, 4)
AND cdr.c2 >= crm.c2
AND (cdr.c2 < crm.c3 OR crm.c3 IS NULL);
++
| SORT-MERGE-JOIN (INNER) TABLES

 |
| CLIENT 794-CHUNK 41607713 ROWS 20709402545 BYTES PARALLEL 794-WAY FULL 
SCAN OVER SIDR_CDR_20160820 
|
| SERVER FILTER BY (C8 IN 
(1,2,3,29,30,31,130,131,132,133,134,135,136,137,150) AND C12 = 
'293400115278890' AND C2 <= TIMESTAMP '2016-08-20 23:59:59.000' |
| SERVER SORTED BY [('0' || SUBSTR(CDR.C5, 4, null))]   

 |
| CLIENT MERGE SORT 

 |
| AND   

 |
| CLIENT 35-CHUNK 1864177 ROWS 865075995 BYTES PARALLEL 1-WAY FULL SCAN 
OVER SIDR_CRM   
 |
| CLIENT FILTER BY (CDR.C2 >= CRM.C2 AND (CDR.C2 < CRM.C3 OR CRM.C3 IS NULL))   

 |
++


When further investigating this, phoenix I think has two errors:


1.   This is a query without a join only selecting from SIDR_CDR_20160820 
cdr. I am not able to mix a real table name (for hints) and synonime (for 
select), the second query has the correct execution plan

explain SELECT /*+ INDEX(SIDR_CDR_20160820 c12_SIDR_CDR_20160820_INDEX)*/ 
cdr.c2, cdr.c4, cdr.c7, cdr.c8, cdr.c9, cdr.c10, cdr.c11, cdr.c12, cdr.c5, 
cdr.c6, cdr.c19, cdr.c20, cdr.c15, cdr.c16, cdr.c13, cdr.c5 FROM 
SIDR_CDR_20160820 cdr WHERE ((cdr.c8 IN (1, 130, 29, 3, 2, 131, 31, 132, 30, 
133, 136, 137, 135, 134, 150)) AND (cdr.c12 = '293400115278890')) AND (cdr.c2 
<= TO_DATE('20.08.2016 23:59:59', 'dd.MM. HH:mm:ss'));

++
| CLIENT 794-CHUNK 41607713 ROWS 20709402545 BYTES PARALLEL 1-WAY ROUND ROBIN 
FULL SCAN OVER SIDR_CDR_20160820
   |
| SERVER FILTER BY ((C8 IN 
(1,2,3,29,30,31,130,131,132,133,134,135,136,137,150) AND C12 = 
'293400115278890') AND C2 <= TIMESTAMP '2016-08-20 23:59:59.000')  |
++

A correct execution plan with the same query only with a changed hint /*+ 
INDEX(cdr c12_SIDR_CDR_20160820_INDEX)*/

0: jdbc:phoenix:lxdr1> explain SELECT /*+ INDEX(cdr 
c12_SIDR_CDR_20160820_INDEX)*/ cdr.c2, cdr.c4, cdr.c7, cdr.c8, cdr.c9, cdr.c10, 
cdr.c11, cdr.c12, cdr.c5, cdr.c6, cdr.c19, cdr.c20, cdr.c15, cdr.c16, cdr.c13, 
cdr.c5 FROM SIDR_CDR_20160820 cdr WHERE ((cdr.c8 IN (1, 130, 29, 3, 2, 131, 31, 
132, 30, 133, 136, 137, 135, 134, 150)) AND (cdr.c12 = '293400115278890')) AND 
(cdr.c2 <= TO_DATE('20.08.2016 23:59:59', 'dd.MM. HH:mm:ss'));
+---+
|   PLAN
|
+---+
| CLIENT 794-CHUNK 4160