[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",   {{OrderPreservingTracker.isOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}}. 

We should modify {{OrderPreservingTracker}} to make 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} could also extract 
partial continuous ordering columns which start from the first column even if 
{{OrderPreservingTracker.isOrderPreserving}} is false.  That is to say, even if 
{{v1}} in "select v1, v2, pk3" does not appear in "order by t.v2, t.pk3", it 
will return the partial ordered "v2, pk3".

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",   {{OrderPreservingTracker.isOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}}. 

We should modify {{OrderPreservingTracker}} to make 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} could also extract 
partial continuous ordering columns which start from the first column even if 
{{OrderPreservingTracker.isOrderPreserving}} is false.  That is to say, even if 
{{v1}} in "select v1, v2, pk3" does not appear in "order by t.v2, t.pk3", it 
will return the partial ordered "v2,pk3".


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",   {{OrderPreservingTracker.isOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}}. 

We should modify {{OrderPreservingTracker}} to make 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} could also extract 
partial continuous ordering columns which start from the first column even if 
{{OrderPreservingTracker.isOrderPreserving}} is false.  That is to say, even if 
{{v1}} in "select v1, v2, pk3" does not appear in "order by t.v2, t.pk3", it 
will return the partial ordered "v2,pk3".

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",   {{OrderPreservingTracker.isOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}}. 

We should modify {{OrderPreservingTracker}} to make 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} could also extract 
partial continuous ordering columns which start from the first column even if 
{{OrderPreservingTracker.isOrderPreserving}} is false.  That is to say, even if 
{{v1}} in "select v1, v2, pk3" does not appear in "order by t.v2, t.pk3", it 
will return the ordered {{v2,pk3}}.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",   {{OrderPreservingTracker.isOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",   {{OrderPreservingTracker.isOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a 

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",   {{OrderPreservingTracker.isOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",  {{OrderPreservingTracker. IsOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a 

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",   {{OrderPreservingTracker.isOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}}. 

We should modify {{OrderPreservingTracker}} to make 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} could also extract 
partial continuous ordering columns which start from the first column even if 
{{OrderPreservingTracker.isOrderPreserving}} is false.  That is to say, even if 
{{v1}} in "select v1, v2, pk3" does not appear in "order by t.v2, t.pk3", it 
will return the ordered {{v2,pk3}}.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",   {{OrderPreservingTracker.isOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> 

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",   {{OrderPreservingTracker.isOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker.getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",   {{OrderPreservingTracker.isOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a 

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3",  {{OrderPreservingTracker. IsOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3", {{OrderPreservingTracker. IsOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a 

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in "select v1, v2, pk3" does not appear in 
"order by t.v2, t.pk3", {{OrderPreservingTracker. IsOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in {{select v1, v2, pk3}} does not appear in 
{{order by t.v2, t.pk3}}, {{OrderPreservingTracker. IsOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given 

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns. That is to say, when it finds that {{v1}} in {{select v1, v2, pk3}} 
does not appear in {{order by t.v2, t.pk3}}, {{OrderPreservingTracker. 
IsOrderPreserving}} returns false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from {{select v1, v2, pk3}} based on {{order by t.v2, t.pk3}}. This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns. That is to say, when it finds that {{v1}} in {{select v1, v2, pk3}} 
does not appear in {{order by t.v2, t.pk3}}, {{OrderPreservingTracker. 
IsOrderPreserving}} returns false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an 

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns, when it finds that {{v1}} in {{select v1, v2, pk3}} does not appear in 
{{order by t.v2, t.pk3}}, {{OrderPreservingTracker. IsOrderPreserving}} returns 
false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from "select v1, v2, pk3" based on "order by t.v2, t.pk3". This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns. That is to say, when it finds that {{v1}} in {{select v1, v2, pk3}} 
does not appear in {{order by t.v2, t.pk3}}, {{OrderPreservingTracker. 
IsOrderPreserving}} returns false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from {{select v1, v2, pk3}} based on {{order by t.v2, t.pk3}}. This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns. That is to say, when it finds that {{v1}} in {{select v1, v2, pk3}} 
does not appear in {{order by t.v2, t.pk3}}, {{OrderPreservingTracker. 
IsOrderPreserving}} returns false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} to make {{OrderPreservingTracker 
getOrderPreservingTrackInfos}} could also extract partial continuous ordering 
columns which start from the first column even if 
{{OrderPreservingTracker#isOrderPreserving}} is false.  That is, even if {{v1}} 
in {{select v1, v2, pk3}} does not appear in {{order by t.v2, t.pk3}}, it will 
return the ordered {{v2,pk3}}.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from {{select v1, v2, pk3}} based on {{order by t.v2, t.pk3}}. This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns. That is to say, when it finds that {{v1}} in {{select v1, v2, pk3}} 
does not appear in {{order by t.v2, t.pk3}}, {{OrderPreservingTracker. 
IsOrderPreserving}} returns false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} and even in 
{{OrderPreservingTracker. IsOrderPreserving}} is false, 
{{OrderPreservingTracker getOrderPreservingTrackInfos}} could also return 
partial of a sequence, That is, even if v1 in {{select v1, v2,pk3}} does not 
appear in {{order by t.v2, t.pk3}}, it will return the ordered {{v2,pk3}}


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table:
> 

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from {{select v1, v2, pk3}} based on {{order by t.v2, t.pk3}}. This 
work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}} . But for 
{{OrderPreservingTracker}}, it could not support extracting partial ordering 
columns. That is to say, when it finds that {{v1}} in {{select v1, v2, pk3}} 
does not appear in {{order by t.v2, t.pk3}}, {{OrderPreservingTracker. 
IsOrderPreserving}} returns false, and 
{{OrderPreservingTracker. getOrderPreservingTrackInfos}} returns an empty list 
in {{TupleProjectionPlan}. 
We should modify {{OrderPreservingTracker}} and even in 
{{OrderPreservingTracker. IsOrderPreserving}} is false, 
{{OrderPreservingTracker getOrderPreservingTrackInfos}} could also return 
partial of a sequence, That is, even if v1 in {{select v1, v2,pk3}} does not 
appear in {{order by t.v2, t.pk3}}, it will return the ordered {{v2,pk3}}

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from {{select v1, v2, pk3}} based on {{}order by t.v2, t.pk3}}. 
This work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}}, but 
for {{OrderPreservingTracker}}, it could not support 


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table:
> {code:java}
>   create table test ( 
>pk1 varchar not null , 
>pk2 varchar not null, 
>pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
>  pk1,
>  pk2,
>  pk3 ))
> {code}
> PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
> v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, 
> t.pk3, t.v1":
> {code:java}
> select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
> t.pk3, t.v1 limit 10) a order by v2, pk3
> {code}
> But if we make a small change to the sql above, just remove the {{t.v1}} in  
> inner query {{OrderBy}} "order by 

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the inner query of the second sql, we infer the output 
{{OrderBy}} from {{select v1, v2, pk3}} based on {{}order by t.v2, t.pk3}}. 
This work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}}, but 
for {{OrderPreservingTracker}}, it could not support 

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the second sql, we infer the output {{OrderBy}} for the 
inner query in {{OrderPreservingTracker}} by {{OrderPreservingTracker}}, but  
{{OrderPreservingTracker}} could not support 


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table:
> {code:java}
>   create table test ( 
>pk1 varchar not null , 
>pk2 varchar not null, 
>pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
>  pk1,
>  pk2,
>  pk3 ))
> {code}
> PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
> v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, 
> t.pk3, t.v1":
> {code:java}
> select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
> t.pk3, t.v1 limit 10) a order by v2, pk3
> {code}
> But if we make a small change to the sql above, just remove the {{t.v1}} in  
> inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
> {code:java}
> select v1 from (select v1, v2, pk3 from test t where pk1 = '6' order by t.v2, 
> t.pk3 limit 10) a order by v2, pk3
> {code}
> Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled 
> out because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
> unfortunately it could not be compiled out now.
> That is because for the inner query of the second sql, we infer the output 
> {{OrderBy}} from {{select v1, v2, pk3}} based on {{}order by t.v2, t.pk3}}. 
> This work is done by {{OrderPreservingTracker}} in {{TupleProjectionPlan}}, 
> but for {{OrderPreservingTracker}}, it could not support 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the second sql, we infer the output {{OrderBy}} for the 
inner query in {{}} by {{OrderPreservingTracker}}, but  
{{OrderPreservingTracker}} could not support 

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table:
> {code:java}
>   create table test ( 
>pk1 varchar not null , 
>pk2 varchar not null, 
>pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
>  pk1,
>  pk2,
>  pk3 ))
> {code}
> PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
> v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, 
> t.pk3, t.v1":
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
> t.pk3, t.v1 limit 10) a order by v2, pk3
> {code}
> But if we make a small change to the sql above, just remove the {{t.v1}} in  
> inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
> t.pk3 limit 10) a order by v2, pk3
> {code}
> Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled 
> out because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
> unfortunately it could not be compiled out now.
> That is because for the second sql, we infer the output {{OrderBy}} for the 
> inner query in {{}} by {{OrderPreservingTracker}}, but  
> {{OrderPreservingTracker}} could not support 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the second sql, we infer the output {{OrderBy}} for the 
inner query in {{OrderPreservingTracker}} by {{OrderPreservingTracker}}, but  
{{OrderPreservingTracker}} could not support 

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

That is because for the second sql, we infer the output {{OrderBy}} for the 
inner query in {{}} by {{OrderPreservingTracker}}, but  
{{OrderPreservingTracker}} could not support 


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table:
> {code:java}
>   create table test ( 
>pk1 varchar not null , 
>pk2 varchar not null, 
>pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
>  pk1,
>  pk2,
>  pk3 ))
> {code}
> PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
> v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, 
> t.pk3, t.v1":
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
> t.pk3, t.v1 limit 10) a order by v2, pk3
> {code}
> But if we make a small change to the sql above, just remove the {{t.v1}} in  
> inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
> t.pk3 limit 10) a order by v2, pk3
> {code}
> Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled 
> out because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
> unfortunately it could not be compiled out now.
> That is because for the second sql, we infer the output {{OrderBy}} for the 
> inner query in {{OrderPreservingTracker}} by {{OrderPreservingTracker}}, but  
> {{OrderPreservingTracker}} could not support 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, t.pk3, 
t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3, t.v1 limit 10) a order by v2, pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
t.pk3 limit 10) a order by v2, pk3
{code}
Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled out 
because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
unfortunately it could not be compiled out now.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2,pk3" because it matchs the inner query {{OrderBy}} "order by 
t.v2,t.pk3,t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3 limit 10) a order by v2,pk3
{code}
Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
unfortunately it could not be compiled out now.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table:
> {code:java}
>   create table test ( 
>pk1 varchar not null , 
>pk2 varchar not null, 
>pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
>  pk1,
>  pk2,
>  pk3 ))
> {code}
> PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
> v2, pk3" because it matches the inner query {{OrderBy}} "order by t.v2, 
> t.pk3, t.v1":
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
> t.pk3, t.v1 limit 10) a order by v2, pk3
> {code}
> But if we make a small change to the sql above, just remove the {{t.v1}} in  
> inner query {{OrderBy}} "order by t.v2, t.pk3, t.v1":
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by t.v2, 
> t.pk3 limit 10) a order by v2, pk3
> {code}
> Obviously, the outer {{OrderBy}} "order by v2,pk3" should still be compiled 
> out because it matches the inner query OrderBy "order by t.v2, t.pk3" , but 
> unfortunately it could not be compiled out now.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2,pk3" because it matchs the inner query {{OrderBy}} "order by 
t.v2,t.pk3,t.v1":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3 limit 10) a order by v2,pk3
{code}
Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
unfortunately it could not be compiled out now.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2,pk3":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3 limit 10) a order by v2,pk3
{code}
Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
unfortunately it could not be compiled out now.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table:
> {code:java}
>   create table test ( 
>pk1 varchar not null , 
>pk2 varchar not null, 
>pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
>  pk1,
>  pk2,
>  pk3 ))
> {code}
> PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
> v2,pk3" because it matchs the inner query {{OrderBy}} "order by 
> t.v2,t.pk3,t.v1":
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
> t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
> {code}
> But if we make a small change to the sql above, just remove the {{t.v1}} in  
> inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
> t.v2,t.pk3 limit 10) a order by v2,pk3
> {code}
> Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
> because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
> unfortunately it could not be compiled out now.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
v2,pk3":
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3 limit 10) a order by v2,pk3
{code}
Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
unfortunately it could not be compiled out now.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  {{order by 
v2,pk3}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3 limit 10) a order by v2,pk3
{code}
Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
unfortunately it could not be compiled out now.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table:
> {code:java}
>   create table test ( 
>pk1 varchar not null , 
>pk2 varchar not null, 
>pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
>  pk1,
>  pk2,
>  pk3 ))
> {code}
> PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  "order by 
> v2,pk3":
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
> t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
> {code}
> But if we make a small change to the sql above, just remove the {{t.v1}} in  
> inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
> t.v2,t.pk3 limit 10) a order by v2,pk3
> {code}
> Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
> because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
> unfortunately it could not be compiled out now.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table:
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  {{order by 
v2,pk3}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3 limit 10) a order by v2,pk3
{code}
Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
unfortunately it could not be compiled out now.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  {{order by 
v2,pk3}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3 limit 10) a order by v2,pk3
{code}
Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
unfortunately it could not be compiled out now.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table:
> {code:java}
>   create table test ( 
>pk1 varchar not null , 
>pk2 varchar not null, 
>pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
>  pk1,
>  pk2,
>  pk3 ))
> {code}
> PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  {{order by 
> v2,pk3}}:
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
> t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
> {code}
> But if we make a small change to the sql above, just remove the {{t.v1}} in  
> inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
> t.v2,t.pk3 limit 10) a order by v2,pk3
> {code}
> Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
> because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
> unfortunately it could not be compiled out now.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  {{order by 
v2,pk3}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we make a small change to the sql above, just remove the {{t.v1}} in  
inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3 limit 10) a order by v2,pk3
{code}
Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
unfortunately it could not be compiled out now.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  {{order by 
v2,pk3}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we , the outer {{OrderBy}} "order by v2,pk3" should be compiled out 
because it matchs the inner query {{OrderBy}} "order by t.v2,t.pk3,t.v1" , but 
unfortunately it is not compiled out.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table
> {code:java}
>   create table test ( 
>pk1 varchar not null , 
>pk2 varchar not null, 
>pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
>  pk1,
>  pk2,
>  pk3 ))
> {code}
> PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  {{order by 
> v2,pk3}}:
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
> t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
> {code}
> But if we make a small change to the sql above, just remove the {{t.v1}} in  
> inner query OrderBy {{order by t.v2,t.pk3,t.v1}}:
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
> t.v2,t.pk3 limit 10) a order by v2,pk3
> {code}
> Obviously, the outer {{OrderBy}} {{order by v2,pk3}} should be compiled out 
> because it matchs the inner query OrderBy {{order by t.v2,t.pk3}} , but 
> unfortunately it could not be compiled out now.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  {{order by 
v2,pk3}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we , the outer {{OrderBy}} "order by v2,pk3" should be compiled out 
because it matchs the inner query {{OrderBy}} "order by t.v2,t.pk3,t.v1" , but 
unfortunately it is not compiled out.

  was:
This is an improvement for  PHOENIX-5148: 
Given a table
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimize following sql to compile outer {{OrderBy}}  {{order by 
v2,pk3}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we , the outer {{OrderBy}} "order by v2,pk3" should be compiled out 
because it matchs the inner query {{OrderBy}} "order by t.v2,t.pk3,t.v1" , but 
unfortunately it is not compiled out.


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table
> {code:java}
>   create table test ( 
>pk1 varchar not null , 
>pk2 varchar not null, 
>pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
>  pk1,
>  pk2,
>  pk3 ))
> {code}
> PHOENIX-5148 optimized following sql to compile outer {{OrderBy}}  {{order by 
> v2,pk3}}:
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
> t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
> {code}
> But if we , the outer {{OrderBy}} "order by v2,pk3" should be compiled out 
> because it matchs the inner query {{OrderBy}} "order by t.v2,t.pk3,t.v1" , 
> but unfortunately it is not compiled out.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: 
This is an improvement for  PHOENIX-5148: 
Given a table
{code:java}
  create table test ( 
   pk1 varchar not null , 
   pk2 varchar not null, 
   pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
   CONSTRAINT TEST_PK PRIMARY KEY ( 
 pk1,
 pk2,
 pk3 ))
{code}
PHOENIX-5148 optimize following sql to compile outer {{OrderBy}}  {{order by 
v2,pk3}}:
{code:java}
select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
{code}
But if we , the outer {{OrderBy}} "order by v2,pk3" should be compiled out 
because it matchs the inner query {{OrderBy}} "order by t.v2,t.pk3,t.v1" , but 
unfortunately it is not compiled out.

  was:This is an improvement for  PHOENIX-5148, 


> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148: 
> Given a table
> {code:java}
>   create table test ( 
>pk1 varchar not null , 
>pk2 varchar not null, 
>pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
>  pk1,
>  pk2,
>  pk3 ))
> {code}
> PHOENIX-5148 optimize following sql to compile outer {{OrderBy}}  {{order by 
> v2,pk3}}:
> {code:java}
> select v1 from (select v1,v2,pk3 from test t where pk1 = '6' order by 
> t.v2,t.pk3,t.v1 limit 10) a order by v2,pk3
> {code}
> But if we , the outer {{OrderBy}} "order by v2,pk3" should be compiled out 
> because it matchs the inner query {{OrderBy}} "order by t.v2,t.pk3,t.v1" , 
> but unfortunately it is not compiled out.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Description: This is an improvement for  PHOENIX-5148, 

> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> This is an improvement for  PHOENIX-5148, 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-7352:
--
Fix Version/s: (was: 5.2.1)

> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

chenglei created PHOENIX-7352:
-

 Summary: Improve OrderPreservingTracker to support extracting 
partial ordering columns for TupleProjectionPlan 
 Key: PHOENIX-7352
 URL: https://issues.apache.org/jira/browse/PHOENIX-7352
 Project: Phoenix
  Issue Type: Improvement
  Components: core
Affects Versions: 5.2.0
Reporter: chenglei
 Fix For: 5.2.1






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (PHOENIX-7352) Improve OrderPreservingTracker to support extracting partial ordering columns for TupleProjectionPlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei reassigned PHOENIX-7352:
-

Assignee: chenglei

> Improve OrderPreservingTracker to support extracting partial ordering columns 
> for TupleProjectionPlan 
> --
>
> Key: PHOENIX-7352
> URL: https://issues.apache.org/jira/browse/PHOENIX-7352
> Project: Phoenix
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.2.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 5.2.1
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (PHOENIX-6800) Remove superfluous semicolon for import statement in UncoveredLocalIndexRegionScanner

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei resolved PHOENIX-6800.
---
  Assignee: chenglei
Resolution: Fixed

> Remove superfluous semicolon for import statement in 
> UncoveredLocalIndexRegionScanner
> -
>
> Key: PHOENIX-6800
> URL: https://issues.apache.org/jira/browse/PHOENIX-6800
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.1.3
>Reporter: chenglei
>Assignee: chenglei
>Priority: Minor
> Fix For: 5.1.3
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (PHOENIX-6800) Remove superfluous semicolon for import statement in UncoveredLocalIndexRegionScanner

[chenglei (Jira)]

chenglei created PHOENIX-6800:
-

 Summary: Remove superfluous semicolon for import statement in 
UncoveredLocalIndexRegionScanner
 Key: PHOENIX-6800
 URL: https://issues.apache.org/jira/browse/PHOENIX-6800
 Project: Phoenix
  Issue Type: Bug
Affects Versions: 5.1.3
Reporter: chenglei
 Fix For: 5.1.3






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-6798) Eliminate unnecessary reversed scan for AggregatePlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6798:
--
Fix Version/s: 5.2.0
   5.1.3

> Eliminate unnecessary reversed scan for AggregatePlan
> -
>
> Key: PHOENIX-6798
> URL: https://issues.apache.org/jira/browse/PHOENIX-6798
> Project: Phoenix
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.1.2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 5.2.0, 5.1.3
>
>
> For HBase, reversed scan is slower than normal forward scan.
> Consider following table:
> {code}
>   create table test_table  
>   (group_id integer not null, 
> keyword varchar not null, 
> cost integer, 
> CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
> {code}
> For the following sql :
> {code}
>select keyword,sum(cost) from test group by keyword order by keyword desc
> {code}
> The compiled {{AggregatePlan}} would set the scan for HBase as reversed scan, 
> but that is very strange, because for this sql, {{GroupBy.isOrderPreserving}} 
> is false, we have no need to set the scan as reversed scan because we have to 
> hash-aggregate the scanned results from HBase in {{Coprocessor}} before 
> sending them to client.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-6798) Eliminate unnecessary reversed scan for AggregatePlan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6798:
--
Summary: Eliminate unnecessary reversed scan for AggregatePlan  (was: 
Eliminate unnecessary reversed scan)

> Eliminate unnecessary reversed scan for AggregatePlan
> -
>
> Key: PHOENIX-6798
> URL: https://issues.apache.org/jira/browse/PHOENIX-6798
> Project: Phoenix
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.1.2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> For HBase, reversed scan is slower than normal forward scan.
> Consider following table:
> {code}
>   create table test_table  
>   (group_id integer not null, 
> keyword varchar not null, 
> cost integer, 
> CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
> {code}
> For the following sql :
> {code}
>select keyword,sum(cost) from test group by keyword order by keyword desc
> {code}
> The compiled {{AggregatePlan}} would set the scan for HBase as reversed scan, 
> but that is very strange, because for this sql, {{GroupBy.isOrderPreserving}} 
> is false, we have no need to set the scan as reversed scan because we have to 
> hash-aggregate the scanned results from HBase in {{Coprocessor}} before 
> sending them to client.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-6798) Eliminate unnecessary reversed scan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6798:
--
Description: 
For HBase, reversed scan is slower than normal forward scan.

Consider following table:
{code}
  create table test_table  
  (group_id integer not null, 
keyword varchar not null, 
cost integer, 
CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
{code}

For the following sql :
{code}
   select keyword,sum(cost) from test group by keyword order by keyword desc
{code}

The compiled {{AggregatePlan}} would set the scan for HBase as reversed scan, 
but that is very strange, because for this sql, {{GroupBy.isOrderPreserving}} 
is false, we have no need to set the scan as reversed scan because we have to 
hash-aggregate the scanned results from HBase in {{Coprocessor}} before sending 
them to client.


  was:
For HBase, reversed scan is slower than normal forward scan.

Consider following table:
{code}
  create table test_table  
  (group_id integer not null, 
keyword varchar not null, 
cost integer, 
CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
{code}

For the following sql :
{code}
   select keyword,sum(cost) from test group by keyword order by keyword desc
{code}

The compiled {{AggregatePlan}} would set the scan for HBase as reversed scan, 
but that is very strange, because for this sql, {{GroupBy.isOrderPreserving}} 
is false, we have no need to set the scan as reversed scan because we have to 
hash-aggregate the scanned results from HBase.



> Eliminate unnecessary reversed scan
> ---
>
> Key: PHOENIX-6798
> URL: https://issues.apache.org/jira/browse/PHOENIX-6798
> Project: Phoenix
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.1.2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> For HBase, reversed scan is slower than normal forward scan.
> Consider following table:
> {code}
>   create table test_table  
>   (group_id integer not null, 
> keyword varchar not null, 
> cost integer, 
> CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
> {code}
> For the following sql :
> {code}
>select keyword,sum(cost) from test group by keyword order by keyword desc
> {code}
> The compiled {{AggregatePlan}} would set the scan for HBase as reversed scan, 
> but that is very strange, because for this sql, {{GroupBy.isOrderPreserving}} 
> is false, we have no need to set the scan as reversed scan because we have to 
> hash-aggregate the scanned results from HBase in {{Coprocessor}} before 
> sending them to client.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-6798) Eliminate unnecessary reversed scan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6798:
--
Description: 
For HBase, reversed scan is slower than normal forward scan.

Consider following table:
{code}
  create table test_table  
  (group_id integer not null, 
keyword varchar not null, 
cost integer, 
CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
{code}

For the following sql :
{code}
   select keyword,sum(cost) from test group by keyword order by keyword desc
{code}

The compiled {{AggregatePlan}} would set the scan for HBase as reversed scan, 
but that is very strange, because for this sql, {{GroupBy.isOrderPreserving}} 
is false, we have no need to set the scan as reversed scan because we have to 
hash-aggregate the scanned results from HBase.


  was:
Consider following table:
{code}
  create table test_table  
  (group_id integer not null, 
keyword varchar not null, 
cost integer, 
CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
{code}

when we query the sql :
{code}
   select keyword,sum(cost) from test group by keyword order by keyword desc
{code}




> Eliminate unnecessary reversed scan
> ---
>
> Key: PHOENIX-6798
> URL: https://issues.apache.org/jira/browse/PHOENIX-6798
> Project: Phoenix
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.1.2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> For HBase, reversed scan is slower than normal forward scan.
> Consider following table:
> {code}
>   create table test_table  
>   (group_id integer not null, 
> keyword varchar not null, 
> cost integer, 
> CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
> {code}
> For the following sql :
> {code}
>select keyword,sum(cost) from test group by keyword order by keyword desc
> {code}
> The compiled {{AggregatePlan}} would set the scan for HBase as reversed scan, 
> but that is very strange, because for this sql, {{GroupBy.isOrderPreserving}} 
> is false, we have no need to set the scan as reversed scan because we have to 
> hash-aggregate the scanned results from HBase.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-6798) Eliminate unnecessary reversed scan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6798:
--
Description: 
Consider following table:
{code}
  create table test_table  
  (group_id integer not null, 
keyword varchar not null, 
cost integer, 
CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
{code}

when we query the sql :
{code}
   select keyword,sum(cost) from test group by keyword order by keyword desc
{code}



  was:
Consider following table and sql:
{code}
   create table test_table  
  (group_id integer not null, 
keyword varchar not null, 
cost integer, 
CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
{code}


> Eliminate unnecessary reversed scan
> ---
>
> Key: PHOENIX-6798
> URL: https://issues.apache.org/jira/browse/PHOENIX-6798
> Project: Phoenix
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.1.2
>Reporter: chenglei
>Priority: Major
>
> Consider following table:
> {code}
>   create table test_table  
>   (group_id integer not null, 
> keyword varchar not null, 
> cost integer, 
> CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
> {code}
> when we query the sql :
> {code}
>select keyword,sum(cost) from test group by keyword order by keyword desc
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (PHOENIX-6798) Eliminate unnecessary reversed scan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei reassigned PHOENIX-6798:
-

Assignee: chenglei

> Eliminate unnecessary reversed scan
> ---
>
> Key: PHOENIX-6798
> URL: https://issues.apache.org/jira/browse/PHOENIX-6798
> Project: Phoenix
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.1.2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> Consider following table:
> {code}
>   create table test_table  
>   (group_id integer not null, 
> keyword varchar not null, 
> cost integer, 
> CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
> {code}
> when we query the sql :
> {code}
>select keyword,sum(cost) from test group by keyword order by keyword desc
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-6798) Eliminate unnecessary reversed scan

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6798:
--
Description: 
Consider following table and sql:
{code}
   create table test_table  
  (group_id integer not null, 
keyword varchar not null, 
cost integer, 
CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
{code}

> Eliminate unnecessary reversed scan
> ---
>
> Key: PHOENIX-6798
> URL: https://issues.apache.org/jira/browse/PHOENIX-6798
> Project: Phoenix
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.1.2
>Reporter: chenglei
>Priority: Major
>
> Consider following table and sql:
> {code}
>create table test_table  
>   (group_id integer not null, 
> keyword varchar not null, 
> cost integer, 
> CONSTRAINT TEST_PK PRIMARY KEY (group_id,keyword))
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (PHOENIX-6798) Eliminate unnecessary reversed scan

[chenglei (Jira)]

chenglei created PHOENIX-6798:
-

 Summary: Eliminate unnecessary reversed scan
 Key: PHOENIX-6798
 URL: https://issues.apache.org/jira/browse/PHOENIX-6798
 Project: Phoenix
  Issue Type: Bug
  Components: core
Affects Versions: 5.1.2
Reporter: chenglei






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PHOENIX-6507) DistinctAggregatingResultIterator should keep original tuple order of the AggregatingResultIterator

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6507:
--
Attachment: PHOENIX-6507_v2-master.patch

> DistinctAggregatingResultIterator should keep original tuple order of the 
> AggregatingResultIterator
> ---
>
> Key: PHOENIX-6507
> URL: https://issues.apache.org/jira/browse/PHOENIX-6507
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.16.1, 5.1.2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 4.17.0, 5.2.0
>
> Attachments: PHOENIX-6507_v2-4.x.patch, PHOENIX-6507_v2-master.patch
>
>
> Given following tables :
> {code:java}
>create table test ( 
>   pk1 varchar not null , 
>   pk2 varchar not null, 
>   pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>   CONSTRAINT TEST_PK PRIMARY KEY ( pk1,pk2,pk3))
> {code}
> The result of the following sql may be incorrect:
> {code:java}
>select distinct pk1,max(v1) from test group by pk1,pk2,pk3 order by pk1
> {code}
> The problem is the {{order by}} is complied out because it is the prefix of 
> the {{group by}}, but for {{distinct}},  
> {{DistinctAggregatingResultIterator}} using {{HashSet}} to rearrange the 
> aggregated tuples.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6507) DistinctAggregatingResultIterator should keep original tuple order of the AggregatingResultIterator

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6507:
--
Attachment: PHOENIX-6507_v2-4.x.patch

> DistinctAggregatingResultIterator should keep original tuple order of the 
> AggregatingResultIterator
> ---
>
> Key: PHOENIX-6507
> URL: https://issues.apache.org/jira/browse/PHOENIX-6507
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.16.1, 5.1.2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 4.17.0, 5.2.0
>
> Attachments: PHOENIX-6507_v2-4.x.patch, PHOENIX-6507_v2-master.patch
>
>
> Given following tables :
> {code:java}
>create table test ( 
>   pk1 varchar not null , 
>   pk2 varchar not null, 
>   pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>   CONSTRAINT TEST_PK PRIMARY KEY ( pk1,pk2,pk3))
> {code}
> The result of the following sql may be incorrect:
> {code:java}
>select distinct pk1,max(v1) from test group by pk1,pk2,pk3 order by pk1
> {code}
> The problem is the {{order by}} is complied out because it is the prefix of 
> the {{group by}}, but for {{distinct}},  
> {{DistinctAggregatingResultIterator}} using {{HashSet}} to rearrange the 
> aggregated tuples.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (PHOENIX-6507) DistinctAggregatingResultIterator should keep original tuple order of the AggregatingResultIterator

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei reassigned PHOENIX-6507:
-

Assignee: chenglei

> DistinctAggregatingResultIterator should keep original tuple order of the 
> AggregatingResultIterator
> ---
>
> Key: PHOENIX-6507
> URL: https://issues.apache.org/jira/browse/PHOENIX-6507
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.16.1, 5.1.2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 4.17.0, 5.2.0
>
>
> Given following tables :
> {code:java}
>create table test ( 
>   pk1 varchar not null , 
>   pk2 varchar not null, 
>   pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>   CONSTRAINT TEST_PK PRIMARY KEY ( pk1,pk2,pk3))
> {code}
> The result of the following sql may be incorrect:
> {code:java}
>select distinct pk1,max(v1) from test group by pk1,pk2,pk3 order by pk1
> {code}
> The problem is the {{order by}} is complied out because it is the prefix of 
> the {{group by}}, but for {{distinct}},  
> {{DistinctAggregatingResultIterator}} using {{HashSet}} to rearrange the 
> aggregated tuples.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6507) DistinctAggregatingResultIterator should keep original tuple order of the AggregatingResultIterator

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6507:
--
Description: 
Given following tables :
{code:java}
   create table test ( 
  pk1 varchar not null , 
  pk2 varchar not null, 
  pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
  CONSTRAINT TEST_PK PRIMARY KEY ( pk1,pk2,pk3))
{code}

The result of the following sql may be incorrect:
{code:java}
   select distinct pk1,max(v1) from test group by pk1,pk2,pk3 order by pk1
{code}

The problem is the {{order by}} is complied out because it is the prefix of the 
{{group by}}, but for {{distinct}},  {{DistinctAggregatingResultIterator}} 
using {{HashSet}} to rearrange the aggregated tuples.


  was:
Given following tables :
{code:java}
   create table test ( 
  pk1 varchar not null , 
  pk2 varchar not null, 
  pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
  CONSTRAINT TEST_PK PRIMARY KEY ( pk1,pk2,pk3))
{code}

The result of the following sql may be incorrect:
{code:java}
   select distinct pk1,max(v1) from test group by pk1,pk2,pk3 order by pk1
{code}

That is because 



> DistinctAggregatingResultIterator should keep original tuple order of the 
> AggregatingResultIterator
> ---
>
> Key: PHOENIX-6507
> URL: https://issues.apache.org/jira/browse/PHOENIX-6507
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.16.1, 5.1.2
>Reporter: chenglei
>Priority: Major
> Fix For: 4.17.0, 5.2.0
>
>
> Given following tables :
> {code:java}
>create table test ( 
>   pk1 varchar not null , 
>   pk2 varchar not null, 
>   pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>   CONSTRAINT TEST_PK PRIMARY KEY ( pk1,pk2,pk3))
> {code}
> The result of the following sql may be incorrect:
> {code:java}
>select distinct pk1,max(v1) from test group by pk1,pk2,pk3 order by pk1
> {code}
> The problem is the {{order by}} is complied out because it is the prefix of 
> the {{group by}}, but for {{distinct}},  
> {{DistinctAggregatingResultIterator}} using {{HashSet}} to rearrange the 
> aggregated tuples.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6507) DistinctAggregatingResultIterator should keep original tuple order of the AggregatingResultIterator

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6507:
--
Description: 
Given following tables :
{code:java}
   create table test ( 
  pk1 varchar not null , 
  pk2 varchar not null, 
  pk3 varchar not null,
   v1 varchar, 
   v2 varchar, 
  CONSTRAINT TEST_PK PRIMARY KEY ( pk1,pk2,pk3))
{code}

The result of the following sql may be incorrect:
{code:java}
   select distinct pk1,max(v1) from test group by pk1,pk2,pk3 order by pk1
{code}

That is because 


> DistinctAggregatingResultIterator should keep original tuple order of the 
> AggregatingResultIterator
> ---
>
> Key: PHOENIX-6507
> URL: https://issues.apache.org/jira/browse/PHOENIX-6507
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.16.1, 5.1.2
>Reporter: chenglei
>Priority: Major
> Fix For: 4.17.0, 5.2.0
>
>
> Given following tables :
> {code:java}
>create table test ( 
>   pk1 varchar not null , 
>   pk2 varchar not null, 
>   pk3 varchar not null,
>v1 varchar, 
>v2 varchar, 
>   CONSTRAINT TEST_PK PRIMARY KEY ( pk1,pk2,pk3))
> {code}
> The result of the following sql may be incorrect:
> {code:java}
>select distinct pk1,max(v1) from test group by pk1,pk2,pk3 order by pk1
> {code}
> That is because 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (PHOENIX-6507) DistinctAggregatingResultIterator should keep original tuple order of the AggregatingResultIterator

[chenglei (Jira)]

chenglei created PHOENIX-6507:
-

 Summary: DistinctAggregatingResultIterator should keep original 
tuple order of the AggregatingResultIterator
 Key: PHOENIX-6507
 URL: https://issues.apache.org/jira/browse/PHOENIX-6507
 Project: Phoenix
  Issue Type: Bug
Affects Versions: 5.1.2, 4.16.1
Reporter: chenglei
 Fix For: 4.17.0, 5.2.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (PHOENIX-6498) Fix incorrect Correlated Exists Subquery rewrite when Subquery is aggregate

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei reassigned PHOENIX-6498:
-

Assignee: chenglei

> Fix incorrect Correlated Exists Subquery rewrite when Subquery is aggregate
> ---
>
> Key: PHOENIX-6498
> URL: https://issues.apache.org/jira/browse/PHOENIX-6498
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.16.1, 5.1.2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 4.17.0, 5.2.0
>
>
> Given following tables :
> {code:java}
>  create table item
>(item_id varchar not null primary key, 
> name varchar, 
> price integer, 
> discount1 integer, 
> discount2 integer, 
> supplier_id varchar, 
>description varchar)
>  
>  create table order
>  (order_id varchar not null primary key, 
>customer_id varchar, 
>item_id varchar, 
>price integer,
>quantity integer, 
>date timestamp)
> {code}
> for the  correlated exists subquery:
> {code:java}
> SELECT item_id, name FROM  item i WHERE exists
> (SELECT 1 FROM  order o  where o.item_id = i.item_id 
> group by customer_id having count(order_id) > 1) ORDER BY name
> {code} 
> Phoenix would throw following exception:
> {code:java}
> java.sql.SQLException: ERROR 1018 (42Y27): Aggregate may not contain columns 
> not in GROUP BY. ITEM_ID
>   at 
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:606)
>   at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:217)
>   at 
> org.apache.phoenix.compile.ExpressionCompiler.throwNonAggExpressionInAggException(ExpressionCompiler.java:1090)
>   at 
> org.apache.phoenix.compile.ProjectionCompiler.compile(ProjectionCompiler.java:445)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:755)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:674)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:251)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:178)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:661)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:289)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:370)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:178)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:672)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:2011)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:2004)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1998)
> {code}
> That is because Phoenix incorrectly rewrite the  subquery as:
> {code:java}
>SELECT DISTINCT 1 $3,O.ITEM_ID $2 FROM ORDER_TABLE O  GROUP BY CUSTOMER_ID 
> HAVING  COUNT(ORDER_ID) > 1
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6498) Fix incorrect Correlated Exists Subquery rewrite when Subquery is aggregate

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6498:
--
Description: 
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
 create table order
 (order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the  correlated exists subquery:
{code:java}
SELECT item_id, name FROM  item i WHERE exists
(SELECT 1 FROM  order o  where o.item_id = i.item_id group 
by customer_id having count(order_id) > 1) ORDER BY name
{code} 

Phoenix would throw following exception:
{code:java}
java.sql.SQLException: ERROR 1018 (42Y27): Aggregate may not contain columns 
not in GROUP BY. ITEM_ID
at 
org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:606)
at 
org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:217)
at 
org.apache.phoenix.compile.ExpressionCompiler.throwNonAggExpressionInAggException(ExpressionCompiler.java:1090)
at 
org.apache.phoenix.compile.ProjectionCompiler.compile(ProjectionCompiler.java:445)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:755)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:674)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:251)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:178)
at 
org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:661)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:289)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:370)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:178)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:672)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:2011)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:2004)
at 
org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1998)

{code}

That is because Phoenix incorrectly rewrite the  subquery as:
{code:java}
   SELECT DISTINCT 1 $3,O.ITEM_ID $2 FROM ORDER_TABLE O  GROUP BY CUSTOMER_ID 
HAVING  COUNT(ORDER_ID) > 1
{code}

> Fix incorrect Correlated Exists Subquery rewrite when Subquery is aggregate
> ---
>
> Key: PHOENIX-6498
> URL: https://issues.apache.org/jira/browse/PHOENIX-6498
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.16.1, 5.1.2
>Reporter: chenglei
>Priority: Major
> Fix For: 4.17.0, 5.2.0
>
>
> Given following tables :
> {code:java}
>  create table item
>(item_id varchar not null primary key, 
> name varchar, 
> price integer, 
> discount1 integer, 
> discount2 integer, 
> supplier_id varchar, 
>description varchar)
>  
>  create table order
>  (order_id varchar not null primary key, 
>customer_id varchar, 
>item_id varchar, 
>price integer,
>quantity integer, 
>date timestamp)
> {code}
> for the  correlated exists subquery:
> {code:java}
> SELECT item_id, name FROM  item i WHERE exists
> (SELECT 1 FROM  order o  where o.item_id = i.item_id 
> group by customer_id having count(order_id) > 1) ORDER BY name
> {code} 
> Phoenix would throw following exception:
> {code:java}
> java.sql.SQLException: ERROR 1018 (42Y27): Aggregate may not contain columns 
> not in GROUP BY. ITEM_ID
>   at 
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:606)
>   at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:217)
>   at 
> org.apache.phoenix.compile.ExpressionCompiler.throwNonAggExpressionInAggException(ExpressionCompiler.java:1090)
>   at 
> 

[jira] [Created] (PHOENIX-6498) Fix incorrect Correlated Exists Subquery rewrite when Subquery is aggregate

[chenglei (Jira)]

chenglei created PHOENIX-6498:
-

 Summary: Fix incorrect Correlated Exists Subquery rewrite when 
Subquery is aggregate
 Key: PHOENIX-6498
 URL: https://issues.apache.org/jira/browse/PHOENIX-6498
 Project: Phoenix
  Issue Type: Bug
Affects Versions: 5.1.2, 4.16.1
Reporter: chenglei
 Fix For: 4.17.0, 5.2.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (PHOENIX-6287) Fix incorrect log in ParallelIterators.submitWork

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei reassigned PHOENIX-6287:
-

Assignee: chenglei

> Fix incorrect log  in ParallelIterators.submitWork
> --
>
> Key: PHOENIX-6287
> URL: https://issues.apache.org/jira/browse/PHOENIX-6287
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.1.0, 4.16.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 4.16.0, 5.10
>
> Attachments: PHOENIX-6287-4.x.patch
>
>
> In {{ParallelIterators.submitWork}}, the following log is critical but is 
> meaningless:
> {code:java}
> 123@Override
> 124public PeekingResultIterator call() throws Exception {
> 125long startTime = 
> EnvironmentEdgeManager.currentTimeMillis();
> 126if (LOGGER.isDebugEnabled()) {
> 127LOGGER.debug(LogUtil.addCustomAnnotations("Id: " + 
> scanId + ", Time: " +
> 128(EnvironmentEdgeManager.currentTimeMillis() - 
> startTime) +
> 129"ms, Scan: " + scan, 
> ScanUtil.getCustomAnnotations(scan)));
> 130}
> {code}
> The log should be placed after the {{iterator.peek}} is called.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6287) Fix incorrect log in ParallelIterators.submitWork

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6287:
--
Attachment: PHOENIX-6287-4.x.patch

> Fix incorrect log  in ParallelIterators.submitWork
> --
>
> Key: PHOENIX-6287
> URL: https://issues.apache.org/jira/browse/PHOENIX-6287
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.1.0, 4.16.0
>Reporter: chenglei
>Priority: Major
> Fix For: 4.16.0, 5.10
>
> Attachments: PHOENIX-6287-4.x.patch
>
>
> In {{ParallelIterators.submitWork}}, the following log is critical but is 
> meaningless:
> {code:java}
> 123@Override
> 124public PeekingResultIterator call() throws Exception {
> 125long startTime = 
> EnvironmentEdgeManager.currentTimeMillis();
> 126if (LOGGER.isDebugEnabled()) {
> 127LOGGER.debug(LogUtil.addCustomAnnotations("Id: " + 
> scanId + ", Time: " +
> 128(EnvironmentEdgeManager.currentTimeMillis() - 
> startTime) +
> 129"ms, Scan: " + scan, 
> ScanUtil.getCustomAnnotations(scan)));
> 130}
> {code}
> The log should be placed after the {{iterator.peek}} is called.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6287) Fix incorrect log in ParallelIterators.submitWork

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6287:
--
Description: 
In {{ParallelIterators.submitWork}}, the following log is critical but is 
meaningless:
{code:java}
123@Override
124public PeekingResultIterator call() throws Exception {
125long startTime = 
EnvironmentEdgeManager.currentTimeMillis();
126if (LOGGER.isDebugEnabled()) {
127LOGGER.debug(LogUtil.addCustomAnnotations("Id: " + 
scanId + ", Time: " +
128(EnvironmentEdgeManager.currentTimeMillis() - 
startTime) +
129"ms, Scan: " + scan, 
ScanUtil.getCustomAnnotations(scan)));
130}
{code}

The log should be placed after the {{iterator.peek}} is called.

  was:In {{}}


> Fix incorrect log  in ParallelIterators.submitWork
> --
>
> Key: PHOENIX-6287
> URL: https://issues.apache.org/jira/browse/PHOENIX-6287
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.1.0, 4.16.0
>Reporter: chenglei
>Priority: Major
> Fix For: 4.16.0, 5.10
>
>
> In {{ParallelIterators.submitWork}}, the following log is critical but is 
> meaningless:
> {code:java}
> 123@Override
> 124public PeekingResultIterator call() throws Exception {
> 125long startTime = 
> EnvironmentEdgeManager.currentTimeMillis();
> 126if (LOGGER.isDebugEnabled()) {
> 127LOGGER.debug(LogUtil.addCustomAnnotations("Id: " + 
> scanId + ", Time: " +
> 128(EnvironmentEdgeManager.currentTimeMillis() - 
> startTime) +
> 129"ms, Scan: " + scan, 
> ScanUtil.getCustomAnnotations(scan)));
> 130}
> {code}
> The log should be placed after the {{iterator.peek}} is called.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6287) Fix incorrect log in ParallelIterators.submitWork

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6287:
--
Description: In {{}}

> Fix incorrect log  in ParallelIterators.submitWork
> --
>
> Key: PHOENIX-6287
> URL: https://issues.apache.org/jira/browse/PHOENIX-6287
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.1.0, 4.16.0
>Reporter: chenglei
>Priority: Major
> Fix For: 4.16.0, 5.10
>
>
> In {{}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (PHOENIX-6287) Fix incorrect log in ParallelIterators.submitWork

[chenglei (Jira)]

chenglei created PHOENIX-6287:
-

 Summary: Fix incorrect log  in ParallelIterators.submitWork
 Key: PHOENIX-6287
 URL: https://issues.apache.org/jira/browse/PHOENIX-6287
 Project: Phoenix
  Issue Type: Bug
Affects Versions: 5.1.0, 4.16.0
Reporter: chenglei
 Fix For: 4.16.0, 5.10






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6232) Correlated subquery should not push to RegionServer as the probe side of the Hash join

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6232:
--
Attachment: PHOENIX-6232_addendum-master.diff

> Correlated subquery should not push to RegionServer as the probe side of the 
> Hash join
> --
>
> Key: PHOENIX-6232
> URL: https://issues.apache.org/jira/browse/PHOENIX-6232
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: Mate Szalay-Beko
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6232_addendum-4.x.patch, 
> PHOENIX-6232_addendum-master.diff, PHOENIX-6232_v1-4.x.patch, 
> PHOENIX-6232_v1-master.patch
>
>
> We were facing an interesting problem when a more complex query (with inner 
> selects in the WHERE clause) succeeds alone, while the same query fails, if 
> it is part of a join. I created a test table / query to reproduce the problem:
> {code:sql}
> DROP TABLE IF EXISTS test;
> CREATE TABLE test (
>   id INTEGER NOT NULL,
>   test_id INTEGER,
>   lastchanged TIMESTAMP,
>   CONSTRAINT my_pk PRIMARY KEY (id));
> UPSERT INTO test VALUES(0, 100, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(1, 101, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(2, 100, '2011-11-11 11:11:11.0');
> {code}
> *Query 1:* Example query, running fine in itself:
> {code:sql}
> SELECT id, test_id, lastchanged FROM test T
> WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> Returns:
> ++-+---+
> | ID | TEST_ID |  LASTCHANGED  |
> ++-+---+
> | 1  | 101 | 2000-01-01 01:00:00.0 |
> | 2  | 100 | 2011-11-11 12:11:11.0 |
> ++-+---+
> {code}
> *Query 2:* Same query fails on the current master branch, when it is part of 
> a larger (implicit) join:
> {code:sql}
> SELECT AAA.*
> FROM 
> (
>   SELECT id, test_id, lastchanged FROM test T
>   WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> ) as AAA,
> (
>   SELECT id FROM test
> ) as BBB
> WHERE AAA.id = BBB.id;
> java.lang.IllegalArgumentException
>   at 
> org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
>   at 
> org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:66)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:663)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:404)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:176)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:504)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:467)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:309)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:298)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:297)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1933)
>   at sqlline.Commands.executeSingleQuery(Commands.java:1054)
>   at sqlline.Commands.execute(Commands.java:1003)
>   at sqlline.Commands.sql(Commands.java:967)
>   at sqlline.SqlLine.dispatch(SqlLine.java:734)
>   at sqlline.SqlLine.begin(SqlLine.java:541)
>   at sqlline.SqlLine.start(SqlLine.java:267)
>   at sqlline.SqlLine.main(SqlLine.java:206)
> {code}
> I am not sure what the problem is exactly. My guess is that Phoenix tries to 
> optimize (flatten) an inner-query, which it shouldn't, if we are inside a 
> join (according to the check in the code which throws the exception).
> The best workaround I found was to define an explicit join in the original 
> query (Query 1), basically change the inner select into a join. This modified 
> query return the same as the original one:
> *Query 3:*
> {code:sql}
> SELECT T.id, T.test_id, T.lastchanged 
> FROM 
>   test T 
>   LEFT JOIN (
> SELECT max(lastchanged) AS max_timestamp, 
>test_id AS max_timestamp_test_id
> FROM test
> GROUP BY test_id
>   ) JOIN_TABLE ON 

[jira] [Updated] (PHOENIX-6232) Correlated subquery should not push to RegionServer as the probe side of the Hash join

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6232:
--
Attachment: PHOENIX-6232_addendum-4.x.patch

> Correlated subquery should not push to RegionServer as the probe side of the 
> Hash join
> --
>
> Key: PHOENIX-6232
> URL: https://issues.apache.org/jira/browse/PHOENIX-6232
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: Mate Szalay-Beko
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6232_addendum-4.x.patch, 
> PHOENIX-6232_addendum-master.diff, PHOENIX-6232_v1-4.x.patch, 
> PHOENIX-6232_v1-master.patch
>
>
> We were facing an interesting problem when a more complex query (with inner 
> selects in the WHERE clause) succeeds alone, while the same query fails, if 
> it is part of a join. I created a test table / query to reproduce the problem:
> {code:sql}
> DROP TABLE IF EXISTS test;
> CREATE TABLE test (
>   id INTEGER NOT NULL,
>   test_id INTEGER,
>   lastchanged TIMESTAMP,
>   CONSTRAINT my_pk PRIMARY KEY (id));
> UPSERT INTO test VALUES(0, 100, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(1, 101, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(2, 100, '2011-11-11 11:11:11.0');
> {code}
> *Query 1:* Example query, running fine in itself:
> {code:sql}
> SELECT id, test_id, lastchanged FROM test T
> WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> Returns:
> ++-+---+
> | ID | TEST_ID |  LASTCHANGED  |
> ++-+---+
> | 1  | 101 | 2000-01-01 01:00:00.0 |
> | 2  | 100 | 2011-11-11 12:11:11.0 |
> ++-+---+
> {code}
> *Query 2:* Same query fails on the current master branch, when it is part of 
> a larger (implicit) join:
> {code:sql}
> SELECT AAA.*
> FROM 
> (
>   SELECT id, test_id, lastchanged FROM test T
>   WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> ) as AAA,
> (
>   SELECT id FROM test
> ) as BBB
> WHERE AAA.id = BBB.id;
> java.lang.IllegalArgumentException
>   at 
> org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
>   at 
> org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:66)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:663)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:404)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:176)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:504)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:467)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:309)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:298)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:297)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1933)
>   at sqlline.Commands.executeSingleQuery(Commands.java:1054)
>   at sqlline.Commands.execute(Commands.java:1003)
>   at sqlline.Commands.sql(Commands.java:967)
>   at sqlline.SqlLine.dispatch(SqlLine.java:734)
>   at sqlline.SqlLine.begin(SqlLine.java:541)
>   at sqlline.SqlLine.start(SqlLine.java:267)
>   at sqlline.SqlLine.main(SqlLine.java:206)
> {code}
> I am not sure what the problem is exactly. My guess is that Phoenix tries to 
> optimize (flatten) an inner-query, which it shouldn't, if we are inside a 
> join (according to the check in the code which throws the exception).
> The best workaround I found was to define an explicit join in the original 
> query (Query 1), basically change the inner select into a join. This modified 
> query return the same as the original one:
> *Query 3:*
> {code:sql}
> SELECT T.id, T.test_id, T.lastchanged 
> FROM 
>   test T 
>   LEFT JOIN (
> SELECT max(lastchanged) AS max_timestamp, 
>test_id AS max_timestamp_test_id
> FROM test
> GROUP BY test_id
>   ) JOIN_TABLE ON 

[jira] [Updated] (PHOENIX-6232) Correlated subquery should not push to RegionServer as the probe side of the Hash join

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6232:
--
Summary: Correlated subquery should not push to RegionServer as the probe 
side of the Hash join  (was: Correlated subquery should not push to 
RegionServer as the probe side of the Hash join.)

> Correlated subquery should not push to RegionServer as the probe side of the 
> Hash join
> --
>
> Key: PHOENIX-6232
> URL: https://issues.apache.org/jira/browse/PHOENIX-6232
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: Mate Szalay-Beko
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6232_v1-4.x.patch, PHOENIX-6232_v1-master.patch
>
>
> We were facing an interesting problem when a more complex query (with inner 
> selects in the WHERE clause) succeeds alone, while the same query fails, if 
> it is part of a join. I created a test table / query to reproduce the problem:
> {code:sql}
> DROP TABLE IF EXISTS test;
> CREATE TABLE test (
>   id INTEGER NOT NULL,
>   test_id INTEGER,
>   lastchanged TIMESTAMP,
>   CONSTRAINT my_pk PRIMARY KEY (id));
> UPSERT INTO test VALUES(0, 100, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(1, 101, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(2, 100, '2011-11-11 11:11:11.0');
> {code}
> *Query 1:* Example query, running fine in itself:
> {code:sql}
> SELECT id, test_id, lastchanged FROM test T
> WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> Returns:
> ++-+---+
> | ID | TEST_ID |  LASTCHANGED  |
> ++-+---+
> | 1  | 101 | 2000-01-01 01:00:00.0 |
> | 2  | 100 | 2011-11-11 12:11:11.0 |
> ++-+---+
> {code}
> *Query 2:* Same query fails on the current master branch, when it is part of 
> a larger (implicit) join:
> {code:sql}
> SELECT AAA.*
> FROM 
> (
>   SELECT id, test_id, lastchanged FROM test T
>   WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> ) as AAA,
> (
>   SELECT id FROM test
> ) as BBB
> WHERE AAA.id = BBB.id;
> java.lang.IllegalArgumentException
>   at 
> org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
>   at 
> org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:66)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:663)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:404)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:176)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:504)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:467)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:309)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:298)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:297)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1933)
>   at sqlline.Commands.executeSingleQuery(Commands.java:1054)
>   at sqlline.Commands.execute(Commands.java:1003)
>   at sqlline.Commands.sql(Commands.java:967)
>   at sqlline.SqlLine.dispatch(SqlLine.java:734)
>   at sqlline.SqlLine.begin(SqlLine.java:541)
>   at sqlline.SqlLine.start(SqlLine.java:267)
>   at sqlline.SqlLine.main(SqlLine.java:206)
> {code}
> I am not sure what the problem is exactly. My guess is that Phoenix tries to 
> optimize (flatten) an inner-query, which it shouldn't, if we are inside a 
> join (according to the check in the code which throws the exception).
> The best workaround I found was to define an explicit join in the original 
> query (Query 1), basically change the inner select into a join. This modified 
> query return the same as the original one:
> *Query 3:*
> {code:sql}
> SELECT T.id, T.test_id, T.lastchanged 
> FROM 
>   test T 
>   LEFT JOIN (
> SELECT max(lastchanged) AS max_timestamp, 
>test_id AS max_timestamp_test_id
>  

[jira] [Updated] (PHOENIX-6232) Correlated subquery should not push to RegionServer as the probe side of the Hash join.

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6232:
--
Attachment: PHOENIX-6232_v1-master.patch

> Correlated subquery should not push to RegionServer as the probe side of the 
> Hash join.
> ---
>
> Key: PHOENIX-6232
> URL: https://issues.apache.org/jira/browse/PHOENIX-6232
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: Mate Szalay-Beko
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6232_v1-4.x.patch, PHOENIX-6232_v1-master.patch
>
>
> We were facing an interesting problem when a more complex query (with inner 
> selects in the WHERE clause) succeeds alone, while the same query fails, if 
> it is part of a join. I created a test table / query to reproduce the problem:
> {code:sql}
> DROP TABLE IF EXISTS test;
> CREATE TABLE test (
>   id INTEGER NOT NULL,
>   test_id INTEGER,
>   lastchanged TIMESTAMP,
>   CONSTRAINT my_pk PRIMARY KEY (id));
> UPSERT INTO test VALUES(0, 100, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(1, 101, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(2, 100, '2011-11-11 11:11:11.0');
> {code}
> *Query 1:* Example query, running fine in itself:
> {code:sql}
> SELECT id, test_id, lastchanged FROM test T
> WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> Returns:
> ++-+---+
> | ID | TEST_ID |  LASTCHANGED  |
> ++-+---+
> | 1  | 101 | 2000-01-01 01:00:00.0 |
> | 2  | 100 | 2011-11-11 12:11:11.0 |
> ++-+---+
> {code}
> *Query 2:* Same query fails on the current master branch, when it is part of 
> a larger (implicit) join:
> {code:sql}
> SELECT AAA.*
> FROM 
> (
>   SELECT id, test_id, lastchanged FROM test T
>   WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> ) as AAA,
> (
>   SELECT id FROM test
> ) as BBB
> WHERE AAA.id = BBB.id;
> java.lang.IllegalArgumentException
>   at 
> org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
>   at 
> org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:66)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:663)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:404)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:176)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:504)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:467)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:309)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:298)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:297)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1933)
>   at sqlline.Commands.executeSingleQuery(Commands.java:1054)
>   at sqlline.Commands.execute(Commands.java:1003)
>   at sqlline.Commands.sql(Commands.java:967)
>   at sqlline.SqlLine.dispatch(SqlLine.java:734)
>   at sqlline.SqlLine.begin(SqlLine.java:541)
>   at sqlline.SqlLine.start(SqlLine.java:267)
>   at sqlline.SqlLine.main(SqlLine.java:206)
> {code}
> I am not sure what the problem is exactly. My guess is that Phoenix tries to 
> optimize (flatten) an inner-query, which it shouldn't, if we are inside a 
> join (according to the check in the code which throws the exception).
> The best workaround I found was to define an explicit join in the original 
> query (Query 1), basically change the inner select into a join. This modified 
> query return the same as the original one:
> *Query 3:*
> {code:sql}
> SELECT T.id, T.test_id, T.lastchanged 
> FROM 
>   test T 
>   LEFT JOIN (
> SELECT max(lastchanged) AS max_timestamp, 
>test_id AS max_timestamp_test_id
> FROM test
> GROUP BY test_id
>   ) JOIN_TABLE ON JOIN_TABLE.max_timestamp_test_id = T.test_id
> WHERE T.lastchanged = JOIN_TABLE.max_timestamp

[jira] [Updated] (PHOENIX-6232) Correlated subquery should not push to RegionServer as the probe side of the Hash join.

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6232:
--
Summary: Correlated subquery should not push to RegionServer as the probe 
side of the Hash join.  (was: Correlated subquery should not as the probe side 
of the Hash join.)

> Correlated subquery should not push to RegionServer as the probe side of the 
> Hash join.
> ---
>
> Key: PHOENIX-6232
> URL: https://issues.apache.org/jira/browse/PHOENIX-6232
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: Mate Szalay-Beko
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6232_v1-4.x.patch
>
>
> We were facing an interesting problem when a more complex query (with inner 
> selects in the WHERE clause) succeeds alone, while the same query fails, if 
> it is part of a join. I created a test table / query to reproduce the problem:
> {code:sql}
> DROP TABLE IF EXISTS test;
> CREATE TABLE test (
>   id INTEGER NOT NULL,
>   test_id INTEGER,
>   lastchanged TIMESTAMP,
>   CONSTRAINT my_pk PRIMARY KEY (id));
> UPSERT INTO test VALUES(0, 100, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(1, 101, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(2, 100, '2011-11-11 11:11:11.0');
> {code}
> *Query 1:* Example query, running fine in itself:
> {code:sql}
> SELECT id, test_id, lastchanged FROM test T
> WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> Returns:
> ++-+---+
> | ID | TEST_ID |  LASTCHANGED  |
> ++-+---+
> | 1  | 101 | 2000-01-01 01:00:00.0 |
> | 2  | 100 | 2011-11-11 12:11:11.0 |
> ++-+---+
> {code}
> *Query 2:* Same query fails on the current master branch, when it is part of 
> a larger (implicit) join:
> {code:sql}
> SELECT AAA.*
> FROM 
> (
>   SELECT id, test_id, lastchanged FROM test T
>   WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> ) as AAA,
> (
>   SELECT id FROM test
> ) as BBB
> WHERE AAA.id = BBB.id;
> java.lang.IllegalArgumentException
>   at 
> org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
>   at 
> org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:66)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:663)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:404)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:176)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:504)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:467)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:309)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:298)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:297)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1933)
>   at sqlline.Commands.executeSingleQuery(Commands.java:1054)
>   at sqlline.Commands.execute(Commands.java:1003)
>   at sqlline.Commands.sql(Commands.java:967)
>   at sqlline.SqlLine.dispatch(SqlLine.java:734)
>   at sqlline.SqlLine.begin(SqlLine.java:541)
>   at sqlline.SqlLine.start(SqlLine.java:267)
>   at sqlline.SqlLine.main(SqlLine.java:206)
> {code}
> I am not sure what the problem is exactly. My guess is that Phoenix tries to 
> optimize (flatten) an inner-query, which it shouldn't, if we are inside a 
> join (according to the check in the code which throws the exception).
> The best workaround I found was to define an explicit join in the original 
> query (Query 1), basically change the inner select into a join. This modified 
> query return the same as the original one:
> *Query 3:*
> {code:sql}
> SELECT T.id, T.test_id, T.lastchanged 
> FROM 
>   test T 
>   LEFT JOIN (
> SELECT max(lastchanged) AS max_timestamp, 
>test_id AS max_timestamp_test_id
> FROM test
> GROUP BY test_id
>   ) 

[jira] [Updated] (PHOENIX-6232) Correlated subquery should not as the probe side of the Hash join.

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6232:
--
Summary: Correlated subquery should not as the probe side of the Hash join. 
 (was: Same query fails with IllegalArgumentException if part of a join)

> Correlated subquery should not as the probe side of the Hash join.
> --
>
> Key: PHOENIX-6232
> URL: https://issues.apache.org/jira/browse/PHOENIX-6232
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: Mate Szalay-Beko
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6232_v1-4.x.patch
>
>
> We were facing an interesting problem when a more complex query (with inner 
> selects in the WHERE clause) succeeds alone, while the same query fails, if 
> it is part of a join. I created a test table / query to reproduce the problem:
> {code:sql}
> DROP TABLE IF EXISTS test;
> CREATE TABLE test (
>   id INTEGER NOT NULL,
>   test_id INTEGER,
>   lastchanged TIMESTAMP,
>   CONSTRAINT my_pk PRIMARY KEY (id));
> UPSERT INTO test VALUES(0, 100, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(1, 101, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(2, 100, '2011-11-11 11:11:11.0');
> {code}
> *Query 1:* Example query, running fine in itself:
> {code:sql}
> SELECT id, test_id, lastchanged FROM test T
> WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> Returns:
> ++-+---+
> | ID | TEST_ID |  LASTCHANGED  |
> ++-+---+
> | 1  | 101 | 2000-01-01 01:00:00.0 |
> | 2  | 100 | 2011-11-11 12:11:11.0 |
> ++-+---+
> {code}
> *Query 2:* Same query fails on the current master branch, when it is part of 
> a larger (implicit) join:
> {code:sql}
> SELECT AAA.*
> FROM 
> (
>   SELECT id, test_id, lastchanged FROM test T
>   WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> ) as AAA,
> (
>   SELECT id FROM test
> ) as BBB
> WHERE AAA.id = BBB.id;
> java.lang.IllegalArgumentException
>   at 
> org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
>   at 
> org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:66)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:663)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:404)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:176)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:504)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:467)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:309)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:298)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:297)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1933)
>   at sqlline.Commands.executeSingleQuery(Commands.java:1054)
>   at sqlline.Commands.execute(Commands.java:1003)
>   at sqlline.Commands.sql(Commands.java:967)
>   at sqlline.SqlLine.dispatch(SqlLine.java:734)
>   at sqlline.SqlLine.begin(SqlLine.java:541)
>   at sqlline.SqlLine.start(SqlLine.java:267)
>   at sqlline.SqlLine.main(SqlLine.java:206)
> {code}
> I am not sure what the problem is exactly. My guess is that Phoenix tries to 
> optimize (flatten) an inner-query, which it shouldn't, if we are inside a 
> join (according to the check in the code which throws the exception).
> The best workaround I found was to define an explicit join in the original 
> query (Query 1), basically change the inner select into a join. This modified 
> query return the same as the original one:
> *Query 3:*
> {code:sql}
> SELECT T.id, T.test_id, T.lastchanged 
> FROM 
>   test T 
>   LEFT JOIN (
> SELECT max(lastchanged) AS max_timestamp, 
>test_id AS max_timestamp_test_id
> FROM test
> GROUP BY test_id
>   ) JOIN_TABLE ON JOIN_TABLE.max_timestamp_test_id = T.test_id
> WHERE 

[jira] [Updated] (PHOENIX-6232) Same query fails with IllegalArgumentException if part of a join

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6232:
--
Affects Version/s: 4.15.0

> Same query fails with IllegalArgumentException if part of a join
> 
>
> Key: PHOENIX-6232
> URL: https://issues.apache.org/jira/browse/PHOENIX-6232
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: Mate Szalay-Beko
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6232_v1-4.x.patch
>
>
> We were facing an interesting problem when a more complex query (with inner 
> selects in the WHERE clause) succeeds alone, while the same query fails, if 
> it is part of a join. I created a test table / query to reproduce the problem:
> {code:sql}
> DROP TABLE IF EXISTS test;
> CREATE TABLE test (
>   id INTEGER NOT NULL,
>   test_id INTEGER,
>   lastchanged TIMESTAMP,
>   CONSTRAINT my_pk PRIMARY KEY (id));
> UPSERT INTO test VALUES(0, 100, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(1, 101, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(2, 100, '2011-11-11 11:11:11.0');
> {code}
> *Query 1:* Example query, running fine in itself:
> {code:sql}
> SELECT id, test_id, lastchanged FROM test T
> WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> Returns:
> ++-+---+
> | ID | TEST_ID |  LASTCHANGED  |
> ++-+---+
> | 1  | 101 | 2000-01-01 01:00:00.0 |
> | 2  | 100 | 2011-11-11 12:11:11.0 |
> ++-+---+
> {code}
> *Query 2:* Same query fails on the current master branch, when it is part of 
> a larger (implicit) join:
> {code:sql}
> SELECT AAA.*
> FROM 
> (
>   SELECT id, test_id, lastchanged FROM test T
>   WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> ) as AAA,
> (
>   SELECT id FROM test
> ) as BBB
> WHERE AAA.id = BBB.id;
> java.lang.IllegalArgumentException
>   at 
> org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
>   at 
> org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:66)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:663)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:404)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:176)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:504)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:467)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:309)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:298)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:297)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1933)
>   at sqlline.Commands.executeSingleQuery(Commands.java:1054)
>   at sqlline.Commands.execute(Commands.java:1003)
>   at sqlline.Commands.sql(Commands.java:967)
>   at sqlline.SqlLine.dispatch(SqlLine.java:734)
>   at sqlline.SqlLine.begin(SqlLine.java:541)
>   at sqlline.SqlLine.start(SqlLine.java:267)
>   at sqlline.SqlLine.main(SqlLine.java:206)
> {code}
> I am not sure what the problem is exactly. My guess is that Phoenix tries to 
> optimize (flatten) an inner-query, which it shouldn't, if we are inside a 
> join (according to the check in the code which throws the exception).
> The best workaround I found was to define an explicit join in the original 
> query (Query 1), basically change the inner select into a join. This modified 
> query return the same as the original one:
> *Query 3:*
> {code:sql}
> SELECT T.id, T.test_id, T.lastchanged 
> FROM 
>   test T 
>   LEFT JOIN (
> SELECT max(lastchanged) AS max_timestamp, 
>test_id AS max_timestamp_test_id
> FROM test
> GROUP BY test_id
>   ) JOIN_TABLE ON JOIN_TABLE.max_timestamp_test_id = T.test_id
> WHERE T.lastchanged = JOIN_TABLE.max_timestamp
> Returns:
> +--+---+---+
> | T.ID | T.TEST_ID | 

[jira] [Updated] (PHOENIX-6232) Same query fails with IllegalArgumentException if part of a join

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6232:
--
Fix Version/s: 4.16.0
   5.1.0

> Same query fails with IllegalArgumentException if part of a join
> 
>
> Key: PHOENIX-6232
> URL: https://issues.apache.org/jira/browse/PHOENIX-6232
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Mate Szalay-Beko
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6232_v1-4.x.patch
>
>
> We were facing an interesting problem when a more complex query (with inner 
> selects in the WHERE clause) succeeds alone, while the same query fails, if 
> it is part of a join. I created a test table / query to reproduce the problem:
> {code:sql}
> DROP TABLE IF EXISTS test;
> CREATE TABLE test (
>   id INTEGER NOT NULL,
>   test_id INTEGER,
>   lastchanged TIMESTAMP,
>   CONSTRAINT my_pk PRIMARY KEY (id));
> UPSERT INTO test VALUES(0, 100, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(1, 101, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(2, 100, '2011-11-11 11:11:11.0');
> {code}
> *Query 1:* Example query, running fine in itself:
> {code:sql}
> SELECT id, test_id, lastchanged FROM test T
> WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> Returns:
> ++-+---+
> | ID | TEST_ID |  LASTCHANGED  |
> ++-+---+
> | 1  | 101 | 2000-01-01 01:00:00.0 |
> | 2  | 100 | 2011-11-11 12:11:11.0 |
> ++-+---+
> {code}
> *Query 2:* Same query fails on the current master branch, when it is part of 
> a larger (implicit) join:
> {code:sql}
> SELECT AAA.*
> FROM 
> (
>   SELECT id, test_id, lastchanged FROM test T
>   WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> ) as AAA,
> (
>   SELECT id FROM test
> ) as BBB
> WHERE AAA.id = BBB.id;
> java.lang.IllegalArgumentException
>   at 
> org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
>   at 
> org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:66)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:663)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:404)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:176)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:504)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:467)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:309)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:298)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:297)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1933)
>   at sqlline.Commands.executeSingleQuery(Commands.java:1054)
>   at sqlline.Commands.execute(Commands.java:1003)
>   at sqlline.Commands.sql(Commands.java:967)
>   at sqlline.SqlLine.dispatch(SqlLine.java:734)
>   at sqlline.SqlLine.begin(SqlLine.java:541)
>   at sqlline.SqlLine.start(SqlLine.java:267)
>   at sqlline.SqlLine.main(SqlLine.java:206)
> {code}
> I am not sure what the problem is exactly. My guess is that Phoenix tries to 
> optimize (flatten) an inner-query, which it shouldn't, if we are inside a 
> join (according to the check in the code which throws the exception).
> The best workaround I found was to define an explicit join in the original 
> query (Query 1), basically change the inner select into a join. This modified 
> query return the same as the original one:
> *Query 3:*
> {code:sql}
> SELECT T.id, T.test_id, T.lastchanged 
> FROM 
>   test T 
>   LEFT JOIN (
> SELECT max(lastchanged) AS max_timestamp, 
>test_id AS max_timestamp_test_id
> FROM test
> GROUP BY test_id
>   ) JOIN_TABLE ON JOIN_TABLE.max_timestamp_test_id = T.test_id
> WHERE T.lastchanged = JOIN_TABLE.max_timestamp
> Returns:
> +--+---+---+
> | T.ID | T.TEST_ID | T.LASTCHANGED 

[jira] [Updated] (PHOENIX-6232) Same query fails with IllegalArgumentException if part of a join

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6232:
--
Attachment: PHOENIX-6232_v1-4.x.patch

> Same query fails with IllegalArgumentException if part of a join
> 
>
> Key: PHOENIX-6232
> URL: https://issues.apache.org/jira/browse/PHOENIX-6232
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Mate Szalay-Beko
>Assignee: chenglei
>Priority: Major
> Attachments: PHOENIX-6232_v1-4.x.patch
>
>
> We were facing an interesting problem when a more complex query (with inner 
> selects in the WHERE clause) succeeds alone, while the same query fails, if 
> it is part of a join. I created a test table / query to reproduce the problem:
> {code:sql}
> DROP TABLE IF EXISTS test;
> CREATE TABLE test (
>   id INTEGER NOT NULL,
>   test_id INTEGER,
>   lastchanged TIMESTAMP,
>   CONSTRAINT my_pk PRIMARY KEY (id));
> UPSERT INTO test VALUES(0, 100, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(1, 101, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(2, 100, '2011-11-11 11:11:11.0');
> {code}
> *Query 1:* Example query, running fine in itself:
> {code:sql}
> SELECT id, test_id, lastchanged FROM test T
> WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> Returns:
> ++-+---+
> | ID | TEST_ID |  LASTCHANGED  |
> ++-+---+
> | 1  | 101 | 2000-01-01 01:00:00.0 |
> | 2  | 100 | 2011-11-11 12:11:11.0 |
> ++-+---+
> {code}
> *Query 2:* Same query fails on the current master branch, when it is part of 
> a larger (implicit) join:
> {code:sql}
> SELECT AAA.*
> FROM 
> (
>   SELECT id, test_id, lastchanged FROM test T
>   WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> ) as AAA,
> (
>   SELECT id FROM test
> ) as BBB
> WHERE AAA.id = BBB.id;
> java.lang.IllegalArgumentException
>   at 
> org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
>   at 
> org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:66)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:663)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:404)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:176)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:504)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:467)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:309)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:298)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:297)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1933)
>   at sqlline.Commands.executeSingleQuery(Commands.java:1054)
>   at sqlline.Commands.execute(Commands.java:1003)
>   at sqlline.Commands.sql(Commands.java:967)
>   at sqlline.SqlLine.dispatch(SqlLine.java:734)
>   at sqlline.SqlLine.begin(SqlLine.java:541)
>   at sqlline.SqlLine.start(SqlLine.java:267)
>   at sqlline.SqlLine.main(SqlLine.java:206)
> {code}
> I am not sure what the problem is exactly. My guess is that Phoenix tries to 
> optimize (flatten) an inner-query, which it shouldn't, if we are inside a 
> join (according to the check in the code which throws the exception).
> The best workaround I found was to define an explicit join in the original 
> query (Query 1), basically change the inner select into a join. This modified 
> query return the same as the original one:
> *Query 3:*
> {code:sql}
> SELECT T.id, T.test_id, T.lastchanged 
> FROM 
>   test T 
>   LEFT JOIN (
> SELECT max(lastchanged) AS max_timestamp, 
>test_id AS max_timestamp_test_id
> FROM test
> GROUP BY test_id
>   ) JOIN_TABLE ON JOIN_TABLE.max_timestamp_test_id = T.test_id
> WHERE T.lastchanged = JOIN_TABLE.max_timestamp
> Returns:
> +--+---+---+
> | T.ID | T.TEST_ID | T.LASTCHANGED |
> +--+---+---+

[jira] [Assigned] (PHOENIX-6232) Same query fails with IllegalArgumentException if part of a join

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei reassigned PHOENIX-6232:
-

Assignee: chenglei

> Same query fails with IllegalArgumentException if part of a join
> 
>
> Key: PHOENIX-6232
> URL: https://issues.apache.org/jira/browse/PHOENIX-6232
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Mate Szalay-Beko
>Assignee: chenglei
>Priority: Major
>
> We were facing an interesting problem when a more complex query (with inner 
> selects in the WHERE clause) succeeds alone, while the same query fails, if 
> it is part of a join. I created a test table / query to reproduce the problem:
> {code:sql}
> DROP TABLE IF EXISTS test;
> CREATE TABLE test (
>   id INTEGER NOT NULL,
>   test_id INTEGER,
>   lastchanged TIMESTAMP,
>   CONSTRAINT my_pk PRIMARY KEY (id));
> UPSERT INTO test VALUES(0, 100, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(1, 101, '2000-01-01 00:00:00.0');
> UPSERT INTO test VALUES(2, 100, '2011-11-11 11:11:11.0');
> {code}
> *Query 1:* Example query, running fine in itself:
> {code:sql}
> SELECT id, test_id, lastchanged FROM test T
> WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> Returns:
> ++-+---+
> | ID | TEST_ID |  LASTCHANGED  |
> ++-+---+
> | 1  | 101 | 2000-01-01 01:00:00.0 |
> | 2  | 100 | 2011-11-11 12:11:11.0 |
> ++-+---+
> {code}
> *Query 2:* Same query fails on the current master branch, when it is part of 
> a larger (implicit) join:
> {code:sql}
> SELECT AAA.*
> FROM 
> (
>   SELECT id, test_id, lastchanged FROM test T
>   WHERE lastchanged = ( SELECT max(lastchanged) FROM test WHERE test_id = 
> T.test_id )
> ) as AAA,
> (
>   SELECT id FROM test
> ) as BBB
> WHERE AAA.id = BBB.id;
> java.lang.IllegalArgumentException
>   at 
> org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
>   at 
> org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:66)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:663)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:404)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:176)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:504)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:467)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:309)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:298)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:297)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1933)
>   at sqlline.Commands.executeSingleQuery(Commands.java:1054)
>   at sqlline.Commands.execute(Commands.java:1003)
>   at sqlline.Commands.sql(Commands.java:967)
>   at sqlline.SqlLine.dispatch(SqlLine.java:734)
>   at sqlline.SqlLine.begin(SqlLine.java:541)
>   at sqlline.SqlLine.start(SqlLine.java:267)
>   at sqlline.SqlLine.main(SqlLine.java:206)
> {code}
> I am not sure what the problem is exactly. My guess is that Phoenix tries to 
> optimize (flatten) an inner-query, which it shouldn't, if we are inside a 
> join (according to the check in the code which throws the exception).
> The best workaround I found was to define an explicit join in the original 
> query (Query 1), basically change the inner select into a join. This modified 
> query return the same as the original one:
> *Query 3:*
> {code:sql}
> SELECT T.id, T.test_id, T.lastchanged 
> FROM 
>   test T 
>   LEFT JOIN (
> SELECT max(lastchanged) AS max_timestamp, 
>test_id AS max_timestamp_test_id
> FROM test
> GROUP BY test_id
>   ) JOIN_TABLE ON JOIN_TABLE.max_timestamp_test_id = T.test_id
> WHERE T.lastchanged = JOIN_TABLE.max_timestamp
> Returns:
> +--+---+---+
> | T.ID | T.TEST_ID | T.LASTCHANGED |
> +--+---+---+
> | 1| 101   | 2000-01-01 01:00:00.0 |
> | 2| 100  

[jira] [Updated] (PHOENIX-6224) Support Correlated IN Subquery

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6224:
--
Attachment: PHOENIX-6224_v1-master.patch

> Support  Correlated IN Subquery
> ---
>
> Key: PHOENIX-6224
> URL: https://issues.apache.org/jira/browse/PHOENIX-6224
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6224_v1-4.x.patch, PHOENIX-6224_v1-master.patch
>
>
> Given following tables :
> {code:java}
>  create table item
>(item_id varchar not null primary key, 
> name varchar, 
> price integer, 
> discount1 integer, 
> discount2 integer, 
> supplier_id varchar, 
>description varchar)
>  
> create table order
>  ( order_id varchar not null primary key, 
>customer_id varchar, 
>item_id varchar, 
>price integer,
>quantity integer, 
>date timestamp)
> {code}
> for the  correlated in subquery:
> {code:java}
>  SELECT item_id, name FROM item i WHERE i.item_id IN 
>   (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
> {code} 
> Phoenix would throw following exception, that is because phoenix only support 
> Non-Correlated In Subquery now:
> {code:java}
> org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
> Undefined column family. familyName=I
>   at 
> org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
>   at 
> org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
>   at 
> org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
>   at 
> org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.resolveColumn(WhereCompiler.java:191)
>   at 
> org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.visit(WhereCompiler.java:177)
>   at 
> org.apache.phoenix.compile.ExpressionCompiler.visit(ExpressionCompiler.java:1)
>   at 
> org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
>   at 
> org.apache.phoenix.parse.CompoundParseNode.acceptChildren(CompoundParseNode.java:64)
>   at 
> org.apache.phoenix.parse.ComparisonParseNode.accept(ComparisonParseNode.java:45)
>   at 
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:138)
>   at 
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:108)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:629)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:574)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:203)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:563)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:239)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:320)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:252)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:201)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:497)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1769)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1762)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1756)
> {code}
> But other SQL engines like Spark SQL and MySQL all support Correlated In 
> Subquery .



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6224) Support Correlated IN Subquery

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6224:
--
Attachment: PHOENIX-6224_v1-4.x.patch

> Support  Correlated IN Subquery
> ---
>
> Key: PHOENIX-6224
> URL: https://issues.apache.org/jira/browse/PHOENIX-6224
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6224_v1-4.x.patch
>
>
> Given following tables :
> {code:java}
>  create table item
>(item_id varchar not null primary key, 
> name varchar, 
> price integer, 
> discount1 integer, 
> discount2 integer, 
> supplier_id varchar, 
>description varchar)
>  
> create table order
>  ( order_id varchar not null primary key, 
>customer_id varchar, 
>item_id varchar, 
>price integer,
>quantity integer, 
>date timestamp)
> {code}
> for the  correlated in subquery:
> {code:java}
>  SELECT item_id, name FROM item i WHERE i.item_id IN 
>   (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
> {code} 
> Phoenix would throw following exception, that is because phoenix only support 
> Non-Correlated In Subquery now:
> {code:java}
> org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
> Undefined column family. familyName=I
>   at 
> org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
>   at 
> org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
>   at 
> org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
>   at 
> org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.resolveColumn(WhereCompiler.java:191)
>   at 
> org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.visit(WhereCompiler.java:177)
>   at 
> org.apache.phoenix.compile.ExpressionCompiler.visit(ExpressionCompiler.java:1)
>   at 
> org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
>   at 
> org.apache.phoenix.parse.CompoundParseNode.acceptChildren(CompoundParseNode.java:64)
>   at 
> org.apache.phoenix.parse.ComparisonParseNode.accept(ComparisonParseNode.java:45)
>   at 
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:138)
>   at 
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:108)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:629)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:574)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:203)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:563)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:239)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:320)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:252)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:201)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:497)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1769)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1762)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1756)
> {code}
> But other SQL engines like Spark SQL and MySQL all support Correlated In 
> Subquery .



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6224) Support Correlated IN Subquery

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6224:
--
Description: 
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
create table order
 ( order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the  correlated in subquery:
{code:java}
 SELECT item_id, name FROM item i WHERE i.item_id IN 
  (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
{code} 

Phoenix would throw following exception, that is because phoenix only support 
Non-Correlated In Subquery now:
{code:java}
org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
Undefined column family. familyName=I
at 
org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
at 
org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
at 
org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
at 
org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.resolveColumn(WhereCompiler.java:191)
at 
org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.visit(WhereCompiler.java:177)
at 
org.apache.phoenix.compile.ExpressionCompiler.visit(ExpressionCompiler.java:1)
at 
org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
at 
org.apache.phoenix.parse.CompoundParseNode.acceptChildren(CompoundParseNode.java:64)
at 
org.apache.phoenix.parse.ComparisonParseNode.accept(ComparisonParseNode.java:45)
at 
org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:138)
at 
org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:108)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:629)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:574)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:203)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
at 
org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:563)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:239)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:320)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:252)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:201)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:497)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1769)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1762)
at 
org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1756)
{code}

But other SQL engines like Spark SQL and MySQL all support Correlated In 
Subquery .







  was:
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
create table order
 ( order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the  correlated in subquery:
{code:java}
 SELECT item_id, name FROM item i WHERE i.item_id IN 
  (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
{code} 

Phoenix would throw following exception, that is because phoenix only support 
Non-Correlated In Subquery now:
{code:java}
org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
Undefined column family. familyName=I
at 
org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
at 
org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
at 
org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
   

[jira] [Assigned] (PHOENIX-6224) Support Correlated IN Subquery

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei reassigned PHOENIX-6224:
-

Assignee: chenglei

> Support  Correlated IN Subquery
> ---
>
> Key: PHOENIX-6224
> URL: https://issues.apache.org/jira/browse/PHOENIX-6224
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
>
> Given following tables :
> {code:java}
>  create table item
>(item_id varchar not null primary key, 
> name varchar, 
> price integer, 
> discount1 integer, 
> discount2 integer, 
> supplier_id varchar, 
>description varchar)
>  
> create table order
>  ( order_id varchar not null primary key, 
>customer_id varchar, 
>item_id varchar, 
>price integer,
>quantity integer, 
>date timestamp)
> {code}
> for the  correlated in subquery:
> {code:java}
>  SELECT item_id, name FROM item i WHERE i.item_id IN 
>   (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
> {code} 
> Phoenix would throw following exception, that is because phoenix only support 
> Non-Correlated In Subquery now:
> {code:java}
> org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
> Undefined column family. familyName=I
>   at 
> org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
>   at 
> org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
>   at 
> org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
>   at 
> org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.resolveColumn(WhereCompiler.java:191)
>   at 
> org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.visit(WhereCompiler.java:177)
>   at 
> org.apache.phoenix.compile.ExpressionCompiler.visit(ExpressionCompiler.java:1)
>   at 
> org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
>   at 
> org.apache.phoenix.parse.CompoundParseNode.acceptChildren(CompoundParseNode.java:64)
>   at 
> org.apache.phoenix.parse.ComparisonParseNode.accept(ComparisonParseNode.java:45)
>   at 
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:138)
>   at 
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:108)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:629)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:574)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:203)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:563)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:239)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:320)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:252)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:201)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:497)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1769)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1762)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1756)
> {code}
> But other sql engines like Spark SQL and MySQL support Correlated In Subquery 
> .



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6224) Support Correlated IN Subquery

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6224:
--
Description: 
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
create table order
 ( order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the  correlated in subquery:
{code:java}
 SELECT item_id, name FROM item i WHERE i.item_id IN 
  (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
{code} 

Phoenix would throw following exception, that is because phoenix only support 
Non-Correlated In Subquery now:
{code:java}
org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
Undefined column family. familyName=I
at 
org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
at 
org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
at 
org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
at 
org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.resolveColumn(WhereCompiler.java:191)
at 
org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.visit(WhereCompiler.java:177)
at 
org.apache.phoenix.compile.ExpressionCompiler.visit(ExpressionCompiler.java:1)
at 
org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
at 
org.apache.phoenix.parse.CompoundParseNode.acceptChildren(CompoundParseNode.java:64)
at 
org.apache.phoenix.parse.ComparisonParseNode.accept(ComparisonParseNode.java:45)
at 
org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:138)
at 
org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:108)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:629)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:574)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:203)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
at 
org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:563)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:239)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:320)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:252)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:201)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:497)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1769)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1762)
at 
org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1756)
{code}

But other sql engines like Spark SQL and MySQL support Correlated In Subquery .







  was:
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
create table order
 ( order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the  correlated in subquery:
{code:java}
 SELECT item_id, name FROM item i WHERE i.item_id IN 
  (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
{code} 

Phoenix would throw following exception, that is because phoenix only support 
Non-Correlated In Subquery now:
{code:java}
org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
Undefined column family. familyName=I
at 
org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
at 
org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
at 
org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
at 

[jira] [Updated] (PHOENIX-6224) Support Correlated IN Subquery

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6224:
--
Description: 
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
create table order
 ( order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the  correlated in subquery:
{code:java}
 SELECT item_id, name FROM item i WHERE i.item_id IN 
  (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
{code} 

Phoenix would throw following exception, that is because phoenix only support 
Non-Correlated In Subquery now:
{code:java}
org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
Undefined column family. familyName=I
at 
org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
at 
org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
at 
org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
at 
org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.resolveColumn(WhereCompiler.java:191)
at 
org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.visit(WhereCompiler.java:177)
at 
org.apache.phoenix.compile.ExpressionCompiler.visit(ExpressionCompiler.java:1)
at 
org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
at 
org.apache.phoenix.parse.CompoundParseNode.acceptChildren(CompoundParseNode.java:64)
at 
org.apache.phoenix.parse.ComparisonParseNode.accept(ComparisonParseNode.java:45)
at 
org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:138)
at 
org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:108)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:629)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:574)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:203)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
at 
org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:563)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:239)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:320)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:252)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:201)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:497)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1769)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1762)
at 
org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1756)
{code}

But other systems like Spark SQL and MySQL support Correlated In Subquery 







  was:
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
create table order
 ( order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the  correlated in subquery:
{code:java}
 SELECT item_id, name FROM item i WHERE i.item_id IN 
  (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
{code} 

Phoenix would throw following exception, that is because phoenix only support 
Non-Correlated InSubquery now:
{code:java}
org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
Undefined column family. familyName=I
at 
org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
at 
org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
at 
org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
at 

[jira] [Updated] (PHOENIX-6224) Support Correlated IN Subquery

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6224:
--
Description: 
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
create table order
 ( order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the  correlated in subquery:
{code:java}
 SELECT item_id, name FROM item i WHERE i.item_id IN 
  (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
{code} 

Phoenix would throw following exception, that is because phoenix only support 
Non-Correlated InSubquery now:
{code:java}
org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
Undefined column family. familyName=I
at 
org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
at 
org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
at 
org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
at 
org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.resolveColumn(WhereCompiler.java:191)
at 
org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.visit(WhereCompiler.java:177)
at 
org.apache.phoenix.compile.ExpressionCompiler.visit(ExpressionCompiler.java:1)
at 
org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
at 
org.apache.phoenix.parse.CompoundParseNode.acceptChildren(CompoundParseNode.java:64)
at 
org.apache.phoenix.parse.ComparisonParseNode.accept(ComparisonParseNode.java:45)
at 
org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:138)
at 
org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:108)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:629)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:574)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:203)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
at 
org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:563)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:239)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:320)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:252)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:201)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:497)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1769)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1762)
at 
org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1756)
{code}

But other systems like Spark SQL and MySQL support Correlated InSubquery 







  was:
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
create table order
 ( order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the  correlated in subquery:
{code:java}
 SELECT item_id, name FROM item i WHERE i.item_id IN 
  (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
{code} 

Phoenix would throw following exception:
{code:java}
org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
Undefined column family. familyName=I
at 
org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
at 
org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
at 
org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
at 

[jira] [Updated] (PHOENIX-6224) Support Correlated IN Subquery

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6224:
--
Summary: Support  Correlated IN Subquery  (was: Support  
Correlated-Subquery for In Clause)

> Support  Correlated IN Subquery
> ---
>
> Key: PHOENIX-6224
> URL: https://issues.apache.org/jira/browse/PHOENIX-6224
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
>
> Given following tables :
> {code:java}
>  create table item
>(item_id varchar not null primary key, 
> name varchar, 
> price integer, 
> discount1 integer, 
> discount2 integer, 
> supplier_id varchar, 
>description varchar)
>  
> create table order
>  ( order_id varchar not null primary key, 
>customer_id varchar, 
>item_id varchar, 
>price integer,
>quantity integer, 
>date timestamp)
> {code}
> for the  correlated in subquery:
> {code:java}
>  SELECT item_id, name FROM item i WHERE i.item_id IN 
>   (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
> {code} 
> Phoenix would throw following exception:
> {code:java}
> org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
> Undefined column family. familyName=I
>   at 
> org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
>   at 
> org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
>   at 
> org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
>   at 
> org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.resolveColumn(WhereCompiler.java:191)
>   at 
> org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.visit(WhereCompiler.java:177)
>   at 
> org.apache.phoenix.compile.ExpressionCompiler.visit(ExpressionCompiler.java:1)
>   at 
> org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
>   at 
> org.apache.phoenix.parse.CompoundParseNode.acceptChildren(CompoundParseNode.java:64)
>   at 
> org.apache.phoenix.parse.ComparisonParseNode.accept(ComparisonParseNode.java:45)
>   at 
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:138)
>   at 
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:108)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:629)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:574)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:203)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:563)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:239)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:320)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:252)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:201)
>   at 
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:497)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1769)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1762)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1756)
> {code}
> But Spark SQL



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6224) Support Correlated-Subquery for In Clause

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6224:
--
Description: 
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
create table order
 ( order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the  correlated in subquery:
{code:java}
 SELECT item_id, name FROM item i WHERE i.item_id IN 
  (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name
{code} 

Phoenix would throw following exception:
{code:java}
org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): 
Undefined column family. familyName=I
at 
org.apache.phoenix.schema.PTableImpl.getColumnFamily(PTableImpl.java:1363)
at 
org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:527)
at 
org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:368)
at 
org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.resolveColumn(WhereCompiler.java:191)
at 
org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.visit(WhereCompiler.java:177)
at 
org.apache.phoenix.compile.ExpressionCompiler.visit(ExpressionCompiler.java:1)
at 
org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
at 
org.apache.phoenix.parse.CompoundParseNode.acceptChildren(CompoundParseNode.java:64)
at 
org.apache.phoenix.parse.ComparisonParseNode.accept(ComparisonParseNode.java:45)
at 
org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:138)
at 
org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:108)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:629)
at 
org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:574)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:203)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
at 
org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:563)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:239)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:320)
at 
org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:252)
at 
org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:201)
at 
org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:497)
at 
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1769)
at 
org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1762)
at 
org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1756)
{code}

But Spark SQL







  was:
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
create table order
 ( order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the 
 SELECT item_id, name FROM item i WHERE i.item_id IN 
  (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name




> Support  Correlated-Subquery for In Clause
> --
>
> Key: PHOENIX-6224
> URL: https://issues.apache.org/jira/browse/PHOENIX-6224
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
>
> Given following tables :
> {code:java}
>  create table item
>(item_id varchar not null primary key, 
> name varchar, 
> price integer, 
> discount1 integer, 
> discount2 integer, 
> supplier_id varchar, 
>description varchar)
>  
> create table order
> 

[jira] [Updated] (PHOENIX-6224) Support Correlated-Subquery for In Clause

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6224:
--
Description: 
Given following tables :
{code:java}
 create table item
   (item_id varchar not null primary key, 
name varchar, 
price integer, 
discount1 integer, 
discount2 integer, 
supplier_id varchar, 
   description varchar)
 
create table order
 ( order_id varchar not null primary key, 
   customer_id varchar, 
   item_id varchar, 
   price integer,
   quantity integer, 
   date timestamp)
{code}

for the 
 SELECT item_id, name FROM item i WHERE i.item_id IN 
  (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name



  was:
For following table :




> Support  Correlated-Subquery for In Clause
> --
>
> Key: PHOENIX-6224
> URL: https://issues.apache.org/jira/browse/PHOENIX-6224
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
>
> Given following tables :
> {code:java}
>  create table item
>(item_id varchar not null primary key, 
> name varchar, 
> price integer, 
> discount1 integer, 
> discount2 integer, 
> supplier_id varchar, 
>description varchar)
>  
> create table order
>  ( order_id varchar not null primary key, 
>customer_id varchar, 
>item_id varchar, 
>price integer,
>quantity integer, 
>date timestamp)
> {code}
> for the 
>  SELECT item_id, name FROM item i WHERE i.item_id IN 
>   (SELECT item_id FROM order o  where o.price = i.price) ORDER BY name



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6224) Support Correlated-Subquery for In Clause

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6224:
--
Summary: Support  Correlated-Subquery for In Clause  (was: Support  
Correlated-Subquery for InParseNode)

> Support  Correlated-Subquery for In Clause
> --
>
> Key: PHOENIX-6224
> URL: https://issues.apache.org/jira/browse/PHOENIX-6224
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
>
> For following table :



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6224) Support Correlated-Subquery for InParseNode

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-6224:
--
Description: 
For following table :



> Support  Correlated-Subquery for InParseNode
> 
>
> Key: PHOENIX-6224
> URL: https://issues.apache.org/jira/browse/PHOENIX-6224
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
>
> For following table :



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (PHOENIX-6224) Support Correlated-Subquery for InParseNode

[chenglei (Jira)]

chenglei created PHOENIX-6224:
-

 Summary: Support  Correlated-Subquery for InParseNode
 Key: PHOENIX-6224
 URL: https://issues.apache.org/jira/browse/PHOENIX-6224
 Project: Phoenix
  Issue Type: Bug
Affects Versions: 4.15.0
Reporter: chenglei
 Fix For: 5.1.0, 4.16.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (PHOENIX-5996) IndexRebuildRegionScanner.prepareIndexMutationsForRebuild may incorrectly delete index row when a delete and put mutation with the same timestamp

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei resolved PHOENIX-5996.
---
Fix Version/s: 4.16.0
   5.1.0
 Assignee: chenglei
   Resolution: Fixed

> IndexRebuildRegionScanner.prepareIndexMutationsForRebuild may incorrectly 
> delete index row when a delete and put mutation with the same timestamp
> -
>
> Key: PHOENIX-5996
> URL: https://issues.apache.org/jira/browse/PHOENIX-5996
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.1.0, 4.16.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> With PHOENIX-5748, 
> {{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}} is responsible 
> for generating index table mutations for rebuild.
> In the processing of data table mutations list, there can be a delete and put 
> mutation with the same timestamp.   If so, the delete and put are processed 
> together in one iteration. First, the delete mutation is applied on the put 
> mutation and current row state, and then the modified put mutation is 
> processed.
> But when the {{modified put mutation}} is empty , even the current row state 
> is not empty after the delete mutation is applied, the whole index row is  
> deleted, just as following line 1191 in 
> {{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}}:
> {code:java}
> 1189  } else {
> 1190if (currentDataRowState != null) {
> 1191Mutation del = 
> indexMaintainer.buildRowDeleteMutation(indexRowKeyForCurrentDataRow,
> 1192IndexMaintainer.DeleteType.ALL_VERSIONS, 
> ts);
> 1193indexMutations.add(del);
> 1194// For the next iteration of the for loop
> 1195currentDataRowState = null;
> 1196indexRowKeyForCurrentDataRow = null;
> 1197}
> 1198  }
> {code} 
> I think above logical is wrong, when the current row state is not empty after 
> the delete mutation is applied, we can not  delete the whole index row, but 
> instead we should reuse the logical of applying a delete mutation on current 
> row state.  I wrote a unit test in {{PrepareIndexMutationsForRebuildTest}} to 
> produce the case:
> {code:java}
> @Test
> public void testPutDeleteOnSameTimeStampAndPutNullifiedByDelete() throws 
> Exception {
> SetupInfo info = setup(
> TABLE_NAME,
> INDEX_NAME,
> "ROW_KEY VARCHAR, CF1.C1 VARCHAR, CF2.C2 VARCHAR",
> "CF2.C2",
> "ROW_KEY",
> "");
> Put dataPut = new Put(Bytes.toBytes(ROW_KEY));
> addCellToPutMutation(
> dataPut,
> Bytes.toBytes("CF2"),
> Bytes.toBytes("C2"),
> 1,
> Bytes.toBytes("v2"));
> addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 1);
> 
> addCellToPutMutation(
> dataPut,
> Bytes.toBytes("CF1"),
> Bytes.toBytes("C1"),
> 2,
> Bytes.toBytes("v1"));
> addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 2);
> Delete dataDel = new Delete(Bytes.toBytes(ROW_KEY));
> addCellToDelMutation(
> dataDel,
> Bytes.toBytes("CF1"),
> null,
> 2,
> KeyValue.Type.DeleteFamily);
> List actualIndexMutations = 
> IndexRebuildRegionScanner.prepareIndexMutationsForRebuild(
> info.indexMaintainer,
> dataPut,
> dataDel);
> List expectedIndexMutations = new ArrayList<>();
> byte[] idxKeyBytes = generateIndexRowKey("v2");
> 
> Put idxPut1 = new Put(idxKeyBytes);
> addEmptyColumnToIndexPutMutation(idxPut1, info.indexMaintainer, 1);
> expectedIndexMutations.add(idxPut1);
> 
> Put idxPut2 = new Put(idxKeyBytes);
> addEmptyColumnToIndexPutMutation(idxPut2, info.indexMaintainer, 2);
> expectedIndexMutations.add(idxPut2);
> assertEqualMutationList(expectedIndexMutations, actualIndexMutations);
> }
> {code} 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5996) IndexRebuildRegionScanner.prepareIndexMutationsForRebuild may incorrectly delete index row when a delete and put mutation with the same timestamp

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5996:
--
Description: 
With PHOENIX-5748, 
{{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}} is responsible 
for generating index table mutations for rebuild.
In the processing of data table mutations list, there can be a delete and put 
mutation with the same timestamp.   If so, the delete and put are processed 
together in one iteration. First, the delete mutation is applied on the put 
mutation and current row state, and then the modified put mutation is processed.
But when the {{modified put mutation}} is empty , even the current row state is 
not empty after the delete mutation is applied, the whole index row is  
deleted, just as following line 1191 in 
{{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}}:
{code:java}
1189  } else {
1190if (currentDataRowState != null) {
1191Mutation del = 
indexMaintainer.buildRowDeleteMutation(indexRowKeyForCurrentDataRow,
1192IndexMaintainer.DeleteType.ALL_VERSIONS, 
ts);
1193indexMutations.add(del);
1194// For the next iteration of the for loop
1195currentDataRowState = null;
1196indexRowKeyForCurrentDataRow = null;
1197}
1198  }
{code} 

I think above logical is wrong, when the current row state is not empty after 
the delete mutation is applied, we can not  delete the whole index row, but 
instead we should reuse the logical of applying a delete mutation on current 
row state.  I wrote a unit test in {{PrepareIndexMutationsForRebuildTest}} to 
produce the case:
{code:java}
@Test
public void testPutDeleteOnSameTimeStampAndPutNullifiedByDelete() throws 
Exception {
SetupInfo info = setup(
TABLE_NAME,
INDEX_NAME,
"ROW_KEY VARCHAR, CF1.C1 VARCHAR, CF2.C2 VARCHAR",
"CF2.C2",
"ROW_KEY",
"");

Put dataPut = new Put(Bytes.toBytes(ROW_KEY));
addCellToPutMutation(
dataPut,
Bytes.toBytes("CF2"),
Bytes.toBytes("C2"),
1,
Bytes.toBytes("v2"));
addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 1);

addCellToPutMutation(
dataPut,
Bytes.toBytes("CF1"),
Bytes.toBytes("C1"),
2,
Bytes.toBytes("v1"));
addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 2);

Delete dataDel = new Delete(Bytes.toBytes(ROW_KEY));
addCellToDelMutation(
dataDel,
Bytes.toBytes("CF1"),
null,
2,
KeyValue.Type.DeleteFamily);

List actualIndexMutations = 
IndexRebuildRegionScanner.prepareIndexMutationsForRebuild(
info.indexMaintainer,
dataPut,
dataDel);

List expectedIndexMutations = new ArrayList<>();
byte[] idxKeyBytes = generateIndexRowKey("v2");


Put idxPut1 = new Put(idxKeyBytes);
addEmptyColumnToIndexPutMutation(idxPut1, info.indexMaintainer, 1);
expectedIndexMutations.add(idxPut1);

Put idxPut2 = new Put(idxKeyBytes);
addEmptyColumnToIndexPutMutation(idxPut2, info.indexMaintainer, 2);
expectedIndexMutations.add(idxPut2);

assertEqualMutationList(expectedIndexMutations, actualIndexMutations);
}
{code} 


  was:
With PHOENIX-5748, 
{{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}} is responsible 
for generating index table mutations for rebuild.
In the processing of data table mutations list, there can be a delete and put 
mutation with the same timestamp.   If so, the delete and put are processed 
together in one iteration. First, the delete mutation is applied on the put 
mutation and current row state, and then the modified put mutation is processed.
But when the {{modified put mutation}} is empty , even the current row state is 
not empty after the delete mutation is applied, the whole index row is  
deleted, just as following line 1191 in 
{{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}}:
{code:java}
1189  } else {
1190if (currentDataRowState != null) {
1191Mutation del = 
indexMaintainer.buildRowDeleteMutation(indexRowKeyForCurrentDataRow,
1192IndexMaintainer.DeleteType.ALL_VERSIONS, 
ts);
1193indexMutations.add(del);
1194// For the next iteration of the for loop
1195currentDataRowState = null;
1196

[jira] [Updated] (PHOENIX-5996) IndexRebuildRegionScanner.prepareIndexMutationsForRebuild may incorrectly delete index row when a delete and put mutation with the same timestamp

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5996:
--
Description: 
With PHOENIX-5748, 
{{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}} is responsible 
for generating index table mutations for rebuild.
In the processing of data table mutations list, there can be a delete and put 
mutation with the same timestamp.   If so, the delete and put are processed 
together in one iteration. First, the delete mutation is applied on the put 
mutation and current row state, and then the modified put mutation is processed.
But when the {{modified put mutation}} is empty , even the current row state is 
not empty after the delete mutation is applied, the whole index row is  
deleted, just as following line 1191 in 
{{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}}:
{code:java}
1189  } else {
1190if (currentDataRowState != null) {
1191Mutation del = 
indexMaintainer.buildRowDeleteMutation(indexRowKeyForCurrentDataRow,
1192IndexMaintainer.DeleteType.ALL_VERSIONS, 
ts);
1193indexMutations.add(del);
1194// For the next iteration of the for loop
1195currentDataRowState = null;
1196indexRowKeyForCurrentDataRow = null;
1197}
1198  }
{code} 

I think above logical is wrong, when the current row state is not empty after 
the delete mutation is applied, we can not  delete the whole index row, but 
instead we should reuse the logical of processing a delete mutation.  I wrote a 
unit test in {{PrepareIndexMutationsForRebuildTest}} to produce the case:
{code:java}
@Test
public void testPutDeleteOnSameTimeStampAndPutNullifiedByDelete() throws 
Exception {
SetupInfo info = setup(
TABLE_NAME,
INDEX_NAME,
"ROW_KEY VARCHAR, CF1.C1 VARCHAR, CF2.C2 VARCHAR",
"CF2.C2",
"ROW_KEY",
"");

Put dataPut = new Put(Bytes.toBytes(ROW_KEY));
addCellToPutMutation(
dataPut,
Bytes.toBytes("CF2"),
Bytes.toBytes("C2"),
1,
Bytes.toBytes("v2"));
addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 1);

addCellToPutMutation(
dataPut,
Bytes.toBytes("CF1"),
Bytes.toBytes("C1"),
2,
Bytes.toBytes("v1"));
addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 2);

Delete dataDel = new Delete(Bytes.toBytes(ROW_KEY));
addCellToDelMutation(
dataDel,
Bytes.toBytes("CF1"),
null,
2,
KeyValue.Type.DeleteFamily);

List actualIndexMutations = 
IndexRebuildRegionScanner.prepareIndexMutationsForRebuild(
info.indexMaintainer,
dataPut,
dataDel);

List expectedIndexMutations = new ArrayList<>();
byte[] idxKeyBytes = generateIndexRowKey("v2");


Put idxPut1 = new Put(idxKeyBytes);
addEmptyColumnToIndexPutMutation(idxPut1, info.indexMaintainer, 1);
expectedIndexMutations.add(idxPut1);

Put idxPut2 = new Put(idxKeyBytes);
addEmptyColumnToIndexPutMutation(idxPut2, info.indexMaintainer, 2);
expectedIndexMutations.add(idxPut2);

assertEqualMutationList(expectedIndexMutations, actualIndexMutations);
}
{code} 


  was:With PHOENIX-5748, IndexRebuildRegionScanner is responsible for 
generating index table mutations


> IndexRebuildRegionScanner.prepareIndexMutationsForRebuild may incorrectly 
> delete index row when a delete and put mutation with the same timestamp
> -
>
> Key: PHOENIX-5996
> URL: https://issues.apache.org/jira/browse/PHOENIX-5996
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.1.0, 4.16.0
>Reporter: chenglei
>Priority: Major
>
> With PHOENIX-5748, 
> {{IndexRebuildRegionScanner.prepareIndexMutationsForRebuild}} is responsible 
> for generating index table mutations for rebuild.
> In the processing of data table mutations list, there can be a delete and put 
> mutation with the same timestamp.   If so, the delete and put are processed 
> together in one iteration. First, the delete mutation is applied on the put 
> mutation and current row state, and then the modified put mutation is 
> processed.
> But when the {{modified put mutation}} is empty , even the current row state 
> is not empty after 

[jira] [Updated] (PHOENIX-5996) IndexRebuildRegionScanner.prepareIndexMutationsForRebuild may incorrectly delete index row when a delete and put mutation with the same timestamp

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5996:
--
Description: With PHOENIX-5748, IndexRebuildRegionScanner is responsible 
for generating index table mutations

> IndexRebuildRegionScanner.prepareIndexMutationsForRebuild may incorrectly 
> delete index row when a delete and put mutation with the same timestamp
> -
>
> Key: PHOENIX-5996
> URL: https://issues.apache.org/jira/browse/PHOENIX-5996
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.1.0, 4.16.0
>Reporter: chenglei
>Priority: Major
>
> With PHOENIX-5748, IndexRebuildRegionScanner is responsible for generating 
> index table mutations



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (PHOENIX-5996) IndexRebuildRegionScanner.prepareIndexMutationsForRebuild may incorrectly delete index row when a delete and put mutation with the same timestamp

[chenglei (Jira)]

chenglei created PHOENIX-5996:
-

 Summary: IndexRebuildRegionScanner.prepareIndexMutationsForRebuild 
may incorrectly delete index row when a delete and put mutation with the same 
timestamp
 Key: PHOENIX-5996
 URL: https://issues.apache.org/jira/browse/PHOENIX-5996
 Project: Phoenix
  Issue Type: Bug
Affects Versions: 5.1.0, 4.16.0
Reporter: chenglei






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests hange

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5970:
--
Description: 
After PHOENIX-5962, the ITtests still hange , I noticed the RegionServer is 
blocked as follows : (by 4.x branch and HBase1.4.10) :
{code:java}
Thread 31512: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - java.lang.Thread.sleep(long, int) @bci=57, line=340 (Compiled frame)
 - java.util.concurrent.TimeUnit.sleep(long) @bci=23, line=386 (Compiled frame)
 - org.apache.hadoop.hbase.util.RetryCounter.sleepUntilNextRetry() @bci=110, 
line=158 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(java.lang.String,
 org.apache.zookeeper.Watcher, org.apache.zookeeper.data.Stat) @bci=180, 
line=367 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.ZKUtil.getData(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 java.lang.String) @bci=11, line=629 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionState(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int) @bci=23, line=487 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionLocation(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int) @bci=6, line=168 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int, long) @bci=63, line=607 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 long) @bci=9, line=588 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 long, org.apache.hadoop.conf.Configuration) @bci=34, line=561 (Compiled frame)
 - org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation() 
@bci=94, line=61 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(org.apache.hadoop.hbase.TableName,
 boolean, int) @bci=205, line=1266 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=119, line=1227 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(org.apache.hadoop.hbase.TableName,
 byte[], int) @bci=93, line=1201 (Interpreted frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.relocateRegion(org.apache.hadoop.hbase.TableName,
 byte[], int) @bci=9, line=41 (Interpreted frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=1668, line=1419 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=138, line=1230 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(boolean,
 int, org.apache.hadoop.hbase.client.ClusterConnection, 
org.apache.hadoop.hbase.TableName, byte[]) @bci=22, line=356 (Compiled frame)
 - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) 
@bci=164, line=153 (Compiled frame)
 - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) @bci=6, 
line=58 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(org.apache.hadoop.hbase.client.RetryingCallable,
 int) @bci=30, line=219 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ClientScanner.call(org.apache.hadoop.hbase.client.ScannerCallableWithReplicas,
 org.apache.hadoop.hbase.client.RpcRetryingCaller, int, boolean) @bci=34, 
line=275 (Compiled frame)
 - org.apache.hadoop.hbase.client.ClientScanner.loadCache() @bci=109, line=436 
(Compiled frame)
 - org.apache.hadoop.hbase.client.ClientScanner.next() @bci=68, line=310 
(Compiled frame)
 - 
org.apache.phoenix.util.ViewUtil.findRelatedViews(org.apache.hadoop.hbase.client.Table,
 byte[], byte[], byte[], org.apache.phoenix.schema.PTable$LinkType, long) 
@bci=281, line=126 (Compiled frame)
 - 
org.apache.phoenix.util.ViewUtil.dropChildViews(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment,
 byte[], byte[], byte[], byte[]) @bci=62, line=198 (Compiled frame)
 - 

[jira] [Resolved] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests hange

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei resolved PHOENIX-5970.
---
Resolution: Fixed

> ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests 
> hange
> 
>
> Key: PHOENIX-5970
> URL: https://issues.apache.org/jira/browse/PHOENIX-5970
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-5970_v1-4.x.patch
>
>
> When ITtests hange recently , I noticed the RegionServer is blocked as 
> follows : (for 4.x branch and HBase1.4.10) :
> {code:java}
> Thread 31512: (state = BLOCKED)
>  - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
> imprecise)
>  - java.lang.Thread.sleep(long, int) @bci=57, line=340 (Compiled frame)
>  - java.util.concurrent.TimeUnit.sleep(long) @bci=23, line=386 (Compiled 
> frame)
>  - org.apache.hadoop.hbase.util.RetryCounter.sleepUntilNextRetry() @bci=110, 
> line=158 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(java.lang.String,
>  org.apache.zookeeper.Watcher, org.apache.zookeeper.data.Stat) @bci=180, 
> line=367 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.ZKUtil.getData(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  java.lang.String) @bci=11, line=629 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionState(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  int) @bci=23, line=487 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionLocation(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  int) @bci=6, line=168 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  int, long) @bci=63, line=607 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  long) @bci=9, line=588 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  long, org.apache.hadoop.conf.Configuration) @bci=34, line=561 (Compiled 
> frame)
>  - org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation() 
> @bci=94, line=61 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(org.apache.hadoop.hbase.TableName,
>  boolean, int) @bci=205, line=1266 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=119, line=1227 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], int) @bci=93, line=1201 (Interpreted frame)
>  - 
> org.apache.hadoop.hbase.client.CoprocessorHConnection.relocateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], int) @bci=9, line=41 (Interpreted frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=1668, line=1419 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=138, line=1230 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(boolean,
>  int, org.apache.hadoop.hbase.client.ClusterConnection, 
> org.apache.hadoop.hbase.TableName, byte[]) @bci=22, line=356 (Compiled frame)
>  - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) 
> @bci=164, line=153 (Compiled frame)
>  - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) 
> @bci=6, line=58 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(org.apache.hadoop.hbase.client.RetryingCallable,
>  int) @bci=30, line=219 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ClientScanner.call(org.apache.hadoop.hbase.client.ScannerCallableWithReplicas,
>  org.apache.hadoop.hbase.client.RpcRetryingCaller, int, boolean) @bci=34, 
> line=275 

[jira] [Assigned] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests hange

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei reassigned PHOENIX-5970:
-

Assignee: chenglei

> ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests 
> hange
> 
>
> Key: PHOENIX-5970
> URL: https://issues.apache.org/jira/browse/PHOENIX-5970
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-5970_v1-4.x.patch
>
>
> When ITtests hange recently , I noticed the RegionServer is blocked as 
> follows : (for 4.x branch and HBase1.4.10) :
> {code:java}
> Thread 31512: (state = BLOCKED)
>  - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
> imprecise)
>  - java.lang.Thread.sleep(long, int) @bci=57, line=340 (Compiled frame)
>  - java.util.concurrent.TimeUnit.sleep(long) @bci=23, line=386 (Compiled 
> frame)
>  - org.apache.hadoop.hbase.util.RetryCounter.sleepUntilNextRetry() @bci=110, 
> line=158 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(java.lang.String,
>  org.apache.zookeeper.Watcher, org.apache.zookeeper.data.Stat) @bci=180, 
> line=367 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.ZKUtil.getData(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  java.lang.String) @bci=11, line=629 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionState(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  int) @bci=23, line=487 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionLocation(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  int) @bci=6, line=168 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  int, long) @bci=63, line=607 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  long) @bci=9, line=588 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  long, org.apache.hadoop.conf.Configuration) @bci=34, line=561 (Compiled 
> frame)
>  - org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation() 
> @bci=94, line=61 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(org.apache.hadoop.hbase.TableName,
>  boolean, int) @bci=205, line=1266 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=119, line=1227 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], int) @bci=93, line=1201 (Interpreted frame)
>  - 
> org.apache.hadoop.hbase.client.CoprocessorHConnection.relocateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], int) @bci=9, line=41 (Interpreted frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=1668, line=1419 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=138, line=1230 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(boolean,
>  int, org.apache.hadoop.hbase.client.ClusterConnection, 
> org.apache.hadoop.hbase.TableName, byte[]) @bci=22, line=356 (Compiled frame)
>  - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) 
> @bci=164, line=153 (Compiled frame)
>  - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) 
> @bci=6, line=58 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(org.apache.hadoop.hbase.client.RetryingCallable,
>  int) @bci=30, line=219 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ClientScanner.call(org.apache.hadoop.hbase.client.ScannerCallableWithReplicas,
>  

[jira] [Updated] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests hange

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5970:
--
Fix Version/s: 4.16.0
   5.1.0

> ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests 
> hange
> 
>
> Key: PHOENIX-5970
> URL: https://issues.apache.org/jira/browse/PHOENIX-5970
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-5970_v1-4.x.patch
>
>
> When ITtests hange recently , I noticed the RegionServer is blocked as 
> follows : (for 4.x branch and HBase1.4.10) :
> {code:java}
> Thread 31512: (state = BLOCKED)
>  - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
> imprecise)
>  - java.lang.Thread.sleep(long, int) @bci=57, line=340 (Compiled frame)
>  - java.util.concurrent.TimeUnit.sleep(long) @bci=23, line=386 (Compiled 
> frame)
>  - org.apache.hadoop.hbase.util.RetryCounter.sleepUntilNextRetry() @bci=110, 
> line=158 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(java.lang.String,
>  org.apache.zookeeper.Watcher, org.apache.zookeeper.data.Stat) @bci=180, 
> line=367 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.ZKUtil.getData(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  java.lang.String) @bci=11, line=629 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionState(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  int) @bci=23, line=487 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionLocation(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  int) @bci=6, line=168 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  int, long) @bci=63, line=607 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  long) @bci=9, line=588 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
>  long, org.apache.hadoop.conf.Configuration) @bci=34, line=561 (Compiled 
> frame)
>  - org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation() 
> @bci=94, line=61 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(org.apache.hadoop.hbase.TableName,
>  boolean, int) @bci=205, line=1266 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=119, line=1227 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], int) @bci=93, line=1201 (Interpreted frame)
>  - 
> org.apache.hadoop.hbase.client.CoprocessorHConnection.relocateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], int) @bci=9, line=41 (Interpreted frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=1668, line=1419 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=138, line=1230 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
>  byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(boolean,
>  int, org.apache.hadoop.hbase.client.ClusterConnection, 
> org.apache.hadoop.hbase.TableName, byte[]) @bci=22, line=356 (Compiled frame)
>  - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) 
> @bci=164, line=153 (Compiled frame)
>  - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) 
> @bci=6, line=58 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(org.apache.hadoop.hbase.client.RetryingCallable,
>  int) @bci=30, line=219 (Compiled frame)
>  - 
> org.apache.hadoop.hbase.client.ClientScanner.call(org.apache.hadoop.hbase.client.ScannerCallableWithReplicas,
>  org.apache.hadoop.hbase.client.RpcRetryingCaller, int, 

[jira] [Updated] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests hange

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5970:
--
Description: 
When ITtests hange recently , I noticed the RegionServer is blocked as follows 
: (for 4.x branch and HBase1.4.10) :
{code:java}
Thread 31512: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - java.lang.Thread.sleep(long, int) @bci=57, line=340 (Compiled frame)
 - java.util.concurrent.TimeUnit.sleep(long) @bci=23, line=386 (Compiled frame)
 - org.apache.hadoop.hbase.util.RetryCounter.sleepUntilNextRetry() @bci=110, 
line=158 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(java.lang.String,
 org.apache.zookeeper.Watcher, org.apache.zookeeper.data.Stat) @bci=180, 
line=367 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.ZKUtil.getData(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 java.lang.String) @bci=11, line=629 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionState(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int) @bci=23, line=487 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionLocation(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int) @bci=6, line=168 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int, long) @bci=63, line=607 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 long) @bci=9, line=588 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 long, org.apache.hadoop.conf.Configuration) @bci=34, line=561 (Compiled frame)
 - org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation() 
@bci=94, line=61 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(org.apache.hadoop.hbase.TableName,
 boolean, int) @bci=205, line=1266 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=119, line=1227 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(org.apache.hadoop.hbase.TableName,
 byte[], int) @bci=93, line=1201 (Interpreted frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.relocateRegion(org.apache.hadoop.hbase.TableName,
 byte[], int) @bci=9, line=41 (Interpreted frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=1668, line=1419 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=138, line=1230 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(boolean,
 int, org.apache.hadoop.hbase.client.ClusterConnection, 
org.apache.hadoop.hbase.TableName, byte[]) @bci=22, line=356 (Compiled frame)
 - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) 
@bci=164, line=153 (Compiled frame)
 - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) @bci=6, 
line=58 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(org.apache.hadoop.hbase.client.RetryingCallable,
 int) @bci=30, line=219 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ClientScanner.call(org.apache.hadoop.hbase.client.ScannerCallableWithReplicas,
 org.apache.hadoop.hbase.client.RpcRetryingCaller, int, boolean) @bci=34, 
line=275 (Compiled frame)
 - org.apache.hadoop.hbase.client.ClientScanner.loadCache() @bci=109, line=436 
(Compiled frame)
 - org.apache.hadoop.hbase.client.ClientScanner.next() @bci=68, line=310 
(Compiled frame)
 - 
org.apache.phoenix.util.ViewUtil.findRelatedViews(org.apache.hadoop.hbase.client.Table,
 byte[], byte[], byte[], org.apache.phoenix.schema.PTable$LinkType, long) 
@bci=281, line=126 (Compiled frame)
 - 
org.apache.phoenix.util.ViewUtil.dropChildViews(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment,
 byte[], byte[], byte[], byte[]) @bci=62, line=198 (Compiled frame)
 - 

[jira] [Updated] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests hange

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5970:
--
Description: 
When ITtests hange recently , I noticed the RegionServer is blocked as follows 
: (for 4.x branch and HBase1.4.10) :
{code:java}
Thread 31512: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - java.lang.Thread.sleep(long, int) @bci=57, line=340 (Compiled frame)
 - java.util.concurrent.TimeUnit.sleep(long) @bci=23, line=386 (Compiled frame)
 - org.apache.hadoop.hbase.util.RetryCounter.sleepUntilNextRetry() @bci=110, 
line=158 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(java.lang.String,
 org.apache.zookeeper.Watcher, org.apache.zookeeper.data.Stat) @bci=180, 
line=367 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.ZKUtil.getData(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 java.lang.String) @bci=11, line=629 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionState(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int) @bci=23, line=487 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionLocation(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int) @bci=6, line=168 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int, long) @bci=63, line=607 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 long) @bci=9, line=588 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 long, org.apache.hadoop.conf.Configuration) @bci=34, line=561 (Compiled frame)
 - org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation() 
@bci=94, line=61 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(org.apache.hadoop.hbase.TableName,
 boolean, int) @bci=205, line=1266 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=119, line=1227 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(org.apache.hadoop.hbase.TableName,
 byte[], int) @bci=93, line=1201 (Interpreted frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.relocateRegion(org.apache.hadoop.hbase.TableName,
 byte[], int) @bci=9, line=41 (Interpreted frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=1668, line=1419 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=138, line=1230 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(boolean,
 int, org.apache.hadoop.hbase.client.ClusterConnection, 
org.apache.hadoop.hbase.TableName, byte[]) @bci=22, line=356 (Compiled frame)
 - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) 
@bci=164, line=153 (Compiled frame)
 - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) @bci=6, 
line=58 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(org.apache.hadoop.hbase.client.RetryingCallable,
 int) @bci=30, line=219 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ClientScanner.call(org.apache.hadoop.hbase.client.ScannerCallableWithReplicas,
 org.apache.hadoop.hbase.client.RpcRetryingCaller, int, boolean) @bci=34, 
line=275 (Compiled frame)
 - org.apache.hadoop.hbase.client.ClientScanner.loadCache() @bci=109, line=436 
(Compiled frame)
 - org.apache.hadoop.hbase.client.ClientScanner.next() @bci=68, line=310 
(Compiled frame)
 - 
org.apache.phoenix.util.ViewUtil.findRelatedViews(org.apache.hadoop.hbase.client.Table,
 byte[], byte[], byte[], org.apache.phoenix.schema.PTable$LinkType, long) 
@bci=281, line=126 (Compiled frame)
 - 
org.apache.phoenix.util.ViewUtil.dropChildViews(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment,
 byte[], byte[], byte[], byte[]) @bci=62, line=198 (Compiled frame)
 - 

[jira] [Updated] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests hange

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5970:
--
Description: 
When ITtests hange recently , I noticed the RegionServer is blocked as follows 
: (for 4.x branch and HBase1.4.10) :
{code:java}
Thread 31512: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - java.lang.Thread.sleep(long, int) @bci=57, line=340 (Compiled frame)
 - java.util.concurrent.TimeUnit.sleep(long) @bci=23, line=386 (Compiled frame)
 - org.apache.hadoop.hbase.util.RetryCounter.sleepUntilNextRetry() @bci=110, 
line=158 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(java.lang.String,
 org.apache.zookeeper.Watcher, org.apache.zookeeper.data.Stat) @bci=180, 
line=367 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.ZKUtil.getData(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 java.lang.String) @bci=11, line=629 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionState(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int) @bci=23, line=487 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaRegionLocation(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int) @bci=6, line=168 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 int, long) @bci=63, line=607 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 long) @bci=9, line=588 (Compiled frame)
 - 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
 long, org.apache.hadoop.conf.Configuration) @bci=34, line=561 (Compiled frame)
 - org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation() 
@bci=94, line=61 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(org.apache.hadoop.hbase.TableName,
 boolean, int) @bci=205, line=1266 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=119, line=1227 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(org.apache.hadoop.hbase.TableName,
 byte[], int) @bci=93, line=1201 (Interpreted frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.relocateRegion(org.apache.hadoop.hbase.TableName,
 byte[], int) @bci=9, line=41 (Interpreted frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=1668, line=1419 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=138, line=1230 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(org.apache.hadoop.hbase.TableName,
 byte[], boolean, boolean, int) @bci=13, line=41 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(boolean,
 int, org.apache.hadoop.hbase.client.ClusterConnection, 
org.apache.hadoop.hbase.TableName, byte[]) @bci=22, line=356 (Compiled frame)
 - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) 
@bci=164, line=153 (Compiled frame)
 - org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(int) @bci=6, 
line=58 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(org.apache.hadoop.hbase.client.RetryingCallable,
 int) @bci=30, line=219 (Compiled frame)
 - 
org.apache.hadoop.hbase.client.ClientScanner.call(org.apache.hadoop.hbase.client.ScannerCallableWithReplicas,
 org.apache.hadoop.hbase.client.RpcRetryingCaller, int, boolean) @bci=34, 
line=275 (Compiled frame)
 - org.apache.hadoop.hbase.client.ClientScanner.loadCache() @bci=109, line=436 
(Compiled frame)
 - org.apache.hadoop.hbase.client.ClientScanner.next() @bci=68, line=310 
(Compiled frame)
 - 
org.apache.phoenix.util.ViewUtil.findRelatedViews(org.apache.hadoop.hbase.client.Table,
 byte[], byte[], byte[], org.apache.phoenix.schema.PTable$LinkType, long) 
@bci=281, line=126 (Compiled frame)
 - 
org.apache.phoenix.util.ViewUtil.dropChildViews(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment,
 byte[], byte[], byte[], byte[]) @bci=62, line=198 (Compiled frame)
 - 

[jira] [Updated] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests hange

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5970:
--
Summary: ViewUtil.dropChildViews may cause HConnection leak which may cause 
ITtests hange  (was: ViewUtil.dropChildViews may cause HConnection leak)

> ViewUtil.dropChildViews may cause HConnection leak which may cause ITtests 
> hange
> 
>
> Key: PHOENIX-5970
> URL: https://issues.apache.org/jira/browse/PHOENIX-5970
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Attachments: PHOENIX-5970_v1-4.x.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5970:
--
Attachment: PHOENIX-5970_v1-4.x.patch

> ViewUtil.dropChildViews may cause HConnection leak
> --
>
> Key: PHOENIX-5970
> URL: https://issues.apache.org/jira/browse/PHOENIX-5970
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Attachments: PHOENIX-5970_v1-4.x.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5970:
--
Attachment: (was: PHOENIX-5970_v1-4.x.patch)

> ViewUtil.dropChildViews may cause HConnection leak
> --
>
> Key: PHOENIX-5970
> URL: https://issues.apache.org/jira/browse/PHOENIX-5970
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Attachments: PHOENIX-5970_v1-4.x.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5970:
--
Attachment: PHOENIX-5970_v1-4.x.patch

> ViewUtil.dropChildViews may cause HConnection leak
> --
>
> Key: PHOENIX-5970
> URL: https://issues.apache.org/jira/browse/PHOENIX-5970
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Attachments: PHOENIX-5970_v1-4.x.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5970) ViewUtil.dropChildViews may cause HConnection leak

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5970:
--
Summary: ViewUtil.dropChildViews may cause HConnection leak  (was: 
org.apache.phoenix.util.ViewUtil.dropChildViews may cause HConnection leak)

> ViewUtil.dropChildViews may cause HConnection leak
> --
>
> Key: PHOENIX-5970
> URL: https://issues.apache.org/jira/browse/PHOENIX-5970
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (PHOENIX-5970) org.apache.phoenix.util.ViewUtil.dropChildViews may cause HConnection leak

[chenglei (Jira)]

chenglei created PHOENIX-5970:
-

 Summary: org.apache.phoenix.util.ViewUtil.dropChildViews may cause 
HConnection leak
 Key: PHOENIX-5970
 URL: https://issues.apache.org/jira/browse/PHOENIX-5970
 Project: Phoenix
  Issue Type: Bug
Affects Versions: 4.15.0
Reporter: chenglei






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5897) SingleKeyValueTuple.toString() returns unexpected result

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5897:
--
Fix Version/s: 4.16.0

> SingleKeyValueTuple.toString() returns unexpected result
> 
>
> Key: PHOENIX-5897
> URL: https://issues.apache.org/jira/browse/PHOENIX-5897
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 5.0.0-alpha
>Reporter: Chen Feng
>Assignee: Chen Feng
>Priority: Minor
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-5897-v1.patch, PHOENIX-5897_v1-4.x.patch
>
>
> In SingleKeyValueTuple.toString(), the code is shown as follows.
> return "SingleKeyValueTuple[" + cell == null ? keyPtr.get() == 
> UNITIALIZED_KEY_BUFFER ? "null" : 
> Bytes.toStringBinary(keyPtr.get(),keyPtr.getOffset(),keyPtr.getLength()) : 
> cell.toString() + "]";
> actually, the code runs in the following order.
> ("SingleKeyValueTuple[" + cell) == null ? keyPtr.get() == 
> UNITIALIZED_KEY_BUFFER ? "null" : Bytes.toStringBinary() : (cell.toString() + 
> "]");
> Therefore the result is weird.
> BTW, value = condition1 ? condition2 ? X : Y : Z is also confusing, using if 
> can be more clear.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5897) SingleKeyValueTuple.toString() returns unexpected result

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5897:
--
Attachment: PHOENIX-5897_v1-4.x.patch

> SingleKeyValueTuple.toString() returns unexpected result
> 
>
> Key: PHOENIX-5897
> URL: https://issues.apache.org/jira/browse/PHOENIX-5897
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 5.0.0-alpha
>Reporter: Chen Feng
>Assignee: Chen Feng
>Priority: Minor
> Fix For: 5.1.0
>
> Attachments: PHOENIX-5897-v1.patch, PHOENIX-5897_v1-4.x.patch
>
>
> In SingleKeyValueTuple.toString(), the code is shown as follows.
> return "SingleKeyValueTuple[" + cell == null ? keyPtr.get() == 
> UNITIALIZED_KEY_BUFFER ? "null" : 
> Bytes.toStringBinary(keyPtr.get(),keyPtr.getOffset(),keyPtr.getLength()) : 
> cell.toString() + "]";
> actually, the code runs in the following order.
> ("SingleKeyValueTuple[" + cell) == null ? keyPtr.get() == 
> UNITIALIZED_KEY_BUFFER ? "null" : Bytes.toStringBinary() : (cell.toString() + 
> "]");
> Therefore the result is weird.
> BTW, value = condition1 ? condition2 ? X : Y : Z is also confusing, using if 
> can be more clear.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5793) Support parallel init and fast null return for SortMergeJoinPlan.

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5793:
--
Fix Version/s: 4.16.0

> Support parallel init and fast null return for SortMergeJoinPlan.
> -
>
> Key: PHOENIX-5793
> URL: https://issues.apache.org/jira/browse/PHOENIX-5793
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 5.0.0
>Reporter: Chen Feng
>Assignee: Chen Feng
>Priority: Minor
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-5793-v2.patch, PHOENIX-5793-v3.patch, 
> PHOENIX-5793-v4.patch, PHOENIX-5793-v5.patch, PHOENIX-5793_v1-4.x.patch
>
>
> For a join sql like A join B. The implementation of SortMergeJoinPlan 
> currently inits the two iterators A and B one by one.
> By initializing A and B in parallel, we can improve performance in two 
> aspects.
> 1) By overlapping the time in initializing.
> 2) If one child query is null, the other child query can be canceled since 
> the final result must be null.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5793) Support parallel init and fast null return for SortMergeJoinPlan.

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5793:
--
Attachment: PHOENIX-5793_v1-4.x.patch

> Support parallel init and fast null return for SortMergeJoinPlan.
> -
>
> Key: PHOENIX-5793
> URL: https://issues.apache.org/jira/browse/PHOENIX-5793
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 5.0.0
>Reporter: Chen Feng
>Assignee: Chen Feng
>Priority: Minor
> Fix For: 5.1.0
>
> Attachments: PHOENIX-5793-v2.patch, PHOENIX-5793-v3.patch, 
> PHOENIX-5793-v4.patch, PHOENIX-5793-v5.patch, PHOENIX-5793_v1-4.x.patch
>
>
> For a join sql like A join B. The implementation of SortMergeJoinPlan 
> currently inits the two iterators A and B one by one.
> By initializing A and B in parallel, we can improve performance in two 
> aspects.
> 1) By overlapping the time in initializing.
> 2) If one child query is null, the other child query can be canceled since 
> the final result must be null.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5956) Optimize LeftSemiJoin For SortMergeJoin

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5956:
--
Description: When LHS left semi join RHS through {{SortMergeJoinPlan}}, if 
RHS is end and returns null, then the left semi join could end early, but for 
{{SortMergeJoinPlan.SemiAntiJoinIterator}} , it would continue until LHS is 
exhausted even if RHS returns null, which is unnecessary.  (was: When LHS left 
semi join RHS through SortMergeJoin, if RHS is end and returns null, then the 
left semi join could end early, but for 
{{SortMergeJoinPlan.SemiAntiJoinIterator}} , it would continue until LHS is 
exhausted even if RHS returns null, which is unnecessary.)

> Optimize LeftSemiJoin For SortMergeJoin 
> 
>
> Key: PHOENIX-5956
> URL: https://issues.apache.org/jira/browse/PHOENIX-5956
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-5956_v1-4.x.patch
>
>
> When LHS left semi join RHS through {{SortMergeJoinPlan}}, if RHS is end and 
> returns null, then the left semi join could end early, but for 
> {{SortMergeJoinPlan.SemiAntiJoinIterator}} , it would continue until LHS is 
> exhausted even if RHS returns null, which is unnecessary.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (PHOENIX-5956) Optimize LeftSemiJoin For SortMergeJoin

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei reassigned PHOENIX-5956:
-

Assignee: chenglei

> Optimize LeftSemiJoin For SortMergeJoin 
> 
>
> Key: PHOENIX-5956
> URL: https://issues.apache.org/jira/browse/PHOENIX-5956
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-5956_v1-4.x.patch
>
>
> When LHS left semi join RHS through SortMergeJoin, if RHS is end and returns 
> null, then the left semi join could end early, but for 
> {{SortMergeJoinPlan.SemiAntiJoinIterator}} , it would continue until LHS is 
> exhausted even if RHS returns null, which is unnecessary.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5956) Optimize LeftSemiJoin For SortMergeJoin

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5956:
--
Attachment: PHOENIX-5956_v1-4.x.patch

> Optimize LeftSemiJoin For SortMergeJoin 
> 
>
> Key: PHOENIX-5956
> URL: https://issues.apache.org/jira/browse/PHOENIX-5956
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-5956_v1-4.x.patch
>
>
> When LHS left semi join RHS through SortMergeJoin, if RHS is end and returns 
> null, then the left semi join could end early, but for 
> {{SortMergeJoinPlan.SemiAntiJoinIterator}} , it would continue until LHS is 
> exhausted even if RHS returns null, which is unnecessary.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5956) Optimize LeftSemiJoin For SortMergeJoin

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5956:
--
Description: When LHS left semi join RHS through SortMergeJoin, if RHS is 
end and returns null, then the left semi join could end early, but for 
{{SortMergeJoinPlan.SemiAntiJoinIterator}} , it would continue until LHS is 
exhausted even if RHS returns null, which is unnecessary.

> Optimize LeftSemiJoin For SortMergeJoin 
> 
>
> Key: PHOENIX-5956
> URL: https://issues.apache.org/jira/browse/PHOENIX-5956
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
>
> When LHS left semi join RHS through SortMergeJoin, if RHS is end and returns 
> null, then the left semi join could end early, but for 
> {{SortMergeJoinPlan.SemiAntiJoinIterator}} , it would continue until LHS is 
> exhausted even if RHS returns null, which is unnecessary.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5956) Optimize LeftSemiJoin For SortMergeJoin

[chenglei (Jira)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-5956:
--
Fix Version/s: 4.16.0

> Optimize LeftSemiJoin For SortMergeJoin 
> 
>
> Key: PHOENIX-5956
> URL: https://issues.apache.org/jira/browse/PHOENIX-5956
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.15.0
>Reporter: chenglei
>Priority: Major
> Fix For: 4.16.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)