date:20161230

[jira] [Commented] (PHOENIX-1386) ANY function only works with absolute value and doesn't work with other parameters

2016-12-30 Thread Kevin Liew (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15788224#comment-15788224
 ] 

Kevin Liew commented on PHOENIX-1386:
-

{quote}
James Taylor added a comment - 14/Mar/16 16:51
Interesting - didn't know about that syntax. Will we get that for free when we 
move to Calcite for parsing/planning, Julian Hyde?

Julian Hyde added a comment - 14/Mar/16 16:54
Yes, I think so.
{quote}

[~jamestaylor] we may want to change the targeted fix version if this depends 
on Calcite.

> ANY function only works with absolute value and doesn't work with other 
> parameters  
> 
>
> Key: PHOENIX-1386
> URL: https://issues.apache.org/jira/browse/PHOENIX-1386
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.2.0
>Reporter: Mohammadreza
>Assignee: ramkrishna.s.vasudevan
>  Labels: features, verify
> Fix For: 4.10.0
>
>
> By this query phoenix doesn't work:
> SELECT *  FROM ac join mat  on  mat.mid=Any(ac.mt);
> ac.mt is an array.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3547) Promote CATALOG.VIEW_INDEX_ID to an int

2016-12-30 Thread Jeremy Huffman (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15788006#comment-15788006
 ] 

Jeremy Huffman commented on PHOENIX-3547:
-

A use case is documented in https://issues.apache.org/jira/browse/PHOENIX-3550 
. The alternative to that method (using tenant views) would be to use a 
non-tenant table and create the views in the global context. That works today, 
but since I logically have multiple tenants in my application layer I'll 
eventually overrun the VIEW_INDEX_ID. 

> Promote CATALOG.VIEW_INDEX_ID to an int
> ---
>
> Key: PHOENIX-3547
> URL: https://issues.apache.org/jira/browse/PHOENIX-3547
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Jeremy Huffman
>Priority: Minor
>
> Increase the size of CATALOG.VIEW_INDEX_ID from smallint to int to support a 
> large number of indexed views on a single table.
> Per James: "The code would just need to be tolerant when reading the data if 
> the length is two byte short versus four byte int. At write time, we'd just 
> always write an int."
> See: 
> https://lists.apache.org/thread.html/22849e4fc73452cee3bea763cf6d5af7164dedcb44573ba6b9f452a2@%3Cuser.phoenix.apache.org%3E



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787802#comment-15787802
 ] 

Hadoop QA commented on PHOENIX-3453:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12845172/PHOENIX-3453_v1.patch
  against master branch at commit 07f92732f9c6d2d9464012cebeb4cefc10da95d5.
  ATTACHMENT ID: 12845172

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
42 warning messages.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+sql="select DISTINCT entity_id, score from ( select entity_id, 
score from "+intTableName+" limit 1)";
+sql="select DISTINCT entity_id, score from ( select entity_id, 
score from "+intTableName+" limit 3) order by entity_id";
+sql="select DISTINCT entity_id, score from ( select entity_id, 
score from "+intTableName+" limit 3) order by entity_id desc";
+sql="select DISTINCT entity_id, score from ( select entity_id, 
score from "+charTableName+" limit 3) order by entity_id";
+sql="select DISTINCT entity_id, score from ( select entity_id, 
score from "+charTableName+" limit 3) order by entity_id desc";
+sql="select DISTINCT entity_id, score from ( select entity_id, 
score from "+tableName+" limit 1)";
+
assertTrue(queryPlan.getGroupBy().getExpressions().get(0).getSortOrder()==SortOrder.DESC);
+
assertTrue(queryPlan.getGroupBy().getExpressions().get(1).getSortOrder()==SortOrder.DESC);
+
assertTrue(queryPlan.getGroupBy().getKeyExpressions().get(0).getSortOrder()==SortOrder.DESC);
+
assertTrue(queryPlan.getGroupBy().getKeyExpressions().get(1).getSortOrder()==SortOrder.DESC);

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/713//testReport/
Javadoc warnings: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/713//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/713//console

This message is automatically generated.

> Secondary index and query using distinct: Outer query results in ERROR 201 
> (22000): Illegal data. CHAR types may only contain single byte characters
> 
>
> Key: PHOENIX-3453
> URL: https://issues.apache.org/jira/browse/PHOENIX-3453
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0, 4.9.0
>Reporter: Joel Palmert
>Assignee: chenglei
> Attachments: PHOENIX-3453_v1.patch
>
>
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=FALSE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (SCORE DESC, ENTITY_ID 
> DESC);
> UPSERT INTO test.test VALUES ('entity1',1.1);
> SELECT DISTINCT entity_id, score
> FROM(
> SELECT entity_id, score
> FROM test.test
> LIMIT 25
> );
> Output (in SQuirreL)
> ���   1.1
> If you run it in SQuirreL it results in the entity_id column getting the 
> above error value. Notice that if you remove the secondary index or DISTINCT 
> you get the correct result.
> I've also run the query through the Phoenix java api. Then I get the 
> following exception:
> Caused by: java.sql.SQLException: ERROR 201 (22000): Illegal data. CHAR types 
> may only contain single byte characters ()
> at 
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:454)
> at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
> at 
> org.apache.phoenix.schema.types.PDataType.newIllegalDataException(PDataType.java:291)
> at org.apache.phoenix.schema.types.PChar.toObject(PChar.java:121)
> at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:997)
> at 
> org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:608)
> at 
> org.apa

[jira] [Comment Edited] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787679#comment-15787679
 ] 

chenglei edited comment on PHOENIX-3491 at 12/30/16 1:36 PM:
-

[~jamestaylor],I noticed this patch had be pushed to master branch and included 
in 4.9.0,Could you please close this issue and mark it resolved ?


was (Author: comnetwork):
[~jamestaylor],I noticed this patch was pushed to master branch and included in 
4.9.0,Could you please close this issue and mark it resolved ?

> OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy 
> is reverse
> 
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch, PHOENIX-3491_v2.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST ( 
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
>  {code}   
>   
> If we execute explain on the following sql: 
> {code:borderStyle=solid}   
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST  GROUP BY ORGANIZATION_ID, 
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC 
> {code}
> the result is :
> {code:borderStyle=solid} 
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY  |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE]  |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC]  |
> +--+
> {code} 
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID 
> DESC, SCORE DESC is not  compiled out,but obviously it should be compiled out 
>  as OrderBy.REV_ROW_KEY_ORDER_BY.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787647#comment-15787647
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 1:34 PM:
-

I uploaded my first patch,[~jamestaylor],please help me have a review,thanks.I 
ran all the existing unit tests and IT tests in my local machine.


was (Author: comnetwork):
I uploaded my first patch,[~jamestaylor],please help me have a review,thanks.I 
ran the all the existing unit tests and IT tests in my local machine

> Secondary index and query using distinct: Outer query results in ERROR 201 
> (22000): Illegal data. CHAR types may only contain single byte characters
> 
>
> Key: PHOENIX-3453
> URL: https://issues.apache.org/jira/browse/PHOENIX-3453
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0, 4.9.0
>Reporter: Joel Palmert
>Assignee: chenglei
> Attachments: PHOENIX-3453_v1.patch
>
>
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=FALSE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (SCORE DESC, ENTITY_ID 
> DESC);
> UPSERT INTO test.test VALUES ('entity1',1.1);
> SELECT DISTINCT entity_id, score
> FROM(
> SELECT entity_id, score
> FROM test.test
> LIMIT 25
> );
> Output (in SQuirreL)
> ���   1.1
> If you run it in SQuirreL it results in the entity_id column getting the 
> above error value. Notice that if you remove the secondary index or DISTINCT 
> you get the correct result.
> I've also run the query through the Phoenix java api. Then I get the 
> following exception:
> Caused by: java.sql.SQLException: ERROR 201 (22000): Illegal data. CHAR types 
> may only contain single byte characters ()
> at 
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:454)
> at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
> at 
> org.apache.phoenix.schema.types.PDataType.newIllegalDataException(PDataType.java:291)
> at org.apache.phoenix.schema.types.PChar.toObject(PChar.java:121)
> at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:997)
> at 
> org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:608)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:621)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787647#comment-15787647
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 1:34 PM:
-

I uploaded my first patch,[~jamestaylor],please help me have a review,thanks.I 
ran the all the existing unit tests and IT tests in my local machine


was (Author: comnetwork):
I uploaded my first patch,[~jamestaylor],please help me have a review,thanks.

> Secondary index and query using distinct: Outer query results in ERROR 201 
> (22000): Illegal data. CHAR types may only contain single byte characters
> 
>
> Key: PHOENIX-3453
> URL: https://issues.apache.org/jira/browse/PHOENIX-3453
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0, 4.9.0
>Reporter: Joel Palmert
>Assignee: chenglei
> Attachments: PHOENIX-3453_v1.patch
>
>
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=FALSE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (SCORE DESC, ENTITY_ID 
> DESC);
> UPSERT INTO test.test VALUES ('entity1',1.1);
> SELECT DISTINCT entity_id, score
> FROM(
> SELECT entity_id, score
> FROM test.test
> LIMIT 25
> );
> Output (in SQuirreL)
> ���   1.1
> If you run it in SQuirreL it results in the entity_id column getting the 
> above error value. Notice that if you remove the secondary index or DISTINCT 
> you get the correct result.
> I've also run the query through the Phoenix java api. Then I get the 
> following exception:
> Caused by: java.sql.SQLException: ERROR 201 (22000): Illegal data. CHAR types 
> may only contain single byte characters ()
> at 
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:454)
> at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
> at 
> org.apache.phoenix.schema.types.PDataType.newIllegalDataException(PDataType.java:291)
> at org.apache.phoenix.schema.types.PChar.toObject(PChar.java:121)
> at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:997)
> at 
> org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:608)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:621)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787679#comment-15787679
 ] 

chenglei commented on PHOENIX-3491:
---

[~jamestaylor],I noticed this patch was pushed to master branch and included in 
4.9.0,Could you please close this issue and mark it resolved ?

> OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy 
> is reverse
> 
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch, PHOENIX-3491_v2.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST ( 
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL, 
>CONSTRAINT TEST_PK PRIMARY KEY ( 
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
>  {code}   
>   
> If we execute explain on the following sql: 
> {code:borderStyle=solid}   
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST  GROUP BY ORGANIZATION_ID, 
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC 
> {code}
> the result is :
> {code:borderStyle=solid} 
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY  |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE]  |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC]  |
> +--+
> {code} 
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID 
> DESC, SCORE DESC is not  compiled out,but obviously it should be compiled out 
>  as OrderBy.REV_ROW_KEY_ORDER_BY.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787222#comment-15787222
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 1:27 PM:
-

I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when the SQL has a SubQuery.

When I debug into the source code,I find the cause of the problem is the 
distinct(or group by) statement in the outer query.By the following code in 
GroupByCompiler.GroupBy.compile method, the "entity" column in GroupBy's 
expressions is ProjectedColumnExpression,but in line 245, the "entity" column 
in GroupBy's keyExpressions is CoerceExpression wrapping 
ProjectedColumnExpression ,which  convert the the ProjectedColumnExpression 
from PInteger to PDecimal:
{code:borderStyle=solid} 
232   for (int i = expressions.size()-2; i >= 0; i--) {
233Expression expression = expressions.get(i);
234PDataType keyType = getGroupByDataType(expression);
235if (keyType == expression.getDataType()) {
236continue;
237}
238// Copy expressions only when keyExpressions will be 
different than expressions
239if (keyExpressions == expressions) {
240keyExpressions = new ArrayList(expressions);
241}
242// Wrap expression in an expression that coerces the 
expression to the required type..
243// This is done so that we have a way of expressing null as 
an empty key when more
244// than one fixed and nullable types are used in a group by 
clause
245keyExpressions.set(i, CoerceExpression.create(expression, 
keyType));
246}
{code} 

When I look into CoerceExpression.create method,in line 68 of the following 
code I observe that the SortOder of the CoerceExpression is 
SortOrder.getDefault(),which is SortOrder.ASC, but it ought to be 
SortOrder.DESC,because the SortOder of ProjectedColumnExpression  is 
SortOrder.DESC:

{code:borderStyle=solid} 
46 public static Expression create(Expression expression, PDataType toType) 
throws  SQLException {
47if (toType == expression.getDataType()) {
48return expression;
49}
50return new CoerceExpression(expression, toType);
51}
  ..
66   //Package protected for tests
67CoerceExpression(Expression expression, PDataType toType) {
68this(expression, toType, SortOrder.getDefault(), null, true);
69}
{code} 

So when we get the query results, in  
ClientGroupedAggregatingResultIterator.getGroupingKey method, we invoke the 
following PDecimal.coerceBytes method to get the coerceBytes of PDecimal, 
noticed that actualType parameter is PInteger,actualModifier parameter  is 
SortOrder.DESC,and expectedModifier parameter is SortOrder.ASC, so in line 
842,we get the PInteger  "1" from the ptr which is got from the HBase 
RegionServer, and in line 845,we convert the PInteger "1"  to PDecimal "1", 
last in line 846, we encode the  PDecimal "1" to bytes, but because the 
expectedModifier parameter is SortOrder.ASC, so the PDecimal "1"  is encoded by 
 SortOrder.ASC.That is to say,the SortOrder of the groupBy key got from 
ClientGroupedAggregatingResultIterator.getGroupingKey method is  SortOrder.ASC.

{code:borderStyle=solid}
826 public void coerceBytes(ImmutableBytesWritable ptr, Object o, PDataType 
actualType, Integer actualMaxLength,
827Integer actualScale, SortOrder actualModifier, Integer 
desiredMaxLength, Integer desiredScale,
828SortOrder expectedModifier) {
   
..

840// Optimization for cases in which we already have the object around
841if (o == null) {
842o = actualType.toObject(ptr, actualType, actualModifier);
843}
844
845o = toObject(o, actualType);
846byte[] b = toBytes(o, expectedModifier);
847ptr.set(b);
848}
{code}

Unfortunately, finally in following PhoenixResult.getObject method, when we 
invoke the ColumnProjector.getValue method in line 524,  the ColumnP

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787222#comment-15787222
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 1:24 PM:
-

I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when the SQL has a SubQuery.

When I debug into the source code,I find the cause of the problem is the 
distinct(or group by) statement in the outer query.By the following code in 
GroupByCompiler.GroupBy.compile method, the "entity" column in GroupBy's 
expressions is ProjectedColumnExpression,but in line 245, the "entity" column 
in GroupBy's keyExpressions is CoerceExpression wrapping 
ProjectedColumnExpression ,which  convert the the ProjectedColumnExpression 
from PInteger to PDecimal:
{code:borderStyle=solid} 
232   for (int i = expressions.size()-2; i >= 0; i--) {
233Expression expression = expressions.get(i);
234PDataType keyType = getGroupByDataType(expression);
235if (keyType == expression.getDataType()) {
236continue;
237}
238// Copy expressions only when keyExpressions will be 
different than expressions
239if (keyExpressions == expressions) {
240keyExpressions = new ArrayList(expressions);
241}
242// Wrap expression in an expression that coerces the 
expression to the required type..
243// This is done so that we have a way of expressing null as 
an empty key when more
244// than one fixed and nullable types are used in a group by 
clause
245keyExpressions.set(i, CoerceExpression.create(expression, 
keyType));
246}
{code} 

When I look into CoerceExpression.create method,in line 68 of the following 
code I observe that the SortOder of the CoerceExpression is 
SortOrder.getDefault(),which is SortOrder.ASC, but it ought to be 
SortOrder.DESC,because the SortOder of ProjectedColumnExpression  is 
SortOrder.DESC:

{code:borderStyle=solid} 
46 public static Expression create(Expression expression, PDataType toType) 
throws  SQLException {
47if (toType == expression.getDataType()) {
48return expression;
49}
50return new CoerceExpression(expression, toType);
51}
  ..
66   //Package protected for tests
67CoerceExpression(Expression expression, PDataType toType) {
68this(expression, toType, SortOrder.getDefault(), null, true);
69}
{code} 

So when we get the query results, in  
ClientGroupedAggregatingResultIterator.getGroupingKey method, we invoke the 
following PDecimal.coerceBytes method to get the coerceBytes of PDecimal, 
noticed that actualType parameter is PInteger,actualModifier parameter  is 
SortOrder.DESC,and expectedModifier parameter is SortOrder.ASC, so in line 
842,we get the
PInteger  "1" from the ptr which is got from the HBase RegionServer, and in 
line 845,we convert the PInteger "1"  to PDecimal "1", last in line 846, we 
encode the  PDecimal "1" to bytes, but because the expectedModifier parameter 
is SortOrder.ASC, so the PDecimal "1"  is encoded by  SortOrder.ASC.That is to 
say,the SortOrder of the groupBy key got from 
ClientGroupedAggregatingResultIterator.getGroupingKey method is  SortOrder.ASC.

{code:borderStyle=solid}
826 public void coerceBytes(ImmutableBytesWritable ptr, Object o, PDataType 
actualType, Integer actualMaxLength,
827Integer actualScale, SortOrder actualModifier, Integer 
desiredMaxLength, Integer desiredScale,
828SortOrder expectedModifier) {
   
..

840// Optimization for cases in which we already have the object around
841if (o == null) {
842o = actualType.toObject(ptr, actualType, actualModifier);
843}
844
845o = toObject(o, actualType);
846byte[] b = toBytes(o, expectedModifier);
847ptr.set(b);
848}
{code}

Unfortunately, finally in following PhoenixResult.getObject method, when we 
invoke the ColumnProjector.getValue method in line 524,  the ColumnP

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787222#comment-15787222
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 1:22 PM:
-

I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when the SQL has a SubQuery.

When I debug into the source code,I find the cause of the problem is the 
distinct(or group by) statement in the outer query.By the following code in 
GroupByCompiler.GroupBy.compile method, the "entity" column in GroupBy's 
expressions is ProjectedColumnExpression,but in line 245, the "entity" column 
in GroupBy's keyExpressions is CoerceExpression wrapping 
ProjectedColumnExpression ,which  convert the the ProjectedColumnExpression 
from PInteger to PDecimal:
{code:borderStyle=solid} 
232   for (int i = expressions.size()-2; i >= 0; i--) {
233Expression expression = expressions.get(i);
234PDataType keyType = getGroupByDataType(expression);
235if (keyType == expression.getDataType()) {
236continue;
237}
238// Copy expressions only when keyExpressions will be 
different than expressions
239if (keyExpressions == expressions) {
240keyExpressions = new ArrayList(expressions);
241}
242// Wrap expression in an expression that coerces the 
expression to the required type..
243// This is done so that we have a way of expressing null as 
an empty key when more
244// than one fixed and nullable types are used in a group by 
clause
245keyExpressions.set(i, CoerceExpression.create(expression, 
keyType));
246}
{code} 

When I look into CoerceExpression.create method,in line 68 of the following 
code I observe that the SortOder of the CoerceExpression is SortOrder.ASC, but 
it ought to be SortOrder.DESC,because the SortOder of ProjectedColumnExpression 
 is SortOrder.DESC:

{code:borderStyle=solid} 
46 public static Expression create(Expression expression, PDataType toType) 
throws  SQLException {
47if (toType == expression.getDataType()) {
48return expression;
49}
50return new CoerceExpression(expression, toType);
51}
  ..
66   //Package protected for tests
67CoerceExpression(Expression expression, PDataType toType) {
68this(expression, toType, SortOrder.getDefault(), null, true);
69}
{code} 

So when we get the query results, in  
ClientGroupedAggregatingResultIterator.getGroupingKey method, we invoke the 
following PDecimal.coerceBytes method to get the coerceBytes of PDecimal, 
noticed that actualType parameter is PInteger,actualModifier parameter  is 
SortOrder.DESC,and expectedModifier parameter is SortOrder.ASC, so in line 
842,we get the
PInteger  "1" from the ptr which is got from the HBase RegionServer, and in 
line 845,we convert the PInteger "1"  to PDecimal "1", last in line 846, we 
encode the  PDecimal "1" to bytes, but because the expectedModifier parameter 
is SortOrder.ASC, so the PDecimal "1"  is encoded by  SortOrder.ASC.That is to 
say,the SortOrder of the groupBy key got from 
ClientGroupedAggregatingResultIterator.getGroupingKey method is  SortOrder.ASC.

{code:borderStyle=solid}
826 public void coerceBytes(ImmutableBytesWritable ptr, Object o, PDataType 
actualType, Integer actualMaxLength,
827Integer actualScale, SortOrder actualModifier, Integer 
desiredMaxLength, Integer desiredScale,
828SortOrder expectedModifier) {
   
..

840// Optimization for cases in which we already have the object around
841if (o == null) {
842o = actualType.toObject(ptr, actualType, actualModifier);
843}
844
845o = toObject(o, actualType);
846byte[] b = toBytes(o, expectedModifier);
847ptr.set(b);
848}
{code}

Unfortunately, finally in following PhoenixResult.getObject method, when we 
invoke the ColumnProjector.getValue method in line 524,  the ColumnProjector's 
Expression is RowKeyC

[jira] [Commented] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787647#comment-15787647
 ] 

chenglei commented on PHOENIX-3453:
---

I uploaded my first patch,[~jamestaylor],please help me have a review,thanks.

> Secondary index and query using distinct: Outer query results in ERROR 201 
> (22000): Illegal data. CHAR types may only contain single byte characters
> 
>
> Key: PHOENIX-3453
> URL: https://issues.apache.org/jira/browse/PHOENIX-3453
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0, 4.9.0
>Reporter: Joel Palmert
>Assignee: chenglei
> Attachments: PHOENIX-3453_v1.patch
>
>
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=FALSE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (SCORE DESC, ENTITY_ID 
> DESC);
> UPSERT INTO test.test VALUES ('entity1',1.1);
> SELECT DISTINCT entity_id, score
> FROM(
> SELECT entity_id, score
> FROM test.test
> LIMIT 25
> );
> Output (in SQuirreL)
> ���   1.1
> If you run it in SQuirreL it results in the entity_id column getting the 
> above error value. Notice that if you remove the secondary index or DISTINCT 
> you get the correct result.
> I've also run the query through the Phoenix java api. Then I get the 
> following exception:
> Caused by: java.sql.SQLException: ERROR 201 (22000): Illegal data. CHAR types 
> may only contain single byte characters ()
> at 
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:454)
> at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
> at 
> org.apache.phoenix.schema.types.PDataType.newIllegalDataException(PDataType.java:291)
> at org.apache.phoenix.schema.types.PChar.toObject(PChar.java:121)
> at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:997)
> at 
> org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:608)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:621)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chenglei updated PHOENIX-3453:
--
Attachment: PHOENIX-3453_v1.patch

> Secondary index and query using distinct: Outer query results in ERROR 201 
> (22000): Illegal data. CHAR types may only contain single byte characters
> 
>
> Key: PHOENIX-3453
> URL: https://issues.apache.org/jira/browse/PHOENIX-3453
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0, 4.9.0
>Reporter: Joel Palmert
>Assignee: chenglei
> Attachments: PHOENIX-3453_v1.patch
>
>
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=FALSE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (SCORE DESC, ENTITY_ID 
> DESC);
> UPSERT INTO test.test VALUES ('entity1',1.1);
> SELECT DISTINCT entity_id, score
> FROM(
> SELECT entity_id, score
> FROM test.test
> LIMIT 25
> );
> Output (in SQuirreL)
> ���   1.1
> If you run it in SQuirreL it results in the entity_id column getting the 
> above error value. Notice that if you remove the secondary index or DISTINCT 
> you get the correct result.
> I've also run the query through the Phoenix java api. Then I get the 
> following exception:
> Caused by: java.sql.SQLException: ERROR 201 (22000): Illegal data. CHAR types 
> may only contain single byte characters ()
> at 
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:454)
> at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
> at 
> org.apache.phoenix.schema.types.PDataType.newIllegalDataException(PDataType.java:291)
> at org.apache.phoenix.schema.types.PChar.toObject(PChar.java:121)
> at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:997)
> at 
> org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:608)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:621)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787222#comment-15787222
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 11:18 AM:
--

I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when the SQL has a SubQuery.

When I debug into the source code,I find the cause of the problem is the 
distinct(or group by) statement in the outer query.By the following code in 
GroupByCompiler.GroupBy.compile method, the "entity" column in GroupBy's 
expressions is ProjectedColumnExpression,but in line 245, the "entity" column 
in GroupBy's keyExpressions is CoerceExpression wrapping 
ProjectedColumnExpression ,which would convert the the 
ProjectedColumnExpression from PInteger to PDecimal:
{code:borderStyle=solid} 
232   for (int i = expressions.size()-2; i >= 0; i--) {
233Expression expression = expressions.get(i);
234PDataType keyType = getGroupByDataType(expression);
235if (keyType == expression.getDataType()) {
236continue;
237}
238// Copy expressions only when keyExpressions will be 
different than expressions
239if (keyExpressions == expressions) {
240keyExpressions = new ArrayList(expressions);
241}
242// Wrap expression in an expression that coerces the 
expression to the required type..
243// This is done so that we have a way of expressing null as 
an empty key when more
244// than one fixed and nullable types are used in a group by 
clause
245keyExpressions.set(i, CoerceExpression.create(expression, 
keyType));
246}
{code} 

When I look into CoerceExpression.create method,in line 68 of the following 
code I observe that the SortOder of the CoerceExpression is SortOrder.ASC, but 
it ought to be SortOrder.DESC,because the SortOder of ProjectedColumnExpression 
 is SortOrder.DESC:

{code:borderStyle=solid} 
46 public static Expression create(Expression expression, PDataType toType) 
throws  SQLException {
47if (toType == expression.getDataType()) {
48return expression;
49}
50return new CoerceExpression(expression, toType);
51}
  ..
66   //Package protected for tests
67CoerceExpression(Expression expression, PDataType toType) {
68this(expression, toType, SortOrder.getDefault(), null, true);
69}
{code} 

So when we get the query results, in  
ClientGroupedAggregatingResultIterator.getGroupingKey method, we invoke the 
following PDecimal.coerceBytes method to get the coerceBytes of PDecimal, 
noticed that actualType parameter is PInteger,actualModifier parameter  is 
SortOrder.DESC,and expectedModifier parameter is SortOrder.ASC, so in line 
842,we get the
PInteger  "1" from the ptr which is got from the HBase RegionServer, and in 
line 845,we convert the PInteger "1"  to PDecimal "1", last in line 846, we 
encode the  PDecimal "1" to bytes, but because the expectedModifier parameter 
is SortOrder.ASC, so the PDecimal "1"  is encoded by  SortOrder.ASC.That is to 
say,the SortOrder of the groupBy key got from 
ClientGroupedAggregatingResultIterator.getGroupingKey method is  SortOrder.ASC.

{code:borderStyle=solid}
826 public void coerceBytes(ImmutableBytesWritable ptr, Object o, PDataType 
actualType, Integer actualMaxLength,
827Integer actualScale, SortOrder actualModifier, Integer 
desiredMaxLength, Integer desiredScale,
828SortOrder expectedModifier) {
   
..

840// Optimization for cases in which we already have the object around
841if (o == null) {
842o = actualType.toObject(ptr, actualType, actualModifier);
843}
844
845o = toObject(o, actualType);
846byte[] b = toBytes(o, expectedModifier);
847ptr.set(b);
848}
{code}

Unfortunately, finally in following PhoenixResult.getObject method, when we 
invoke the ColumnProjector.getValue method in line 524,  the ColumnProjector's 
Expression is

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787222#comment-15787222
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 11:15 AM:
--

I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when the SQL has a SubQuery.

When I debug into the source code,I find the cause of the problem is the 
distinct(or group by) statement in the outer query.By the following code in 
GroupByCompiler.GroupBy.compile method, the "entity" column in GroupBy's 
expressions is ProjectedColumnExpression,but in line 245, the "entity" column 
in GroupBy's keyExpressions is CoerceExpression wrapping 
ProjectedColumnExpression ,which would convert the the 
ProjectedColumnExpression from PInteger to PDecimal:
{code:borderStyle=solid} 
232   for (int i = expressions.size()-2; i >= 0; i--) {
233Expression expression = expressions.get(i);
234PDataType keyType = getGroupByDataType(expression);
235if (keyType == expression.getDataType()) {
236continue;
237}
238// Copy expressions only when keyExpressions will be 
different than expressions
239if (keyExpressions == expressions) {
240keyExpressions = new ArrayList(expressions);
241}
242// Wrap expression in an expression that coerces the 
expression to the required type..
243// This is done so that we have a way of expressing null as 
an empty key when more
244// than one fixed and nullable types are used in a group by 
clause
245keyExpressions.set(i, CoerceExpression.create(expression, 
keyType));
246}
{code} 

When I look into CoerceExpression.create method,in line 68 of the following 
code I observe that the SortOder of the CoerceExpression is SortOrder.ASC, but 
it ought to be SortOrder.DESC,because the SortOder of ProjectedColumnExpression 
 is SortOrder.DESC:

{code:borderStyle=solid} 
46 public static Expression create(Expression expression, PDataType toType) 
throws  SQLException {
47if (toType == expression.getDataType()) {
48return expression;
49}
50return new CoerceExpression(expression, toType);
51}
  ..
66   //Package protected for tests
67CoerceExpression(Expression expression, PDataType toType) {
68this(expression, toType, SortOrder.getDefault(), null, true);
69}
{code} 

So when we get the query results, in  
ClientGroupedAggregatingResultIterator.getGroupingKey method, we invoke the 
following PDecimal.coerceBytes method to get the coerceBytes of PDecimal, 
noticed that actualType parameter is PInteger,actualModifier parameter  is 
SortOrder.DESC,and expectedModifier parameter is SortOrder.ASC, so in line 
842,we get the
PInteger  "1" from the ptr which is got from the HBase RegionServer, and in 
line 845,we convert the PInteger "1"  to PDecimal "1", last in line 846, we 
encode the  PDecimal "1" to bytes, but because the expectedModifier parameter 
is SortOrder.ASC, so the PDecimal "1"  is encoded by  SortOrder.ASC.That is to 
say,the SortOrder of the groupBy key got from 
ClientGroupedAggregatingResultIterator.getGroupingKey method is  SortOrder.ASC.

{code:borderStyle=solid}
826 public void coerceBytes(ImmutableBytesWritable ptr, Object o, PDataType 
actualType, Integer actualMaxLength,
827Integer actualScale, SortOrder actualModifier, Integer 
desiredMaxLength, Integer desiredScale,
828SortOrder expectedModifier) {
   
..

840// Optimization for cases in which we already have the object around
841if (o == null) {
842o = actualType.toObject(ptr, actualType, actualModifier);
843}
844
845o = toObject(o, actualType);
846byte[] b = toBytes(o, expectedModifier);
847ptr.set(b);
848}
{code}

Unfortunately, finally in PhoenixResult.getObject method, when we invoke the 
ColumnProjector.getValue method,  the columnProjector's Expression is 
RowKeyColumnExpression

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787222#comment-15787222
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 11:07 AM:
--

I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when the SQL has a SubQuery.

When I debug into the source code,I find the cause of the problem is the 
distinct(or group by) statement in the outer query.By the following code in 
GroupByCompiler.GroupBy.compile method, the "entity" column in GroupBy's 
expressions is ProjectedColumnExpression,but in line 245, the "entity" column 
in GroupBy's keyExpressions is CoerceExpression wrapping 
ProjectedColumnExpression ,which would convert the the 
ProjectedColumnExpression from PInteger to PDecimal:
{code:borderStyle=solid} 
232   for (int i = expressions.size()-2; i >= 0; i--) {
233Expression expression = expressions.get(i);
234PDataType keyType = getGroupByDataType(expression);
235if (keyType == expression.getDataType()) {
236continue;
237}
238// Copy expressions only when keyExpressions will be 
different than expressions
239if (keyExpressions == expressions) {
240keyExpressions = new ArrayList(expressions);
241}
242// Wrap expression in an expression that coerces the 
expression to the required type..
243// This is done so that we have a way of expressing null as 
an empty key when more
244// than one fixed and nullable types are used in a group by 
clause
245keyExpressions.set(i, CoerceExpression.create(expression, 
keyType));
246}
{code} 

When I look into CoerceExpression.create method,in line 68 of the following 
code I observe that the SortOder of the CoerceExpression is SortOrder.ASC, but 
it ought to be SortOrder.DESC,because the SortOder of ProjectedColumnExpression 
 is SortOrder.DESC:

{code:borderStyle=solid} 
46 public static Expression create(Expression expression, PDataType toType) 
throws  SQLException {
47if (toType == expression.getDataType()) {
48return expression;
49}
50return new CoerceExpression(expression, toType);
51}
  ..
66   //Package protected for tests
67CoerceExpression(Expression expression, PDataType toType) {
68this(expression, toType, SortOrder.getDefault(), null, true);
69}
{code} 

So when we get the query results, in 
ClientGroupedAggregatingResultIterator.getGroupingKey method, we invoke the 
PDecimal.coerceBytes method to get the coerceBytes of PDecimal, noticed that 
actualType parameter is PInteger,actualModifier parameter  is 
SortOrder.DESC,and expectedModifier parameter is SortOrder.ASC, so in line 
842,we get the
PInteger  "1" from the ptr which is got from the RegionServer, and in line 
845,we convert the PInteger "1"  to PDecimal "1", last in line 846, we encode 
the  PDecimal "1" to bytes, but because the expectedModifier parameter is 
SortOrder.ASC, so the PDecimal "1"  is encoded by  SortOrder.ASC.

{code:borderStyle=solid}
826 public void coerceBytes(ImmutableBytesWritable ptr, Object o, PDataType 
actualType, Integer actualMaxLength,
827Integer actualScale, SortOrder actualModifier, Integer 
desiredMaxLength, Integer desiredScale,
828SortOrder expectedModifier) {
   
..

840// Optimization for cases in which we already have the object around
841if (o == null) {
842o = actualType.toObject(ptr, actualType, actualModifier);
843}
844
845o = toObject(o, actualType);
846byte[] b = toBytes(o, expectedModifier);
847ptr.set(b);
848}
{code}

Unfortunately, finally in PhoenixResult.getObject method, when we invoke the 
ColumnProjector.getValue method,  the columnProjector's Expression is 
RowKeyColumnExpression,which thinks the SortOrder of bytes got from the 
above-mentioned ClientGroupedAggregatingResultIterator.getGroupingKey method is 
SortOrder.DESC, so it d

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787222#comment-15787222
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 11:03 AM:
--

I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when the SQL has a SubQuery.

When I debug into the source code,I find the cause of the problem is the 
distinct(or group by) statement in the outer query.By the following code in 
GroupByCompiler.GroupBy.compile method, the "entity" column in GroupBy's 
expressions is ProjectedColumnExpression,but in line 245, the "entity" column 
in GroupBy's keyExpressions is CoerceExpression wrapping 
ProjectedColumnExpression ,which would convert the the 
ProjectedColumnExpression from PInteger to PDecimal:
{code:borderStyle=solid} 
232   for (int i = expressions.size()-2; i >= 0; i--) {
233Expression expression = expressions.get(i);
234PDataType keyType = getGroupByDataType(expression);
235if (keyType == expression.getDataType()) {
236continue;
237}
238// Copy expressions only when keyExpressions will be 
different than expressions
239if (keyExpressions == expressions) {
240keyExpressions = new ArrayList(expressions);
241}
242// Wrap expression in an expression that coerces the 
expression to the required type..
243// This is done so that we have a way of expressing null as 
an empty key when more
244// than one fixed and nullable types are used in a group by 
clause
245keyExpressions.set(i, CoerceExpression.create(expression, 
keyType));
246}
{code} 

When I look into CoerceExpression.create method,in line 68 of the following 
code I observe that the SortOder of the CoerceExpression is SortOrder.ASC, but 
it ought to be SortOrder.DESC,because the SortOder of ProjectedColumnExpression 
 is SortOrder.DESC:

{code:borderStyle=solid} 
46 public static Expression create(Expression expression, PDataType toType) 
throws  SQLException {
47if (toType == expression.getDataType()) {
48return expression;
49}
50return new CoerceExpression(expression, toType);
51}
  ..
66   //Package protected for tests
67CoerceExpression(Expression expression, PDataType toType) {
68this(expression, toType, SortOrder.getDefault(), null, true);
69}
{code} 

So when we get the query results, in 
ClientGroupedAggregatingResultIterator.getGroupingKey method, we invoke the 
PDecimal.coerceBytes method to get the coerceBytes of PDecimal, noticed that 
actualType parameter is PInteger,actualModifier parameter  is 
SortOrder.DESC,and expectedModifier parameter is SortOrder.ASC, so in line 
842,we get the
PInteger  "1" from the ptr which is got from the RegionServer, and in line 
845,we convert the PInteger "1"  to PDecimal "1", last in line 846, we encode 
the  PDecimal "1" to bytes, but because the expectedModifier parameter is 
SortOrder.ASC, so the PDecimal "1"  is encoded by  SortOrder.ASC.

{code:borderStyle=solid}
826 public void coerceBytes(ImmutableBytesWritable ptr, Object o, PDataType 
actualType, Integer actualMaxLength,
827Integer actualScale, SortOrder actualModifier, Integer 
desiredMaxLength, Integer desiredScale,
828SortOrder expectedModifier) {
   
..

840// Optimization for cases in which we already have the object around
841if (o == null) {
842o = actualType.toObject(ptr, actualType, actualModifier);
843}
844
845o = toObject(o, actualType);
846byte[] b = toBytes(o, expectedModifier);
847ptr.set(b);
848}
{code:borderStyle=solid}

Unfortunately, finally in PhoenixResult.getObject method, when we invoke the 
ColumnProjector.getValue method,  the columnProjector's Expression is 
RowKeyColumnExpression,which expects the SortOrder of bytes got from the 
above-mentioned ClientGroupedAggregatingResultIterator.getGroupingKey method is 
Sort

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787222#comment-15787222
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 11:04 AM:
--

I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when the SQL has a SubQuery.

When I debug into the source code,I find the cause of the problem is the 
distinct(or group by) statement in the outer query.By the following code in 
GroupByCompiler.GroupBy.compile method, the "entity" column in GroupBy's 
expressions is ProjectedColumnExpression,but in line 245, the "entity" column 
in GroupBy's keyExpressions is CoerceExpression wrapping 
ProjectedColumnExpression ,which would convert the the 
ProjectedColumnExpression from PInteger to PDecimal:
{code:borderStyle=solid} 
232   for (int i = expressions.size()-2; i >= 0; i--) {
233Expression expression = expressions.get(i);
234PDataType keyType = getGroupByDataType(expression);
235if (keyType == expression.getDataType()) {
236continue;
237}
238// Copy expressions only when keyExpressions will be 
different than expressions
239if (keyExpressions == expressions) {
240keyExpressions = new ArrayList(expressions);
241}
242// Wrap expression in an expression that coerces the 
expression to the required type..
243// This is done so that we have a way of expressing null as 
an empty key when more
244// than one fixed and nullable types are used in a group by 
clause
245keyExpressions.set(i, CoerceExpression.create(expression, 
keyType));
246}
{code} 

When I look into CoerceExpression.create method,in line 68 of the following 
code I observe that the SortOder of the CoerceExpression is SortOrder.ASC, but 
it ought to be SortOrder.DESC,because the SortOder of ProjectedColumnExpression 
 is SortOrder.DESC:

{code:borderStyle=solid} 
46 public static Expression create(Expression expression, PDataType toType) 
throws  SQLException {
47if (toType == expression.getDataType()) {
48return expression;
49}
50return new CoerceExpression(expression, toType);
51}
  ..
66   //Package protected for tests
67CoerceExpression(Expression expression, PDataType toType) {
68this(expression, toType, SortOrder.getDefault(), null, true);
69}
{code} 

So when we get the query results, in 
ClientGroupedAggregatingResultIterator.getGroupingKey method, we invoke the 
PDecimal.coerceBytes method to get the coerceBytes of PDecimal, noticed that 
actualType parameter is PInteger,actualModifier parameter  is 
SortOrder.DESC,and expectedModifier parameter is SortOrder.ASC, so in line 
842,we get the
PInteger  "1" from the ptr which is got from the RegionServer, and in line 
845,we convert the PInteger "1"  to PDecimal "1", last in line 846, we encode 
the  PDecimal "1" to bytes, but because the expectedModifier parameter is 
SortOrder.ASC, so the PDecimal "1"  is encoded by  SortOrder.ASC.

{code:borderStyle=solid}
826 public void coerceBytes(ImmutableBytesWritable ptr, Object o, PDataType 
actualType, Integer actualMaxLength,
827Integer actualScale, SortOrder actualModifier, Integer 
desiredMaxLength, Integer desiredScale,
828SortOrder expectedModifier) {
   
..

840// Optimization for cases in which we already have the object around
841if (o == null) {
842o = actualType.toObject(ptr, actualType, actualModifier);
843}
844
845o = toObject(o, actualType);
846byte[] b = toBytes(o, expectedModifier);
847ptr.set(b);
848}
{code}

Unfortunately, finally in PhoenixResult.getObject method, when we invoke the 
ColumnProjector.getValue method,  the columnProjector's Expression is 
RowKeyColumnExpression,which expects the SortOrder of bytes got from the 
above-mentioned ClientGroupedAggregatingResultIterator.getGroupingKey method is 
SortOrder.DESC, so it

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787222#comment-15787222
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 9:09 AM:
-

I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when the SQL has a SubQuery.

When I debuged into the source code,I found the cause of the problem is the 
distinct(or group by) statement in the outer query.By the following code in 
GroupByCompiler.GroupBy.compile method, the "entity" column in GroupBy's 
expressions is ProjectedColumnExpression,but in line 245, the "entity" column 
in GroupBy's keyExpressions is replaced by CoerceExpression,which would convert 
the the "entity" column from PInteger to PDecimal:
{code:borderStyle=solid} 
232   for (int i = expressions.size()-2; i >= 0; i--) {
233Expression expression = expressions.get(i);
234PDataType keyType = getGroupByDataType(expression);
235if (keyType == expression.getDataType()) {
236continue;
237}
238// Copy expressions only when keyExpressions will be 
different than expressions
239if (keyExpressions == expressions) {
240keyExpressions = new ArrayList(expressions);
241}
242// Wrap expression in an expression that coerces the 
expression to the required type..
243// This is done so that we have a way of expressing null as 
an empty key when more
244// than one fixed and nullable types are used in a group by 
clause
245keyExpressions.set(i, CoerceExpression.create(expression, 
keyType));
246}
{code} 


was (Author: comnetwork):
I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when the SQL has a SubQuery.


> Secondary index and query using distinct: Outer query results in ERROR 201 
> (22000): Illegal data. CHAR types may only contain single byte characters
> 
>
> Key: PHOENIX-3453
> URL: https://issues.apache.org/jira/browse/PHOENIX-3453
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Joel Palmert
>Assignee: chenglei
>
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=FALSE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (SCORE DESC, ENTITY_ID 
> DESC);
> UPSERT INTO test.test VALUES ('entity1',1.1);
> SELECT DISTINCT entity_id, score
> FROM(
> SELECT entity_id, score
> FROM test.test
> LIMIT 25
> );
> Output (in SQuirreL)
> ���   1.1
> If you run it in SQuirreL it results in the entity_id column getting the 
> above error value. Notice that if you remove the secondary index or DISTINCT 
> you get the correct result.
> I've also run the query through the Phoenix java api. Then I get the 
> following exception:
> Caused by: java.sql.SQLException: ERROR 201 (22000): Illegal data. CHAR types 
> may only contain single byte characters ()

[jira] [Commented] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787222#comment-15787222
 ] 

chenglei commented on PHOENIX-3453:
---

I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when there SQL has a SubQuery.


> Secondary index and query using distinct: Outer query results in ERROR 201 
> (22000): Illegal data. CHAR types may only contain single byte characters
> 
>
> Key: PHOENIX-3453
> URL: https://issues.apache.org/jira/browse/PHOENIX-3453
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Joel Palmert
>Assignee: chenglei
>
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=FALSE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (SCORE DESC, ENTITY_ID 
> DESC);
> UPSERT INTO test.test VALUES ('entity1',1.1);
> SELECT DISTINCT entity_id, score
> FROM(
> SELECT entity_id, score
> FROM test.test
> LIMIT 25
> );
> Output (in SQuirreL)
> ���   1.1
> If you run it in SQuirreL it results in the entity_id column getting the 
> above error value. Notice that if you remove the secondary index or DISTINCT 
> you get the correct result.
> I've also run the query through the Phoenix java api. Then I get the 
> following exception:
> Caused by: java.sql.SQLException: ERROR 201 (22000): Illegal data. CHAR types 
> may only contain single byte characters ()
> at 
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:454)
> at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
> at 
> org.apache.phoenix.schema.types.PDataType.newIllegalDataException(PDataType.java:291)
> at org.apache.phoenix.schema.types.PChar.toObject(PChar.java:121)
> at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:997)
> at 
> org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:608)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:621)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

2016-12-30 Thread chenglei (JIRA)


[ 
https://issues.apache.org/jira/browse/PHOENIX-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787222#comment-15787222
 ] 

chenglei edited comment on PHOENIX-3453 at 12/30/16 8:40 AM:
-

I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when the SQL has a SubQuery.



was (Author: comnetwork):
I wrote following test case to make this problem can be reproduced under 4.9.0, 
simplifying the original test case by removing the index table and change the 
type from CHAR(15) to Integer,  which is more easier to debug:

{code:borderStyle=solid} 
  CREATE TABLE GROUPBY3453_INT (
ENTITY_ID INTEGER NOT NULL,
CONTAINER_ID INTEGER NOT NULL,
SCORE INTEGER NOT NULL,
CONSTRAINT TEST_PK PRIMARY KEY (ENTITY_ID DESC,CONTAINER_ID 
DESC,SCORE DESC)
  )
 
  UPSERT INTO  GROUPBY3453_INT  VALUES (1,1,1)
  select DISTINCT entity_id, score from ( select entity_id, score from 
GROUPBY3453_INT limit 1)
{code} 

the expecting result is : 
{code:borderStyle=solid} 
   1  1
{code} 

but the actual result is:
{code:borderStyle=solid}
  -104  1
{code} 

This problem can only be reproduced when there SQL has a SubQuery.


> Secondary index and query using distinct: Outer query results in ERROR 201 
> (22000): Illegal data. CHAR types may only contain single byte characters
> 
>
> Key: PHOENIX-3453
> URL: https://issues.apache.org/jira/browse/PHOENIX-3453
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Joel Palmert
>Assignee: chenglei
>
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=FALSE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (SCORE DESC, ENTITY_ID 
> DESC);
> UPSERT INTO test.test VALUES ('entity1',1.1);
> SELECT DISTINCT entity_id, score
> FROM(
> SELECT entity_id, score
> FROM test.test
> LIMIT 25
> );
> Output (in SQuirreL)
> ���   1.1
> If you run it in SQuirreL it results in the entity_id column getting the 
> above error value. Notice that if you remove the secondary index or DISTINCT 
> you get the correct result.
> I've also run the query through the Phoenix java api. Then I get the 
> following exception:
> Caused by: java.sql.SQLException: ERROR 201 (22000): Illegal data. CHAR types 
> may only contain single byte characters ()
> at 
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:454)
> at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
> at 
> org.apache.phoenix.schema.types.PDataType.newIllegalDataException(PDataType.java:291)
> at org.apache.phoenix.schema.types.PChar.toObject(PChar.java:121)
> at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:997)
> at 
> org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:608)
> at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:621)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-1386) ANY function only works with absolute value and doesn't work with other parameters

[jira] [Commented] (PHOENIX-3547) Promote CATALOG.VIEW_INDEX_ID to an int

[jira] [Commented] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Comment Edited] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Commented] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Commented] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Updated] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Commented] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

[jira] [Comment Edited] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters

20 matches

Site Navigation

Mail list logo

Footer information