[jira] [Comment Edited] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15678453#comment-15678453
]
chenglei edited comment on PHOENIX-3491 at 11/19/16 3:11 AM:
-
[~jamestaylor],because I have some time difference with you,sorry for late
response,yes ,I ran the all the existing unit tests and IT tests in my local
machine,seems the https://builds.apache.org/job/PreCommit-PHOENIX-Build/ is
hanging.
was (Author: comnetwork):
[~jamestaylor],because I have time difference with you,sorry for late
response,yes ,I ran the all the existing unit tests and IT tests in my local
machine,seems the https://builds.apache.org/job/PreCommit-PHOENIX-Build/ is
hanging.
> OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy
> is reverse
>
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch, PHOENIX-3491_v2.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST (
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL,
>CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
> {code}
>
> If we execute explain on the following sql:
> {code:borderStyle=solid}
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST GROUP BY ORGANIZATION_ID,
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC
> {code}
> the result is :
> {code:borderStyle=solid}
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE] |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC] |
> +--+
> {code}
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID
> DESC, SCORE DESC is not compiled out,but obviously it should be compiled out
> as OrderBy.REV_ROW_KEY_ORDER_BY.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15678453#comment-15678453
]
chenglei commented on PHOENIX-3491:
---
[~jamestaylor],because I have time difference with you,sorry for late
response,yes ,I ran the all the existing unit tests and IT tests in my local
machine,seems the https://builds.apache.org/job/PreCommit-PHOENIX-Build/ is
hanging.
> OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy
> is reverse
>
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch, PHOENIX-3491_v2.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST (
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL,
>CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
> {code}
>
> If we execute explain on the following sql:
> {code:borderStyle=solid}
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST GROUP BY ORGANIZATION_ID,
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC
> {code}
> the result is :
> {code:borderStyle=solid}
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE] |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC] |
> +--+
> {code}
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID
> DESC, SCORE DESC is not compiled out,but obviously it should be compiled out
> as OrderBy.REV_ROW_KEY_ORDER_BY.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15678332#comment-15678332
]
Hudson commented on PHOENIX-3491:
-
SUCCESS: Integrated in Jenkins build Phoenix-master #1507 (See
[https://builds.apache.org/job/Phoenix-master/1507/])
PHOENIX-3491 OrderBy can not be compiled out if GroupBy is not (jamestaylor:
rev 9109026240ef0a0ece534f0b625a34c49be44568)
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/OrderByIT.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/compile/OrderPreservingTracker.java
* (edit)
phoenix-core/src/test/java/org/apache/phoenix/compile/QueryCompilerTest.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/execute/AggregatePlan.java
> OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy
> is reverse
>
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch, PHOENIX-3491_v2.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST (
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL,
>CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
> {code}
>
> If we execute explain on the following sql:
> {code:borderStyle=solid}
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST GROUP BY ORGANIZATION_ID,
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC
> {code}
> the result is :
> {code:borderStyle=solid}
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE] |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC] |
> +--+
> {code}
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID
> DESC, SCORE DESC is not compiled out,but obviously it should be compiled out
> as OrderBy.REV_ROW_KEY_ORDER_BY.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3501) Push limit to server for distinct optimization
[ https://issues.apache.org/jira/browse/PHOENIX-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15678331#comment-15678331 ] Hudson commented on PHOENIX-3501: - SUCCESS: Integrated in Jenkins build Phoenix-master #1507 (See [https://builds.apache.org/job/Phoenix-master/1507/]) PHOENIX-3501 Push limit to server for distinct optimization (jamestaylor: rev 279eb170761dbe081a0ffe93205f3a9294f4e3ea) * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/DistinctPrefixFilterIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java > Push limit to server for distinct optimization > -- > > Key: PHOENIX-3501 > URL: https://issues.apache.org/jira/browse/PHOENIX-3501 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Fix For: 4.9.0 > > Attachments: PHOENIX-3501.patch > > > Simple addendum fix for PHOENIX-3451 that pushes the limit (through a > PageFilter) when the DistinctPrefixFilter is used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[VOTE] Release of Apache Phoenix 4.9.0 RC5
Hello Everyone, This is a call for a vote on Apache Phoenix 4.9.0 RC5. This is the next minor release of Phoenix 4, compatible with Apache HBase 0.98, 1.1 & 1.2. The release includes both a source-only release and a convenience binary release for each supported HBase version. The previous RC was sunk due to PHOENIX-3461 which has now been fixed. This release has feature parity with supported HBase versions and includes the following improvements: - Atomic UPSERT through new ON DUPLICATE KEY syntax [1] - Support for DEFAULT declaration in DDL statements [2] - Specify guidepost width per table [3] - Over 40 bugs fixed The source tarball, including signatures, digests, etc can be found at: https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.9.0-HBase-0.98-rc5/src/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.9.0-HBase-1.1-rc5/src/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.9.0-HBase-1.2-rc5/src/ The binary artifacts can be found at: https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.9.0-HBase-0.98-rc5/bin/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.9.0-HBase-1.1-rc5/bin/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.9.0-HBase-1.2-rc5/bin/ For a complete list of changes, see: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120&version=12335845 Release artifacts are signed with the following key: https://people.apache.org/keys/committer/mujtaba.asc https://dist.apache.org/repos/dist/release/phoenix/KEYS The hash and tag to be voted upon: https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=30bb79b7abce29f415494214fc980208ff20e25f https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.9.0-HBase-0.98-rc5 https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=4d2602ae5f112105034fa071ce1a7cfb8b113389 https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.9.0-HBase-1.1-rc5 https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=279eb170761dbe081a0ffe93205f3a9294f4e3ea https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.9.0-HBase-1.2-rc5 Vote will be open for at least 72 hours. Please vote: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Thanks, The Apache Phoenix Team [1] https://issues.apache.org/jira/browse/PHOENIX-6 [2] https://issues.apache.org/jira/browse/PHOENIX-476 [3] https://issues.apache.org/jira/browse/PHOENIX-2675
Re: [jira] [Updated] (PHOENIX-3501) Push limit to server for distinct optimization
+1 On Fri, Nov 18, 2016 at 3:20 PM James Taylor (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/PHOENIX-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel > ] > > James Taylor updated PHOENIX-3501: > -- > Attachment: PHOENIX-3501.patch > > Please review, [~samarthjain]. > > > Push limit to server for distinct optimization > > -- > > > > Key: PHOENIX-3501 > > URL: https://issues.apache.org/jira/browse/PHOENIX-3501 > > Project: Phoenix > > Issue Type: Bug > >Reporter: James Taylor > >Assignee: James Taylor > > Fix For: 4.9.0 > > > > Attachments: PHOENIX-3501.patch > > > > > > Simple addendum fix for PHOENIX-3451 that pushes the limit (through a > PageFilter) when the DistinctPrefixFilter is used. > > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332) >
[jira] [Updated] (PHOENIX-3501) Push limit to server for distinct optimization
[ https://issues.apache.org/jira/browse/PHOENIX-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-3501: -- Attachment: PHOENIX-3501.patch Please review, [~samarthjain]. > Push limit to server for distinct optimization > -- > > Key: PHOENIX-3501 > URL: https://issues.apache.org/jira/browse/PHOENIX-3501 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Fix For: 4.9.0 > > Attachments: PHOENIX-3501.patch > > > Simple addendum fix for PHOENIX-3451 that pushes the limit (through a > PageFilter) when the DistinctPrefixFilter is used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-3501) Push limit to server for distinct optimization
James Taylor created PHOENIX-3501: - Summary: Push limit to server for distinct optimization Key: PHOENIX-3501 URL: https://issues.apache.org/jira/browse/PHOENIX-3501 Project: Phoenix Issue Type: Bug Reporter: James Taylor Assignee: James Taylor Fix For: 4.9.0 Simple addendum fix for PHOENIX-3451 that pushes the limit (through a PageFilter) when the DistinctPrefixFilter is used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3484) Major compaction does not update statistics based on table specific GUIDE_POSTS_WIDTH
[
https://issues.apache.org/jira/browse/PHOENIX-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15678017#comment-15678017
]
Hudson commented on PHOENIX-3484:
-
SUCCESS: Integrated in Jenkins build Phoenix-master #1506 (See
[https://builds.apache.org/job/Phoenix-master/1506/])
PHOENIX-3484 Major compaction does not update statistics based on table
(jamestaylor: rev 66d6bba6d8fa1d58eb636458f9f7095b8f4f5059)
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/schema/stats/DefaultStatisticsCollector.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/GroupByCaseIT.java
> Major compaction does not update statistics based on table specific
> GUIDE_POSTS_WIDTH
> -
>
> Key: PHOENIX-3484
> URL: https://issues.apache.org/jira/browse/PHOENIX-3484
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.9.0
> Environment: HBase 0.98.17
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3484.patch
>
>
> 1. Create table T and upsert data
> 2. Set {{GUIDE_POSTS_WIDTH}} using {{ALTER T SET GUIDE_POSTS_WIDTH=x}}
> 3. truncate {{SYSTEM.STATS}} and restart HBase
> 4. Major compact T
> Stats are not updated in {{SYSTEM.STATS}} however running {{UPDATE
> STATISTICS}} work fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Phoenix upgrade 4.6.1 to 4.8.1
Hi, We are upgrading apache phoenix from 4.6.1 to 4.8.1. While connection is getting initialized, it's trying to upgrader system tables with missing columns related to 4.7 and 4.8. While altering SYSTEM.CATALOG to add the columns related to 4.7 and 4.8 we are facing the following issues. 1. Its throwing "NEWER_TABLE_FOUND" and the columns are not being added. 2. Its trying to disable Indexes on Views during this process its adding "_IDX_" to the physical table of the view. Whereas at HBase side the table would be "_LOCAL_IDX" because of this reason it's not able to find the table and throwing the exception. Kindly suggest the options to make it work. Thanks Venkat
[jira] [Updated] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-3491:
--
Attachment: PHOENIX-3491_v2.patch
> OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy
> is reverse
>
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch, PHOENIX-3491_v2.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST (
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL,
>CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
> {code}
>
> If we execute explain on the following sql:
> {code:borderStyle=solid}
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST GROUP BY ORGANIZATION_ID,
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC
> {code}
> the result is :
> {code:borderStyle=solid}
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE] |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC] |
> +--+
> {code}
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID
> DESC, SCORE DESC is not compiled out,but obviously it should be compiled out
> as OrderBy.REV_ROW_KEY_ORDER_BY.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (PHOENIX-3484) Major compaction does not update statistics based on table specific GUIDE_POSTS_WIDTH
[
https://issues.apache.org/jira/browse/PHOENIX-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor resolved PHOENIX-3484.
---
Resolution: Fixed
> Major compaction does not update statistics based on table specific
> GUIDE_POSTS_WIDTH
> -
>
> Key: PHOENIX-3484
> URL: https://issues.apache.org/jira/browse/PHOENIX-3484
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.9.0
> Environment: HBase 0.98.17
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3484.patch
>
>
> 1. Create table T and upsert data
> 2. Set {{GUIDE_POSTS_WIDTH}} using {{ALTER T SET GUIDE_POSTS_WIDTH=x}}
> 3. truncate {{SYSTEM.STATS}} and restart HBase
> 4. Major compact T
> Stats are not updated in {{SYSTEM.STATS}} however running {{UPDATE
> STATISTICS}} work fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3484) Major compaction does not update statistics based on table specific GUIDE_POSTS_WIDTH
[
https://issues.apache.org/jira/browse/PHOENIX-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677786#comment-15677786
]
James Taylor commented on PHOENIX-3484:
---
FYI - a little explanation. For the case of major compaction, we depend on the
DefaultStatisticsCollector constructor setting up the guidePostsInfoWriterMap
here:
{code}
if (family != null) {
ImmutableBytesPtr cfKey = new ImmutableBytesPtr(family);
cachedGuidePosts = new Pair(0l, new
GuidePostsInfoBuilder());
guidePostsInfoWriterMap.put(cfKey, cachedGuidePosts);
{code}
The relatively new init() method was clearing this map which wasn't correct.
The DefaultStatisticsCollector is used once during compaction or update
statistics, so there's no need to reset any state.
> Major compaction does not update statistics based on table specific
> GUIDE_POSTS_WIDTH
> -
>
> Key: PHOENIX-3484
> URL: https://issues.apache.org/jira/browse/PHOENIX-3484
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.9.0
> Environment: HBase 0.98.17
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3484.patch
>
>
> 1. Create table T and upsert data
> 2. Set {{GUIDE_POSTS_WIDTH}} using {{ALTER T SET GUIDE_POSTS_WIDTH=x}}
> 3. truncate {{SYSTEM.STATS}} and restart HBase
> 4. Major compact T
> Stats are not updated in {{SYSTEM.STATS}} however running {{UPDATE
> STATISTICS}} work fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3484) Major compaction does not update statistics based on table specific GUIDE_POSTS_WIDTH
[
https://issues.apache.org/jira/browse/PHOENIX-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677768#comment-15677768
]
Samarth Jain commented on PHOENIX-3484:
---
+1, looks good.
> Major compaction does not update statistics based on table specific
> GUIDE_POSTS_WIDTH
> -
>
> Key: PHOENIX-3484
> URL: https://issues.apache.org/jira/browse/PHOENIX-3484
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.9.0
> Environment: HBase 0.98.17
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3484.patch
>
>
> 1. Create table T and upsert data
> 2. Set {{GUIDE_POSTS_WIDTH}} using {{ALTER T SET GUIDE_POSTS_WIDTH=x}}
> 3. truncate {{SYSTEM.STATS}} and restart HBase
> 4. Major compact T
> Stats are not updated in {{SYSTEM.STATS}} however running {{UPDATE
> STATISTICS}} work fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3498) Query with index failed when query back to data table with desc PK column
[
https://issues.apache.org/jira/browse/PHOENIX-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677742#comment-15677742
]
Hudson commented on PHOENIX-3498:
-
SUCCESS: Integrated in Jenkins build Phoenix-master #1505 (See
[https://builds.apache.org/job/Phoenix-master/1505/])
PHOENIX-3498 Query with index failed when query back to data table with
(maryannxue: rev 3b3d8ad3ab3bb0fae013154878ba0467691c5b30)
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/IndexIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/join/HashCacheClient.java
> Query with index failed when query back to data table with desc PK column
> -
>
> Key: PHOENIX-3498
> URL: https://issues.apache.org/jira/browse/PHOENIX-3498
> Project: Phoenix
> Issue Type: Bug
>Reporter: William Yang
>Assignee: William Yang
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3498.patch
>
>
> This bug can be re-pro against two conditions:
> 1. select must hit index table and have to query back to data table
> 2. at least one of the data table's PK column is in DESC order
> see the following SQLs:
> {code}
> create table tt (p1 integer not null, p2 integer not null, a integer, b
> integer constraint pk primary key (p1,p2));
> create index tti on tt (a);
> upsert into tt values (0, 1, 2, 3);
> select /*+index(tt tti)*/ b from tt where a = 2; // will query back
> to data table
> //this SELECT works fine, will return b=3
> {code}
> if we declare ether p1/p2 as DESC, then the same SELECT will return nothing.
> {code}
> create table tt (p1 integer not null, p2 integer not null, a integer, b
> integer constraint pk primary key (p1 desc, p2));
> create index tti on tt (a);
> upsert into tt values (0, 1, 2, 3);
> select /*+index(tt tti)*/ b from tt where a = 2; // return nothing
> {code}
> if p1 is not DESC, but p2 is, SELECT will fail too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3484) Major compaction does not update statistics based on table specific GUIDE_POSTS_WIDTH
[
https://issues.apache.org/jira/browse/PHOENIX-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-3484:
--
Attachment: PHOENIX-3484.patch
Please review, [~samarthjain].
> Major compaction does not update statistics based on table specific
> GUIDE_POSTS_WIDTH
> -
>
> Key: PHOENIX-3484
> URL: https://issues.apache.org/jira/browse/PHOENIX-3484
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.9.0
> Environment: HBase 0.98.17
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3484.patch
>
>
> 1. Create table T and upsert data
> 2. Set {{GUIDE_POSTS_WIDTH}} using {{ALTER T SET GUIDE_POSTS_WIDTH=x}}
> 3. truncate {{SYSTEM.STATS}} and restart HBase
> 4. Major compact T
> Stats are not updated in {{SYSTEM.STATS}} however running {{UPDATE
> STATISTICS}} work fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3484) Major compaction does not update statistics based on table specific GUIDE_POSTS_WIDTH
[
https://issues.apache.org/jira/browse/PHOENIX-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-3484:
--
Fix Version/s: 4.9.0
> Major compaction does not update statistics based on table specific
> GUIDE_POSTS_WIDTH
> -
>
> Key: PHOENIX-3484
> URL: https://issues.apache.org/jira/browse/PHOENIX-3484
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.9.0
> Environment: HBase 0.98.17
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3484.patch
>
>
> 1. Create table T and upsert data
> 2. Set {{GUIDE_POSTS_WIDTH}} using {{ALTER T SET GUIDE_POSTS_WIDTH=x}}
> 3. truncate {{SYSTEM.STATS}} and restart HBase
> 4. Major compact T
> Stats are not updated in {{SYSTEM.STATS}} however running {{UPDATE
> STATISTICS}} work fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3498) Query with index failed when query back to data table with desc PK column
[
https://issues.apache.org/jira/browse/PHOENIX-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-3498:
--
Fix Version/s: 4.9.0
> Query with index failed when query back to data table with desc PK column
> -
>
> Key: PHOENIX-3498
> URL: https://issues.apache.org/jira/browse/PHOENIX-3498
> Project: Phoenix
> Issue Type: Bug
>Reporter: William Yang
>Assignee: William Yang
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3498.patch
>
>
> This bug can be re-pro against two conditions:
> 1. select must hit index table and have to query back to data table
> 2. at least one of the data table's PK column is in DESC order
> see the following SQLs:
> {code}
> create table tt (p1 integer not null, p2 integer not null, a integer, b
> integer constraint pk primary key (p1,p2));
> create index tti on tt (a);
> upsert into tt values (0, 1, 2, 3);
> select /*+index(tt tti)*/ b from tt where a = 2; // will query back
> to data table
> //this SELECT works fine, will return b=3
> {code}
> if we declare ether p1/p2 as DESC, then the same SELECT will return nothing.
> {code}
> create table tt (p1 integer not null, p2 integer not null, a integer, b
> integer constraint pk primary key (p1 desc, p2));
> create index tti on tt (a);
> upsert into tt values (0, 1, 2, 3);
> select /*+index(tt tti)*/ b from tt where a = 2; // return nothing
> {code}
> if p1 is not DESC, but p2 is, SELECT will fail too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (PHOENIX-3498) Query with index failed when query back to data table with desc PK column
[
https://issues.apache.org/jira/browse/PHOENIX-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maryann Xue resolved PHOENIX-3498.
--
Resolution: Fixed
> Query with index failed when query back to data table with desc PK column
> -
>
> Key: PHOENIX-3498
> URL: https://issues.apache.org/jira/browse/PHOENIX-3498
> Project: Phoenix
> Issue Type: Bug
>Reporter: William Yang
>Assignee: William Yang
> Attachments: PHOENIX-3498.patch
>
>
> This bug can be re-pro against two conditions:
> 1. select must hit index table and have to query back to data table
> 2. at least one of the data table's PK column is in DESC order
> see the following SQLs:
> {code}
> create table tt (p1 integer not null, p2 integer not null, a integer, b
> integer constraint pk primary key (p1,p2));
> create index tti on tt (a);
> upsert into tt values (0, 1, 2, 3);
> select /*+index(tt tti)*/ b from tt where a = 2; // will query back
> to data table
> //this SELECT works fine, will return b=3
> {code}
> if we declare ether p1/p2 as DESC, then the same SELECT will return nothing.
> {code}
> create table tt (p1 integer not null, p2 integer not null, a integer, b
> integer constraint pk primary key (p1 desc, p2));
> create index tti on tt (a);
> upsert into tt values (0, 1, 2, 3);
> select /*+index(tt tti)*/ b from tt where a = 2; // return nothing
> {code}
> if p1 is not DESC, but p2 is, SELECT will fail too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3497) Provide a work around for HBASE-17122
[ https://issues.apache.org/jira/browse/PHOENIX-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677413#comment-15677413 ] Hudson commented on PHOENIX-3497: - SUCCESS: Integrated in Jenkins build Phoenix-4.8-HBase-1.2 #64 (See [https://builds.apache.org/job/Phoenix-4.8-HBase-1.2/64/]) Revert "PHOENIX-3497 Provide a work around for HBASE-17122" (Samarth: rev 3d4205123f763fdfd875211d551da42deeb78412) * (edit) phoenix-core/src/main/java/org/apache/phoenix/util/ServerUtil.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/AlterTableIT.java > Provide a work around for HBASE-17122 > - > > Key: PHOENIX-3497 > URL: https://issues.apache.org/jira/browse/PHOENIX-3497 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Fix For: 4.9.0, 4.8.2 > > Attachments: PHOENIX-3497.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-3498) Query with index failed when query back to data table with desc PK column
[
https://issues.apache.org/jira/browse/PHOENIX-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677392#comment-15677392
]
Maryann Xue edited comment on PHOENIX-3498 at 11/18/16 6:36 PM:
+1 to the patch. Thank you very much for fixing this issue, [~yhxx511]! I'll go
ahead and commit it.
was (Author: maryannxue):
+1 to the patch. Thank you very much for fixing this issue, [~yhxx511]!
> Query with index failed when query back to data table with desc PK column
> -
>
> Key: PHOENIX-3498
> URL: https://issues.apache.org/jira/browse/PHOENIX-3498
> Project: Phoenix
> Issue Type: Bug
>Reporter: William Yang
>Assignee: William Yang
> Attachments: PHOENIX-3498.patch
>
>
> This bug can be re-pro against two conditions:
> 1. select must hit index table and have to query back to data table
> 2. at least one of the data table's PK column is in DESC order
> see the following SQLs:
> {code}
> create table tt (p1 integer not null, p2 integer not null, a integer, b
> integer constraint pk primary key (p1,p2));
> create index tti on tt (a);
> upsert into tt values (0, 1, 2, 3);
> select /*+index(tt tti)*/ b from tt where a = 2; // will query back
> to data table
> //this SELECT works fine, will return b=3
> {code}
> if we declare ether p1/p2 as DESC, then the same SELECT will return nothing.
> {code}
> create table tt (p1 integer not null, p2 integer not null, a integer, b
> integer constraint pk primary key (p1 desc, p2));
> create index tti on tt (a);
> upsert into tt values (0, 1, 2, 3);
> select /*+index(tt tti)*/ b from tt where a = 2; // return nothing
> {code}
> if p1 is not DESC, but p2 is, SELECT will fail too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3498) Query with index failed when query back to data table with desc PK column
[
https://issues.apache.org/jira/browse/PHOENIX-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677392#comment-15677392
]
Maryann Xue commented on PHOENIX-3498:
--
+1 to the patch. Thank you very much for fixing this issue, [~yhxx511]!
> Query with index failed when query back to data table with desc PK column
> -
>
> Key: PHOENIX-3498
> URL: https://issues.apache.org/jira/browse/PHOENIX-3498
> Project: Phoenix
> Issue Type: Bug
>Reporter: William Yang
>Assignee: William Yang
> Attachments: PHOENIX-3498.patch
>
>
> This bug can be re-pro against two conditions:
> 1. select must hit index table and have to query back to data table
> 2. at least one of the data table's PK column is in DESC order
> see the following SQLs:
> {code}
> create table tt (p1 integer not null, p2 integer not null, a integer, b
> integer constraint pk primary key (p1,p2));
> create index tti on tt (a);
> upsert into tt values (0, 1, 2, 3);
> select /*+index(tt tti)*/ b from tt where a = 2; // will query back
> to data table
> //this SELECT works fine, will return b=3
> {code}
> if we declare ether p1/p2 as DESC, then the same SELECT will return nothing.
> {code}
> create table tt (p1 integer not null, p2 integer not null, a integer, b
> integer constraint pk primary key (p1 desc, p2));
> create index tti on tt (a);
> upsert into tt values (0, 1, 2, 3);
> select /*+index(tt tti)*/ b from tt where a = 2; // return nothing
> {code}
> if p1 is not DESC, but p2 is, SELECT will fail too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677357#comment-15677357
]
James Taylor commented on PHOENIX-3491:
---
Really excellent work, [~comnetwork]. The tests are very thorough. Did you get
a chance to run all unit and IT tests as well?
> OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy
> is reverse
>
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST (
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL,
>CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
> {code}
>
> If we execute explain on the following sql:
> {code:borderStyle=solid}
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST GROUP BY ORGANIZATION_ID,
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC
> {code}
> the result is :
> {code:borderStyle=solid}
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE] |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC] |
> +--+
> {code}
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID
> DESC, SCORE DESC is not compiled out,but obviously it should be compiled out
> as OrderBy.REV_ROW_KEY_ORDER_BY.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-3497) Provide a work around for HBASE-17122
[ https://issues.apache.org/jira/browse/PHOENIX-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677317#comment-15677317 ] Samarth Jain edited comment on PHOENIX-3497 at 11/18/16 6:09 PM: - Filed PHOENIX-3500 to address the flapper. was (Author: samarthjain): Filed PHOENIX-3550 to address the flapper. > Provide a work around for HBASE-17122 > - > > Key: PHOENIX-3497 > URL: https://issues.apache.org/jira/browse/PHOENIX-3497 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Fix For: 4.9.0, 4.8.2 > > Attachments: PHOENIX-3497.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3497) Provide a work around for HBASE-17122
[ https://issues.apache.org/jira/browse/PHOENIX-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677317#comment-15677317 ] Samarth Jain commented on PHOENIX-3497: --- Filed PHOENIX-3550 to address the flapper. > Provide a work around for HBASE-17122 > - > > Key: PHOENIX-3497 > URL: https://issues.apache.org/jira/browse/PHOENIX-3497 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Fix For: 4.9.0, 4.8.2 > > Attachments: PHOENIX-3497.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-3500) Fix flapping MutableIndexFailureIT
Samarth Jain created PHOENIX-3500: - Summary: Fix flapping MutableIndexFailureIT Key: PHOENIX-3500 URL: https://issues.apache.org/jira/browse/PHOENIX-3500 Project: Phoenix Issue Type: Bug Reporter: Samarth Jain -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3497) Provide a work around for HBASE-17122
[ https://issues.apache.org/jira/browse/PHOENIX-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677281#comment-15677281 ] James Taylor commented on PHOENIX-3497: --- Failing due to NamespaceNotFoundException: 67.195.81.154:39534 org.apache.hadoop.hbase.NamespaceNotFoundException: T01 at org.apache.hadoop.hbase.master.HMaster.getNamespaceDescriptor(HMaster.java:2499) at org.apache.hadoop.hbase.master.MasterRpcServices.getNamespaceDescriptor(MasterRpcServices.java:810) > Provide a work around for HBASE-17122 > - > > Key: PHOENIX-3497 > URL: https://issues.apache.org/jira/browse/PHOENIX-3497 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Fix For: 4.9.0, 4.8.2 > > Attachments: PHOENIX-3497.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3497) Provide a work around for HBASE-17122
[ https://issues.apache.org/jira/browse/PHOENIX-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677215#comment-15677215 ] Hudson commented on PHOENIX-3497: - FAILURE: Integrated in Jenkins build Phoenix-master #1504 (See [https://builds.apache.org/job/Phoenix-master/1504/]) Revert "PHOENIX-3497 Provide a work around for HBASE-17122" (Samarth: rev 8152131e41f9944940b1fd071d815dd5199d5a63) * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/AlterTableIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/util/ServerUtil.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java > Provide a work around for HBASE-17122 > - > > Key: PHOENIX-3497 > URL: https://issues.apache.org/jira/browse/PHOENIX-3497 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Fix For: 4.9.0, 4.8.2 > > Attachments: PHOENIX-3497.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3495) Ignore NamespaceExistException when namespace mapping enabled
[ https://issues.apache.org/jira/browse/PHOENIX-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677081#comment-15677081 ] Ankit Singhal commented on PHOENIX-3495: Actually, the problem is that when CSVBulkLoadTool run without hbase-client.jar/phoenix-client.jar in classpath, it is not finding NamespaceExistException class to initialize at client even though class exists in phoenix-client.jar. And, during unwrapping of remote exception by using hadoop API , ClassNotFoundException(or any exception come due to initialization) is absorbed and unwrapping of RemoteWithExtrasException from NamespaceExistsException is ignored . As a result, RpcRetryingCaller will keep on calling remote method for creating namespace till hbase.client.retries.number as RemoteWithExtrasException doesn't extend DoNotRetryIOException and at the end RemoteWithExtrasException is thrown which is not expected to be caught by the client. I'm suspecting it is due to the difference in classloader used by runJar(context classloader) than the hadoop(current classloader). Let me see tomorrow if a fix is possible at Phoenix side or we can take help from hadoop. It works fine with sqlline.py and others because they include phoenix-client.jar always in their class path. > Ignore NamespaceExistException when namespace mapping enabled > - > > Key: PHOENIX-3495 > URL: https://issues.apache.org/jira/browse/PHOENIX-3495 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-3495.patch, PHOENIX-3495_v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3495) Ignore NamespaceExistException when namespace mapping enabled
[ https://issues.apache.org/jira/browse/PHOENIX-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677030#comment-15677030 ] Samarth Jain commented on PHOENIX-3495: --- I haven't committed the v2 patch yet. > Ignore NamespaceExistException when namespace mapping enabled > - > > Key: PHOENIX-3495 > URL: https://issues.apache.org/jira/browse/PHOENIX-3495 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-3495.patch, PHOENIX-3495_v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3495) Ignore NamespaceExistException when namespace mapping enabled
[ https://issues.apache.org/jira/browse/PHOENIX-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-3495: -- Fix Version/s: (was: 4.8.2) (was: 4.9.0) > Ignore NamespaceExistException when namespace mapping enabled > - > > Key: PHOENIX-3495 > URL: https://issues.apache.org/jira/browse/PHOENIX-3495 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-3495.patch, PHOENIX-3495_v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3495) Ignore NamespaceExistException when namespace mapping enabled
[ https://issues.apache.org/jira/browse/PHOENIX-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677014#comment-15677014 ] James Taylor commented on PHOENIX-3495: --- Thanks for the follow up, [~an...@apache.org]. Should [~samarthjain] revert his change or leave it in? > Ignore NamespaceExistException when namespace mapping enabled > - > > Key: PHOENIX-3495 > URL: https://issues.apache.org/jira/browse/PHOENIX-3495 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Fix For: 4.9.0, 4.8.2 > > Attachments: PHOENIX-3495.patch, PHOENIX-3495_v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676508#comment-15676508
]
chenglei edited comment on PHOENIX-3491 at 11/18/16 1:17 PM:
-
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
123/*
124 * When a GROUP BY is not order preserving, we
cannot do a reverse
125 * scan to eliminate the ORDER BY since our
server-side scan is not
126 * ordered in that case.
127 */
128if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
129isOrderPreserving = false;
130isReverse = false;
131return;
132}
{code}
The above code and its comment seem very strange, in fact,if GroupBy is not
orderPreserving, AggregatePlan would sort the aggregated Keys after geting
results from RegionServer at the client side, but AggregatePlan always uses ASC
order to sort the aggregated Keys, just as the following code,so if we just
remove above code, the query result will be error when OrderBy is
OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
142 }
{code}
Therefore,we can modify the AggregatePlan to make it can sort the aggregated
Keys ASC or DESC:
{code:borderStyle=solid}
public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
Expression expression = RowKeyExpression.INSTANCE;
boolean isNullsLast=false;
boolean isAscending=true;
if(this.orderBy==OrderBy.REV_ROW_KEY_ORDER_BY) {
isNullsLast=true; //which is needed for the whole rowKey.
isAscending=false;
}
OrderByExpression orderByExpression = new
OrderByExpression(expression, isNullsLast, isAscending);
int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
}
{code}
After modifying AggregatePlan ,It seems we can remove the code in
OrderPreservingTracker,
I include a lot unit tests and IT tests in my patch to check it, such as
ASC/DESC, salted Table, mult-region table and NULLS FIRST/LAST.
was (Author: comnetwork):
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
123/*
124 * When a GROUP BY is not order preserving, we
cannot do a reverse
125 * scan to eliminate the ORDER BY since our
server-side scan is not
126 * ordered in that case.
127 */
128if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
129isOrderPreserving = false;
130isReverse = false;
131return;
132}
{code}
The above code and its comment seems very strange, in fact,if GroupBy is not
orderPreserving, AggregatePlan would sort the aggregated Keys after geting
results from RegionServer at the client side, but AggregatePlan always uses ASC
order to sort the aggregated Keys, just as the following code,so if we just
remove above code, the query result will be error when OrderBy is
OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new OrderedR
[jira] [Comment Edited] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676508#comment-15676508
]
chenglei edited comment on PHOENIX-3491 at 11/18/16 1:16 PM:
-
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
123/*
124 * When a GROUP BY is not order preserving, we
cannot do a reverse
125 * scan to eliminate the ORDER BY since our
server-side scan is not
126 * ordered in that case.
127 */
128if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
129isOrderPreserving = false;
130isReverse = false;
131return;
132}
{code}
The above code and its comment seems very strange, in fact,if GroupBy is not
orderPreserving, AggregatePlan would sort the aggregated Keys after geting
results from RegionServer at the client side, but AggregatePlan always uses ASC
order to sort the aggregated Keys, just as the following code,so if we just
remove above code, the query result will be error when OrderBy is
OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
142 }
{code}
Therefore,we can modify the AggregatePlan to make it can sort the aggregated
Keys ASC or DESC:
{code:borderStyle=solid}
public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
Expression expression = RowKeyExpression.INSTANCE;
boolean isNullsLast=false;
boolean isAscending=true;
if(this.orderBy==OrderBy.REV_ROW_KEY_ORDER_BY) {
isNullsLast=true; //which is needed for the whole rowKey.
isAscending=false;
}
OrderByExpression orderByExpression = new
OrderByExpression(expression, isNullsLast, isAscending);
int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
}
{code}
After modifying AggregatePlan ,It seems we can remove the code in
OrderPreservingTracker,
I include a lot unit tests and IT tests in my patch to check it, such as
ASC/DESC, salted Table, mult-region table and NULLS FIRST/LAST.
was (Author: comnetwork):
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
123/*
124 * When a GROUP BY is not order preserving, we
cannot do a reverse
125 * scan to eliminate the ORDER BY since our
server-side scan is not
126 * ordered in that case.
127 */
128if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
129isOrderPreserving = false;
130isReverse = false;
131return;
132}
{code}
The above code and its comment seems very strange, in fact,if GroupBy is not
orderPreserving, AggregatePlan would sort the aggregated Keys after geting
results from RegionServer at the client side, but AggregatePlan always uses ASC
order to sort the aggregated Keys, just as the following code,so if we just
remove above code, the query result will be error when OrderBy is
OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new Ordered
[jira] [Comment Edited] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676508#comment-15676508
]
chenglei edited comment on PHOENIX-3491 at 11/18/16 1:15 PM:
-
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
123/*
124 * When a GROUP BY is not order preserving, we
cannot do a reverse
125 * scan to eliminate the ORDER BY since our
server-side scan is not
126 * ordered in that case.
127 */
128if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
129isOrderPreserving = false;
130isReverse = false;
131return;
132}
{code}
The above code and its comment seems very strange, in fact,if GroupBy is not
orderPreserving, AggregatePlan would sort the aggregated Keys after geting
results from RegionServer at the client side, but AggregatePlan always uses ASC
order to sort the aggregated Keys, just as the following code,so if we just
remove above code, the query result will be error when OrderBy is
OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
142 }
{code}
Therefore,we can modify the AggregatePlan to make it can sort the aggregated
Keys ASC or DESC:
{code:borderStyle=solid}
public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
Expression expression = RowKeyExpression.INSTANCE;
boolean isNullsLast=false;
boolean isAscending=true;
if(this.orderBy==OrderBy.REV_ROW_KEY_ORDER_BY) {
isNullsLast=true; //which is needed for the whole rowKey.
isAscending=false;
}
OrderByExpression orderByExpression = new
OrderByExpression(expression, isNullsLast, isAscending);
int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
}
{code}
After modifying AggregatePlan ,It seems we can remove the code in
OrderPreservingTracker,
I include a lot unit tests and IT tests in my patch to check it, such as
ASC/DESC, Salted Table, mult-region table and NULLS FIRST/LAST.
was (Author: comnetwork):
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
123/*
124 * When a GROUP BY is not order preserving, we
cannot do a reverse
125 * scan to eliminate the ORDER BY since our
server-side scan is not
126 * ordered in that case.
127 */
128if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
129isOrderPreserving = false;
130isReverse = false;
131return;
132}
{code}
The above code and its comment seems very strange, in fact,if GroupBy is not
orderPreserving, AggregatePlan would sort the aggregated Keys after geting
results from RegionServer at the client side, but AggregatePlan always uses ASC
order to sort the aggregated Keys, just as the following code,so if we just
remove above code, the query result will be error when OrderBy is
OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new Ordered
[jira] [Updated] (PHOENIX-3491) OrderBy should be compiled out if GroupBy is not orderPreserving and OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-3491:
--
Summary: OrderBy should be compiled out if GroupBy is not orderPreserving
and OrderBy is reverse (was: OrderBy should be compiled out when GroupBy is
not orderPreserving but OrderBy is reverse)
> OrderBy should be compiled out if GroupBy is not orderPreserving and OrderBy
> is reverse
> ---
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST (
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL,
>CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
> {code}
>
> If we execute explain on the following sql:
> {code:borderStyle=solid}
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST GROUP BY ORGANIZATION_ID,
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC
> {code}
> the result is :
> {code:borderStyle=solid}
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE] |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC] |
> +--+
> {code}
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID
> DESC, SCORE DESC is not compiled out,but obviously it should be compiled out
> as OrderBy.REV_ROW_KEY_ORDER_BY.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3491) OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-3491:
--
Summary: OrderBy can not be compiled out if GroupBy is not orderPreserving
and OrderBy is reverse (was: OrderBy should be compiled out if GroupBy is not
orderPreserving and OrderBy is reverse)
> OrderBy can not be compiled out if GroupBy is not orderPreserving and OrderBy
> is reverse
>
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST (
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL,
>CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
> {code}
>
> If we execute explain on the following sql:
> {code:borderStyle=solid}
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST GROUP BY ORGANIZATION_ID,
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC
> {code}
> the result is :
> {code:borderStyle=solid}
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE] |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC] |
> +--+
> {code}
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID
> DESC, SCORE DESC is not compiled out,but obviously it should be compiled out
> as OrderBy.REV_ROW_KEY_ORDER_BY.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-3491) OrderBy should be compiled out when GroupBy is not orderPreserving but OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676508#comment-15676508
]
chenglei edited comment on PHOENIX-3491 at 11/18/16 12:07 PM:
--
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
123/*
124 * When a GROUP BY is not order preserving, we
cannot do a reverse
125 * scan to eliminate the ORDER BY since our
server-side scan is not
126 * ordered in that case.
127 */
128if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
129isOrderPreserving = false;
130isReverse = false;
131return;
132}
{code}
The above code and its comment seems very strange, in fact,if GroupBy is not
orderPreserving, AggregatePlan would sort the aggregated Keys after geting
results from RegionServer at the client side, but AggregatePlan always uses ASC
order to sort the aggregated Keys, just as the following code,so if we just
remove above code, the query result will be error when OrderBy is
OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
142 }
{code}
Therefore,we can modify the AggregatePlan to make it can sort the aggregated
Keys ASC or DESC:
{code:borderStyle=solid}
public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
Expression expression = RowKeyExpression.INSTANCE;
boolean isNullsLast=false;
boolean isAscending=true;
if(this.orderBy==OrderBy.REV_ROW_KEY_ORDER_BY) {
isNullsLast=true; //which is needed for the whole rowKey.
isAscending=false;
}
OrderByExpression orderByExpression = new
OrderByExpression(expression, isNullsLast, isAscending);
int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
}
{code}
After modifying AggregatePlan ,It seems we can remove the code in
OrderPreservingTracker,
I include a lot of test cases in my patch to prove it, such as ASC/DESC, Salted
Table and NULLS FIRST/LAST.
was (Author: comnetwork):
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
123/*
124 * When a GROUP BY is not order preserving, we
cannot do a reverse
125 * scan to eliminate the ORDER BY since our
server-side scan is not
126 * ordered in that case.
127 */
128if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
129isOrderPreserving = false;
130isReverse = false;
131return;
132}
{code}
The above code and its comment seems very strange, in fact,if GroupBy is not
orderPreserving, AggregatePlan would sort the aggregated Keys after geting
results from RegionServer at the client side, but AggregatePlan always uses ASC
order to sort the aggregated Keys, just as the following code,so if we just
remove above code, the query result will be error when OrderBy is
OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new OrderedResultIterator(scanner,
Co
[jira] [Commented] (PHOENIX-3451) Incorrect determination of preservation of order for an aggregate query leads to incorrect query results
[
https://issues.apache.org/jira/browse/PHOENIX-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676576#comment-15676576
]
chenglei commented on PHOENIX-3451:
---
[~jamestaylor], I filed a JIRA PHOENIX-3491 to remove the following code you
mentioned, and uploaded my patch, please help me review,thank you.
{code:borderStyle=solid}
-/*
- * When a GROUP BY is not order preserving, we cannot
do a reverse
- * scan to eliminate the ORDER BY since our
server-side scan is not
- * ordered in that case.
- */
-if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
-isOrderPreserving = false;
-isReverse = false;
-return;
-}
{code}
> Incorrect determination of preservation of order for an aggregate query leads
> to incorrect query results
>
>
> Key: PHOENIX-3451
> URL: https://issues.apache.org/jira/browse/PHOENIX-3451
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Joel Palmert
>Assignee: chenglei
> Fix For: 4.9.0, 4.8.2
>
> Attachments: PHOENIX-3451_v1.patch, PHOENIX-3451_v2.patch
>
>
> This may be related to PHOENIX-3452 but the behavior is different so filing
> it separately.
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ORGANIZATION_ID CHAR(15) NOT NULL,
> CONTAINER_ID CHAR(15) NOT NULL,
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
> ENTITY_ID DESC);
> UPSERT INTO test.test VALUES ('org2','container2','entityId6',1.1);
> UPSERT INTO test.test VALUES ('org2','container1','entityId5',1.2);
> UPSERT INTO test.test VALUES ('org2','container2','entityId4',1.3);
> UPSERT INTO test.test VALUES ('org2','container1','entityId3',1.4);
> UPSERT INTO test.test VALUES ('org2','container3','entityId7',1.35);
> UPSERT INTO test.test VALUES ('org2','container3','entityId8',1.45);
> EXPLAIN
> SELECT DISTINCT entity_id, score
> FROM test.test
> WHERE organization_id = 'org2'
> AND container_id IN ( 'container1','container2','container3' )
> ORDER BY score DESC
> LIMIT 2
> OUTPUT
> entityId51.2
> entityId31.4
> The expected out out would be
> entityId81.45
> entityId31.4
> You will get the expected output if you remove the secondary index from the
> table or remove distinct from the query.
> As described in PHOENIX-3452 if you run the query without the LIMIT the
> ordering is not correct. However, the 2first results in that ordering is
> still not the onces returned by the limit clause, which makes me think there
> are multiple issues here and why I filed both separately. The rows being
> returned are the ones assigned to container1. It looks like Phoenix is first
> getting the rows from the first container and when it finds that to be enough
> it stops the scan. What it should be doing is getting 2 results for each
> container and then merge then and then limit again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3491) OrderBy should be compiled out when GroupBy is not orderPreserving but OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-3491:
--
Attachment: (was: PHOENIX-3491_v1.patch)
> OrderBy should be compiled out when GroupBy is not orderPreserving but
> OrderBy is reverse
> -
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST (
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL,
>CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
> {code}
>
> If we execute explain on the following sql:
> {code:borderStyle=solid}
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST GROUP BY ORGANIZATION_ID,
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC
> {code}
> the result is :
> {code:borderStyle=solid}
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE] |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC] |
> +--+
> {code}
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID
> DESC, SCORE DESC is not compiled out,but obviously it should be compiled out
> as OrderBy.REV_ROW_KEY_ORDER_BY.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3491) OrderBy should be compiled out when GroupBy is not orderPreserving but OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-3491:
--
Attachment: PHOENIX-3491_v1.patch
> OrderBy should be compiled out when GroupBy is not orderPreserving but
> OrderBy is reverse
> -
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST (
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL,
>CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
> {code}
>
> If we execute explain on the following sql:
> {code:borderStyle=solid}
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST GROUP BY ORGANIZATION_ID,
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC
> {code}
> the result is :
> {code:borderStyle=solid}
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE] |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC] |
> +--+
> {code}
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID
> DESC, SCORE DESC is not compiled out,but obviously it should be compiled out
> as OrderBy.REV_ROW_KEY_ORDER_BY.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3491) OrderBy should be compiled out when GroupBy is not orderPreserving but OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-3491:
--
Attachment: PHOENIX-3491_v1.patch
> OrderBy should be compiled out when GroupBy is not orderPreserving but
> OrderBy is reverse
> -
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Attachments: PHOENIX-3491_v1.patch
>
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST (
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL,
>CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
> {code}
>
> If we execute explain on the following sql:
> {code:borderStyle=solid}
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST GROUP BY ORGANIZATION_ID,
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC
> {code}
> the result is :
> {code:borderStyle=solid}
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE] |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC] |
> +--+
> {code}
> from the above explain result, we can see that the ORDER BY ORGANIZATION_ID
> DESC, SCORE DESC is not compiled out,but obviously it should be compiled out
> as OrderBy.REV_ROW_KEY_ORDER_BY.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-3491) OrderBy should be compiled out when GroupBy is not orderPreserving but OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676508#comment-15676508
]
chenglei edited comment on PHOENIX-3491 at 11/18/16 11:54 AM:
--
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
123/*
124 * When a GROUP BY is not order preserving, we
cannot do a reverse
125 * scan to eliminate the ORDER BY since our
server-side scan is not
126 * ordered in that case.
127 */
128if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
129isOrderPreserving = false;
130isReverse = false;
131return;
132}
{code}
The above code and its comment seems very strange, in fact,if GroupBy is not
orderPreserving, AggregatePlan would sort the aggregated Keys after geting
results from RegionServer at the client side, but AggregatePlan always uses ASC
order to sort the aggregated Keys, just as the following code,so if we just
remove above code, the query result will be error when OrderBy is
OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
142 }
{code}
Therefore,we can modify the AggregatePlan to make it can sort the aggregated
Keys ASC or DESC:
{code:borderStyle=solid}
public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
Expression expression = RowKeyExpression.INSTANCE;
boolean isNullsLast=false;
boolean isAscending=true;
if(this.orderBy==OrderBy.REV_ROW_KEY_ORDER_BY) {
isNullsLast=true; //which is needed for the whole rowKey.
isAscending=false;
}
OrderByExpression orderByExpression = new
OrderByExpression(expression, isNullsLast, isAscending);
int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
}
{code}
After modifying AggregatePlan ,It seems we can remove the code in
OrderPreservingTracker,
I include a lot of test cases to prove it, such as ASC/DESC, Salted Table and
NULLS FIRST/LAST.
was (Author: comnetwork):
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
123/*
124 * When a GROUP BY is not order preserving, we
cannot do a reverse
125 * scan to eliminate the ORDER BY since our
server-side scan is not
126 * ordered in that case.
127 */
128if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
129isOrderPreserving = false;
130isReverse = false;
131return;
132}
{code}
In fact,if GroupBy is not orderPreserving, AggregatePlan would sort the
aggregated Keys after geting results from RegionServer at the client side, but
AggregatePlan always uses ASC order to sort the aggregated Keys, just as the
following code,so if we just remove above code, the query result will be error
when the OrderBy is OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
142
[jira] [Comment Edited] (PHOENIX-3491) OrderBy should be compiled out when GroupBy is not orderPreserving but OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676508#comment-15676508
]
chenglei edited comment on PHOENIX-3491 at 11/18/16 11:24 AM:
--
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
123/*
124 * When a GROUP BY is not order preserving, we
cannot do a reverse
125 * scan to eliminate the ORDER BY since our
server-side scan is not
126 * ordered in that case.
127 */
128if (!groupBy.isEmpty() &&
!groupBy.isOrderPreserving()) {
129isOrderPreserving = false;
130isReverse = false;
131return;
132}
{code}
In fact,if GroupBy is not orderPreserving, AggregatePlan would sort the
aggregated Keys after geting results from RegionServer at the client side, but
AggregatePlan always uses ASC order to sort the aggregated Keys, just as the
following code,so if we just remove above code, the query result will be error
when the OrderBy is OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
142 }
{code}
Therefore,we can modify the AggregatePlan to make it can sort the aggregated
Keys ASC or DESC:
{code:borderStyle=solid}
public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
Expression expression = RowKeyExpression.INSTANCE;
boolean isNullsLast=false;
boolean isAscending=true;
if(this.orderBy==OrderBy.REV_ROW_KEY_ORDER_BY) {
isNullsLast=true; //which is needed for the whole rowKey.
isAscending=false;
}
OrderByExpression orderByExpression = new
OrderByExpression(expression, isNullsLast, isAscending);
int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
}
{code}
was (Author: comnetwork):
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
/*
* When a GROUP BY is not order preserving, we cannot
do a reverse
* scan to eliminate the ORDER BY since our server-side
scan is not
* ordered in that case.
*/
if (!groupBy.isEmpty() && !groupBy.isOrderPreserving())
{
isOrderPreserving = false;
isReverse = false;
return;
}
{code}
In fact,if GroupBy is not orderPreserving, AggregatePlan would sort the
aggregated Keys after geting results from RegionServer at the client side, but
AggregatePlan always uses ASC order to sort the aggregated Keys, just as the
following code,so if we just remove above code, the query result will be error
when the OrderBy is OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
142 }
{code}
Therefore,we can modify the AggregatePlan to make it can sort the aggregated
Keys ASC or DESC:
{code:borderStyle=solid}
public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName,
[jira] [Updated] (PHOENIX-3499) Enable null value for quote character for CSVBulkLoad tool
[ https://issues.apache.org/jira/browse/PHOENIX-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Soldatov updated PHOENIX-3499: - Attachment: PHOENIX-3499-1.patch Added support for null values for escape and quote characters. > Enable null value for quote character for CSVBulkLoad tool > -- > > Key: PHOENIX-3499 > URL: https://issues.apache.org/jira/browse/PHOENIX-3499 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.7.0, 4.8.0 >Reporter: Sergey Soldatov >Assignee: Sergey Soldatov > Attachments: PHOENIX-3499-1.patch > > > It's quite often that CSV data contains '"' character as part of the user > data. At the moment user has to replace quote character with something that > is not used in the data. More convenient if we allow to set it to null (using > empty character). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3491) OrderBy should be compiled out when GroupBy is not orderPreserving but OrderBy is reverse
[
https://issues.apache.org/jira/browse/PHOENIX-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676508#comment-15676508
]
chenglei commented on PHOENIX-3491:
---
The OrderBy could not be compiled out because the following code in
OrderPreservingTracker :
{code:borderStyle=solid}
/*
* When a GROUP BY is not order preserving, we cannot
do a reverse
* scan to eliminate the ORDER BY since our server-side
scan is not
* ordered in that case.
*/
if (!groupBy.isEmpty() && !groupBy.isOrderPreserving())
{
isOrderPreserving = false;
isReverse = false;
return;
}
{code}
In fact,if GroupBy is not orderPreserving, AggregatePlan would sort the
aggregated Keys after geting results from RegionServer at the client side, but
AggregatePlan always uses ASC order to sort the aggregated Keys, just as the
following code,so if we just remove above code, the query result will be error
when the OrderBy is OrderBy.REV_ROW_KEY_ORDER_BY :
{code:borderStyle=solid}
137 public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
138Expression expression = RowKeyExpression.INSTANCE;
139OrderByExpression orderByExpression = new
OrderByExpression(expression, false, true);
140int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
141return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
142 }
{code}
Therefore,we can modify the AggregatePlan to make it can sort the aggregated
Keys ASC or DESC:
{code:borderStyle=solid}
public PeekingResultIterator newIterator(StatementContext context,
ResultIterator scanner, Scan scan, String tableName, QueryPlan plan) throws
SQLException {
Expression expression = RowKeyExpression.INSTANCE;
boolean isNullsLast=false;
boolean isAscending=true;
if(this.orderBy==OrderBy.REV_ROW_KEY_ORDER_BY) {
isNullsLast=true; //which is needed for the whole rowKey.
isAscending=false;
}
OrderByExpression orderByExpression = new
OrderByExpression(expression, isNullsLast, isAscending);
int threshold =
services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
return new OrderedResultIterator(scanner,
Collections.singletonList(orderByExpression), threshold);
}
{code}
> OrderBy should be compiled out when GroupBy is not orderPreserving but
> OrderBy is reverse
> -
>
> Key: PHOENIX-3491
> URL: https://issues.apache.org/jira/browse/PHOENIX-3491
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
>
> for the following table:
> {code:borderStyle=solid}
> CREATE TABLE ORDERBY_TEST (
> ORGANIZATION_ID INTEGER NOT NULL,
> CONTAINER_ID INTEGER NOT NULL,
> SCORE INTEGER NOT NULL,
> ENTITY_ID INTEGER NOT NULL,
>CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> SCORE,
> ENTITY_ID
> ));
> {code}
>
> If we execute explain on the following sql:
> {code:borderStyle=solid}
> SELECT ORGANIZATION_ID,SCORE FROM ORDERBY_TEST GROUP BY ORGANIZATION_ID,
> SCORE ORDER BY ORGANIZATION_ID DESC, SCORE DESC
> {code}
> the result is :
> {code:borderStyle=solid}
> --+
> | PLAN |
> +--+
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER ORDERBY_TEST|
> | SERVER FILTER BY FIRST KEY ONLY |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [ORGANIZATION_ID, SCORE] |
> | CLIENT MERGE SORT|
> | CLIENT SORTED BY [ORGANIZATION_ID DESC, SCORE DESC] |
> +--+
> {code}
> from the above explain result, we can see that the ORDER BY ORG
[jira] [Updated] (PHOENIX-3499) Enable null value for quote character for CSVBulkLoad tool
[ https://issues.apache.org/jira/browse/PHOENIX-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Soldatov updated PHOENIX-3499: - Summary: Enable null value for quote character for CSVBulkLoad tool (was: Enable null value for escape character for CSVBulkLoad tool) > Enable null value for quote character for CSVBulkLoad tool > -- > > Key: PHOENIX-3499 > URL: https://issues.apache.org/jira/browse/PHOENIX-3499 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.7.0, 4.8.0 >Reporter: Sergey Soldatov >Assignee: Sergey Soldatov > > It's quite often that CSV data contains '"' character as part of the user > data. At the moment user has to replace escape character with something that > is not used in the data. More convenient if we allow to set it to null (using > empty character). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3499) Enable null value for quote character for CSVBulkLoad tool
[ https://issues.apache.org/jira/browse/PHOENIX-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Soldatov updated PHOENIX-3499: - Description: It's quite often that CSV data contains '"' character as part of the user data. At the moment user has to replace quote character with something that is not used in the data. More convenient if we allow to set it to null (using empty character). (was: It's quite often that CSV data contains '"' character as part of the user data. At the moment user has to replace escape character with something that is not used in the data. More convenient if we allow to set it to null (using empty character).) > Enable null value for quote character for CSVBulkLoad tool > -- > > Key: PHOENIX-3499 > URL: https://issues.apache.org/jira/browse/PHOENIX-3499 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.7.0, 4.8.0 >Reporter: Sergey Soldatov >Assignee: Sergey Soldatov > > It's quite often that CSV data contains '"' character as part of the user > data. At the moment user has to replace quote character with something that > is not used in the data. More convenient if we allow to set it to null (using > empty character). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-3499) Enable null value for escape character for CSVBulkLoad tool
Sergey Soldatov created PHOENIX-3499: Summary: Enable null value for escape character for CSVBulkLoad tool Key: PHOENIX-3499 URL: https://issues.apache.org/jira/browse/PHOENIX-3499 Project: Phoenix Issue Type: Bug Affects Versions: 4.8.0, 4.7.0 Reporter: Sergey Soldatov Assignee: Sergey Soldatov It's quite often that CSV data contains '"' character as part of the user data. At the moment user has to replace escape character with something that is not used in the data. More convenient if we allow to set it to null (using empty character). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3497) Provide a work around for HBASE-17122
[ https://issues.apache.org/jira/browse/PHOENIX-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676415#comment-15676415 ] Hudson commented on PHOENIX-3497: - FAILURE: Integrated in Jenkins build Phoenix-4.8-HBase-1.2 #63 (See [https://builds.apache.org/job/Phoenix-4.8-HBase-1.2/63/]) PHOENIX-3497 Provide a work around for HBASE-17122 (Samarth: rev cd48969c9ef6784f6448c884981e78519eebf1dd) * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/AlterTableIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/util/ServerUtil.java > Provide a work around for HBASE-17122 > - > > Key: PHOENIX-3497 > URL: https://issues.apache.org/jira/browse/PHOENIX-3497 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Fix For: 4.9.0, 4.8.2 > > Attachments: PHOENIX-3497.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3495) Ignore NamespaceExistException when namespace mapping enabled
[
https://issues.apache.org/jira/browse/PHOENIX-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676394#comment-15676394
]
Ankit Singhal commented on PHOENIX-3495:
Yep, patch will certainly avoid race condition but I also did the same change
but reverted in subsequent push to pull request after [~speleato] reported
below error while he was testing namespace with CSVBulkLoad.
{code}
2016-04-26
21:10:38,067|beaver.machine|INFO|2566|140169846208256|MainThread|16/04/26
21:10:38 INFO client.RpcRetryingCaller: Call exception, tries=10, retries=35,
started=48781 ms ago, cancelled=false, msg=
2016-04-26
21:10:58,115|beaver.machine|INFO|2566|140169846208256|MainThread|16/04/26
21:10:58 INFO client.RpcRetryingCaller: Call exception, tries=11, retries=35,
started=68831 ms ago, cancelled=false, msg=
2016-04-26
21:11:18,148|beaver.machine|INFO|2566|140169846208256|MainThread|16/04/26
21:11:18 INFO client.RpcRetryingCaller: Call exception, tries=12, retries=35,
started=88864 ms ago, cancelled=false, msg=
2016-04-26
21:11:38,281|beaver.machine|INFO|2566|140169846208256|MainThread|16/04/26
21:11:38 INFO client.RpcRetryingCaller: Call exception, tries=13, retries=35,
started=108998 ms ago, cancelled=false, msg=
...
...
...
2016-04-26
21:18:00,824|beaver.machine|INFO|2566|140169846208256|MainThread|16/04/26
21:18:00 INFO client.RpcRetryingCaller: Call exception, tries=32, retries=35,
started=491541 ms ago, cancelled=false, msg=
2016-04-26
21:18:21,031|beaver.machine|INFO|2566|140169846208256|MainThread|16/04/26
21:18:21 INFO client.RpcRetryingCaller: Call exception, tries=33, retries=35,
started=511747 ms ago, cancelled=false, msg=
2016-04-26
21:18:41,104|beaver.machine|INFO|2566|140169846208256|MainThread|16/04/26
21:18:41 INFO client.RpcRetryingCaller: Call exception, tries=34, retries=35,
started=531820 ms ago, cancelled=false, msg=
2016-04-26
21:18:41,214|beaver.machine|INFO|2566|140169846208256|MainThread|Exception in
thread "main" org.apache.phoenix.exception.PhoenixIOException: Failed after
attempts=35, exceptions:
2016-04-26 21:18:41,214|beaver.machine|INFO|2566|140169846208256|MainThread|Tue
Apr 26 21:09:49 UTC 2016, RpcRetryingCaller{globalStartTime=1461704989282,
pause=100, retries=35},
org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.NamespaceExistException):
org.apache.hadoop.hbase.NamespaceExistException: SYSTEM
2016-04-26 21:18:41,215|beaver.machine|INFO|2566|140169846208256|MainThread|at
org.apache.hadoop.hbase.master.TableNamespaceManager.create(TableNamespaceManager.java:156)
2016-04-26 21:18:41,215|beaver.machine|INFO|2566|140169846208256|MainThread|at
org.apache.hadoop.hbase.master.TableNamespaceManager.create(TableNamespaceManager.java:131)
2016-04-26 21:18:41,215|beaver.machine|INFO|2566|140169846208256|MainThread|at
org.apache.hadoop.hbase.master.HMaster.createNamespace(HMaster.java:2553)
2016-04-26 21:18:41,216|beaver.machine|INFO|2566|140169846208256|MainThread|at
org.apache.hadoop.hbase.master.MasterRpcServices.createNamespace(MasterRpcServices.java:447)
2016-04-26 21:18:41,216|beaver.machine|INFO|2566|140169846208256|MainThread|at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:58043)
2016-04-26 21:18:41,216|beaver.machine|INFO|2566|140169846208256|MainThread|at
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2115)
2016-04-26 21:18:41,216|beaver.machine|INFO|2566|140169846208256|MainThread|at
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
2016-04-26 21:18:41,217|beaver.machine|INFO|2566|140169846208256|MainThread|at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
2016-04-26 21:18:41,217|beaver.machine|INFO|2566|140169846208256|MainThread|at
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
2016-04-26 21:18:41,217|beaver.machine|INFO|2566|140169846208256|MainThread|at
java.lang.Thread.run(Thread.java:745)
2016-04-26 21:18:41,217|beaver.machine|INFO|2566|140169846208256|MainThread|
{code}
It is expected that HBase should not retry after NamespaceExistsException but
it's not the case when executed with CSVBulkLoadTool. I'll try to reproduce
same with recent version and get some RCA.
> Ignore NamespaceExistException when namespace mapping enabled
> -
>
> Key: PHOENIX-3495
> URL: https://issues.apache.org/jira/browse/PHOENIX-3495
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
>Assignee: Samarth Jain
> Fix For: 4.9.0, 4.8.2
>
> Attachments: PHOENIX-3495.patch, PHOENIX-3495_v2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3497) Provide a work around for HBASE-17122
[ https://issues.apache.org/jira/browse/PHOENIX-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676322#comment-15676322 ] Hudson commented on PHOENIX-3497: - FAILURE: Integrated in Jenkins build Phoenix-master #1503 (See [https://builds.apache.org/job/Phoenix-master/1503/]) PHOENIX-3497 Provide a work around for HBASE-17122 (Samarth: rev 3aa9c43fd5a8612b63edc42507b057ba8215dac4) * (edit) phoenix-core/src/main/java/org/apache/phoenix/util/ServerUtil.java * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/AlterTableIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java > Provide a work around for HBASE-17122 > - > > Key: PHOENIX-3497 > URL: https://issues.apache.org/jira/browse/PHOENIX-3497 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Fix For: 4.9.0, 4.8.2 > > Attachments: PHOENIX-3497.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
