[jira] [Commented] (PHOENIX-4100) Simplify mutable secondary index implementation
[ https://issues.apache.org/jira/browse/PHOENIX-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212186#comment-16212186 ] James Taylor commented on PHOENIX-4100: --- One thought I had which would be pretty straightforward would be to use the PhoenixTransactionalIndexer for global, mutable non transactional index maintenance too. Everything would be the same, we'd just have to allow for the lack of the transaction state to be passed over. > Simplify mutable secondary index implementation > --- > > Key: PHOENIX-4100 > URL: https://issues.apache.org/jira/browse/PHOENIX-4100 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor > Labels: HBase-2.0 > > There's a lot of code for mutable secondary indexes, a lot of which is > commented out (because it's not working under load). With all the latest > patches, in particular PHOENIX-4089, we don't need most of it. Instead, we > can do a simple scan to find the current value of the data row. We won't have > mutations at the same time due to our locking and due to us timestamping the > rows. > This will get rid of many private HBase classes used by Phoenix, such as > IndexMemStore, KeyValueStore, etc. and thus is likely a prerequisite to run > on top of HBase 2.0. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4280) Delete doesn't work when immutable indexes are in building state
[ https://issues.apache.org/jira/browse/PHOENIX-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-4280: -- Attachment: PHOENIX-4280_v11.patch > Delete doesn't work when immutable indexes are in building state > > > Key: PHOENIX-4280 > URL: https://issues.apache.org/jira/browse/PHOENIX-4280 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Fix For: 4.12.1 > > Attachments: PHOENIX-4280.patch, PHOENIX-4280_v10.patch, > PHOENIX-4280_v11.patch, PHOENIX-4280_v2.patch, PHOENIX-4280_v3.patch, > PHOENIX-4280_v4.patch, PHOENIX-4280_v5.patch, PHOENIX-4280_v6.patch, > PHOENIX-4280_v7.patch, PHOENIX-4280_v8.patch, PHOENIX-4280_v9.patch, > PHOENIX-4280_wip.patch > > > We don't generate the Delete mutations correctly in this case as we assume > we're scanning over the index table. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4269) IndexScrutinyToolIT is flapping
[
https://issues.apache.org/jira/browse/PHOENIX-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212112#comment-16212112
]
Hudson commented on PHOENIX-4269:
-
FAILURE: Integrated in Jenkins build Phoenix-master #1845 (See
[https://builds.apache.org/job/Phoenix-master/1845/])
PHOENIX-4269 IndexScrutinyToolIT is flapping (vincentpoon: rev
e50b357a0f8d9c6e8e2e803881e4cc197559b822)
* (edit)
phoenix-core/src/it/java/org/apache/phoenix/end2end/IndexScrutinyToolIT.java
> IndexScrutinyToolIT is flapping
> ---
>
> Key: PHOENIX-4269
> URL: https://issues.apache.org/jira/browse/PHOENIX-4269
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.0
>Reporter: James Taylor
>Assignee: Vincent Poon
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4269.master.patch
>
>
> In a local test run (not able to repro when run separately), I saw the
> following failure:
> {code}
> [ERROR] Tests run: 20, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 193.228 s <<< FAILURE! - in org.apache.phoenix.end2end.IndexScrutinyToolIT
> [ERROR]
> testBothDataAndIndexAsSource[0](org.apache.phoenix.end2end.IndexScrutinyToolIT)
> Time elapsed: 11.708 s <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<0>
> at
> org.apache.phoenix.end2end.IndexScrutinyToolIT.testBothDataAndIndexAsSource(IndexScrutinyToolIT.java:344)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4280) Delete doesn't work when immutable indexes are in building state
[
https://issues.apache.org/jira/browse/PHOENIX-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212105#comment-16212105
]
Hadoop QA commented on PHOENIX-4280:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12893144/PHOENIX-4280_v10.patch
against master branch at commit e50b357a0f8d9c6e8e2e803881e4cc197559b822.
ATTACHMENT ID: 12893144
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+rs = conn.createStatement().executeQuery("SELECT /*+
NO_INDEX*/ COUNT(*) FROM " + fullTableName);
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+String dml = "DELETE from " + fullTableName + " WHERE
varchar_col1='varchar_a' AND varchar_pk='varchar1'";
+
TestUtil.dumpTable(conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(Bytes.toBytes(fullTableName)));
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+Iterator>> iterator =
PhoenixRuntime.getUncommittedDataIterator(conn);
+
assertEquals(SQLExceptionCode.INVALID_FILTER_ON_IMMUTABLE_ROWS.getErrorCode(),
e.getErrorCode());
+IndexMaintainer maintainer =
indexTableRefs.get(i).getTable().getIndexMaintainer(table, connection);
+indexPtr.set(maintainer.buildRowKey(null, indexPtr,
null, null, HConstants.LATEST_TIMESTAMP));
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.DropColumnIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.LocalImmutableNonTxIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SysTableNamespaceMappedStatsCollectorIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1561//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1561//console
This message is automatically generated.
> Delete doesn't work when immutable indexes are in building state
>
>
> Key: PHOENIX-4280
> URL: https://issues.apache.org/jira/browse/PHOENIX-4280
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4280.patch, PHOENIX-4280_v10.patch,
> PHOENIX-4280_v2.patch, PHOENIX-4280_v3.patch, PHOENIX-4280_v4.patch,
> PHOENIX-4280_v5.patch, PHOENIX-4280_v6.patch, PHOENIX-4280_v7.patch,
> PHOENIX-4280_v8.patch, PHOENIX-4280_v9.patch, PHOENIX-4280_wip.patch
>
>
> We don't generate the Delete mutations correctly in this case as we assume
> we're scanning over the index table.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4280) Delete doesn't work when immutable indexes are in building state
[
https://issues.apache.org/jira/browse/PHOENIX-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212080#comment-16212080
]
Hadoop QA commented on PHOENIX-4280:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12893144/PHOENIX-4280_v10.patch
against master branch at commit e50b357a0f8d9c6e8e2e803881e4cc197559b822.
ATTACHMENT ID: 12893144
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+rs = conn.createStatement().executeQuery("SELECT /*+
NO_INDEX*/ COUNT(*) FROM " + fullTableName);
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+String dml = "DELETE from " + fullTableName + " WHERE
varchar_col1='varchar_a' AND varchar_pk='varchar1'";
+
TestUtil.dumpTable(conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(Bytes.toBytes(fullTableName)));
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+Iterator>> iterator =
PhoenixRuntime.getUncommittedDataIterator(conn);
+
assertEquals(SQLExceptionCode.INVALID_FILTER_ON_IMMUTABLE_ROWS.getErrorCode(),
e.getErrorCode());
+IndexMaintainer maintainer =
indexTableRefs.get(i).getTable().getIndexMaintainer(table, connection);
+indexPtr.set(maintainer.buildRowKey(null, indexPtr,
null, null, HConstants.LATEST_TIMESTAMP));
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.DropColumnIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.LocalImmutableNonTxIndexIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1560//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1560//console
This message is automatically generated.
> Delete doesn't work when immutable indexes are in building state
>
>
> Key: PHOENIX-4280
> URL: https://issues.apache.org/jira/browse/PHOENIX-4280
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4280.patch, PHOENIX-4280_v10.patch,
> PHOENIX-4280_v2.patch, PHOENIX-4280_v3.patch, PHOENIX-4280_v4.patch,
> PHOENIX-4280_v5.patch, PHOENIX-4280_v6.patch, PHOENIX-4280_v7.patch,
> PHOENIX-4280_v8.patch, PHOENIX-4280_v9.patch, PHOENIX-4280_wip.patch
>
>
> We don't generate the Delete mutations correctly in this case as we assume
> we're scanning over the index table.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4237) Allow sorting on (Java) collation keys for non-English locales
[ https://issues.apache.org/jira/browse/PHOENIX-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212015#comment-16212015 ] ASF GitHub Bot commented on PHOENIX-4237: - Github user snakhoda-sfdc commented on the issue: https://github.com/apache/phoenix/pull/275 @JamesRTaylor I'm not sure how to do that within this PR. Looking at https://github.com/blog/2141-squash-your-commits, I believe at the time you merge the PR, github should give you the option to squash all commits into one. Will that suffice? > Allow sorting on (Java) collation keys for non-English locales > -- > > Key: PHOENIX-4237 > URL: https://issues.apache.org/jira/browse/PHOENIX-4237 > Project: Phoenix > Issue Type: Improvement >Reporter: Shehzaad Nakhoda > Fix For: 4.12.0 > > > Strings stored via Phoenix can be composed from a subset of the entire set of > Unicode characters. The natural sort order for strings for different > languages often differs from the order dictated by the binary representation > of the characters of these strings. Java provides the idea of a Collator > which given an input string and a (language) locale can generate a Collation > Key which can then be used to compare strings in that natural order. > Salesforce has recently open-sourced grammaticus. IBM has open-sourced ICU4J > some time ago. These technologies can be combined to provide a robust new > Phoenix function that can be used in an ORDER BY clause to sort strings > according to the user's locale. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] phoenix issue #275: PHOENIX-4237: Add function to calculate Java collation k...
Github user snakhoda-sfdc commented on the issue: https://github.com/apache/phoenix/pull/275 @JamesRTaylor I'm not sure how to do that within this PR. Looking at https://github.com/blog/2141-squash-your-commits, I believe at the time you merge the PR, github should give you the option to squash all commits into one. Will that suffice? ---
[jira] [Commented] (PHOENIX-4237) Allow sorting on (Java) collation keys for non-English locales
[ https://issues.apache.org/jira/browse/PHOENIX-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212002#comment-16212002 ] ASF GitHub Bot commented on PHOENIX-4237: - Github user JamesRTaylor commented on the issue: https://github.com/apache/phoenix/pull/275 Would you mind squashing all the commits into a single commit, @shehzaadn and I'll get this committed? > Allow sorting on (Java) collation keys for non-English locales > -- > > Key: PHOENIX-4237 > URL: https://issues.apache.org/jira/browse/PHOENIX-4237 > Project: Phoenix > Issue Type: Improvement >Reporter: Shehzaad Nakhoda > Fix For: 4.12.0 > > > Strings stored via Phoenix can be composed from a subset of the entire set of > Unicode characters. The natural sort order for strings for different > languages often differs from the order dictated by the binary representation > of the characters of these strings. Java provides the idea of a Collator > which given an input string and a (language) locale can generate a Collation > Key which can then be used to compare strings in that natural order. > Salesforce has recently open-sourced grammaticus. IBM has open-sourced ICU4J > some time ago. These technologies can be combined to provide a robust new > Phoenix function that can be used in an ORDER BY clause to sort strings > according to the user's locale. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] phoenix issue #275: PHOENIX-4237: Add function to calculate Java collation k...
Github user JamesRTaylor commented on the issue: https://github.com/apache/phoenix/pull/275 Would you mind squashing all the commits into a single commit, @shehzaadn and I'll get this committed? ---
[jira] [Updated] (PHOENIX-4280) Delete doesn't work when immutable indexes are in building state
[ https://issues.apache.org/jira/browse/PHOENIX-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-4280: -- Attachment: PHOENIX-4280_v10.patch > Delete doesn't work when immutable indexes are in building state > > > Key: PHOENIX-4280 > URL: https://issues.apache.org/jira/browse/PHOENIX-4280 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Fix For: 4.12.1 > > Attachments: PHOENIX-4280.patch, PHOENIX-4280_v10.patch, > PHOENIX-4280_v2.patch, PHOENIX-4280_v3.patch, PHOENIX-4280_v4.patch, > PHOENIX-4280_v5.patch, PHOENIX-4280_v6.patch, PHOENIX-4280_v7.patch, > PHOENIX-4280_v8.patch, PHOENIX-4280_v9.patch, PHOENIX-4280_wip.patch > > > We don't generate the Delete mutations correctly in this case as we assume > we're scanning over the index table. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4242) Fix Indexer post-compact hook logging of NPE and TableNotFound
[ https://issues.apache.org/jira/browse/PHOENIX-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211974#comment-16211974 ] Hudson commented on PHOENIX-4242: - SUCCESS: Integrated in Jenkins build Phoenix-master #1844 (See [https://builds.apache.org/job/Phoenix-master/1844/]) PHOENIX-4242 Fix Indexer post-compact hook logging of NPE and (vincentpoon: rev 53910f9f88cd47f51f68691042306c89118b6ab3) * (edit) phoenix-core/src/main/java/org/apache/phoenix/hbase/index/Indexer.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionObserver.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java * (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/UngroupedAggregateRegionObserverIT.java > Fix Indexer post-compact hook logging of NPE and TableNotFound > -- > > Key: PHOENIX-4242 > URL: https://issues.apache.org/jira/browse/PHOENIX-4242 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Vincent Poon >Assignee: Vincent Poon > Fix For: 4.13.0, 4.12.1 > > Attachments: PHOENIX-4242.v2.master.patch, > PHOENIX-4242.v3.master.patch, PHOENIX-4242.v4.master.patch, > PHOENIX-4747.v1.master.patch > > > The post-compact hook in the Indexer seems to log extraneous log messages > indicating NPE or TableNotFound. The TableNotFound exceptions seem to > indicate actual table names prefixed with MERGE or RESTORE, and sometimes > suffixed with a digit, so perhaps these are views or something similar. > Examples: > 2017-09-28 13:35:03,118 WARN [ctions-1506410238599] index.Indexer - Unable > to permanently disable indexes being partially rebuild for SYSTEM.SEQUENCE > java.lang.NullPointerException > 2017-09-28 10:20:56,406 WARN [ctions-1506410238415] index.Indexer - Unable > to permanently disable indexes being partially rebuild for > MERGE_PLATFORM_ENTITY.PLATFORM_IMMUTABLE_ENTITY_DATA2 > org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03): Table > undefined. tableName=MERGE_PLATFORM_ENTITY.PLATFORM_IMMUTABLE_ENTITY_DATA2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211932#comment-16211932
]
Lars Hofhansl commented on PHOENIX-4292:
At the risk of prolonging the discussion here... It's clear that all of us are
looking for ways to work together, and there's simply some disagreement to the
extent and who does what.
I agree that tests should be part of any *new* feature. Not sure that each bug
fix necessarily needs a test as part of the fix, when there's another jira to
provide tests for the general area. The answer - as so often - it depends, use
good judgement. :)
Same with who should provide a test. It can well be suggested to committers to
provide some tests for the areas they're interested in. Is it worth a big
argument? No.
It seems overall we're good...? We'll have a fix here, and tests in
PHOENIX-3383.
Overall I'd prefer to see more involvement from all committers.
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -- Returns 5 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 >= 'testa'; -- INCORRECT RESULT: This
> query returns 1 record, expected to return 5
> SELECT * FROM TEST."abc" WHERE pk1 <= 'testa'; -- Returns 4 rows as expected
>
Re: [DISCUSS] looking for RM for HBase 0.98 Phoenix releases
Sounds good James. On Thu, Oct 19, 2017 at 2:32 PM, James Taylor wrote: > Looks like PHOENIX-4237 is ready, so this new built-in requires a minor > release. I'll volunteer to RM for a 4.13 release for 0.98 and 1.3 and we > can maintain patch releases for the 4.13-HBase-0.98 release going forward. > > I'd like to target an RC at the end of next week (10/27). > > Sound ok? > > Thanks, > James > > On Mon, Oct 16, 2017 at 5:38 PM, James Taylor > wrote: > > > FYI, fixes for these issues will be ported to the 4.12-HBase-0.98 and > > 4.x-HBase-0.98 branch by the assignee, so no need for you to do that. > > > > On Mon, Oct 16, 2017 at 4:02 PM, Andrew Purtell > > wrote: > > > >> Thanks James. > >> Will start this week, then. > >> > >> > >> On Mon, Oct 16, 2017 at 3:36 PM, James Taylor > >> wrote: > >> > >>> We've found some pretty serious bugs [1] that would be good to get > fixes > >>> out for in a patch release. I was thinking perhaps a simultaneous > 4.12.1 > >>> release for 0.98 and 1.3 in two or three weeks? Might be one or two > other > >>> issues that make it in opportunistically. > >>> > >>> Thanks for asking! > >>> > >>> [1] > >>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20P > >>> HOENIX%20AND%20resolution%20%3D%20Unresolved%20and% > 20fixVersion%3D4.12.1 > >>> > >>> On Mon, Oct 16, 2017 at 3:13 PM, Andrew Purtell > >>> wrote: > >>> > >>> > When are we planning to have the 4.13 release? Any timeline yet. > >>> Probably > >>> > premature to ask, but why not. One month from now? Two? > >>> > > >>> > This will help me plan out time to attend to backports/maintenence of > >>> the > >>> > 0.98 branch ahead of the next RC. > >>> > > >>> > Thanks in advance! > >>> > > >>> > > >>> > > >>> > On Wed, Oct 11, 2017 at 11:09 AM, Andrew Purtell < > apurt...@apache.org> > >>> > wrote: > >>> > > >>> >> I volunteer. > >>> >> I may need help on the backports in the beginning. Consider me a new > >>> guy. > >>> >> > >>> >> > >>> >> On Wed, Oct 11, 2017 at 10:10 AM, James Taylor < > >>> jamestay...@apache.org> > >>> >> wrote: > >>> >> > >>> >>> There's a need to continue doing HBase 0.98 releases. I'm looking > >>> for a > >>> >>> volunteer to be the RM. This would entail keeping an eye on JIRAs > >>> that > >>> >>> make > >>> >>> sense to back port to either the 4.12-HBase-0.98 for a point > release > >>> or > >>> >>> the > >>> >>> 4.x-HBase-0.98 branch for a minor release. Developers/committers > may > >>> back > >>> >>> port a JIRA on their own, however this isn't mandatory, so it'd be > >>> >>> important to be on the dev list and keep an eye out for important > >>> JIRAs. > >>> >>> Once the RM deems it's time (or perhaps after a set amount of time > >>> has > >>> >>> passed), the RM would put together the release bits, run the vote, > >>> push > >>> >>> the > >>> >>> bits out, etc (see [1]). > >>> >>> > >>> >>> Please let us know if you're interested. > >>> >>> > >>> >>> Thanks, > >>> >>> James > >>> >>> > >>> >>> [1] https://phoenix.apache.org/release.html > >>> >>> > >>> > >> > > > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Commented] (PHOENIX-4280) Delete doesn't work when immutable indexes are in building state
[
https://issues.apache.org/jira/browse/PHOENIX-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211852#comment-16211852
]
Hadoop QA commented on PHOENIX-4280:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12893105/PHOENIX-4280_v9.patch
against master branch at commit 0461fe855aaed27e23cb5d17e8be022a82626162.
ATTACHMENT ID: 12893105
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+rs = conn.createStatement().executeQuery("SELECT /*+
NO_INDEX*/ COUNT(*) FROM " + fullTableName);
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+String dml = "DELETE from " + fullTableName + " WHERE
varchar_col1='varchar_a' AND varchar_pk='varchar1'";
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+Iterator>> iterator =
PhoenixRuntime.getUncommittedDataIterator(conn);
+
assertEquals(SQLExceptionCode.INVALID_FILTER_ON_IMMUTABLE_ROWS.getErrorCode(),
e.getErrorCode());
+IndexMaintainer maintainer =
indexTableRefs.get(i).getTable().getIndexMaintainer(table, connection);
+indexPtr.set(maintainer.buildRowKey(null, indexPtr,
null, null, HConstants.LATEST_TIMESTAMP));
+if (index.getIndexState() != PIndexState.DISABLE &&
index.getIndexType() == IndexType.GLOBAL) {
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.tx.TxCheckpointIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.txn.MutableRollbackIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1559//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1559//console
This message is automatically generated.
> Delete doesn't work when immutable indexes are in building state
>
>
> Key: PHOENIX-4280
> URL: https://issues.apache.org/jira/browse/PHOENIX-4280
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4280.patch, PHOENIX-4280_v2.patch,
> PHOENIX-4280_v3.patch, PHOENIX-4280_v4.patch, PHOENIX-4280_v5.patch,
> PHOENIX-4280_v6.patch, PHOENIX-4280_v7.patch, PHOENIX-4280_v8.patch,
> PHOENIX-4280_v9.patch, PHOENIX-4280_wip.patch
>
>
> We don't generate the Delete mutations correctly in this case as we assume
> we're scanning over the index table.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Re: [DISCUSS] looking for RM for HBase 0.98 Phoenix releases
Looks like PHOENIX-4237 is ready, so this new built-in requires a minor release. I'll volunteer to RM for a 4.13 release for 0.98 and 1.3 and we can maintain patch releases for the 4.13-HBase-0.98 release going forward. I'd like to target an RC at the end of next week (10/27). Sound ok? Thanks, James On Mon, Oct 16, 2017 at 5:38 PM, James Taylor wrote: > FYI, fixes for these issues will be ported to the 4.12-HBase-0.98 and > 4.x-HBase-0.98 branch by the assignee, so no need for you to do that. > > On Mon, Oct 16, 2017 at 4:02 PM, Andrew Purtell > wrote: > >> Thanks James. >> Will start this week, then. >> >> >> On Mon, Oct 16, 2017 at 3:36 PM, James Taylor >> wrote: >> >>> We've found some pretty serious bugs [1] that would be good to get fixes >>> out for in a patch release. I was thinking perhaps a simultaneous 4.12.1 >>> release for 0.98 and 1.3 in two or three weeks? Might be one or two other >>> issues that make it in opportunistically. >>> >>> Thanks for asking! >>> >>> [1] >>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20P >>> HOENIX%20AND%20resolution%20%3D%20Unresolved%20and%20fixVersion%3D4.12.1 >>> >>> On Mon, Oct 16, 2017 at 3:13 PM, Andrew Purtell >>> wrote: >>> >>> > When are we planning to have the 4.13 release? Any timeline yet. >>> Probably >>> > premature to ask, but why not. One month from now? Two? >>> > >>> > This will help me plan out time to attend to backports/maintenence of >>> the >>> > 0.98 branch ahead of the next RC. >>> > >>> > Thanks in advance! >>> > >>> > >>> > >>> > On Wed, Oct 11, 2017 at 11:09 AM, Andrew Purtell >>> > wrote: >>> > >>> >> I volunteer. >>> >> I may need help on the backports in the beginning. Consider me a new >>> guy. >>> >> >>> >> >>> >> On Wed, Oct 11, 2017 at 10:10 AM, James Taylor < >>> jamestay...@apache.org> >>> >> wrote: >>> >> >>> >>> There's a need to continue doing HBase 0.98 releases. I'm looking >>> for a >>> >>> volunteer to be the RM. This would entail keeping an eye on JIRAs >>> that >>> >>> make >>> >>> sense to back port to either the 4.12-HBase-0.98 for a point release >>> or >>> >>> the >>> >>> 4.x-HBase-0.98 branch for a minor release. Developers/committers may >>> back >>> >>> port a JIRA on their own, however this isn't mandatory, so it'd be >>> >>> important to be on the dev list and keep an eye out for important >>> JIRAs. >>> >>> Once the RM deems it's time (or perhaps after a set amount of time >>> has >>> >>> passed), the RM would put together the release bits, run the vote, >>> push >>> >>> the >>> >>> bits out, etc (see [1]). >>> >>> >>> >>> Please let us know if you're interested. >>> >>> >>> >>> Thanks, >>> >>> James >>> >>> >>> >>> [1] https://phoenix.apache.org/release.html >>> >>> >>> >> >
[jira] [Commented] (PHOENIX-4237) Allow sorting on (Java) collation keys for non-English locales
[ https://issues.apache.org/jira/browse/PHOENIX-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211765#comment-16211765 ] ASF GitHub Bot commented on PHOENIX-4237: - Github user JamesRTaylor commented on the issue: https://github.com/apache/phoenix/pull/275 +1. Nice work, @shehzaadn! > Allow sorting on (Java) collation keys for non-English locales > -- > > Key: PHOENIX-4237 > URL: https://issues.apache.org/jira/browse/PHOENIX-4237 > Project: Phoenix > Issue Type: Improvement >Reporter: Shehzaad Nakhoda > Fix For: 4.12.0 > > > Strings stored via Phoenix can be composed from a subset of the entire set of > Unicode characters. The natural sort order for strings for different > languages often differs from the order dictated by the binary representation > of the characters of these strings. Java provides the idea of a Collator > which given an input string and a (language) locale can generate a Collation > Key which can then be used to compare strings in that natural order. > Salesforce has recently open-sourced grammaticus. IBM has open-sourced ICU4J > some time ago. These technologies can be combined to provide a robust new > Phoenix function that can be used in an ORDER BY clause to sort strings > according to the user's locale. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] phoenix issue #275: PHOENIX-4237: Add function to calculate Java collation k...
Github user JamesRTaylor commented on the issue: https://github.com/apache/phoenix/pull/275 +1. Nice work, @shehzaadn! ---
[jira] [Updated] (PHOENIX-4289) UPDATE STATISTICS command does not collect stats for local indexes
[
https://issues.apache.org/jira/browse/PHOENIX-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-4289:
--
Labels: localIndex (was: )
> UPDATE STATISTICS command does not collect stats for local indexes
> --
>
> Key: PHOENIX-4289
> URL: https://issues.apache.org/jira/browse/PHOENIX-4289
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.12.0
> Environment: HBase 1.3.1, Phoenix 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Samarth Jain
> Labels: localIndex
>
> With clean {{SYSTEM.STATS}} table and restarted HBase server+Phoenix client.
> Ran {{UPDATE STATISTICS T ALL}} command. Global guidepost width is set to
> 100M. No stats are generated for any of the local indexes on table T.
> {noformat}
> explain select count(*) from T;
> +---+-++--+
> | PLAN|
> EST_BYTES_READ | EST_ROWS_READ | EST_INFO_TS |
> +---+-++--+
> | CLIENT 8-CHUNK PARALLEL 8-WAY RANGE SCAN OVER T [1] |
> null| null | null |
> | SERVER FILTER BY FIRST KEY ONLY |
> null| null | null |
> | SERVER AGGREGATE INTO SINGLE ROW |
> null| null | null |
> +---+-++--+
> select * from system.stats;
> +--++-++--++
> |PHYSICAL_NAME | COLUMN_FAMILY | GUIDE_POST_KEY |
> GUIDE_POSTS_WIDTH | LAST_STATS_UPDATE_TIME | GUIDE_POSTS_ROW_COUNT |
> +--++-++--++
> | T || | null |
> 2017-10-16 18:36:57.884 | null |
> | T | 0 | [B@9bd0fa6 | 10099 |
>| 75756 |
> | T | 0 | [B@59d2103b | 10057 |
>| 75748 |
> | T | 0 | [B@39dcf4b0 | 10058 |
>| 75748 |
> | T | 0 | [B@6e4de19b | 10081 |
>| 75743 |
> | T | 0 | [B@f6c03cb | 10044 |
>| 75744 |
> | T | 0 | [B@46f699d5 | 10023 |
>| 75741 |
> | T | 0 | [B@18518ccf | 10019 |
>| 75749 |
> | T | 0 | [B@1991f767 | 10097 |
>| 75740 |
> | T | 0 | [B@768ccdc5 | 10092 |
>| 75740 |
> | T | 0 | [B@4c6daf0 | 10026 |
>| 75739 |
> | T | 0 | [B@10650953 | 10054 |
>| 75731 |
> | T | 0 | [B@659eef7 | 10092 |
>| 75741 |
> | T | 0 | [B@162be91c | 10023 |
>| 75752 |
> | T | 0 | [B@2488b073 | 10096 |
>| 75743 |
> | T | 0 | [B@1c9f0a20 | 10025 |
>| 75745 |
> | T | 0 | [B@55787112 | 10104 |
>| 75725 |
> | T | 0 | [B@1cd201a8 | 10019 |
>| 75748 |
> | T | 0 | [B@7db82169 | 10080 |
>| 75740 |
> | T | 0 | [B@1992eaf4 | 10079 |
>| 75733 |
> | T | 0 | [B@f74e835 | 10003 |
>| 75746
[jira] [Updated] (PHOENIX-4287) Incorrect aggregate query results when stats are disable for parallelization
[
https://issues.apache.org/jira/browse/PHOENIX-4287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-4287:
--
Labels: localIndex (was: )
> Incorrect aggregate query results when stats are disable for parallelization
>
>
> Key: PHOENIX-4287
> URL: https://issues.apache.org/jira/browse/PHOENIX-4287
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.12.0
> Environment: HBase 1.3.1
>Reporter: Mujtaba Chohan
>Assignee: Samarth Jain
> Labels: localIndex
> Fix For: 4.12.1
>
>
> With {{phoenix.use.stats.parallelization}} set to {{false}}, aggregate query
> returns incorrect results when stats are available.
> With local index and stats disabled for parallelization:
> {noformat}
> explain select count(*) from TABLE_T;
> +---+-++---+
> | PLAN
> | EST_BYTES_READ | EST_ROWS_READ | EST_INFO |
> +---+-++---+
> | CLIENT 0-CHUNK 332170 ROWS 625043899 BYTES PARALLEL 0-WAY RANGE SCAN OVER
> TABLE_T [1] | 625043899 | 332170 | 150792825 |
> | SERVER FILTER BY FIRST KEY ONLY
> | 625043899 | 332170 | 150792825 |
> | SERVER AGGREGATE INTO SINGLE ROW
> | 625043899 | 332170 | 150792825 |
> +---+-++---+
> select count(*) from TABLE_T;
> +---+
> | COUNT(1) |
> +---+
> | 0 |
> +---+
> {noformat}
> Using data table
> {noformat}
> explain select /*+NO_INDEX*/ count(*) from TABLE_T;
> +--+-+++
> | PLAN
> | EST_BYTES_READ | EST_ROWS_READ | EST_INFO_TS |
> +--+-+++
> | CLIENT 2-CHUNK 332151 ROWS 438492470 BYTES PARALLEL 1-WAY FULL SCAN OVER
> TABLE_T | 438492470 | 332151 | 1507928257617 |
> | SERVER FILTER BY FIRST KEY ONLY
> | 438492470 | 332151 | 1507928257617 |
> | SERVER AGGREGATE INTO SINGLE ROW
> | 438492470 | 332151 | 1507928257617 |
> +--+-+++
> select /*+NO_INDEX*/ count(*) from TABLE_T;
> +---+
> | COUNT(1) |
> +---+
> | 14|
> +---+
> {noformat}
> Without stats available, results are correct:
> {noformat}
> explain select /*+NO_INDEX*/ count(*) from TABLE_T;
> +--+-++--+
> | PLAN |
> EST_BYTES_READ | EST_ROWS_READ | EST_INFO_TS |
> +--+-++--+
> | CLIENT 2-CHUNK PARALLEL 1-WAY FULL SCAN OVER TABLE_T | null|
> null | null |
> | SERVER FILTER BY FIRST KEY ONLY | null
>| null | null |
> | SERVER AGGREGATE INTO SINGLE ROW | null
>| null | null |
> +--+-++--+
> select /*+NO_INDEX*/ count(*) from TABLE_T;
> +---+
> | COUNT(1) |
> +---+
> | 27|
> +---+
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4242) Fix Indexer post-compact hook logging of NPE and TableNotFound
[ https://issues.apache.org/jira/browse/PHOENIX-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211749#comment-16211749 ] James Taylor commented on PHOENIX-4242: --- +1. Thanks, [~vincentpoon]. Please commit to 4.12-HBase-0.98, 4.x-HBase-0.98, 4.12-HBase-1.3, and master. > Fix Indexer post-compact hook logging of NPE and TableNotFound > -- > > Key: PHOENIX-4242 > URL: https://issues.apache.org/jira/browse/PHOENIX-4242 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Vincent Poon >Assignee: Vincent Poon > Attachments: PHOENIX-4242.v2.master.patch, > PHOENIX-4242.v3.master.patch, PHOENIX-4242.v4.master.patch, > PHOENIX-4747.v1.master.patch > > > The post-compact hook in the Indexer seems to log extraneous log messages > indicating NPE or TableNotFound. The TableNotFound exceptions seem to > indicate actual table names prefixed with MERGE or RESTORE, and sometimes > suffixed with a digit, so perhaps these are views or something similar. > Examples: > 2017-09-28 13:35:03,118 WARN [ctions-1506410238599] index.Indexer - Unable > to permanently disable indexes being partially rebuild for SYSTEM.SEQUENCE > java.lang.NullPointerException > 2017-09-28 10:20:56,406 WARN [ctions-1506410238415] index.Indexer - Unable > to permanently disable indexes being partially rebuild for > MERGE_PLATFORM_ENTITY.PLATFORM_IMMUTABLE_ENTITY_DATA2 > org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03): Table > undefined. tableName=MERGE_PLATFORM_ENTITY.PLATFORM_IMMUTABLE_ENTITY_DATA2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211704#comment-16211704
]
Jan Fernando commented on PHOENIX-4292:
---
I think things have gotten lost in translation here and I think this discussion
is no longer productive as my intent of the question has gotten lost. I'm not
going to comment any further on this thread. I apologize if I wasn't clear. All
points taken.
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -- Returns 5 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 >= 'testa'; -- INCORRECT RESULT: This
> query returns 1 record, expected to return 5
> SELECT * FROM TEST."abc" WHERE pk1 <= 'testa'; -- Returns 4 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 > 'testa'; -- Returns 1 row as expected
> SELECT * FROM TEST."abc" WHERE pk1 < 'testa'; -- INCORRECT RESULT: This query
> returns 1 record, expected to return 0
> -- The following are cases where everything works as expected and which don't
> have composite VARCHAR PKs
> -- DEMONOSTRATE NO QUERY PROBLEMS WHEN HAVE VIEW WITH SINGLE DESC TEXT PK:
> View with composite PK with single pk value DESC
> upsert into TEST."ab2" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211698#comment-16211698
]
Andrew Purtell commented on PHOENIX-4292:
-
[~jfernando_sfdc] I'm not arguing with you either about this bug report, and
the specific tests for it. Your comment about insufficient quality/testing in
Phoenix derailed us from that narrow focus into a larger discussion. You don't
define what you mean by "adequate" when you say this:
{quote}
It's great we address bugs quickly, don't get me wrong and that's always
appreciated. However, If we find a bug that exposes holes that have been there
from the very beginning, I believe part of fixing that bug is exploring where
else that bug might have implications. As part of this we should make sure
there is adequate test coverage. That way we really solve the problem and have
confidence in the quality of what we are shipping. Otherwise we are just
plugging leaks as they are found. As Phoenix matures and used in more
production environments I believe this kind of approach is becoming even more
important.
{quote}
As a Phoenix committer and community member it's on you as much as everyone
else to define what 'adequate' means and your thoughts are more than welcome.
Later, when you said this:
{quote}
My concern, came when the answer to my question was that there might be other
cases that could still not be addressed and that would be handled by filing
another JIRA. If the other cases are not pertinent to this particular issue and
we feel we have locked this one down then great and let's state that. I don't
believe asking community members to submit their own tests is the right
response here. Everyone committing code should care about quality holistically.
{quote}
I do think you are exactly backwards about what we should ask our community
members, and seem to suggest that committers are the responsible ones for test
coverage, which is a misunderstanding of the committer role at Apache in my
opinion. However as I said we have been derailed a bit from the issue at hand.
If more needs to be said or hashed out probably best to start a discussion on
dev@ or continue in that thread that Lars started a couple of weeks ago.
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211675#comment-16211675
]
James Taylor commented on PHOENIX-4292:
---
[~jfernando_sfdc] - we appreciate you spending the time to narrow down the
cause and provide the basis for a test. I'm not disagreeing with you too, that
if there's still a gap in testing it should be filled. My point is that there's
a lot more than 1 test testing descending row keys. If you have a *specific*
idea on which test or tests are missing or which scenarios aren't being tested,
please share that (ideally as unit tests).
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -- Returns 5 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 >= 'testa'; -- INCORRECT RESULT: This
> query returns 1 record, expected to return 5
> SELECT * FROM TEST."abc" WHERE pk1 <= 'testa'; -- Returns 4 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 > 'testa'; -- Returns 1 row as expected
> SELECT * FROM TEST."abc" WHERE pk1 < 'testa'; -- INCORRECT RESULT: This query
> returns 1 record, expected to return 0
> -- The following are cases where everything works as expected and which don't
> have composite VARCHAR PKs
> -- DEMONOSTRATE NO
[jira] [Updated] (PHOENIX-4280) Delete doesn't work when immutable indexes are in building state
[ https://issues.apache.org/jira/browse/PHOENIX-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-4280: -- Attachment: PHOENIX-4280_v9.patch > Delete doesn't work when immutable indexes are in building state > > > Key: PHOENIX-4280 > URL: https://issues.apache.org/jira/browse/PHOENIX-4280 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Fix For: 4.12.1 > > Attachments: PHOENIX-4280.patch, PHOENIX-4280_v2.patch, > PHOENIX-4280_v3.patch, PHOENIX-4280_v4.patch, PHOENIX-4280_v5.patch, > PHOENIX-4280_v6.patch, PHOENIX-4280_v7.patch, PHOENIX-4280_v8.patch, > PHOENIX-4280_v9.patch, PHOENIX-4280_wip.patch > > > We don't generate the Delete mutations correctly in this case as we assume > we're scanning over the index table. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4280) Delete doesn't work when immutable indexes are in building state
[
https://issues.apache.org/jira/browse/PHOENIX-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211643#comment-16211643
]
Hadoop QA commented on PHOENIX-4280:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12893099/PHOENIX-4280_v8.patch
against master branch at commit 0461fe855aaed27e23cb5d17e8be022a82626162.
ATTACHMENT ID: 12893099
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+rs = conn.createStatement().executeQuery("SELECT /*+
NO_INDEX*/ COUNT(*) FROM " + fullTableName);
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+String dml = "DELETE from " + fullTableName + " WHERE
varchar_col1='varchar_a' AND varchar_pk='varchar1'";
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+Iterator>> iterator =
PhoenixRuntime.getUncommittedDataIterator(conn);
+
assertEquals(SQLExceptionCode.INVALID_FILTER_ON_IMMUTABLE_ROWS.getErrorCode(),
e.getErrorCode());
+IndexMaintainer maintainer =
indexTableRefs.get(i).getTable().getIndexMaintainer(table, connection);
+indexPtr.set(maintainer.buildRowKey(null, indexPtr,
null, null, HConstants.LATEST_TIMESTAMP));
+if (index.getIndexState() != PIndexState.DISABLE &&
index.getIndexType() == IndexType.GLOBAL) {
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.phoenix.hbase.index.covered.LocalTableStateTest
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1558//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1558//console
This message is automatically generated.
> Delete doesn't work when immutable indexes are in building state
>
>
> Key: PHOENIX-4280
> URL: https://issues.apache.org/jira/browse/PHOENIX-4280
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4280.patch, PHOENIX-4280_v2.patch,
> PHOENIX-4280_v3.patch, PHOENIX-4280_v4.patch, PHOENIX-4280_v5.patch,
> PHOENIX-4280_v6.patch, PHOENIX-4280_v7.patch, PHOENIX-4280_v8.patch,
> PHOENIX-4280_wip.patch
>
>
> We don't generate the Delete mutations correctly in this case as we assume
> we're scanning over the index table.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211637#comment-16211637
]
Andrew Purtell commented on PHOENIX-4292:
-
In fact if a contributor decided to improve test coverage and submit this work
as patches this would be so amazing it would virtually guarantee a fast track
to committership.
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -- Returns 5 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 >= 'testa'; -- INCORRECT RESULT: This
> query returns 1 record, expected to return 5
> SELECT * FROM TEST."abc" WHERE pk1 <= 'testa'; -- Returns 4 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 > 'testa'; -- Returns 1 row as expected
> SELECT * FROM TEST."abc" WHERE pk1 < 'testa'; -- INCORRECT RESULT: This query
> returns 1 record, expected to return 0
> -- The following are cases where everything works as expected and which don't
> have composite VARCHAR PKs
> -- DEMONOSTRATE NO QUERY PROBLEMS WHEN HAVE VIEW WITH SINGLE DESC TEXT PK:
> View with composite PK with single pk value DESC
> upsert into TEST."ab2" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert i
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211636#comment-16211636
]
Jan Fernando commented on PHOENIX-4292:
---
I'm not arguing with that [~apurtell] at all. I was simply asking if the bug
fix was complete. If that's not okay to ask then I stand corrected. FWIW I
spent several hours coming up with a detailed repro for this which was turned
into a test. I just wasn't sure if that was sufficient which is why I asked, as
I'm an not super familiar with that code.
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -- Returns 5 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 >= 'testa'; -- INCORRECT RESULT: This
> query returns 1 record, expected to return 5
> SELECT * FROM TEST."abc" WHERE pk1 <= 'testa'; -- Returns 4 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 > 'testa'; -- Returns 1 row as expected
> SELECT * FROM TEST."abc" WHERE pk1 < 'testa'; -- INCORRECT RESULT: This query
> returns 1 record, expected to return 0
> -- The following are cases where everything works as expected and which don't
> have composite VARCHAR PKs
> -- DEMONOSTRATE NO QUERY PROBLEMS WHEN HAVE VIEW WITH SINGLE DESC TEXT PK:
> View with composite PK
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211627#comment-16211627
]
Andrew Purtell commented on PHOENIX-4292:
-
bq. I don't believe asking community members to submit their own tests is the
right response here.
This is backwards, IMHO. It is the *community's* responsibility to maintain the
code base and Apache is a do-ocracy where the best course of action is to put
up a patch, with second best being to suggest something. Committers are here to
review and guide contributions not respond to feature requests as if this is a
professional support organization. Of course not everyone is capable of code
contribution, so suggestions are always welcome, however absolutely if anyone
is concerned about or interested in growing test coverage the best thing to do
is contribute a patch. I mean this as general feedback.
In this case, adding a new unit to cover the fix for the specific problem, with
secondary considerations like improving test coverage for a feature in general
pushed to another JIRA, is appropriate and very common practice across all
Apache projects I participate in, FWIW.
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211612#comment-16211612
]
Jan Fernando commented on PHOENIX-4292:
---
I am not suggesting we test everything. I was just concerned when I looked at
the review that there was only 1 test. This is the normal kind of feedback I'd
given any developer who's code I was reviewing. Seems to me this bug might
warrant more than 1 test, I think it's a fair question to ask any developer and
if 1 test makes sense they should be able to argue for that.
My concern, came when the answer to my question was that there might be other
cases that could still not be addressed and that would be handled by filing
another JIRA. If the other cases are not pertinent to this particular issue and
we feel we have locked this one down then great and let's state that. I don't
believe asking community members to submit their own tests is the right
response here. Everyone committing code should care about quality holistically.
I was merely trying to ask questions to think about things from that
perspective.
I'm simply responding to thread on the dev list a few weeks back about quality
principles [~lhofhansl] brought up, my take away from that was try and engage
more with these kind of questions when opportunities arose.
Apologies if this was not how my comments sounded. This was my 2 cents and I
appreciate the fast turn around on the fix. I'm happy to defer to you guys as
you are in the code everyday and I am not these days.
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('tes
[jira] [Updated] (PHOENIX-4280) Delete doesn't work when immutable indexes are in building state
[ https://issues.apache.org/jira/browse/PHOENIX-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-4280: -- Attachment: PHOENIX-4280_v8.patch [~tdsilva] - would you mind reviewing this one? I'll wait to commit until after the test run too. I've removed some dead code, ensured that index mutations don't get generated and send over from the client for deletes involving local indexes, and fixed the check for determining whether or not we need to scan for the prior data row. > Delete doesn't work when immutable indexes are in building state > > > Key: PHOENIX-4280 > URL: https://issues.apache.org/jira/browse/PHOENIX-4280 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Fix For: 4.12.1 > > Attachments: PHOENIX-4280.patch, PHOENIX-4280_v2.patch, > PHOENIX-4280_v3.patch, PHOENIX-4280_v4.patch, PHOENIX-4280_v5.patch, > PHOENIX-4280_v6.patch, PHOENIX-4280_v7.patch, PHOENIX-4280_v8.patch, > PHOENIX-4280_wip.patch > > > We don't generate the Delete mutations correctly in this case as we assume > we're scanning over the index table. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4280) Delete doesn't work when immutable indexes are in building state
[
https://issues.apache.org/jira/browse/PHOENIX-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211590#comment-16211590
]
Hadoop QA commented on PHOENIX-4280:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12893071/PHOENIX-4280_v7.patch
against master branch at commit 0461fe855aaed27e23cb5d17e8be022a82626162.
ATTACHMENT ID: 12893071
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+rs = conn.createStatement().executeQuery("SELECT /*+
NO_INDEX*/ COUNT(*) FROM " + fullTableName);
+
TestUtil.dumpTable(conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(Bytes.toBytes(fullTableName)));
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+String dml = "DELETE from " + fullTableName + " WHERE
varchar_col1='varchar_a' AND varchar_pk='varchar1'";
+
TestUtil.dumpTable(conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(Bytes.toBytes(fullTableName)));
+rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX*/
COUNT(*) FROM " + fullTableName);
+
assertEquals(SQLExceptionCode.INVALID_FILTER_ON_IMMUTABLE_ROWS.getErrorCode(),
e.getErrorCode());
+IndexMaintainer maintainer =
indexTableRefs.get(i).getTable().getIndexMaintainer(table, connection);
+indexPtr.set(maintainer.buildRowKey(null, indexPtr,
null, null, HConstants.LATEST_TIMESTAMP));
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1557//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1557//console
This message is automatically generated.
> Delete doesn't work when immutable indexes are in building state
>
>
> Key: PHOENIX-4280
> URL: https://issues.apache.org/jira/browse/PHOENIX-4280
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4280.patch, PHOENIX-4280_v2.patch,
> PHOENIX-4280_v3.patch, PHOENIX-4280_v4.patch, PHOENIX-4280_v5.patch,
> PHOENIX-4280_v6.patch, PHOENIX-4280_v7.patch, PHOENIX-4280_wip.patch
>
>
> We don't generate the Delete mutations correctly in this case as we assume
> we're scanning over the index table.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211587#comment-16211587
]
James Taylor commented on PHOENIX-4292:
---
There are 1335 references to DESC in our unit and integration tests. We have
5,690 tests. Saying that "you should add more tests" is not particularly
helpful at this point in the project lifecycle. There's an enormous surface
area to SQL and it gets even more enormous with features like local/global
indexes, immutable/mutable tables, salted tables, global views, multi-tenant
views, multi-tenant views on views, atomic/non atomic upserts, hash joins, sort
merge joins, skip scans, range scans, with stats, without stats, with stats not
used for parallelization, encoded columns (1-4 bytes for qualifiers), fat
client, thin client, namespaces/no namespaces, transactional/non transactional.
Best would be to submit a patch with some of the tests you have in mind. Second
best would be to comment on the JIRA filed by Thomas on which specific tests
you'd like to see added.
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -- Returns 5 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 >= 'testa'; -- INCORRECT RESULT: This
> query re
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211555#comment-16211555
]
Jan Fernando commented on PHOENIX-4292:
---
I guess my 2 cents is I disagree with that approach. It's great we address bugs
quickly, don't get me wrong and that's always appreciated. However, If we find
a bug that exposes holes that have been there from the very beginning, I
believe part of fixing that bug is exploring where else that bug might have
implications. As part of this we should make sure there is adequate test
coverage. That way we really solve the problem and have confidence in the
quality of what we are shipping. Otherwise we are just plugging leaks as they
are found. As Phoenix matures and used in more production environments I
believe this kind of approach is becoming even more important. But as I said
that's my 2 cents.
In this case, I just wasn't clear if we had looked beyond the couple of repros
I supplied. Hence, my question.
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -- Returns 5 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 >= 'testa'; -- INCORRECT RESULT: This
> query returns 1 record, expected to return 5
> SELECT * FROM TEST."a
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211548#comment-16211548
]
Thomas D'Silva commented on PHOENIX-4292:
-
This has been broken from the beginning as this bug was introduced in the
original sort order implementation see
https://issues.apache.org/jira/browse/PHOENIX-32.
I added a test case based on the steps you documented to repro this issue. This
patch just contains a bug fix for the issue you reported and no additional dev
work.
We could use more test coverage for the Sort Order feature, but I think it
should be handled in separate JIRA.
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -- Returns 5 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 >= 'testa'; -- INCORRECT RESULT: This
> query returns 1 record, expected to return 5
> SELECT * FROM TEST."abc" WHERE pk1 <= 'testa'; -- Returns 4 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 > 'testa'; -- Returns 1 row as expected
> SELECT * FROM TEST."abc" WHERE pk1 < 'testa'; -- INCORRECT RESULT: This query
> returns 1 record, expected to return 0
> -- The following are cases where everything works as expected and which don't
> have composite VARCHAR PKs
> --
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211516#comment-16211516
]
Jan Fernando commented on PHOENIX-4292:
---
Okay. I'm not a big fan of doing testing as a separate work item, we should
have everything we need with this patch to make sure we feel we caught all the
cases where this might be an issue. Do you feel we have enough test coverage
with this patch?
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -- Returns 5 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 >= 'testa'; -- INCORRECT RESULT: This
> query returns 1 record, expected to return 5
> SELECT * FROM TEST."abc" WHERE pk1 <= 'testa'; -- Returns 4 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 > 'testa'; -- Returns 1 row as expected
> SELECT * FROM TEST."abc" WHERE pk1 < 'testa'; -- INCORRECT RESULT: This query
> returns 1 record, expected to return 0
> -- The following are cases where everything works as expected and which don't
> have composite VARCHAR PKs
> -- DEMONOSTRATE NO QUERY PROBLEMS WHEN HAVE VIEW WITH SINGLE DESC TEXT PK:
> View with composite PK with single pk value DESC
> upsert into TEST."ab2" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
>
[jira] [Updated] (PHOENIX-4306) Add more tests for DESC sort order
[ https://issues.apache.org/jira/browse/PHOENIX-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas D'Silva updated PHOENIX-4306: Labels: DESC (was: ) > Add more tests for DESC sort order > -- > > Key: PHOENIX-4306 > URL: https://issues.apache.org/jira/browse/PHOENIX-4306 > Project: Phoenix > Issue Type: Test >Reporter: Thomas D'Silva > Labels: DESC > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211378#comment-16211378
]
Thomas D'Silva commented on PHOENIX-4292:
-
Its possible, I filed PHOENIX-3383 to add more tests.
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -- Returns 5 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 >= 'testa'; -- INCORRECT RESULT: This
> query returns 1 record, expected to return 5
> SELECT * FROM TEST."abc" WHERE pk1 <= 'testa'; -- Returns 4 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 > 'testa'; -- Returns 1 row as expected
> SELECT * FROM TEST."abc" WHERE pk1 < 'testa'; -- INCORRECT RESULT: This query
> returns 1 record, expected to return 0
> -- The following are cases where everything works as expected and which don't
> have composite VARCHAR PKs
> -- DEMONOSTRATE NO QUERY PROBLEMS WHEN HAVE VIEW WITH SINGLE DESC TEXT PK:
> View with composite PK with single pk value DESC
> upsert into TEST."ab2" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."ab2" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10
[jira] [Created] (PHOENIX-4306) Add more tests for DESC sort order
Thomas D'Silva created PHOENIX-4306: --- Summary: Add more tests for DESC sort order Key: PHOENIX-4306 URL: https://issues.apache.org/jira/browse/PHOENIX-4306 Project: Phoenix Issue Type: Test Reporter: Thomas D'Silva -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-3383) Comparison between descending row keys used in RVC is reverse
[ https://issues.apache.org/jira/browse/PHOENIX-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas D'Silva updated PHOENIX-3383: Labels: DESC (was: ) > Comparison between descending row keys used in RVC is reverse > - > > Key: PHOENIX-3383 > URL: https://issues.apache.org/jira/browse/PHOENIX-3383 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Labels: DESC > Fix For: 4.13.0 > > Attachments: PHOENIX-3383_wip.patch, PHOENIX-3383_wip2.patch, > PHOENIX-3383_wip3.patch > > > See PHOENIX-3382, but the comparison for RVC with descending row key columns > is the reverse of what it should be. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-3996) DECIMAL TYPE DESC index compare wrong
[ https://issues.apache.org/jira/browse/PHOENIX-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas D'Silva updated PHOENIX-3996: Labels: DESC (was: ) > DECIMAL TYPE DESC index compare wrong > - > > Key: PHOENIX-3996 > URL: https://issues.apache.org/jira/browse/PHOENIX-3996 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.8.1 >Reporter: steve, Oh >Priority: Blocker > Labels: DESC > > I created decimal type descending index column. Some case compare result is > odd. > # table schema > CREATE TABLE IF NOT EXISTS test_tbl2 ( > num DECIMAL NOT NULL > CONSTRAINT pk PRIMARY KEY (num DESC) > ) > ; > # explain query > explain > select * from test_tbl2 > where > num >= 149875500 and num <= 1498755999000; > # explain result > DEGENERATE SCAN OVER TEST_TBL2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4280) Delete doesn't work when immutable indexes are in building state
[ https://issues.apache.org/jira/browse/PHOENIX-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-4280: -- Attachment: PHOENIX-4280_v7.patch Still needs some cleanup, but hopefully tests will all pass now. > Delete doesn't work when immutable indexes are in building state > > > Key: PHOENIX-4280 > URL: https://issues.apache.org/jira/browse/PHOENIX-4280 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Fix For: 4.12.1 > > Attachments: PHOENIX-4280.patch, PHOENIX-4280_v2.patch, > PHOENIX-4280_v3.patch, PHOENIX-4280_v4.patch, PHOENIX-4280_v5.patch, > PHOENIX-4280_v6.patch, PHOENIX-4280_v7.patch, PHOENIX-4280_wip.patch > > > We don't generate the Delete mutations correctly in this case as we assume > we're scanning over the index table. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4237) Allow sorting on (Java) collation keys for non-English locales
[
https://issues.apache.org/jira/browse/PHOENIX-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211282#comment-16211282
]
ASF GitHub Bot commented on PHOENIX-4237:
-
Github user snakhoda-sfdc commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/275#discussion_r145747600
--- Diff:
phoenix-core/src/main/java/com/ibm/icu/impl/jdkadapter/NumberFormatICU.java ---
@@ -0,0 +1,229 @@
+// © 2016 and later: Unicode, Inc. and others.
+// License & terms of use: http://www.unicode.org/copyright.html#License
+/*
+
***
+ * Copyright (C) 2008, International Business Machines Corporation and
*
+ * others. All Rights Reserved.
*
+
***
+ */
+package com.ibm.icu.impl.jdkadapter;
+
+import java.math.RoundingMode;
+import java.text.FieldPosition;
+import java.text.ParseException;
+import java.text.ParsePosition;
+import java.util.Currency;
+
+import com.ibm.icu.impl.icuadapter.NumberFormatJDK;
+import com.ibm.icu.text.NumberFormat;
+
+/**
+ * NumberFormatICU is an adapter class which wraps ICU4J NumberFormat and
+ * implements java.text.NumberFormat APIs.
+ */
+public class NumberFormatICU extends java.text.NumberFormat {
+
+private static final long serialVersionUID = 4892903815641574060L;
+
+private NumberFormat fIcuNfmt;
+
+private NumberFormatICU(NumberFormat icuNfmt) {
+fIcuNfmt = icuNfmt;
+}
+
+public static java.text.NumberFormat wrap(NumberFormat icuNfmt) {
+if (icuNfmt instanceof NumberFormatJDK) {
+return ((NumberFormatJDK)icuNfmt).unwrap();
+}
+return new NumberFormatICU(icuNfmt);
+}
+
+public NumberFormat unwrap() {
+return fIcuNfmt;
+}
+
+@Override
+public Object clone() {
+NumberFormatICU other = (NumberFormatICU)super.clone();
+other.fIcuNfmt = (NumberFormat)fIcuNfmt.clone();
+return other;
+}
+
+@Override
+public boolean equals(Object obj) {
+if (obj instanceof NumberFormatICU) {
+return ((NumberFormatICU)obj).fIcuNfmt.equals(fIcuNfmt);
+}
+return false;
+}
+
+//public String format(double number)
--- End diff --
Thanks for taking a look at this PR, @solzy. This code is external and
simply copied over from ICU4J 59.1. The reason it's here at all is that that
project doesn't have all its artifacts in maven. I'm hoping to have a new PR in
the near future to remove this external code and replace it with maven
dependencies. CC: @JamesRTaylor
> Allow sorting on (Java) collation keys for non-English locales
> --
>
> Key: PHOENIX-4237
> URL: https://issues.apache.org/jira/browse/PHOENIX-4237
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Shehzaad Nakhoda
> Fix For: 4.12.0
>
>
> Strings stored via Phoenix can be composed from a subset of the entire set of
> Unicode characters. The natural sort order for strings for different
> languages often differs from the order dictated by the binary representation
> of the characters of these strings. Java provides the idea of a Collator
> which given an input string and a (language) locale can generate a Collation
> Key which can then be used to compare strings in that natural order.
> Salesforce has recently open-sourced grammaticus. IBM has open-sourced ICU4J
> some time ago. These technologies can be combined to provide a robust new
> Phoenix function that can be used in an ORDER BY clause to sort strings
> according to the user's locale.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] phoenix pull request #275: PHOENIX-4237: Add function to calculate Java coll...
Github user snakhoda-sfdc commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/275#discussion_r145747600
--- Diff:
phoenix-core/src/main/java/com/ibm/icu/impl/jdkadapter/NumberFormatICU.java ---
@@ -0,0 +1,229 @@
+// © 2016 and later: Unicode, Inc. and others.
+// License & terms of use: http://www.unicode.org/copyright.html#License
+/*
+
***
+ * Copyright (C) 2008, International Business Machines Corporation and
*
+ * others. All Rights Reserved.
*
+
***
+ */
+package com.ibm.icu.impl.jdkadapter;
+
+import java.math.RoundingMode;
+import java.text.FieldPosition;
+import java.text.ParseException;
+import java.text.ParsePosition;
+import java.util.Currency;
+
+import com.ibm.icu.impl.icuadapter.NumberFormatJDK;
+import com.ibm.icu.text.NumberFormat;
+
+/**
+ * NumberFormatICU is an adapter class which wraps ICU4J NumberFormat and
+ * implements java.text.NumberFormat APIs.
+ */
+public class NumberFormatICU extends java.text.NumberFormat {
+
+private static final long serialVersionUID = 4892903815641574060L;
+
+private NumberFormat fIcuNfmt;
+
+private NumberFormatICU(NumberFormat icuNfmt) {
+fIcuNfmt = icuNfmt;
+}
+
+public static java.text.NumberFormat wrap(NumberFormat icuNfmt) {
+if (icuNfmt instanceof NumberFormatJDK) {
+return ((NumberFormatJDK)icuNfmt).unwrap();
+}
+return new NumberFormatICU(icuNfmt);
+}
+
+public NumberFormat unwrap() {
+return fIcuNfmt;
+}
+
+@Override
+public Object clone() {
+NumberFormatICU other = (NumberFormatICU)super.clone();
+other.fIcuNfmt = (NumberFormat)fIcuNfmt.clone();
+return other;
+}
+
+@Override
+public boolean equals(Object obj) {
+if (obj instanceof NumberFormatICU) {
+return ((NumberFormatICU)obj).fIcuNfmt.equals(fIcuNfmt);
+}
+return false;
+}
+
+//public String format(double number)
--- End diff --
Thanks for taking a look at this PR, @solzy. This code is external and
simply copied over from ICU4J 59.1. The reason it's here at all is that that
project doesn't have all its artifacts in maven. I'm hoping to have a new PR in
the near future to remove this external code and replace it with maven
dependencies. CC: @JamesRTaylor
---
[jira] [Created] (PHOENIX-4305) Make use of Cell interface than KeyValue
Rajeshbabu Chintaguntla created PHOENIX-4305: Summary: Make use of Cell interface than KeyValue Key: PHOENIX-4305 URL: https://issues.apache.org/jira/browse/PHOENIX-4305 Project: Phoenix Issue Type: Sub-task Affects Versions: 4.12.0 Reporter: Rajeshbabu Chintaguntla Assignee: Rajeshbabu Chintaguntla -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4292) Filters on Tables and Views with composite PK of VARCHAR fields with sort direction DESC do not work
[
https://issues.apache.org/jira/browse/PHOENIX-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211191#comment-16211191
]
Jan Fernando commented on PHOENIX-4292:
---
[~tdsilva] Should we add more test coverage? This was one example of how this
was triggered. Does this fact this slipped through point to some other gaps in
our test coverage around DESC keys and we should add tests for other data types
too and some combos of data types to make sure no regressions are introduced in
the future?
> Filters on Tables and Views with composite PK of VARCHAR fields with sort
> direction DESC do not work
>
>
> Key: PHOENIX-4292
> URL: https://issues.apache.org/jira/browse/PHOENIX-4292
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Jan Fernando
>Assignee: Thomas D'Silva
> Fix For: 4.13.0, 4.12.1
>
> Attachments: PHOENIX-4292.patch
>
>
> We noticed that in certain instances on tables and views that were defined
> with a Composite PK and where the elements of the PK were all DESC that
> queries exhibited strange behavior and did not return results when expected.
> A simple query on the first element of the PK returned 0 results e.g SELECT *
> FROM MY_TABLE WHERE PK1 = 'myvaluethatexists' would return 0 results.
> After some investigation it appears that querying tables and views with a
> Composite PK that :
> a) have multiple VARCHAR columns in the PK
> b) the sort direction of all the VARCHAR columns is defined as DESC
> does not work correctly and the filters are not honored and SQL appears
> broken to the end user.
> Detailed repro steps:
> ---
> -- 1. Create Global Base Table
> CREATE TABLE IF NOT EXISTS TEST.BASETABLE (
> TENANT_ID CHAR(15) NOT NULL,
> KEY_PREFIX CHAR(3) NOT NULL,
> CREATED_DATE DATE,
> CREATED_BY CHAR(15),
> SYSTEM_MODSTAMP DATE
> CONSTRAINT PK PRIMARY KEY (
> TENANT_ID,
> KEY_PREFIX
> )
> ) VERSIONS=1, MULTI_TENANT=true, IMMUTABLE_ROWS=TRUE, REPLICATION_SCOPE=1
> -- 2. Create various TENANT SPECIFIC VIEWS i.e. use a tenant specific
> connection
> CREATE VIEW IF NOT EXISTS TEST."abc" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'abc';
> CREATE VIEW IF NOT EXISTS TEST."ab2" (pk1 VARCHAR(10) NOT NULL, pk2
> VARCHAR(10) NOT NULL, col1 DATE, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 ASC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab2';
> CREATE VIEW IF NOT EXISTS TEST."ab3" (pk1 DATE(10) NOT NULL, pk2 DATE(10) NOT
> NULL, col1 VARCHAR(10), col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1 DESC,
> pk2 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX = 'ab3';
> CREATE VIEW IF NOT EXISTS TEST."ab4" (pk1 DATE(10) NOT NULL, pk2 DECIMAL NOT
> NULL, pk3 VARCHAR(10) NOT NULL, col3 DECIMAL CONSTRAINT PK PRIMARY KEY (pk1
> DESC, pk2 DESC, pk3 DESC)) AS SELECT * FROM TEST.BASETABLE WHERE KEY_PREFIX =
> 'ab4';
> -- 3. Test cases that exhibit this issues
> -- SIMULATE EQUALITY QUERY PROBLEMS: View with composite PK with multiple PK
> values of VARCHAR values DESC
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testb',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testc',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'testd',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testa', 'teste',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> upsert into TEST."abc" (pk1, pk2, col1, col3) VALUES ('testb', 'testa',
> TO_DATE('2017-10-16 22:00:00', '-MM-dd HH:mm:ss'), 10);
> SELECT * FROM TEST."abc" WHERE pk1 = 'testa'; -- INCORRECT RESULT: This query
> returns no records, expected to return 4
> SELECT * FROM TEST."abc"; -- Returns 5 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 >= 'testa'; -- INCORRECT RESULT: This
> query returns 1 record, expected to return 5
> SELECT * FROM TEST."abc" WHERE pk1 <= 'testa'; -- Returns 4 rows as expected
> SELECT * FROM TEST."abc" WHERE pk1 > 'testa'; -- Returns 1 row as expected
> SELECT * FROM TEST."abc" WHERE pk1 < 'testa'; -- INCORRECT RESULT: This query
> returns 1 record, expected to return 0
> -- The following are cases where everything works as expected and which don't
> have composite VARCHAR PKs
> -- DEMONOSTRATE NO QUERY PROBLEMS WHEN HAVE VIEW WITH SINGLE DESC TEXT PK:
> View with composite PK with single pk value DE
[jira] [Created] (PHOENIX-4304) Refactoring to avoid using deprecated HTableDescriptor, HColumnDescriptor, HRegionInfo
Rajeshbabu Chintaguntla created PHOENIX-4304: Summary: Refactoring to avoid using deprecated HTableDescriptor, HColumnDescriptor, HRegionInfo Key: PHOENIX-4304 URL: https://issues.apache.org/jira/browse/PHOENIX-4304 Project: Phoenix Issue Type: Sub-task Reporter: Rajeshbabu Chintaguntla Assignee: Rajeshbabu Chintaguntla We can change HTableDescriptor - > TableDescriptor HColumnDescriptor -> ColumnFamilyDescriptor HRegioInfo -> RegionInfo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4304) Refactoring to avoid using deprecated HTableDescriptor, HColumnDescriptor, HRegionInfo
[ https://issues.apache.org/jira/browse/PHOENIX-4304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated PHOENIX-4304: - Labels: HBase-2.0 (was: ) > Refactoring to avoid using deprecated HTableDescriptor, HColumnDescriptor, > HRegionInfo > -- > > Key: PHOENIX-4304 > URL: https://issues.apache.org/jira/browse/PHOENIX-4304 > Project: Phoenix > Issue Type: Sub-task >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla > Labels: HBase-2.0 > Fix For: 4.13.0 > > > We can change > HTableDescriptor - > TableDescriptor > HColumnDescriptor -> ColumnFamilyDescriptor > HRegioInfo -> RegionInfo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (PHOENIX-4303) Replace HTableInterface,HConnection with Table,Connection interfaces respectively
Rajeshbabu Chintaguntla created PHOENIX-4303: Summary: Replace HTableInterface,HConnection with Table,Connection interfaces respectively Key: PHOENIX-4303 URL: https://issues.apache.org/jira/browse/PHOENIX-4303 Project: Phoenix Issue Type: Sub-task Reporter: Rajeshbabu Chintaguntla Assignee: Rajeshbabu Chintaguntla In latest versions of HBase HTableInterface,HConnection are replaced with Table and Connection respectively. We can make use of new interfaces. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4303) Replace HTableInterface,HConnection with Table,Connection interfaces respectively
[ https://issues.apache.org/jira/browse/PHOENIX-4303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated PHOENIX-4303: - Labels: HBase-2.0 (was: ) > Replace HTableInterface,HConnection with Table,Connection interfaces > respectively > - > > Key: PHOENIX-4303 > URL: https://issues.apache.org/jira/browse/PHOENIX-4303 > Project: Phoenix > Issue Type: Sub-task >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla > Labels: HBase-2.0 > Fix For: 4.13.0 > > > In latest versions of HBase HTableInterface,HConnection are replaced with > Table and Connection respectively. We can make use of new interfaces. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Date issue when copying data from Phoenix to Spark
I don't think there is a GMT+216:00 time zone. Pavani, can you maybe tell which version of Phoenix and which version of Spark you used when observing this? What exact command you used to insert into spark? On 18/10/17 23:51, "sergey.solda...@gmail.com on behalf of Sergey Soldatov" wrote: >Just a guess. Date is storying as the regular timestamp in GMT. So Phoenix >shows you the GMT time while Spark is convert GMT time to you local time >zone. > >Thanks, >Sergey > >On Tue, Oct 17, 2017 at 3:54 PM, Pavani Addanki >wrote: > >> I am trying to insert phoenix data into spark, found that the date is >> changing after inserting into spark. For ex: Date in Phoenix is showing as >> 2017-10-10 but when same is seen in spark it is showing as 2017-10-19. >> >> Did anyone else face the same issue. >> >> >> Thanks and Regards, >> >> Pavani A >>
[jira] [Commented] (PHOENIX-4237) Allow sorting on (Java) collation keys for non-English locales
[
https://issues.apache.org/jira/browse/PHOENIX-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16210750#comment-16210750
]
ASF GitHub Bot commented on PHOENIX-4237:
-
Github user solzy commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/275#discussion_r145639624
--- Diff:
phoenix-core/src/main/java/com/ibm/icu/impl/jdkadapter/NumberFormatICU.java ---
@@ -0,0 +1,229 @@
+// © 2016 and later: Unicode, Inc. and others.
+// License & terms of use: http://www.unicode.org/copyright.html#License
+/*
+
***
+ * Copyright (C) 2008, International Business Machines Corporation and
*
+ * others. All Rights Reserved.
*
+
***
+ */
+package com.ibm.icu.impl.jdkadapter;
+
+import java.math.RoundingMode;
+import java.text.FieldPosition;
+import java.text.ParseException;
+import java.text.ParsePosition;
+import java.util.Currency;
+
+import com.ibm.icu.impl.icuadapter.NumberFormatJDK;
+import com.ibm.icu.text.NumberFormat;
+
+/**
+ * NumberFormatICU is an adapter class which wraps ICU4J NumberFormat and
+ * implements java.text.NumberFormat APIs.
+ */
+public class NumberFormatICU extends java.text.NumberFormat {
+
+private static final long serialVersionUID = 4892903815641574060L;
+
+private NumberFormat fIcuNfmt;
+
+private NumberFormatICU(NumberFormat icuNfmt) {
+fIcuNfmt = icuNfmt;
+}
+
+public static java.text.NumberFormat wrap(NumberFormat icuNfmt) {
+if (icuNfmt instanceof NumberFormatJDK) {
+return ((NumberFormatJDK)icuNfmt).unwrap();
+}
+return new NumberFormatICU(icuNfmt);
+}
+
+public NumberFormat unwrap() {
+return fIcuNfmt;
+}
+
+@Override
+public Object clone() {
+NumberFormatICU other = (NumberFormatICU)super.clone();
+other.fIcuNfmt = (NumberFormat)fIcuNfmt.clone();
+return other;
+}
+
+@Override
+public boolean equals(Object obj) {
+if (obj instanceof NumberFormatICU) {
+return ((NumberFormatICU)obj).fIcuNfmt.equals(fIcuNfmt);
+}
+return false;
+}
+
+//public String format(double number)
--- End diff --
delete this unusable lien, keep clean!
> Allow sorting on (Java) collation keys for non-English locales
> --
>
> Key: PHOENIX-4237
> URL: https://issues.apache.org/jira/browse/PHOENIX-4237
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Shehzaad Nakhoda
> Fix For: 4.12.0
>
>
> Strings stored via Phoenix can be composed from a subset of the entire set of
> Unicode characters. The natural sort order for strings for different
> languages often differs from the order dictated by the binary representation
> of the characters of these strings. Java provides the idea of a Collator
> which given an input string and a (language) locale can generate a Collation
> Key which can then be used to compare strings in that natural order.
> Salesforce has recently open-sourced grammaticus. IBM has open-sourced ICU4J
> some time ago. These technologies can be combined to provide a robust new
> Phoenix function that can be used in an ORDER BY clause to sort strings
> according to the user's locale.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] phoenix pull request #275: PHOENIX-4237: Add function to calculate Java coll...
Github user solzy commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/275#discussion_r145639624
--- Diff:
phoenix-core/src/main/java/com/ibm/icu/impl/jdkadapter/NumberFormatICU.java ---
@@ -0,0 +1,229 @@
+// © 2016 and later: Unicode, Inc. and others.
+// License & terms of use: http://www.unicode.org/copyright.html#License
+/*
+
***
+ * Copyright (C) 2008, International Business Machines Corporation and
*
+ * others. All Rights Reserved.
*
+
***
+ */
+package com.ibm.icu.impl.jdkadapter;
+
+import java.math.RoundingMode;
+import java.text.FieldPosition;
+import java.text.ParseException;
+import java.text.ParsePosition;
+import java.util.Currency;
+
+import com.ibm.icu.impl.icuadapter.NumberFormatJDK;
+import com.ibm.icu.text.NumberFormat;
+
+/**
+ * NumberFormatICU is an adapter class which wraps ICU4J NumberFormat and
+ * implements java.text.NumberFormat APIs.
+ */
+public class NumberFormatICU extends java.text.NumberFormat {
+
+private static final long serialVersionUID = 4892903815641574060L;
+
+private NumberFormat fIcuNfmt;
+
+private NumberFormatICU(NumberFormat icuNfmt) {
+fIcuNfmt = icuNfmt;
+}
+
+public static java.text.NumberFormat wrap(NumberFormat icuNfmt) {
+if (icuNfmt instanceof NumberFormatJDK) {
+return ((NumberFormatJDK)icuNfmt).unwrap();
+}
+return new NumberFormatICU(icuNfmt);
+}
+
+public NumberFormat unwrap() {
+return fIcuNfmt;
+}
+
+@Override
+public Object clone() {
+NumberFormatICU other = (NumberFormatICU)super.clone();
+other.fIcuNfmt = (NumberFormat)fIcuNfmt.clone();
+return other;
+}
+
+@Override
+public boolean equals(Object obj) {
+if (obj instanceof NumberFormatICU) {
+return ((NumberFormatICU)obj).fIcuNfmt.equals(fIcuNfmt);
+}
+return false;
+}
+
+//public String format(double number)
--- End diff --
delete this unusable lien, keep clean!
---
[jira] [Created] (PHOENIX-4302) Phoenix interface raises SQLException without any errorCode in case of IOException
Patrick Cavrel created PHOENIX-4302:
---
Summary: Phoenix interface raises SQLException without any
errorCode in case of IOException
Key: PHOENIX-4302
URL: https://issues.apache.org/jira/browse/PHOENIX-4302
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.10.0
Reporter: Patrick Cavrel
Priority: Minor
In Phoenix there is a path where a SQLException is thrown without any errorCode.
The expected value is SQLExceptionCode.IO_EXCEPTION in case the original
exception is an IOException.
The purpose is to allow client retries on this error code.
Here after are the SQLException source code path:
Cause0: java.sql.SQLException: hbase.client.HBaseClientException: Could not
create HBase connection
at
org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2464)
} catch (Exception e) {
if (e instanceof SQLException) {
initializationException = (SQLException)e;
} else {
// wrap every other exception into a SQLException
* initializationException = new SQLException(e);*
}
The e exception can be an exception which cause is an IOException raised by
HBase.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
