[CANCEL][VOTE] Release of Apache Phoenix 4.9.0 RC2
We found a showstopper in PHOENIX-3454. The fix has been checked in and we'll get a new RC up soon. Thanks, James On Thu, Nov 3, 2016 at 8:21 PM, James Taylor wrote: > Hello Everyone, > > This is a call for a vote on Apache Phoenix 4.9.0 RC2. This is the next > minor release of Phoenix 4, compatible with Apache HBase 0.98, 1.1 & 1.2. > The release includes both a source-only release and a convenience binary > release for each supported HBase version. The previous RC was sunk due to > PHOENIX-3421 and PHOENIX-3439 which have now been fixed. > > This release has feature parity with supported HBase versions and includes > the following improvements: > - Atomic upsert through new ON DUPLICATE KEY syntax [1] > - Support for DEFAULT declaration in DDL statements [2] > - Specify guidepost width per table [3] > - ~40 bugs resolved > > The source tarball, including signatures, digests, etc can be found at: > https://dist.apache.org/repos/dist/dev/phoenix/apache- > phoenix-4.9.0-HBase-0.98-rc2/src/ > https://dist.apache.org/repos/dist/dev/phoenix/apache- > phoenix-4.9.0-HBase-1.1-rc2/src/ > https://dist.apache.org/repos/dist/dev/phoenix/apache- > phoenix-4.9.0-HBase-1.2-rc2/src/ > > The binary artifacts can be found at: > https://dist.apache.org/repos/dist/dev/phoenix/apache- > phoenix-4.9.0-HBase-0.98-rc2/bin/ > https://dist.apache.org/repos/dist/dev/phoenix/apache- > phoenix-4.9.0-HBase-1.1-rc2/bin/ > https://dist.apache.org/repos/dist/dev/phoenix/apache- > phoenix-4.9.0-HBase-1.2-rc2/bin/ > > For a complete list of changes, see: > https://issues.apache.org/jira/secure/ReleaseNote.jspa? > projectId=12315120&version=12335845 > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/mujtaba.asc > https://dist.apache.org/repos/dist/release/phoenix/KEYS > > The hash and tag to be voted upon: > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h= > 83ed28f4e526171f1906c99e4fbc184c0b2e7569 > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag; > h=refs/tags/v4.9.0-HBase-0.98-rc2 > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h= > cc2781d7da758a287dc37998340d856c4ddfc38d > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag; > h=refs/tags/v4.9.0-HBase-1.1-rc2 > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h= > 51918bb812f950f6882e9b731967fecf930b3674 > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag; > h=refs/tags/v4.9.0-HBase-1.2-rc2 > > Vote will be open for at least 72 hours. Please vote: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and reason why) > > Thanks, > The Apache Phoenix Team > > > [1] https://issues.apache.org/jira/browse/PHOENIX-6 > [2] https://issues.apache.org/jira/browse/PHOENIX-476 > [3] https://issues.apache.org/jira/browse/PHOENIX-2675 >
[jira] [Created] (PHOENIX-3457) Disable parallel run of tests and increase memory
James Taylor created PHOENIX-3457: - Summary: Disable parallel run of tests and increase memory Key: PHOENIX-3457 URL: https://issues.apache.org/jira/browse/PHOENIX-3457 Project: Phoenix Issue Type: Test Reporter: James Taylor Assignee: James Taylor Fix For: 4.9.0 Attachments: PHOENIX-3457.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3457) Disable parallel run of tests and increase memory
[ https://issues.apache.org/jira/browse/PHOENIX-3457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-3457: -- Attachment: PHOENIX-3457.patch > Disable parallel run of tests and increase memory > - > > Key: PHOENIX-3457 > URL: https://issues.apache.org/jira/browse/PHOENIX-3457 > Project: Phoenix > Issue Type: Test >Reporter: James Taylor >Assignee: James Taylor > Fix For: 4.9.0 > > Attachments: PHOENIX-3457.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3449) Ignore hanging IndexExtendedIT tests until they can be investigated
[ https://issues.apache.org/jira/browse/PHOENIX-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638761#comment-15638761 ] Hudson commented on PHOENIX-3449: - FAILURE: Integrated in Jenkins build Phoenix-master #1474 (See [https://builds.apache.org/job/Phoenix-master/1474/]) PHOENIX-3449 Ignore hanging IndexExtendedIT tests until they can be (jamestaylor: rev 77ab7dfa233d57cfc4e5dcff33d2e4d0dad7959c) * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/IndexExtendedIT.java * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ReadOnlyIndexFailureIT.java > Ignore hanging IndexExtendedIT tests until they can be investigated > --- > > Key: PHOENIX-3449 > URL: https://issues.apache.org/jira/browse/PHOENIX-3449 > Project: Phoenix > Issue Type: Test >Reporter: James Taylor >Assignee: James Taylor > Fix For: 4.9.0 > > > Main hanging appears to be related to local indexes. Needs to be investigated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638762#comment-15638762
]
Hudson commented on PHOENIX-3454:
-
FAILURE: Integrated in Jenkins build Phoenix-master #1474 (See
[https://builds.apache.org/job/Phoenix-master/1474/])
PHOENIX-3454 ON DUPLICATE KEY construct doesn't work correctly when
(jamestaylor: rev b157c485d5fc821b38319eb3f497063c1e9f0ffa)
* (edit)
phoenix-core/src/it/java/org/apache/phoenix/end2end/OnDuplicateKeyIT.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexBuilder.java
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
>Assignee: James Taylor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3454.patch, Screen Shot 2016-11-04 at 1.29.43
> PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3450) Fix hanging tests in IndexExtendedIT for 4.x-HBase-0.98
[ https://issues.apache.org/jira/browse/PHOENIX-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-3450: -- Summary: Fix hanging tests in IndexExtendedIT for 4.x-HBase-0.98 (was: Fix hanging tests in IndexExtendedIT) > Fix hanging tests in IndexExtendedIT for 4.x-HBase-0.98 > --- > > Key: PHOENIX-3450 > URL: https://issues.apache.org/jira/browse/PHOENIX-3450 > Project: Phoenix > Issue Type: Test >Reporter: James Taylor > > It appears that many of the tests in IndexExtendedIT are hanging when run > against local indexes. We should investigate this and get the tests working > again to make sure there are no lurking issues here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-3456) Use unique table names for MutableIndexFailureIT
James Taylor created PHOENIX-3456: - Summary: Use unique table names for MutableIndexFailureIT Key: PHOENIX-3456 URL: https://issues.apache.org/jira/browse/PHOENIX-3456 Project: Phoenix Issue Type: Test Reporter: James Taylor Assignee: James Taylor The MutableIndexFailureIT is failing due to the same table name being used multiple times. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-3454:
--
Fix Version/s: 4.9.0
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3454.patch, Screen Shot 2016-11-04 at 1.29.43
> PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor reassigned PHOENIX-3454:
-
Assignee: James Taylor
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
>Assignee: James Taylor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3454.patch, Screen Shot 2016-11-04 at 1.29.43
> PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3199) ServerCacheClient sends cache to all regions unnecessarily
[
https://issues.apache.org/jira/browse/PHOENIX-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-3199:
--
Fix Version/s: (was: 4.10.0)
4.9.0
> ServerCacheClient sends cache to all regions unnecessarily
> --
>
> Key: PHOENIX-3199
> URL: https://issues.apache.org/jira/browse/PHOENIX-3199
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Fix For: 4.9.0, 4.8.2
>
> Attachments: PHOENIX-3199_v8.patch, PHOENIX-3199_v9.patch
>
>
> The issue is caused by the htable's coprocessorService method,the third
> parameter endKey is inclusive,not exclusive.When both startKey and endKey
> are HConstants.EMPTY_START_ROW,the coprocessorService method may send
> callable to all regions:
> {code:borderStyle=solid}
> coprocessorService(final Class service,byte[] startKey, byte[] endKey,
> final Batch.Call callable)
> {code}
> In the addServerCache method of org.apache.phoenix.cache.ServerCacheClient
> class, once the first region can pass the if test in line 174, because the
> startKey of the first region is HConstants.EMPTY_START_ROW, so the key is
> also HConstants.EMPTY_START_ROW in line 180. When we invoke the htable's
> coprocessorService method in line 189, the startKey and endKey(Inclusive)
> parameters are both HConstants.EMPTY_START_ROW,and the htable's
> coprocessorService method internally uses getKeysAndRegionsInRange method to
> locate
> [HConstants.EMPTY_START_ROW,HConstants.EMPTY_START_ROW] to all regions, so
> cache would be sent to all regions :
> {code:borderStyle=solid}
> 170 for (HRegionLocation entry : locations) {
> 171 // Keep track of servers we've sent to and only send once
> 172byte[] regionStartKey =
> entry.getRegionInfo().getStartKey();
> 173byte[] regionEndKey = entry.getRegionInfo().getEndKey();
> 174 if ( ! servers.contains(entry) &&
> 175keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> 176 cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> 177 // Call RPC once per server
> 178 servers.add(entry);
> 179if (LOG.isDebugEnabled())
> {LOG.debug(addCustomAnnotations("Adding cache entry to be sent for " + entry,
> connection));}
> 180final byte[] key = entry.getRegionInfo().getStartKey();
> 181final HTableInterface htable =
> services.getTable(cacheUsingTableRef.getTable().getPhysicalName().getBytes());
> 182closeables.add(htable);
> 183futures.add(executor.submit(new JobCallable()
> {
> 184
> 185@Override
> 186public Boolean call() throws Exception {
> 187final Map
> results;
> 188try {
> 189results =
> htable.coprocessorService(ServerCachingService.class, key, key,
> 190new
> Batch.Call() {
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3455) MutableIndexFailureIT is hanging on 4.x-HBase-1.1 branch
[ https://issues.apache.org/jira/browse/PHOENIX-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638604#comment-15638604 ] James Taylor commented on PHOENIX-3455: --- [~an...@apache.org] and [~rajeshbabu] - can you please take a look? > MutableIndexFailureIT is hanging on 4.x-HBase-1.1 branch > > > Key: PHOENIX-3455 > URL: https://issues.apache.org/jira/browse/PHOENIX-3455 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor > > MutableIndexFailureIT is hanging on 4.x-HBase-1.1 branch. I reverted back to > the last version that passes along with making the tables unique and > preventing them from being deleted -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-3455) MutableIndexFailureIT is hanging on 4.x-HBase-1.1 branch
James Taylor created PHOENIX-3455: - Summary: MutableIndexFailureIT is hanging on 4.x-HBase-1.1 branch Key: PHOENIX-3455 URL: https://issues.apache.org/jira/browse/PHOENIX-3455 Project: Phoenix Issue Type: Bug Reporter: James Taylor MutableIndexFailureIT is hanging on 4.x-HBase-1.1 branch. I reverted back to the last version that passes along with making the tables unique and preventing them from being deleted -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638401#comment-15638401
]
James Taylor commented on PHOENIX-3454:
---
Yes, definitely a sort is needed. It'll do the sort at the top of loop (before
expression evaluation happens again), unless the loop is being exited (where a
sort is not required). The list is live, so though it's been set already on the
Tuple, the sort will reorder the KeyValues.
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Attachments: PHOENIX-3454.patch, Screen Shot 2016-11-04 at 1.29.43
> PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638376#comment-15638376
]
Samarth Jain commented on PHOENIX-3454:
---
Actually, it is possible that the mutations returned by row.toRowMutations()
may not be sorted wrt the key values. So a sort may still be needed at the
place I mentioned above.
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Attachments: PHOENIX-3454.patch, Screen Shot 2016-11-04 at 1.29.43
> PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638340#comment-15638340
]
Samarth Jain commented on PHOENIX-3454:
---
I see, makes sense. +1 to the patch, then.
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Attachments: PHOENIX-3454.patch, Screen Shot 2016-11-04 at 1.29.43
> PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638328#comment-15638328
]
James Taylor commented on PHOENIX-3454:
---
I only want to sort if the expressions are going to be evaluated again. There's
no need to sort if the loop is going to be exited. I'll add a comment, as
that's not obvious.
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Attachments: PHOENIX-3454.patch, Screen Shot 2016-11-04 at 1.29.43
> PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638324#comment-15638324
]
Samarth Jain commented on PHOENIX-3454:
---
Thanks for the patch, [~jamestaylor]. Wouldn't it be safer (and probably the
correct thing to do too) to sort later :
{code}
for (Mutation source : mutations) {
flattenCells(source, flattenedCells);
}
if (flattenedCells != null) {
Collections.sort(flattenedCells,KeyValue.COMPARATOR);
}
{code}
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Attachments: PHOENIX-3454.patch, Screen Shot 2016-11-04 at 1.29.43
> PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Fwd: Hadoop Summit EU 2017
-- Forwarded message -- From: Alan Gates Date: Fri, Nov 4, 2016 at 10:19 AM Subject: Hadoop Summit EU 2017 To: d...@tephra.incubator.apache.org The DataWorks Summit EU 2017 (including Hadoop Summit) is going to be in Munich April 5-6 2017. I’ve pasted the text from the CFP below. Would you like to share your knowledge with the best and brightest in the data community? If so, we encourage you to submit an abstract for DataWorks Summit with Hadoop Summit being held on April 5-6, 2017 at The ICM – International Congress Center Munich. DataWorks Summit with Hadoop Summit is the premier event for business and technical audiences who want to learn how data is transforming business and the underlying technologies that are driving that change. Our 2017 tracks include: · Applications · Enterprise Adoption · Data Processing & Warehousing · Apache Hadoop Core Internals · Governance & Security · IoT & Streaming · Cloud & Operations · Apache Spark & Data Science For questions or additional information, please contact Joshua Woodward. Deadline: Friday, November 11, 2017. Submission Link: http://dataworkssummit.com/munich-2017/abstracts/submit- abstract/ Alan.
[jira] [Updated] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-3454:
--
Attachment: PHOENIX-3454.patch
Good find, [~samarthjain]. Please review. The problem was that we weren't
resorting the List as required by the underlying HBase code that looks up
column qualifiers. I think we better roll a new RC, [~lhofhansl] &
[~mujtabachohan], as this can lead to silently not incrementing columns which
is bad.
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Attachments: PHOENIX-3454.patch, Screen Shot 2016-11-04 at 1.29.43
> PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3443) Support different encoding schemes (BYTE, SHORT, INTEGER) for storing encoded column qualifiers
[ https://issues.apache.org/jira/browse/PHOENIX-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638100#comment-15638100 ] James Taylor commented on PHOENIX-3443: --- I think the encoding scheme should be specified by a number 0-4 that indicates how many bytes will be used to encode the column qualifier. A value of 0 would indicate that the feature is disabled. Otherwise, we'd detect when the column number value goes beyond the max value based on the number of bytes (we can just use a long when we do the arithmetic), and in that case throw an error. Eventually we could potentially support a kind of clean-up procedure that would reclaim unused columns, but that could be future work if we see demand for it. The poor mans alternative would be to create a new table and do an upsert select into it. > Support different encoding schemes (BYTE, SHORT, INTEGER) for storing encoded > column qualifiers > --- > > Key: PHOENIX-3443 > URL: https://issues.apache.org/jira/browse/PHOENIX-3443 > Project: Phoenix > Issue Type: Sub-task >Reporter: Samarth Jain > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638079#comment-15638079
]
James Taylor commented on PHOENIX-3454:
---
Let me look a little closer, [~lhofhansl] & [~samarthjain] about this one. If
it silently ignores the case sensitive column names and doesn't give an error,
that'd be different. I'll post an update here when I get to the bottom of it.
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Attachments: Screen Shot 2016-11-04 at 1.29.43 PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637690#comment-15637690
]
Samarth Jain commented on PHOENIX-3454:
---
I don't expect the majority of phoenix users to be using case sensitive column
names. So not really a blocker. But it needs to be fixed since it is causing
some test failures in the encodecolumns2 branch ;)
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Attachments: Screen Shot 2016-11-04 at 1.29.43 PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637659#comment-15637659
]
James Taylor commented on PHOENIX-3454:
---
Not a blocker IMHO.
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Attachments: Screen Shot 2016-11-04 at 1.29.43 PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637647#comment-15637647
]
Lars Hofhansl commented on PHOENIX-3454:
Blocker?
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Attachments: Screen Shot 2016-11-04 at 1.29.43 PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[VOTE] Release of Apache Phoenix 4.8.2 RC0
Hello Fellow Phoenix'ers, The first RC for Apache Phoenix 4.8.2 is available. This is a patch release for the Phoenix 4.8 release line, compatible with Apache HBase 0.98, 1.0, 1.1 & 1.2. This release fixes the following 17 issues: [PHOENIX-2996] - Process name of PQS should indicate its role [PHOENIX-3108] - ImmutableIndexIT fails when run on its own [PHOENIX-3159] - CachingHTableFactory may close HTable during eviction even if it is getting used for writing by another thread. [PHOENIX-3199] - ServerCacheClient sends cache to all regions unnecessarily [PHOENIX-3240] - ClassCastException from Pig loader [PHOENIX-3287] - SecureUserConnectionsTest is flapping [PHOENIX-3331] - Bug in calculating minDisabledTimestamp for a batch [PHOENIX-3334] - ConnectionQueryServicesImpl should close HConnection if init fails [PHOENIX-3342] - ORDER BY and LIMIT+OFFSET doesnt work on second column from compound key [PHOENIX-3349] - DROP TABLE and DROP SEQUENCE not working with schemas [PHOENIX-3374] - Wrong data row key is getting generated for local indexes for functions with fixed non null columns [PHOENIX-3382] - Query using Row Value Constructor comprised of non leading PK columns returns incorrect results [PHOENIX-3407] - Read only RowLock may lead to corrupting SYSTEM.CATALOG and non atomic sequences in HBase 1.2 [PHOENIX-3432] - Upgrade Phoenix 4.8.0 to 4.9.0 fails because of illegal characters in snapshot name [PHOENIX-3436] - Port snapshot and upgrade mutex fixes to 4.8 branches [PHOENIX-3439] - Query using an RVC based on the base table PK is incorrectly using an index and doing a full scan instead of a point query [PHOENIX-3296] - ArrayConstructor expression does not serialize arrays with leading nulls correctly. (the release notes are also available here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120&version=12338233) The release includes both a source-only release and a convenience binary release for each supported HBase version. The source tarball for supported HBase versions, including signatures, digests, etc can be found at: https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.8.2-HBase-1.2-rc0/src/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.8.2-HBase-1.1-rc0/src/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.8.2-HBase-1.0-rc0/src/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.8.3-HBase-0.98-rc0/src/ The binary artifacts for supported HBase versions can be found at: https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.8.2-HBase-1.2-rc0/bin/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.8.2-HBase-1.1-rc0/bin/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.8.2-HBase-1.0-rc0/bin/ https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.8.2-HBase-0.98-rc0/bin/ Both artifacts are signed with my "CODE SIGNING KEY": C7CFE328 KEYS file available here: https://dist.apache.org/repos/dist/dev/phoenix/KEYS The hashes and tags to be voted upon: https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.8.2-HBase-1.2-rc0 https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=192de60cfbcb8ca8074ee05906086a685798c76d https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.8.2-HBase-1.1-rc0 https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=5f6929edca9c0bfe4908b2070ccad5805135a2b6 https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.8.2-HBase-1.0-rc0 https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=051fb52b26ac2a87eaad33579f3cd967b40c0c17 https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.8.2-HBase-0.98-rc0 https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=a9a3416a445a0a3e940504159190932ed2c53dfa Vote will be open until, Tue, Nov 8th (inclusive). Please inspect the tarballs, poke around, check the signatures, install and run your favorite workloads, etc. Then please vote: [ ] +1 approve [ ] +-0 no opinion [ ] -1 disapprove (and reason why) Thanks, Your friendly release manager & The Apache Phoenix Team
[jira] [Updated] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Samarth Jain updated PHOENIX-3454:
--
Attachment: Screen Shot 2016-11-04 at 1.29.43 PM.png
See the attached debugger screenshot for PhoenixIndexBuilder.executeAtomicOp().
The reason is because the the mutations returned by PRowImpl.toRowMutations()
isn't returning a sorted list of mutations. The code in
PhoenixIndexBuilder.executeAtomicOp() inherently expects the mutations to be
sorted wrt column qualifier cells since it is doing the following:
{code}
for (Mutation source : mutations) {
flattenCells(source, flattenedCells);
}
tuple.setKeyValues(flattenedCells);
{code}
One option would be to sort the list of flattened cells over in
PhoenixIndexBuilder before setting in the tuple. But then
PRowImpl.toRowMutations() is called in other places too.
I ended up discovering this as part of my work for encoding column qualifiers
where we use the column qualifier "0" instead of "_0" for the empty key value
column.
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
> Attachments: Screen Shot 2016-11-04 at 1.29.43 PM.png
>
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(ddl);
> String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
> DUPLICATE KEY UPDATE \"counter1\" = null";
> conn.createStatement().execute(dml);
> conn.createStatement().execute(dml);
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals(null,rs.getString(2));
> assertFalse(rs.next());
>
> dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
> KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
> conn.createStatement().execute(dml);
> conn.commit();
> rs = conn.createStatement().executeQuery("SELECT * FROM " +
> tableName);
> assertTrue(rs.next());
> assertEquals("a",rs.getString(1));
> assertEquals("c",rs.getString(2));
> assertEquals(2,rs.getInt(3));
> assertFalse(rs.next());
> conn.close();
> }
> {code}
> After changing the column names to upper case (or removing the quotes), the
> test passes.
> FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
[
https://issues.apache.org/jira/browse/PHOENIX-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Samarth Jain updated PHOENIX-3454:
--
Description:
See this test case for a repro:
{code}
@Test
public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
Connection conn = DriverManager.getConnection(getUrl(), props);
conn.setAutoCommit(false);
String tableName = generateUniqueName();
String ddl = " create table " + tableName + "(pk varchar primary key,
\"counter1\" varchar, \"counter2\" smallint)";
conn.createStatement().execute(ddl);
String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
DUPLICATE KEY UPDATE \"counter1\" = null";
conn.createStatement().execute(dml);
conn.createStatement().execute(dml);
conn.commit();
ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
tableName);
assertTrue(rs.next());
assertEquals("a",rs.getString(1));
assertEquals(null,rs.getString(2));
assertFalse(rs.next());
dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
conn.createStatement().execute(dml);
dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
conn.createStatement().execute(dml);
dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
conn.createStatement().execute(dml);
conn.commit();
rs = conn.createStatement().executeQuery("SELECT * FROM " + tableName);
assertTrue(rs.next());
assertEquals("a",rs.getString(1));
assertEquals("c",rs.getString(2));
assertEquals(2,rs.getInt(3));
assertFalse(rs.next());
conn.close();
}
{code}
After changing the column names to upper case (or removing the quotes), the
test passes.
FYI, [~jamestaylor]
was:
See this test case for a repro:
{code}
@Test
private void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
Connection conn = DriverManager.getConnection(getUrl(), props);
conn.setAutoCommit(false);
String tableName = generateUniqueName();
String ddl = " create table " + tableName + "(pk varchar primary key,
\"counter1\" varchar, \"counter2\" smallint)";
conn.createStatement().execute(ddl);
String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
DUPLICATE KEY UPDATE \"counter1\" = null";
conn.createStatement().execute(dml);
conn.createStatement().execute(dml);
conn.commit();
ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
tableName);
assertTrue(rs.next());
assertEquals("a",rs.getString(1));
assertEquals(null,rs.getString(2));
assertFalse(rs.next());
dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
conn.createStatement().execute(dml);
dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
conn.createStatement().execute(dml);
dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
conn.createStatement().execute(dml);
conn.commit();
rs = conn.createStatement().executeQuery("SELECT * FROM " + tableName);
assertTrue(rs.next());
assertEquals("a",rs.getString(1));
assertEquals("c",rs.getString(2));
assertEquals(2,rs.getInt(3));
assertFalse(rs.next());
conn.close();
}
{code}
After changing the column names to upper case (or removing the quotes), the
test passes.
FYI, [~jamestaylor]
> ON DUPLICATE KEY construct doesn't work correctly when using lower case
> column names
>
>
> Key: PHOENIX-3454
> URL: https://issues.apache.org/jira/browse/PHOENIX-3454
> Project: Phoenix
> Issue Type: Bug
>Reporter: Samarth Jain
>
> See this test case for a repro:
> {code}
> @Test
> public void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> conn.setAutoCommit(false);
> String tableName = generateUniqueName();
> String ddl = " create table " + tableName + "(pk varchar primary key,
> \"counter1\" varchar, \"counter2\" smallint)";
> conn.createStatement().execute(
[jira] [Created] (PHOENIX-3454) ON DUPLICATE KEY construct doesn't work correctly when using lower case column names
Samarth Jain created PHOENIX-3454:
-
Summary: ON DUPLICATE KEY construct doesn't work correctly when
using lower case column names
Key: PHOENIX-3454
URL: https://issues.apache.org/jira/browse/PHOENIX-3454
Project: Phoenix
Issue Type: Bug
Reporter: Samarth Jain
See this test case for a repro:
{code}
@Test
private void testDeleteOnSingleLowerCaseVarcharColumn() throws Exception {
Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
Connection conn = DriverManager.getConnection(getUrl(), props);
conn.setAutoCommit(false);
String tableName = generateUniqueName();
String ddl = " create table " + tableName + "(pk varchar primary key,
\"counter1\" varchar, \"counter2\" smallint)";
conn.createStatement().execute(ddl);
String dml = "UPSERT INTO " + tableName + " VALUES('a','b') ON
DUPLICATE KEY UPDATE \"counter1\" = null";
conn.createStatement().execute(dml);
conn.createStatement().execute(dml);
conn.commit();
ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
tableName);
assertTrue(rs.next());
assertEquals("a",rs.getString(1));
assertEquals(null,rs.getString(2));
assertFalse(rs.next());
dml = "UPSERT INTO " + tableName + " VALUES('a','b',0)";
conn.createStatement().execute(dml);
dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
KEY UPDATE \"counter1\" = null, \"counter2\" = \"counter2\" + 1";
conn.createStatement().execute(dml);
dml = "UPSERT INTO " + tableName + " VALUES('a','b', 0) ON DUPLICATE
KEY UPDATE \"counter1\" = 'c', \"counter2\" = \"counter2\" + 1";
conn.createStatement().execute(dml);
conn.commit();
rs = conn.createStatement().executeQuery("SELECT * FROM " + tableName);
assertTrue(rs.next());
assertEquals("a",rs.getString(1));
assertEquals("c",rs.getString(2));
assertEquals(2,rs.getInt(3));
assertFalse(rs.next());
conn.close();
}
{code}
After changing the column names to upper case (or removing the quotes), the
test passes.
FYI, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3199) ServerCacheClient sends cache to all regions unnecessarily
[
https://issues.apache.org/jira/browse/PHOENIX-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637479#comment-15637479
]
Hudson commented on PHOENIX-3199:
-
FAILURE: Integrated in Jenkins build Phoenix-master #1473 (See
[https://builds.apache.org/job/Phoenix-master/1473/])
PHOENIX-3199 ServerCacheClient sends cache to all regions unnecessarily
(jamestaylor: rev e4e1570b83ca0141fc19421a0bd5217ebb37f512)
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/cache/ServerCacheClient.java
> ServerCacheClient sends cache to all regions unnecessarily
> --
>
> Key: PHOENIX-3199
> URL: https://issues.apache.org/jira/browse/PHOENIX-3199
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Fix For: 4.10.0, 4.8.2
>
> Attachments: PHOENIX-3199_v8.patch, PHOENIX-3199_v9.patch
>
>
> The issue is caused by the htable's coprocessorService method,the third
> parameter endKey is inclusive,not exclusive.When both startKey and endKey
> are HConstants.EMPTY_START_ROW,the coprocessorService method may send
> callable to all regions:
> {code:borderStyle=solid}
> coprocessorService(final Class service,byte[] startKey, byte[] endKey,
> final Batch.Call callable)
> {code}
> In the addServerCache method of org.apache.phoenix.cache.ServerCacheClient
> class, once the first region can pass the if test in line 174, because the
> startKey of the first region is HConstants.EMPTY_START_ROW, so the key is
> also HConstants.EMPTY_START_ROW in line 180. When we invoke the htable's
> coprocessorService method in line 189, the startKey and endKey(Inclusive)
> parameters are both HConstants.EMPTY_START_ROW,and the htable's
> coprocessorService method internally uses getKeysAndRegionsInRange method to
> locate
> [HConstants.EMPTY_START_ROW,HConstants.EMPTY_START_ROW] to all regions, so
> cache would be sent to all regions :
> {code:borderStyle=solid}
> 170 for (HRegionLocation entry : locations) {
> 171 // Keep track of servers we've sent to and only send once
> 172byte[] regionStartKey =
> entry.getRegionInfo().getStartKey();
> 173byte[] regionEndKey = entry.getRegionInfo().getEndKey();
> 174 if ( ! servers.contains(entry) &&
> 175keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> 176 cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> 177 // Call RPC once per server
> 178 servers.add(entry);
> 179if (LOG.isDebugEnabled())
> {LOG.debug(addCustomAnnotations("Adding cache entry to be sent for " + entry,
> connection));}
> 180final byte[] key = entry.getRegionInfo().getStartKey();
> 181final HTableInterface htable =
> services.getTable(cacheUsingTableRef.getTable().getPhysicalName().getBytes());
> 182closeables.add(htable);
> 183futures.add(executor.submit(new JobCallable()
> {
> 184
> 185@Override
> 186public Boolean call() throws Exception {
> 187final Map
> results;
> 188try {
> 189results =
> htable.coprocessorService(ServerCachingService.class, key, key,
> 190new
> Batch.Call() {
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3427) rdd.saveToPhoenix gives table undefined error when attempting to write to a tenant-specific view (TenantId defined in configuration object and passed to saveToPhoenix
[ https://issues.apache.org/jira/browse/PHOENIX-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637476#comment-15637476 ] James Taylor commented on PHOENIX-3427: --- So cute! Thanks for sharing. > rdd.saveToPhoenix gives table undefined error when attempting to write to a > tenant-specific view (TenantId defined in configuration object and passed to > saveToPhoenix) > --- > > Key: PHOENIX-3427 > URL: https://issues.apache.org/jira/browse/PHOENIX-3427 > Project: Phoenix > Issue Type: Bug >Reporter: Nico Pappagianis > > Although we can read from a tenant-specific view by passing TenantId in the > conf object when calling sc.phoenixTableAsRDD the same does not hold for > rdd.saveToPhoenix. Calling saveToPhoenix with a tenant-specific view as the > table name gives a table undefined error, even when passing in the TenantId > with the conf object. > It appears that TenantId is lost during the execution path of saveToPhoenix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3427) rdd.saveToPhoenix gives table undefined error when attempting to write to a tenant-specific view (TenantId defined in configuration object and passed to saveToPhoenix
[ https://issues.apache.org/jira/browse/PHOENIX-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637446#comment-15637446 ] Josh Mahonin commented on PHOENIX-3427: --- Thanks [~nico.pappagianis], glad it worked out. [~jamestaylor] Thanks as well. I don't think there's any ASF guidelines against linking to your children's pictures on open issues, right? :) It expires in a few days anyway: https://www.dropbox.com/s/xj2kiyzodpyjoz1/Photo%202016-10-31%2C%201%2037%2052%20PM.jpg?dl=0 > rdd.saveToPhoenix gives table undefined error when attempting to write to a > tenant-specific view (TenantId defined in configuration object and passed to > saveToPhoenix) > --- > > Key: PHOENIX-3427 > URL: https://issues.apache.org/jira/browse/PHOENIX-3427 > Project: Phoenix > Issue Type: Bug >Reporter: Nico Pappagianis > > Although we can read from a tenant-specific view by passing TenantId in the > conf object when calling sc.phoenixTableAsRDD the same does not hold for > rdd.saveToPhoenix. Calling saveToPhoenix with a tenant-specific view as the > table name gives a table undefined error, even when passing in the TenantId > with the conf object. > It appears that TenantId is lost during the execution path of saveToPhoenix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (PHOENIX-3419) phoenix client (sqlline) adds a duplicate table schema to the query
[ https://issues.apache.org/jira/browse/PHOENIX-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Van Wely updated PHOENIX-3419: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > phoenix client (sqlline) adds a duplicate table schema to the query > --- > > Key: PHOENIX-3419 > URL: https://issues.apache.org/jira/browse/PHOENIX-3419 > Project: Phoenix > Issue Type: Bug > Environment: HBase 0.98.20 >Reporter: Saurabh > > phoenix client (sqlline) adds a duplicate table schema to the query. This > causes the select query to fail with table not found exception. Other DDL > operations also are failing. This appears to occur only with tables which > have a secondary index. > describe still works correctly. > 0: jdbc:phoenix:istbd4-mnds1-1-crd.eng.sfdc.n> select count(1) from > archive.field_history_archive; > Error: ERROR 1012 (42M03): Table undefined. > tableName=ARCHIVE.ARCHIVE.FIELD_HISTORY_ARCHIVE (state=42M03,code=1012) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3427) rdd.saveToPhoenix gives table undefined error when attempting to write to a tenant-specific view (TenantId defined in configuration object and passed to saveToPhoenix
[ https://issues.apache.org/jira/browse/PHOENIX-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637383#comment-15637383 ] Nico Pappagianis commented on PHOENIX-3427: --- [~jmahonin] Congratulations! That's awesome! Your suggestion on passing the tenantId to the outConfig worked - that's what I was missing. I was only passing tenantId to the partitionConfig. Thanks for that! I should have a cleaned up version of this sometime today or this weekend. Hopefully a patch will be ready soon! Thanks again :) > rdd.saveToPhoenix gives table undefined error when attempting to write to a > tenant-specific view (TenantId defined in configuration object and passed to > saveToPhoenix) > --- > > Key: PHOENIX-3427 > URL: https://issues.apache.org/jira/browse/PHOENIX-3427 > Project: Phoenix > Issue Type: Bug >Reporter: Nico Pappagianis > > Although we can read from a tenant-specific view by passing TenantId in the > conf object when calling sc.phoenixTableAsRDD the same does not hold for > rdd.saveToPhoenix. Calling saveToPhoenix with a tenant-specific view as the > table name gives a table undefined error, even when passing in the TenantId > with the conf object. > It appears that TenantId is lost during the execution path of saveToPhoenix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3199) ServerCacheClient sends cache to all regions unnecessarily
[
https://issues.apache.org/jira/browse/PHOENIX-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637047#comment-15637047
]
Lars Hofhansl commented on PHOENIX-3199:
Great... I'll pull this into 4.8.2
> ServerCacheClient sends cache to all regions unnecessarily
> --
>
> Key: PHOENIX-3199
> URL: https://issues.apache.org/jira/browse/PHOENIX-3199
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Fix For: 4.10.0, 4.8.2
>
> Attachments: PHOENIX-3199_v8.patch, PHOENIX-3199_v9.patch
>
>
> The issue is caused by the htable's coprocessorService method,the third
> parameter endKey is inclusive,not exclusive.When both startKey and endKey
> are HConstants.EMPTY_START_ROW,the coprocessorService method may send
> callable to all regions:
> {code:borderStyle=solid}
> coprocessorService(final Class service,byte[] startKey, byte[] endKey,
> final Batch.Call callable)
> {code}
> In the addServerCache method of org.apache.phoenix.cache.ServerCacheClient
> class, once the first region can pass the if test in line 174, because the
> startKey of the first region is HConstants.EMPTY_START_ROW, so the key is
> also HConstants.EMPTY_START_ROW in line 180. When we invoke the htable's
> coprocessorService method in line 189, the startKey and endKey(Inclusive)
> parameters are both HConstants.EMPTY_START_ROW,and the htable's
> coprocessorService method internally uses getKeysAndRegionsInRange method to
> locate
> [HConstants.EMPTY_START_ROW,HConstants.EMPTY_START_ROW] to all regions, so
> cache would be sent to all regions :
> {code:borderStyle=solid}
> 170 for (HRegionLocation entry : locations) {
> 171 // Keep track of servers we've sent to and only send once
> 172byte[] regionStartKey =
> entry.getRegionInfo().getStartKey();
> 173byte[] regionEndKey = entry.getRegionInfo().getEndKey();
> 174 if ( ! servers.contains(entry) &&
> 175keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> 176 cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> 177 // Call RPC once per server
> 178 servers.add(entry);
> 179if (LOG.isDebugEnabled())
> {LOG.debug(addCustomAnnotations("Adding cache entry to be sent for " + entry,
> connection));}
> 180final byte[] key = entry.getRegionInfo().getStartKey();
> 181final HTableInterface htable =
> services.getTable(cacheUsingTableRef.getTable().getPhysicalName().getBytes());
> 182closeables.add(htable);
> 183futures.add(executor.submit(new JobCallable()
> {
> 184
> 185@Override
> 186public Boolean call() throws Exception {
> 187final Map
> results;
> 188try {
> 189results =
> htable.coprocessorService(ServerCachingService.class, key, key,
> 190new
> Batch.Call() {
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3439) Query using an RVC based on the base table PK is incorrectly using an index and doing a full scan instead of a point query
[
https://issues.apache.org/jira/browse/PHOENIX-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637043#comment-15637043
]
Lars Hofhansl commented on PHOENIX-3439:
Awesome. Thanks [~jamestaylor]!
> Query using an RVC based on the base table PK is incorrectly using an index
> and doing a full scan instead of a point query
> --
>
> Key: PHOENIX-3439
> URL: https://issues.apache.org/jira/browse/PHOENIX-3439
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.1
>Reporter: Jan Fernando
>Assignee: James Taylor
> Fix For: 4.9.0, 4.8.2
>
> Attachments: PHOENIX-3439.patch
>
>
> We use Phoenix RVCs to support paginated queries. This performance of this
> functionality relies on Phoenix predictably generating scans against a table
> or index with a PK that matches the RVC specified for each page.
> What we do is that on the initial query we use
> PhoenixRuntime.getPkColsDataTypesForSql() to get the list of PK columns and
> persist that and use those to generate RVCs for paginated queries.
> We have discovered that for queries where:
> a) the user doesn't specify an ORDER BY
> b) for tables where secondary indexes are present
> Phoenix returns pk cols for the base table via getPkColsDataTypesForSql() but
> then subsequent queries using the RVCs to paginate execute against a
> secondary index doing a full scan.
> We have a table with a secondary index where this is an issue. The base table
> has a PK of PKCOL1, PKCOL2, PKCOL3 and
> PKCOL4. We have an immutable secondary index where the PK is PKCOL1, PKCOL3,
> PKCOL2, PKCOL4.
> Here's what happens:
> Here is our query we run to get the Query plan from which we generate the
> RVCs to be used for paging:
> EXPLAIN SELECT pkcol1, pkcol2, pkcol3, pkcol4, col1, col2
> FROM MY_TABLES."MYTABLE"
> LIMIT 501;
> I get the following explain:
> CLIENT 6-CHUNK SERIAL 1-WAY ROUND ROBIN RANGE SCAN OVER MY_TABLES.MY_TABLE
> ['00Dxx001gFA']
> SERVER 501 ROW LIMIT
> CLIENT 501 ROW LIMIT
> Therefore the columns we record for RVCs for paging are PK1, PK2, PK3, PK4
> from MY_TABLES.MY_TABLE
> However when I generate the RVC query to page through the data:
> EXPLAIN SELECT pkcol1, pkcol2, pkcol3, pkcol4, col1, col2
> FROM MY_TABLES."MYTABLE"
> (pkcol1, pkcol2, pkcol3,pkcol4) >
> ('001','001xx03DHml',to_date('2015-10-21 09
> (tel:2015102109):50:55.0'),'017xx022FuI')
> LIMIT 501;
> I get the follow explain plan:
> CLIENT 24-CHUNK 7811766 ROWS 6291457403 BYTES PARALLEL 1-WAY ROUND ROBIN
> RANGE SCAN OVER MY_TABLES.MY_SECONDARY_INDEX ['00Dxx001gFA','001'] -
> ['00Dxx001gFA',*]
> SERVER FILTER BY ("PKCOL1", "PKCOL2, "PKCOL3", "PKCOL4") > (TO_CHAR('001'),
> TO_CHAR('001xx03DHml'), DATE '2015-10-21 09 (tel:2015102109):50:55.000',
> TO_CHAR('017xx022FuI'))
> SERVER 501 ROW LIMIT
> CLIENT 501 ROW LIMIT
> We expected that the second query with RVCs above would execute against the
> base table as the base table PK is PKCOL1, PKCOL2, PKCOL3, PKCOL4 and the
> index PK is PKCOL1, PKCOL3, PKCOL2, PKCOL4.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3421) Column name lookups fail when on an indexed table
[ https://issues.apache.org/jira/browse/PHOENIX-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637042#comment-15637042 ] Lars Hofhansl commented on PHOENIX-3421: OK... Changed the affected version to 4.9.0. (Let's try to do that, so we can gauge whether we need to backport stuff) > Column name lookups fail when on an indexed table > - > > Key: PHOENIX-3421 > URL: https://issues.apache.org/jira/browse/PHOENIX-3421 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.9.0 >Reporter: Cody Marcel >Assignee: James Taylor > Fix For: 4.9.0 > > Attachments: DebuggerWindow.png, PHOENIX-3421.patch, > PHOENIX-3421_addendum1.patch > > > Using an index the lookup for encoded values fails. > This happens on tables when using an index. > The conflict is essentially between the two methods below. The pkColsList > create by getPkColsDataTypesForSql() returns column names without a ":", but > the encodeValues() method does a lookup on PTable for the column and cannot > find it. > PhoenixRuntime.getPkColsDataTypesForSql(pkColsList, dataTypesList, queryPlan, > connection, true); > PhoenixRuntime.encodeValues(connection, queryPlanTableName, objects , > pkColsList); -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3421) Column name lookups fail when on an indexed table
[ https://issues.apache.org/jira/browse/PHOENIX-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated PHOENIX-3421: --- Affects Version/s: (was: 4.8.0) 4.9.0 > Column name lookups fail when on an indexed table > - > > Key: PHOENIX-3421 > URL: https://issues.apache.org/jira/browse/PHOENIX-3421 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.9.0 >Reporter: Cody Marcel >Assignee: James Taylor > Fix For: 4.9.0 > > Attachments: DebuggerWindow.png, PHOENIX-3421.patch, > PHOENIX-3421_addendum1.patch > > > Using an index the lookup for encoded values fails. > This happens on tables when using an index. > The conflict is essentially between the two methods below. The pkColsList > create by getPkColsDataTypesForSql() returns column names without a ":", but > the encodeValues() method does a lookup on PTable for the column and cannot > find it. > PhoenixRuntime.getPkColsDataTypesForSql(pkColsList, dataTypesList, queryPlan, > connection, true); > PhoenixRuntime.encodeValues(connection, queryPlanTableName, objects , > pkColsList); -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3199) ServerCacheClient sends cache to all regions unnecessarily
[
https://issues.apache.org/jira/browse/PHOENIX-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637026#comment-15637026
]
Hudson commented on PHOENIX-3199:
-
FAILURE: Integrated in Jenkins build Phoenix-4.8-HBase-1.2 #46 (See
[https://builds.apache.org/job/Phoenix-4.8-HBase-1.2/46/])
PHOENIX-3199 ServerCacheClient sends cache to all regions unnecessarily
(jamestaylor: rev 30e6673d95b95d7b17c2ae73aa8be7ab08725d5b)
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/cache/ServerCacheClient.java
> ServerCacheClient sends cache to all regions unnecessarily
> --
>
> Key: PHOENIX-3199
> URL: https://issues.apache.org/jira/browse/PHOENIX-3199
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Fix For: 4.10.0, 4.8.2
>
> Attachments: PHOENIX-3199_v8.patch, PHOENIX-3199_v9.patch
>
>
> The issue is caused by the htable's coprocessorService method,the third
> parameter endKey is inclusive,not exclusive.When both startKey and endKey
> are HConstants.EMPTY_START_ROW,the coprocessorService method may send
> callable to all regions:
> {code:borderStyle=solid}
> coprocessorService(final Class service,byte[] startKey, byte[] endKey,
> final Batch.Call callable)
> {code}
> In the addServerCache method of org.apache.phoenix.cache.ServerCacheClient
> class, once the first region can pass the if test in line 174, because the
> startKey of the first region is HConstants.EMPTY_START_ROW, so the key is
> also HConstants.EMPTY_START_ROW in line 180. When we invoke the htable's
> coprocessorService method in line 189, the startKey and endKey(Inclusive)
> parameters are both HConstants.EMPTY_START_ROW,and the htable's
> coprocessorService method internally uses getKeysAndRegionsInRange method to
> locate
> [HConstants.EMPTY_START_ROW,HConstants.EMPTY_START_ROW] to all regions, so
> cache would be sent to all regions :
> {code:borderStyle=solid}
> 170 for (HRegionLocation entry : locations) {
> 171 // Keep track of servers we've sent to and only send once
> 172byte[] regionStartKey =
> entry.getRegionInfo().getStartKey();
> 173byte[] regionEndKey = entry.getRegionInfo().getEndKey();
> 174 if ( ! servers.contains(entry) &&
> 175keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> 176 cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> 177 // Call RPC once per server
> 178 servers.add(entry);
> 179if (LOG.isDebugEnabled())
> {LOG.debug(addCustomAnnotations("Adding cache entry to be sent for " + entry,
> connection));}
> 180final byte[] key = entry.getRegionInfo().getStartKey();
> 181final HTableInterface htable =
> services.getTable(cacheUsingTableRef.getTable().getPhysicalName().getBytes());
> 182closeables.add(htable);
> 183futures.add(executor.submit(new JobCallable()
> {
> 184
> 185@Override
> 186public Boolean call() throws Exception {
> 187final Map
> results;
> 188try {
> 189results =
> htable.coprocessorService(ServerCachingService.class, key, key,
> 190new
> Batch.Call() {
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3439) Query using an RVC based on the base table PK is incorrectly using an index and doing a full scan instead of a point query
[
https://issues.apache.org/jira/browse/PHOENIX-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637025#comment-15637025
]
Hudson commented on PHOENIX-3439:
-
FAILURE: Integrated in Jenkins build Phoenix-4.8-HBase-1.2 #46 (See
[https://builds.apache.org/job/Phoenix-4.8-HBase-1.2/46/])
PHOENIX-3439 Query using an RVC based on the base table PK is (jamestaylor: rev
eb871beefef375dad2059f4ba553bdd2407835f8)
* (edit)
phoenix-core/src/test/java/org/apache/phoenix/compile/QueryOptimizerTest.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/compile/ScanRanges.java
> Query using an RVC based on the base table PK is incorrectly using an index
> and doing a full scan instead of a point query
> --
>
> Key: PHOENIX-3439
> URL: https://issues.apache.org/jira/browse/PHOENIX-3439
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.1
>Reporter: Jan Fernando
>Assignee: James Taylor
> Fix For: 4.9.0, 4.8.2
>
> Attachments: PHOENIX-3439.patch
>
>
> We use Phoenix RVCs to support paginated queries. This performance of this
> functionality relies on Phoenix predictably generating scans against a table
> or index with a PK that matches the RVC specified for each page.
> What we do is that on the initial query we use
> PhoenixRuntime.getPkColsDataTypesForSql() to get the list of PK columns and
> persist that and use those to generate RVCs for paginated queries.
> We have discovered that for queries where:
> a) the user doesn't specify an ORDER BY
> b) for tables where secondary indexes are present
> Phoenix returns pk cols for the base table via getPkColsDataTypesForSql() but
> then subsequent queries using the RVCs to paginate execute against a
> secondary index doing a full scan.
> We have a table with a secondary index where this is an issue. The base table
> has a PK of PKCOL1, PKCOL2, PKCOL3 and
> PKCOL4. We have an immutable secondary index where the PK is PKCOL1, PKCOL3,
> PKCOL2, PKCOL4.
> Here's what happens:
> Here is our query we run to get the Query plan from which we generate the
> RVCs to be used for paging:
> EXPLAIN SELECT pkcol1, pkcol2, pkcol3, pkcol4, col1, col2
> FROM MY_TABLES."MYTABLE"
> LIMIT 501;
> I get the following explain:
> CLIENT 6-CHUNK SERIAL 1-WAY ROUND ROBIN RANGE SCAN OVER MY_TABLES.MY_TABLE
> ['00Dxx001gFA']
> SERVER 501 ROW LIMIT
> CLIENT 501 ROW LIMIT
> Therefore the columns we record for RVCs for paging are PK1, PK2, PK3, PK4
> from MY_TABLES.MY_TABLE
> However when I generate the RVC query to page through the data:
> EXPLAIN SELECT pkcol1, pkcol2, pkcol3, pkcol4, col1, col2
> FROM MY_TABLES."MYTABLE"
> (pkcol1, pkcol2, pkcol3,pkcol4) >
> ('001','001xx03DHml',to_date('2015-10-21 09
> (tel:2015102109):50:55.0'),'017xx022FuI')
> LIMIT 501;
> I get the follow explain plan:
> CLIENT 24-CHUNK 7811766 ROWS 6291457403 BYTES PARALLEL 1-WAY ROUND ROBIN
> RANGE SCAN OVER MY_TABLES.MY_SECONDARY_INDEX ['00Dxx001gFA','001'] -
> ['00Dxx001gFA',*]
> SERVER FILTER BY ("PKCOL1", "PKCOL2, "PKCOL3", "PKCOL4") > (TO_CHAR('001'),
> TO_CHAR('001xx03DHml'), DATE '2015-10-21 09 (tel:2015102109):50:55.000',
> TO_CHAR('017xx022FuI'))
> SERVER 501 ROW LIMIT
> CLIENT 501 ROW LIMIT
> We expected that the second query with RVCs above would execute against the
> base table as the base table PK is PKCOL1, PKCOL2, PKCOL3, PKCOL4 and the
> index PK is PKCOL1, PKCOL3, PKCOL2, PKCOL4.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3439) Query using an RVC based on the base table PK is incorrectly using an index and doing a full scan instead of a point query
[
https://issues.apache.org/jira/browse/PHOENIX-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636924#comment-15636924
]
James Taylor commented on PHOENIX-3439:
---
Thanks, [~lhofhansl] - I back ported this to 4.8 branches now.
> Query using an RVC based on the base table PK is incorrectly using an index
> and doing a full scan instead of a point query
> --
>
> Key: PHOENIX-3439
> URL: https://issues.apache.org/jira/browse/PHOENIX-3439
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.1
>Reporter: Jan Fernando
>Assignee: James Taylor
> Fix For: 4.9.0, 4.8.2
>
> Attachments: PHOENIX-3439.patch
>
>
> We use Phoenix RVCs to support paginated queries. This performance of this
> functionality relies on Phoenix predictably generating scans against a table
> or index with a PK that matches the RVC specified for each page.
> What we do is that on the initial query we use
> PhoenixRuntime.getPkColsDataTypesForSql() to get the list of PK columns and
> persist that and use those to generate RVCs for paginated queries.
> We have discovered that for queries where:
> a) the user doesn't specify an ORDER BY
> b) for tables where secondary indexes are present
> Phoenix returns pk cols for the base table via getPkColsDataTypesForSql() but
> then subsequent queries using the RVCs to paginate execute against a
> secondary index doing a full scan.
> We have a table with a secondary index where this is an issue. The base table
> has a PK of PKCOL1, PKCOL2, PKCOL3 and
> PKCOL4. We have an immutable secondary index where the PK is PKCOL1, PKCOL3,
> PKCOL2, PKCOL4.
> Here's what happens:
> Here is our query we run to get the Query plan from which we generate the
> RVCs to be used for paging:
> EXPLAIN SELECT pkcol1, pkcol2, pkcol3, pkcol4, col1, col2
> FROM MY_TABLES."MYTABLE"
> LIMIT 501;
> I get the following explain:
> CLIENT 6-CHUNK SERIAL 1-WAY ROUND ROBIN RANGE SCAN OVER MY_TABLES.MY_TABLE
> ['00Dxx001gFA']
> SERVER 501 ROW LIMIT
> CLIENT 501 ROW LIMIT
> Therefore the columns we record for RVCs for paging are PK1, PK2, PK3, PK4
> from MY_TABLES.MY_TABLE
> However when I generate the RVC query to page through the data:
> EXPLAIN SELECT pkcol1, pkcol2, pkcol3, pkcol4, col1, col2
> FROM MY_TABLES."MYTABLE"
> (pkcol1, pkcol2, pkcol3,pkcol4) >
> ('001','001xx03DHml',to_date('2015-10-21 09
> (tel:2015102109):50:55.0'),'017xx022FuI')
> LIMIT 501;
> I get the follow explain plan:
> CLIENT 24-CHUNK 7811766 ROWS 6291457403 BYTES PARALLEL 1-WAY ROUND ROBIN
> RANGE SCAN OVER MY_TABLES.MY_SECONDARY_INDEX ['00Dxx001gFA','001'] -
> ['00Dxx001gFA',*]
> SERVER FILTER BY ("PKCOL1", "PKCOL2, "PKCOL3", "PKCOL4") > (TO_CHAR('001'),
> TO_CHAR('001xx03DHml'), DATE '2015-10-21 09 (tel:2015102109):50:55.000',
> TO_CHAR('017xx022FuI'))
> SERVER 501 ROW LIMIT
> CLIENT 501 ROW LIMIT
> We expected that the second query with RVCs above would execute against the
> base table as the base table PK is PKCOL1, PKCOL2, PKCOL3, PKCOL4 and the
> index PK is PKCOL1, PKCOL3, PKCOL2, PKCOL4.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3439) Query using an RVC based on the base table PK is incorrectly using an index and doing a full scan instead of a point query
[
https://issues.apache.org/jira/browse/PHOENIX-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-3439:
--
Fix Version/s: 4.8.2
> Query using an RVC based on the base table PK is incorrectly using an index
> and doing a full scan instead of a point query
> --
>
> Key: PHOENIX-3439
> URL: https://issues.apache.org/jira/browse/PHOENIX-3439
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.1
>Reporter: Jan Fernando
>Assignee: James Taylor
> Fix For: 4.9.0, 4.8.2
>
> Attachments: PHOENIX-3439.patch
>
>
> We use Phoenix RVCs to support paginated queries. This performance of this
> functionality relies on Phoenix predictably generating scans against a table
> or index with a PK that matches the RVC specified for each page.
> What we do is that on the initial query we use
> PhoenixRuntime.getPkColsDataTypesForSql() to get the list of PK columns and
> persist that and use those to generate RVCs for paginated queries.
> We have discovered that for queries where:
> a) the user doesn't specify an ORDER BY
> b) for tables where secondary indexes are present
> Phoenix returns pk cols for the base table via getPkColsDataTypesForSql() but
> then subsequent queries using the RVCs to paginate execute against a
> secondary index doing a full scan.
> We have a table with a secondary index where this is an issue. The base table
> has a PK of PKCOL1, PKCOL2, PKCOL3 and
> PKCOL4. We have an immutable secondary index where the PK is PKCOL1, PKCOL3,
> PKCOL2, PKCOL4.
> Here's what happens:
> Here is our query we run to get the Query plan from which we generate the
> RVCs to be used for paging:
> EXPLAIN SELECT pkcol1, pkcol2, pkcol3, pkcol4, col1, col2
> FROM MY_TABLES."MYTABLE"
> LIMIT 501;
> I get the following explain:
> CLIENT 6-CHUNK SERIAL 1-WAY ROUND ROBIN RANGE SCAN OVER MY_TABLES.MY_TABLE
> ['00Dxx001gFA']
> SERVER 501 ROW LIMIT
> CLIENT 501 ROW LIMIT
> Therefore the columns we record for RVCs for paging are PK1, PK2, PK3, PK4
> from MY_TABLES.MY_TABLE
> However when I generate the RVC query to page through the data:
> EXPLAIN SELECT pkcol1, pkcol2, pkcol3, pkcol4, col1, col2
> FROM MY_TABLES."MYTABLE"
> (pkcol1, pkcol2, pkcol3,pkcol4) >
> ('001','001xx03DHml',to_date('2015-10-21 09
> (tel:2015102109):50:55.0'),'017xx022FuI')
> LIMIT 501;
> I get the follow explain plan:
> CLIENT 24-CHUNK 7811766 ROWS 6291457403 BYTES PARALLEL 1-WAY ROUND ROBIN
> RANGE SCAN OVER MY_TABLES.MY_SECONDARY_INDEX ['00Dxx001gFA','001'] -
> ['00Dxx001gFA',*]
> SERVER FILTER BY ("PKCOL1", "PKCOL2, "PKCOL3", "PKCOL4") > (TO_CHAR('001'),
> TO_CHAR('001xx03DHml'), DATE '2015-10-21 09 (tel:2015102109):50:55.000',
> TO_CHAR('017xx022FuI'))
> SERVER 501 ROW LIMIT
> CLIENT 501 ROW LIMIT
> We expected that the second query with RVCs above would execute against the
> base table as the base table PK is PKCOL1, PKCOL2, PKCOL3, PKCOL4 and the
> index PK is PKCOL1, PKCOL3, PKCOL2, PKCOL4.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3421) Column name lookups fail when on an indexed table
[ https://issues.apache.org/jira/browse/PHOENIX-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636918#comment-15636918 ] James Taylor commented on PHOENIX-3421: --- This fix was to fix a b/w compat issue in a change that was made in the 4.x branches only, so it's not applicable to 4.8. > Column name lookups fail when on an indexed table > - > > Key: PHOENIX-3421 > URL: https://issues.apache.org/jira/browse/PHOENIX-3421 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.8.0 >Reporter: Cody Marcel >Assignee: James Taylor > Fix For: 4.9.0 > > Attachments: DebuggerWindow.png, PHOENIX-3421.patch, > PHOENIX-3421_addendum1.patch > > > Using an index the lookup for encoded values fails. > This happens on tables when using an index. > The conflict is essentially between the two methods below. The pkColsList > create by getPkColsDataTypesForSql() returns column names without a ":", but > the encodeValues() method does a lookup on PTable for the column and cannot > find it. > PhoenixRuntime.getPkColsDataTypesForSql(pkColsList, dataTypesList, queryPlan, > connection, true); > PhoenixRuntime.encodeValues(connection, queryPlanTableName, objects , > pkColsList); -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3199) ServerCacheClient sends cache to all regions unnecessarily
[
https://issues.apache.org/jira/browse/PHOENIX-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636883#comment-15636883
]
Hadoop QA commented on PHOENIX-3199:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12837169/PHOENIX-3199_v9.patch
against master branch at commit e4e1570b83ca0141fc19421a0bd5217ebb37f512.
ATTACHMENT ID: 12837169
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/676//console
This message is automatically generated.
> ServerCacheClient sends cache to all regions unnecessarily
> --
>
> Key: PHOENIX-3199
> URL: https://issues.apache.org/jira/browse/PHOENIX-3199
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Fix For: 4.10.0, 4.8.2
>
> Attachments: PHOENIX-3199_v8.patch, PHOENIX-3199_v9.patch
>
>
> The issue is caused by the htable's coprocessorService method,the third
> parameter endKey is inclusive,not exclusive.When both startKey and endKey
> are HConstants.EMPTY_START_ROW,the coprocessorService method may send
> callable to all regions:
> {code:borderStyle=solid}
> coprocessorService(final Class service,byte[] startKey, byte[] endKey,
> final Batch.Call callable)
> {code}
> In the addServerCache method of org.apache.phoenix.cache.ServerCacheClient
> class, once the first region can pass the if test in line 174, because the
> startKey of the first region is HConstants.EMPTY_START_ROW, so the key is
> also HConstants.EMPTY_START_ROW in line 180. When we invoke the htable's
> coprocessorService method in line 189, the startKey and endKey(Inclusive)
> parameters are both HConstants.EMPTY_START_ROW,and the htable's
> coprocessorService method internally uses getKeysAndRegionsInRange method to
> locate
> [HConstants.EMPTY_START_ROW,HConstants.EMPTY_START_ROW] to all regions, so
> cache would be sent to all regions :
> {code:borderStyle=solid}
> 170 for (HRegionLocation entry : locations) {
> 171 // Keep track of servers we've sent to and only send once
> 172byte[] regionStartKey =
> entry.getRegionInfo().getStartKey();
> 173byte[] regionEndKey = entry.getRegionInfo().getEndKey();
> 174 if ( ! servers.contains(entry) &&
> 175keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> 176 cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> 177 // Call RPC once per server
> 178 servers.add(entry);
> 179if (LOG.isDebugEnabled())
> {LOG.debug(addCustomAnnotations("Adding cache entry to be sent for " + entry,
> connection));}
> 180final byte[] key = entry.getRegionInfo().getStartKey();
> 181final HTableInterface htable =
> services.getTable(cacheUsingTableRef.getTable().getPhysicalName().getBytes());
> 182closeables.add(htable);
> 183futures.add(executor.submit(new JobCallable()
> {
> 184
> 185@Override
> 186public Boolean call() throws Exception {
> 187final Map
> results;
> 188try {
> 189results =
> htable.coprocessorService(ServerCachingService.class, key, key,
> 190new
> Batch.Call() {
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3199) ServerCacheClient sends cache to all regions unnecessarily
[
https://issues.apache.org/jira/browse/PHOENIX-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-3199:
--
Attachment: PHOENIX-3199_v9.patch
Version of patch committed (just renamed constant and method).
> ServerCacheClient sends cache to all regions unnecessarily
> --
>
> Key: PHOENIX-3199
> URL: https://issues.apache.org/jira/browse/PHOENIX-3199
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Fix For: 4.10.0, 4.8.2
>
> Attachments: PHOENIX-3199_v8.patch, PHOENIX-3199_v9.patch
>
>
> The issue is caused by the htable's coprocessorService method,the third
> parameter endKey is inclusive,not exclusive.When both startKey and endKey
> are HConstants.EMPTY_START_ROW,the coprocessorService method may send
> callable to all regions:
> {code:borderStyle=solid}
> coprocessorService(final Class service,byte[] startKey, byte[] endKey,
> final Batch.Call callable)
> {code}
> In the addServerCache method of org.apache.phoenix.cache.ServerCacheClient
> class, once the first region can pass the if test in line 174, because the
> startKey of the first region is HConstants.EMPTY_START_ROW, so the key is
> also HConstants.EMPTY_START_ROW in line 180. When we invoke the htable's
> coprocessorService method in line 189, the startKey and endKey(Inclusive)
> parameters are both HConstants.EMPTY_START_ROW,and the htable's
> coprocessorService method internally uses getKeysAndRegionsInRange method to
> locate
> [HConstants.EMPTY_START_ROW,HConstants.EMPTY_START_ROW] to all regions, so
> cache would be sent to all regions :
> {code:borderStyle=solid}
> 170 for (HRegionLocation entry : locations) {
> 171 // Keep track of servers we've sent to and only send once
> 172byte[] regionStartKey =
> entry.getRegionInfo().getStartKey();
> 173byte[] regionEndKey = entry.getRegionInfo().getEndKey();
> 174 if ( ! servers.contains(entry) &&
> 175keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> 176 cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> 177 // Call RPC once per server
> 178 servers.add(entry);
> 179if (LOG.isDebugEnabled())
> {LOG.debug(addCustomAnnotations("Adding cache entry to be sent for " + entry,
> connection));}
> 180final byte[] key = entry.getRegionInfo().getStartKey();
> 181final HTableInterface htable =
> services.getTable(cacheUsingTableRef.getTable().getPhysicalName().getBytes());
> 182closeables.add(htable);
> 183futures.add(executor.submit(new JobCallable()
> {
> 184
> 185@Override
> 186public Boolean call() throws Exception {
> 187final Map
> results;
> 188try {
> 189results =
> htable.coprocessorService(ServerCachingService.class, key, key,
> 190new
> Batch.Call() {
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3439) Query using an RVC based on the base table PK is incorrectly using an index and doing a full scan instead of a point query
[
https://issues.apache.org/jira/browse/PHOENIX-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636832#comment-15636832
]
Lars Hofhansl commented on PHOENIX-3439:
Since this affect 4.8.1 should it be in 4.8.2 as well?
> Query using an RVC based on the base table PK is incorrectly using an index
> and doing a full scan instead of a point query
> --
>
> Key: PHOENIX-3439
> URL: https://issues.apache.org/jira/browse/PHOENIX-3439
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.1
>Reporter: Jan Fernando
>Assignee: James Taylor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3439.patch
>
>
> We use Phoenix RVCs to support paginated queries. This performance of this
> functionality relies on Phoenix predictably generating scans against a table
> or index with a PK that matches the RVC specified for each page.
> What we do is that on the initial query we use
> PhoenixRuntime.getPkColsDataTypesForSql() to get the list of PK columns and
> persist that and use those to generate RVCs for paginated queries.
> We have discovered that for queries where:
> a) the user doesn't specify an ORDER BY
> b) for tables where secondary indexes are present
> Phoenix returns pk cols for the base table via getPkColsDataTypesForSql() but
> then subsequent queries using the RVCs to paginate execute against a
> secondary index doing a full scan.
> We have a table with a secondary index where this is an issue. The base table
> has a PK of PKCOL1, PKCOL2, PKCOL3 and
> PKCOL4. We have an immutable secondary index where the PK is PKCOL1, PKCOL3,
> PKCOL2, PKCOL4.
> Here's what happens:
> Here is our query we run to get the Query plan from which we generate the
> RVCs to be used for paging:
> EXPLAIN SELECT pkcol1, pkcol2, pkcol3, pkcol4, col1, col2
> FROM MY_TABLES."MYTABLE"
> LIMIT 501;
> I get the following explain:
> CLIENT 6-CHUNK SERIAL 1-WAY ROUND ROBIN RANGE SCAN OVER MY_TABLES.MY_TABLE
> ['00Dxx001gFA']
> SERVER 501 ROW LIMIT
> CLIENT 501 ROW LIMIT
> Therefore the columns we record for RVCs for paging are PK1, PK2, PK3, PK4
> from MY_TABLES.MY_TABLE
> However when I generate the RVC query to page through the data:
> EXPLAIN SELECT pkcol1, pkcol2, pkcol3, pkcol4, col1, col2
> FROM MY_TABLES."MYTABLE"
> (pkcol1, pkcol2, pkcol3,pkcol4) >
> ('001','001xx03DHml',to_date('2015-10-21 09
> (tel:2015102109):50:55.0'),'017xx022FuI')
> LIMIT 501;
> I get the follow explain plan:
> CLIENT 24-CHUNK 7811766 ROWS 6291457403 BYTES PARALLEL 1-WAY ROUND ROBIN
> RANGE SCAN OVER MY_TABLES.MY_SECONDARY_INDEX ['00Dxx001gFA','001'] -
> ['00Dxx001gFA',*]
> SERVER FILTER BY ("PKCOL1", "PKCOL2, "PKCOL3", "PKCOL4") > (TO_CHAR('001'),
> TO_CHAR('001xx03DHml'), DATE '2015-10-21 09 (tel:2015102109):50:55.000',
> TO_CHAR('017xx022FuI'))
> SERVER 501 ROW LIMIT
> CLIENT 501 ROW LIMIT
> We expected that the second query with RVCs above would execute against the
> base table as the base table PK is PKCOL1, PKCOL2, PKCOL3, PKCOL4 and the
> index PK is PKCOL1, PKCOL3, PKCOL2, PKCOL4.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3421) Column name lookups fail when on an indexed table
[ https://issues.apache.org/jira/browse/PHOENIX-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636839#comment-15636839 ] Lars Hofhansl commented on PHOENIX-3421: Affects 4.8.0... Should this be in 4.8.2 as well? > Column name lookups fail when on an indexed table > - > > Key: PHOENIX-3421 > URL: https://issues.apache.org/jira/browse/PHOENIX-3421 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.8.0 >Reporter: Cody Marcel >Assignee: James Taylor > Fix For: 4.9.0 > > Attachments: DebuggerWindow.png, PHOENIX-3421.patch, > PHOENIX-3421_addendum1.patch > > > Using an index the lookup for encoded values fails. > This happens on tables when using an index. > The conflict is essentially between the two methods below. The pkColsList > create by getPkColsDataTypesForSql() returns column names without a ":", but > the encodeValues() method does a lookup on PTable for the column and cannot > find it. > PhoenixRuntime.getPkColsDataTypesForSql(pkColsList, dataTypesList, queryPlan, > connection, true); > PhoenixRuntime.encodeValues(connection, queryPlanTableName, objects , > pkColsList); -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3452) Secondary index and query using distinct: ORDER BY doesn't work correctly
[
https://issues.apache.org/jira/browse/PHOENIX-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Palmert updated PHOENIX-3452:
--
Summary: Secondary index and query using distinct: ORDER BY doesn't work
correctly (was: Secondary index and query using distinct: ORDE BY doesn't work
correctly)
> Secondary index and query using distinct: ORDER BY doesn't work correctly
> -
>
> Key: PHOENIX-3452
> URL: https://issues.apache.org/jira/browse/PHOENIX-3452
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Joel Palmert
>
> This may be related to PHOENIX-3451 but the behavior is different so filing
> it separately.
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ORGANIZATION_ID CHAR(15) NOT NULL,
> CONTAINER_ID CHAR(15) NOT NULL,
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
> ENTITY_ID DESC);
> UPSERT INTO test.test VALUES ('org1','container1','entityId6',1.1);
> UPSERT INTO test.test VALUES ('org1','container1','entityId5',1.2);
> UPSERT INTO test.test VALUES ('org1','container1','entityId4',1.3);
> UPSERT INTO test.test VALUES ('org1','container1','entityId3',1.4);
> UPSERT INTO test.test VALUES ('org1','container1','entityId2',1.5);
> UPSERT INTO test.test VALUES ('org1','container1','entityId1',1.6);
> SELECT DISTINCT entity_id, score
> FROM test.test
> WHERE organization_id = 'org1'
> AND container_id = 'container1'
> ORDER BY score DESC
> Notice that the returned results are not returned in descending score order.
> Instead they are returned in descending entity_id order. If I remove the
> DISTINCT or remove the secondary index the result is correct.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3199) ServerCacheClient sends cache to all regions unnecessarily
[
https://issues.apache.org/jira/browse/PHOENIX-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636545#comment-15636545
]
James Taylor commented on PHOENIX-3199:
---
+1. Nice work, [~comnetwork]. I'll commit this today.
> ServerCacheClient sends cache to all regions unnecessarily
> --
>
> Key: PHOENIX-3199
> URL: https://issues.apache.org/jira/browse/PHOENIX-3199
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: chenglei
>Assignee: chenglei
> Fix For: 4.10.0, 4.8.2
>
> Attachments: PHOENIX-3199_v8.patch
>
>
> The issue is caused by the htable's coprocessorService method,the third
> parameter endKey is inclusive,not exclusive.When both startKey and endKey
> are HConstants.EMPTY_START_ROW,the coprocessorService method may send
> callable to all regions:
> {code:borderStyle=solid}
> coprocessorService(final Class service,byte[] startKey, byte[] endKey,
> final Batch.Call callable)
> {code}
> In the addServerCache method of org.apache.phoenix.cache.ServerCacheClient
> class, once the first region can pass the if test in line 174, because the
> startKey of the first region is HConstants.EMPTY_START_ROW, so the key is
> also HConstants.EMPTY_START_ROW in line 180. When we invoke the htable's
> coprocessorService method in line 189, the startKey and endKey(Inclusive)
> parameters are both HConstants.EMPTY_START_ROW,and the htable's
> coprocessorService method internally uses getKeysAndRegionsInRange method to
> locate
> [HConstants.EMPTY_START_ROW,HConstants.EMPTY_START_ROW] to all regions, so
> cache would be sent to all regions :
> {code:borderStyle=solid}
> 170 for (HRegionLocation entry : locations) {
> 171 // Keep track of servers we've sent to and only send once
> 172byte[] regionStartKey =
> entry.getRegionInfo().getStartKey();
> 173byte[] regionEndKey = entry.getRegionInfo().getEndKey();
> 174 if ( ! servers.contains(entry) &&
> 175keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> 176 cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> 177 // Call RPC once per server
> 178 servers.add(entry);
> 179if (LOG.isDebugEnabled())
> {LOG.debug(addCustomAnnotations("Adding cache entry to be sent for " + entry,
> connection));}
> 180final byte[] key = entry.getRegionInfo().getStartKey();
> 181final HTableInterface htable =
> services.getTable(cacheUsingTableRef.getTable().getPhysicalName().getBytes());
> 182closeables.add(htable);
> 183futures.add(executor.submit(new JobCallable()
> {
> 184
> 185@Override
> 186public Boolean call() throws Exception {
> 187final Map
> results;
> 188try {
> 189results =
> htable.coprocessorService(ServerCachingService.class, key, key,
> 190new
> Batch.Call() {
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (PHOENIX-3453) Secondary index and query using distinct: Outer query results in ERROR 201 (22000): Illegal data. CHAR types may only contain single byte characters
Joel Palmert created PHOENIX-3453:
-
Summary: Secondary index and query using distinct: Outer query
results in ERROR 201 (22000): Illegal data. CHAR types may only contain single
byte characters
Key: PHOENIX-3453
URL: https://issues.apache.org/jira/browse/PHOENIX-3453
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.8.0
Reporter: Joel Palmert
Steps to repro:
CREATE TABLE IF NOT EXISTS TEST.TEST (
ENTITY_ID CHAR(15) NOT NULL,
SCORE DOUBLE,
CONSTRAINT TEST_PK PRIMARY KEY (
ENTITY_ID
)
) VERSIONS=1, MULTI_TENANT=FALSE, REPLICATION_SCOPE=1, TTL=31536000;
CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (SCORE DESC, ENTITY_ID DESC);
UPSERT INTO test.test VALUES ('entity1',1.1);
SELECT DISTINCT entity_id, score
FROM(
SELECT entity_id, score
FROM test.test
LIMIT 25
);
Output (in SQuirreL)
��� 1.1
If you run it in SQuirreL it results in the entity_id column getting the above
error value. Notice that if you remove the secondary index or DISTINCT you get
the correct result.
I've also run the query through the Phoenix java api. Then I get the following
exception:
Caused by: java.sql.SQLException: ERROR 201 (22000): Illegal data. CHAR types
may only contain single byte characters ()
at
org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:454)
at
org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
at
org.apache.phoenix.schema.types.PDataType.newIllegalDataException(PDataType.java:291)
at org.apache.phoenix.schema.types.PChar.toObject(PChar.java:121)
at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:997)
at
org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
at
org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:608)
at
org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:621)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3452) Secondary index and query using distinct: ORDE BY doesn't work correctly
[
https://issues.apache.org/jira/browse/PHOENIX-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Palmert updated PHOENIX-3452:
--
Description:
This may be related to PHOENIX-3451 but the behavior is different so filing it
separately.
Steps to repro:
CREATE TABLE IF NOT EXISTS TEST.TEST (
ORGANIZATION_ID CHAR(15) NOT NULL,
CONTAINER_ID CHAR(15) NOT NULL,
ENTITY_ID CHAR(15) NOT NULL,
SCORE DOUBLE,
CONSTRAINT TEST_PK PRIMARY KEY (
ORGANIZATION_ID,
CONTAINER_ID,
ENTITY_ID
)
) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
ENTITY_ID DESC);
UPSERT INTO test.test VALUES ('org1','container1','entityId6',1.1);
UPSERT INTO test.test VALUES ('org1','container1','entityId5',1.2);
UPSERT INTO test.test VALUES ('org1','container1','entityId4',1.3);
UPSERT INTO test.test VALUES ('org1','container1','entityId3',1.4);
UPSERT INTO test.test VALUES ('org1','container1','entityId2',1.5);
UPSERT INTO test.test VALUES ('org1','container1','entityId1',1.6);
SELECT DISTINCT entity_id, score
FROM test.test
WHERE organization_id = 'org1'
AND container_id = 'container1'
ORDER BY score DESC
Notice that the returned results are not returned in descending score order.
Instead they are returned in descending entity_id order. If I remove the
DISTINCT or remove the secondary index the result is correct.
was:
This may be related to PHOENIX-3451 but the behavior is different so filing it
separately.
Steps to repro:
DROP TABLE TEST.TEST;
CREATE TABLE IF NOT EXISTS TEST.TEST (
ORGANIZATION_ID CHAR(15) NOT NULL,
CONTAINER_ID CHAR(15) NOT NULL,
ENTITY_ID CHAR(15) NOT NULL,
SCORE DOUBLE,
CONSTRAINT TEST_PK PRIMARY KEY (
ORGANIZATION_ID,
CONTAINER_ID,
ENTITY_ID
)
) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
ENTITY_ID DESC);
UPSERT INTO test.test VALUES ('org1','container1','entityId6',1.1);
UPSERT INTO test.test VALUES ('org1','container1','entityId5',1.2);
UPSERT INTO test.test VALUES ('org1','container1','entityId4',1.3);
UPSERT INTO test.test VALUES ('org1','container1','entityId3',1.4);
UPSERT INTO test.test VALUES ('org1','container1','entityId2',1.5);
UPSERT INTO test.test VALUES ('org1','container1','entityId1',1.6);
SELECT DISTINCT entity_id, score
FROM test.test
WHERE organization_id = 'org1'
AND container_id = 'container1'
ORDER BY score DESC
Notice that the returned results are not returned in descending score order.
Instead they are returned in descending entity_id order. If I remove the
DISTINCT or remove the secondary index the result is correct.
> Secondary index and query using distinct: ORDE BY doesn't work correctly
>
>
> Key: PHOENIX-3452
> URL: https://issues.apache.org/jira/browse/PHOENIX-3452
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Joel Palmert
>
> This may be related to PHOENIX-3451 but the behavior is different so filing
> it separately.
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ORGANIZATION_ID CHAR(15) NOT NULL,
> CONTAINER_ID CHAR(15) NOT NULL,
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
> ENTITY_ID DESC);
> UPSERT INTO test.test VALUES ('org1','container1','entityId6',1.1);
> UPSERT INTO test.test VALUES ('org1','container1','entityId5',1.2);
> UPSERT INTO test.test VALUES ('org1','container1','entityId4',1.3);
> UPSERT INTO test.test VALUES ('org1','container1','entityId3',1.4);
> UPSERT INTO test.test VALUES ('org1','container1','entityId2',1.5);
> UPSERT INTO test.test VALUES ('org1','container1','entityId1',1.6);
> SELECT DISTINCT entity_id, score
> FROM test.test
> WHERE organization_id = 'org1'
> AND container_id = 'container1'
> ORDER BY score DESC
> Notice that the returned results are not returned in descending score order.
> Instead they are returned in descending entity_id order. If I remove the
> DISTINCT or remove the secondary index the result is correct.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3451) Secondary index and query using distinct: LIMIT doesn't return the first rows
[
https://issues.apache.org/jira/browse/PHOENIX-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636031#comment-15636031
]
Joel Palmert commented on PHOENIX-3451:
---
[~giacomotaylor]
> Secondary index and query using distinct: LIMIT doesn't return the first rows
> -
>
> Key: PHOENIX-3451
> URL: https://issues.apache.org/jira/browse/PHOENIX-3451
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Joel Palmert
>
> This may be related to PHOENIX-3452 but the behavior is different so filing
> it separately.
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ORGANIZATION_ID CHAR(15) NOT NULL,
> CONTAINER_ID CHAR(15) NOT NULL,
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
> ENTITY_ID DESC);
> UPSERT INTO test.test VALUES ('org2','container2','entityId6',1.1);
> UPSERT INTO test.test VALUES ('org2','container1','entityId5',1.2);
> UPSERT INTO test.test VALUES ('org2','container2','entityId4',1.3);
> UPSERT INTO test.test VALUES ('org2','container1','entityId3',1.4);
> UPSERT INTO test.test VALUES ('org2','container3','entityId7',1.35);
> UPSERT INTO test.test VALUES ('org2','container3','entityId8',1.45);
> EXPLAIN
> SELECT DISTINCT entity_id, score
> FROM test.test
> WHERE organization_id = 'org2'
> AND container_id IN ( 'container1','container2','container3' )
> ORDER BY score DESC
> LIMIT 2
> OUTPUT
> entityId51.2
> entityId31.4
> The expected out out would be
> entityId81.45
> entityId31.4
> You will get the expected output if you remove the secondary index from the
> table or remove distinct from the query.
> As described in PHOENIX-3452 if you run the query without the LIMIT the
> ordering is not correct. However, the 2first results in that ordering is
> still not the onces returned by the limit clause, which makes me think there
> are multiple issues here and why I filed both separately. The rows being
> returned are the ones assigned to container1. It looks like Phoenix is first
> getting the rows from the first container and when it finds that to be enough
> it stops the scan. What it should be doing is getting 2 results for each
> container and then merge then and then limit again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3452) Secondary index and query using distinct: ORDE BY doesn't work correctly
[
https://issues.apache.org/jira/browse/PHOENIX-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636029#comment-15636029
]
Joel Palmert commented on PHOENIX-3452:
---
[~giacomotaylor]
> Secondary index and query using distinct: ORDE BY doesn't work correctly
>
>
> Key: PHOENIX-3452
> URL: https://issues.apache.org/jira/browse/PHOENIX-3452
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Joel Palmert
>
> This may be related to PHOENIX-3451 but the behavior is different so filing
> it separately.
> Steps to repro:
> DROP TABLE TEST.TEST;
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ORGANIZATION_ID CHAR(15) NOT NULL,
> CONTAINER_ID CHAR(15) NOT NULL,
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
> ENTITY_ID DESC);
> UPSERT INTO test.test VALUES ('org1','container1','entityId6',1.1);
> UPSERT INTO test.test VALUES ('org1','container1','entityId5',1.2);
> UPSERT INTO test.test VALUES ('org1','container1','entityId4',1.3);
> UPSERT INTO test.test VALUES ('org1','container1','entityId3',1.4);
> UPSERT INTO test.test VALUES ('org1','container1','entityId2',1.5);
> UPSERT INTO test.test VALUES ('org1','container1','entityId1',1.6);
> SELECT DISTINCT entity_id, score
> FROM test.test
> WHERE organization_id = 'org1'
> AND container_id = 'container1'
> ORDER BY score DESC
> Notice that the returned results are not returned in descending score order.
> Instead they are returned in descending entity_id order. If I remove the
> DISTINCT or remove the secondary index the result is correct.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3451) Secondary index and query using distinct: LIMIT doesn't return the first rows
[
https://issues.apache.org/jira/browse/PHOENIX-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Palmert updated PHOENIX-3451:
--
Description:
This may be related to PHOENIX-3452 but the behavior is different so filing it
separately.
Steps to repro:
CREATE TABLE IF NOT EXISTS TEST.TEST (
ORGANIZATION_ID CHAR(15) NOT NULL,
CONTAINER_ID CHAR(15) NOT NULL,
ENTITY_ID CHAR(15) NOT NULL,
SCORE DOUBLE,
CONSTRAINT TEST_PK PRIMARY KEY (
ORGANIZATION_ID,
CONTAINER_ID,
ENTITY_ID
)
) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
ENTITY_ID DESC);
UPSERT INTO test.test VALUES ('org2','container2','entityId6',1.1);
UPSERT INTO test.test VALUES ('org2','container1','entityId5',1.2);
UPSERT INTO test.test VALUES ('org2','container2','entityId4',1.3);
UPSERT INTO test.test VALUES ('org2','container1','entityId3',1.4);
UPSERT INTO test.test VALUES ('org2','container3','entityId7',1.35);
UPSERT INTO test.test VALUES ('org2','container3','entityId8',1.45);
EXPLAIN
SELECT DISTINCT entity_id, score
FROM test.test
WHERE organization_id = 'org2'
AND container_id IN ( 'container1','container2','container3' )
ORDER BY score DESC
LIMIT 2
OUTPUT
entityId51.2
entityId31.4
The expected out out would be
entityId81.45
entityId31.4
You will get the expected output if you remove the secondary index from the
table or remove distinct from the query.
As described in PHOENIX-3452 if you run the query without the LIMIT the
ordering is not correct. However, the 2first results in that ordering is still
not the onces returned by the limit clause, which makes me think there are
multiple issues here and why I filed both separately. The rows being returned
are the ones assigned to container1. It looks like Phoenix is first getting the
rows from the first container and when it finds that to be enough it stops the
scan. What it should be doing is getting 2 results for each container and then
merge then and then limit again.
was:
Steps to repro:
CREATE TABLE IF NOT EXISTS TEST.TEST (
ORGANIZATION_ID CHAR(15) NOT NULL,
CONTAINER_ID CHAR(15) NOT NULL,
ENTITY_ID CHAR(15) NOT NULL,
SCORE DOUBLE,
CONSTRAINT TEST_PK PRIMARY KEY (
ORGANIZATION_ID,
CONTAINER_ID,
ENTITY_ID
)
) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
ENTITY_ID DESC);
UPSERT INTO test.test VALUES ('org2','container2','entityId6',1.1);
UPSERT INTO test.test VALUES ('org2','container1','entityId5',1.2);
UPSERT INTO test.test VALUES ('org2','container2','entityId4',1.3);
UPSERT INTO test.test VALUES ('org2','container1','entityId3',1.4);
UPSERT INTO test.test VALUES ('org2','container3','entityId7',1.35);
UPSERT INTO test.test VALUES ('org2','container3','entityId8',1.45);
EXPLAIN
SELECT DISTINCT entity_id, score
FROM test.test
WHERE organization_id = 'org2'
AND container_id IN ( 'container1','container2','container3' )
ORDER BY score DESC
LIMIT 2
OUTPUT
entityId51.2
entityId31.4
The expected out out would be
entityId81.45
entityId31.4
You will get the expected output if you remove the secondary index from the
table or remove distinct from the query.
Further notice that the outputed 2 rows is not the first two rows if you run
the query without the limit.
The rows being returned are the ones assigned to container1. It looks like
Phoenix is first getting the rows from the first container and when it finds
that to be enough it stops the scan. What it should be doing is getting 2
results for each container and then merge then and then limit again.
> Secondary index and query using distinct: LIMIT doesn't return the first rows
> -
>
> Key: PHOENIX-3451
> URL: https://issues.apache.org/jira/browse/PHOENIX-3451
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Joel Palmert
>
> This may be related to PHOENIX-3452 but the behavior is different so filing
> it separately.
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ORGANIZATION_ID CHAR(15) NOT NULL,
> CONTAINER_ID CHAR(15) NOT NULL,
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
> ENTITY_ID DESC);
> UPSERT INTO test.test VALUES ('org2','container2','entityId6',1.1);
> UPSERT INTO test.test VALUES ('org2','container1','entityId5',1.2);
> UPSERT INTO test.test VALUES ('
[jira] [Created] (PHOENIX-3452) Secondary index and query using distinct: ORDE BY doesn't work correctly
Joel Palmert created PHOENIX-3452:
-
Summary: Secondary index and query using distinct: ORDE BY doesn't
work correctly
Key: PHOENIX-3452
URL: https://issues.apache.org/jira/browse/PHOENIX-3452
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.8.0
Reporter: Joel Palmert
This may be related to PHOENIX-3451 but the behavior is different so filing it
separately.
Steps to repro:
DROP TABLE TEST.TEST;
CREATE TABLE IF NOT EXISTS TEST.TEST (
ORGANIZATION_ID CHAR(15) NOT NULL,
CONTAINER_ID CHAR(15) NOT NULL,
ENTITY_ID CHAR(15) NOT NULL,
SCORE DOUBLE,
CONSTRAINT TEST_PK PRIMARY KEY (
ORGANIZATION_ID,
CONTAINER_ID,
ENTITY_ID
)
) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
ENTITY_ID DESC);
UPSERT INTO test.test VALUES ('org1','container1','entityId6',1.1);
UPSERT INTO test.test VALUES ('org1','container1','entityId5',1.2);
UPSERT INTO test.test VALUES ('org1','container1','entityId4',1.3);
UPSERT INTO test.test VALUES ('org1','container1','entityId3',1.4);
UPSERT INTO test.test VALUES ('org1','container1','entityId2',1.5);
UPSERT INTO test.test VALUES ('org1','container1','entityId1',1.6);
SELECT DISTINCT entity_id, score
FROM test.test
WHERE organization_id = 'org1'
AND container_id = 'container1'
ORDER BY score DESC
Notice that the returned results are not returned in descending score order.
Instead they are returned in descending entity_id order. If I remove the
DISTINCT or remove the secondary index the result is correct.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3451) Secondary index and query using distinct: LIMIT doesn't return the first rows
[
https://issues.apache.org/jira/browse/PHOENIX-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Palmert updated PHOENIX-3451:
--
Summary: Secondary index and query using distinct: LIMIT doesn't return the
first rows (was: Secondary index brakes ordering when using distinct)
> Secondary index and query using distinct: LIMIT doesn't return the first rows
> -
>
> Key: PHOENIX-3451
> URL: https://issues.apache.org/jira/browse/PHOENIX-3451
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Joel Palmert
>
> Steps to repro:
> CREATE TABLE IF NOT EXISTS TEST.TEST (
> ORGANIZATION_ID CHAR(15) NOT NULL,
> CONTAINER_ID CHAR(15) NOT NULL,
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE,
> CONSTRAINT TEST_PK PRIMARY KEY (
> ORGANIZATION_ID,
> CONTAINER_ID,
> ENTITY_ID
> )
> ) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
> CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
> ENTITY_ID DESC);
> UPSERT INTO test.test VALUES ('org2','container2','entityId6',1.1);
> UPSERT INTO test.test VALUES ('org2','container1','entityId5',1.2);
> UPSERT INTO test.test VALUES ('org2','container2','entityId4',1.3);
> UPSERT INTO test.test VALUES ('org2','container1','entityId3',1.4);
> UPSERT INTO test.test VALUES ('org2','container3','entityId7',1.35);
> UPSERT INTO test.test VALUES ('org2','container3','entityId8',1.45);
> EXPLAIN
> SELECT DISTINCT entity_id, score
> FROM test.test
> WHERE organization_id = 'org2'
> AND container_id IN ( 'container1','container2','container3' )
> ORDER BY score DESC
> LIMIT 2
> OUTPUT
> entityId51.2
> entityId31.4
> The expected out out would be
> entityId81.45
> entityId31.4
> You will get the expected output if you remove the secondary index from the
> table or remove distinct from the query.
> Further notice that the outputed 2 rows is not the first two rows if you run
> the query without the limit.
> The rows being returned are the ones assigned to container1. It looks like
> Phoenix is first getting the rows from the first container and when it finds
> that to be enough it stops the scan. What it should be doing is getting 2
> results for each container and then merge then and then limit again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (PHOENIX-3451) Secondary index brakes ordering when using distinct
Joel Palmert created PHOENIX-3451:
-
Summary: Secondary index brakes ordering when using distinct
Key: PHOENIX-3451
URL: https://issues.apache.org/jira/browse/PHOENIX-3451
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.8.0
Reporter: Joel Palmert
Steps to repro:
CREATE TABLE IF NOT EXISTS TEST.TEST (
ORGANIZATION_ID CHAR(15) NOT NULL,
CONTAINER_ID CHAR(15) NOT NULL,
ENTITY_ID CHAR(15) NOT NULL,
SCORE DOUBLE,
CONSTRAINT TEST_PK PRIMARY KEY (
ORGANIZATION_ID,
CONTAINER_ID,
ENTITY_ID
)
) VERSIONS=1, MULTI_TENANT=TRUE, REPLICATION_SCOPE=1, TTL=31536000;
CREATE INDEX IF NOT EXISTS TEST_SCORE ON TEST.TEST (CONTAINER_ID, SCORE DESC,
ENTITY_ID DESC);
UPSERT INTO test.test VALUES ('org2','container2','entityId6',1.1);
UPSERT INTO test.test VALUES ('org2','container1','entityId5',1.2);
UPSERT INTO test.test VALUES ('org2','container2','entityId4',1.3);
UPSERT INTO test.test VALUES ('org2','container1','entityId3',1.4);
UPSERT INTO test.test VALUES ('org2','container3','entityId7',1.35);
UPSERT INTO test.test VALUES ('org2','container3','entityId8',1.45);
EXPLAIN
SELECT DISTINCT entity_id, score
FROM test.test
WHERE organization_id = 'org2'
AND container_id IN ( 'container1','container2','container3' )
ORDER BY score DESC
LIMIT 2
OUTPUT
entityId51.2
entityId31.4
The expected out out would be
entityId81.45
entityId31.4
You will get the expected output if you remove the secondary index from the
table or remove distinct from the query.
Further notice that the outputed 2 rows is not the first two rows if you run
the query without the limit.
The rows being returned are the ones assigned to container1. It looks like
Phoenix is first getting the rows from the first container and when it finds
that to be enough it stops the scan. What it should be doing is getting 2
results for each container and then merge then and then limit again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
