[GitHub] [phoenix-thirdparty] stoty commented on a change in pull request #1: PHOENIX-6152 Add phoenix-shaded-protobuf to phoenix-thirdpart.
stoty commented on a change in pull request #1: URL: https://github.com/apache/phoenix-thirdparty/pull/1#discussion_r494061406 ## File path: phoenix-shaded-protobuf/pom.xml ## @@ -0,0 +1,104 @@ + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd";> + + 4.0.0 + +org.apache.phoenix.thirdparty +phoenix-thirdparty +1.0.0-SNAPSHOT +.. + + phoenix-shaded-protobuf + Apache Phoenix Relocated (Shaded) Protobuf + +Pulls down protobuf, compiles, and then relocates/shades. Review comment: This comment is still not true ## File path: phoenix-shaded-protobuf/pom.xml ## @@ -0,0 +1,104 @@ + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd";> + + 4.0.0 + +org.apache.phoenix.thirdparty +phoenix-thirdparty +1.0.0-SNAPSHOT +.. + + phoenix-shaded-protobuf + Apache Phoenix Relocated (Shaded) Protobuf + +Pulls down protobuf, compiles, and then relocates/shades. + + + + + +maven-clean-plugin Review comment: The clean plugin settings can also be take from guava. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (PHOENIX-6157) Fix dependecy:analyze error introduced by PHOENIX-5909
Istvan Toth created PHOENIX-6157: Summary: Fix dependecy:analyze error introduced by PHOENIX-5909 Key: PHOENIX-6157 URL: https://issues.apache.org/jira/browse/PHOENIX-6157 Project: Phoenix Issue Type: Bug Components: core Affects Versions: 5.1.0 Reporter: Istvan Toth Assignee: Istvan Toth PHOENIX-5909 adds an unshaded com.google.common import, which breaks mvn verify. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Please don't use unshaded Guava classes in Phoenix master
Hi! We have recently switched to shaded Guava from phoenix-thirdparty in master. Using unshaded Guava will break maven verify (and cause dependency problems down the line), so please take care not to commit patches that use unshaded Guava classes. Yetus does flag these patches with -1. I know that the tests are in bad shape, and the IT results are kind of useless in the PreCommit check right now, but Yetus flags this problem as an *mvninstall* error, which should be taken seriously: Patch Compile Tests -1 mvninstall 1m 14s root in the patch failed. you'll see the following line in the linked mvn install log: [WARNING] Used undeclared dependencies found: [WARNING]com.google.guava:guava:jar:11.0.2:compile regards Istvan
[jira] [Updated] (PHOENIX-6156) IndexOutOfBoundsException, Expression 'AND' with 'RVC'
[
https://issues.apache.org/jira/browse/PHOENIX-6156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hahafam updated PHOENIX-6156:
-
Description:
Bug in WhereOptimizer$KeyExpressionVisitor.intersectTrailing included in
PHOENIX-3383(https://issues.apache.org/jira/browse/PHOENIX-3383) patch
Using AndExpression with RVCExpression in a query results in an exception.
- Test Query
{code:java}
CREATE TABLE TEST.TABLE1 (CREATE TABLE TEST.TABLE1 ( ID1 BIGINT NOT NULL,
MMDD INTEGER NOT NULL, ID2 VARCHAR NOT NULL, ID3 VARCHAR NOT NULL,
CONSTRAINT PK PRIMARY KEY (ID1,MMDD,ID2,ID3));
SELECT ID1, MMDD, ID2, ID3FROM TEST.TABLE1WHERE (ID1 = 100200300 AND
MMDD = 20200101 AND ID2 = 'LENGTH_8_BYTE_OVER' AND ID3 = 'ABCDEFGHIJK')
AND (ID1, MMDD, ID2, ID3) IN ((100200300, 20200101,'LENGTH_8_BYTE_OVER',
'ABCDEFGHIJK'),(200200300, 20200201,'ABC', 'ABCDEFGHIJK'));
{code}
- raise IndexOutOfExceptin
{code:java}
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.intersectTrailing(WhereOptimizer.java:1362)
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.andKeySlots(WhereOptimizer.java:974)
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.visitLeave(WhereOptimizer.java:1501)
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.visitLeave(WhereOptimizer.java:512)
at
org.apache.phoenix.expression.AndExpression.accept(AndExpression.java:100)
at
org.apache.phoenix.compile.WhereOptimizer.pushKeyExpressionsToScan(WhereOptimizer.java:142)
at org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:157)
at org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:232)
at
org.apache.phoenix.execute.DelegateQueryPlan.iterator(DelegateQueryPlan.java:144)
at
org.apache.phoenix.execute.DelegateQueryPlan.iterator(DelegateQueryPlan.java:139)
at
org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:312)
at
org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:291)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at
org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
at
org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:283)
at
org.apache.phoenix.jdbc.PhoenixPreparedStatement.executeQuery(PhoenixPreparedStatement.java:194)
{code}
Exception occurs when the value of the ID2 column is 8 bytes or more.
This is because of the way bytes are read in the function below.
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.intersectTrailing
- My log
{code:java}
andExpression=((ID1 = 100200300 AND MMDD = 20200101 AND ID2 =
'LENGTH_8_BYTE_OVER' AND ID3 = 'ABCDEFGHIJK') AND (ID1, MMDD, ID2, ID3) IN
([128,0,0,0,5,248,239,108,129,52,58,165,76,69,78,71,84,72,95,56,95,66,89,84,69,95,79,86,69,82,0,65,66,67,68,69,70,71,72,73,74,75],[128,0,0,0,11,238,208,108,129,52,59,9,65,66,67,0,65,66,67,68,69,70,71,72,73,74,75],...)){code}
- bytes, 8byte(bigint) + 4byte(int) + LENGTH_8_BYTE_OVER + delimeter(\0) +
ABCDEFGHIJK
{code:java}
128,0,0,0,5,248,239,108,129,52,58,165,76,69,78,71,84,72,95,56,95,66,89,84,69,95,79,86,69,82,0,65,66,67,68,69,70,71,72,73,74,75{code}
intersectTrailing function reads byte : 8byte > 8byte > 4byte > remain
If the value of the ID2 column is more than 8 bytes, the delimeter is
encountered when reading the 'remain' byte, and the loop is executed once more.
{code:java}
2020-09-23 17:27:36,907 compile.WhereOptimizer$KeyExpressionVisitor(1358) -
MYLOG intersectTrailing :
result=\x80\x00\x00\x00\x05\xF8\xEFl\x814:\xA5LENGTH_8_BYTE_OVER\x00ABCDEFGHIJK,
pkPos=0, otherRange=\x814:\xA5, otherPKPos=1
2020-09-23 17:27:36,907 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=1, getOffset=8, getLength=4, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56, 95, 66, 89, 84, 69, 95, 79, 86, 69, 82, 0,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75],
str=�4:�LENGTH_8_BYTE_OVERABCDEFGHIJK
2020-09-23 17:27:36,908 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=2, getOffset=12, getLength=10, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56, 95, 66, 89, 84, 69, 95, 79, 86, 69, 82, 0,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75],
str=�4:�LENGTH_8_BYTE_OVERABCDEFGHIJK
2020-09-23 17:27:36,908 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=3, getOffset=23, getLength=11, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56, 95, 66, 89, 84, 69, 95, 79, 86, 69, 82, 0,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75],
str=�4:�LENGTH_8_BYTE_OVERABCDEFGHIJK
2020-09-23 17:27:36,908 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=4, getOffset=34, getLength=0, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56,
[jira] [Updated] (PHOENIX-6055) Improve error reporting for index validation when there are "Not matching index rows"
[ https://issues.apache.org/jira/browse/PHOENIX-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gokcen Iskender updated PHOENIX-6055: - Attachment: PHOENIX-6055.master.003.patch > Improve error reporting for index validation when there are "Not matching > index rows" > - > > Key: PHOENIX-6055 > URL: https://issues.apache.org/jira/browse/PHOENIX-6055 > Project: Phoenix > Issue Type: Improvement >Reporter: Gokcen Iskender >Priority: Major > Attachments: PHOENIX-6055.master.001.patch, > PHOENIX-6055.master.002.patch, PHOENIX-6055.master.003.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > In the IndexRebuildRegionScanner, in the following line we return the > timestamp of the latest expected mutation rather than the expected mutation > itself. > > byte[] dataKey = indexMaintainer.buildDataRowKey(new > ImmutableBytesWritable(indexRow.getRow()), viewConstants); > String errorMsg = "Not matching index row"; > logToIndexToolOutputTable(dataKey, indexRow.getRow(), > getTimestamp(expectedMutationList.get(0)), 0L, errorMsg, isBeforeRebuild, > INVALID_ROW); > > Instead of expectedMutationList.get(0), expected should be used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6124) Block adding/dropping a column on a parent view for clients <4.15 and for clients that have phoenix.allow.system.catalog.rollback=true
[ https://issues.apache.org/jira/browse/PHOENIX-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated PHOENIX-6124: -- Priority: Blocker (was: Major) > Block adding/dropping a column on a parent view for clients <4.15 and for > clients that have phoenix.allow.system.catalog.rollback=true > -- > > Key: PHOENIX-6124 > URL: https://issues.apache.org/jira/browse/PHOENIX-6124 > Project: Phoenix > Issue Type: Bug >Affects Versions: 5.0.0, 4.15.0 >Reporter: Chinmay Kulkarni >Priority: Blocker > Fix For: 5.1.0, 4.16.0 > > > For pre-4.15 clients, we have to block adding/dropping a column to a table if > it has child views since we can’t do any checkAndPut distributed locking (see > [this|https://github.com/apache/phoenix/blob/6ecc66738e576a5349605c2f5b20003df03f95de/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2595-L2616]). > > However, this is only prevented if the parent is a table and not if the > parent is a view. We should extend [the > condition|https://github.com/apache/phoenix/blob/6ecc66738e576a5349605c2f5b20003df03f95de/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2591] > to also cover views since conflicting mutations on its children can also > lead to inconsistencies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6055) Improve error reporting for index validation when there are "Not matching index rows"
[ https://issues.apache.org/jira/browse/PHOENIX-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gokcen Iskender updated PHOENIX-6055: - Attachment: PHOENIX-6055.master.002.patch > Improve error reporting for index validation when there are "Not matching > index rows" > - > > Key: PHOENIX-6055 > URL: https://issues.apache.org/jira/browse/PHOENIX-6055 > Project: Phoenix > Issue Type: Improvement >Reporter: Gokcen Iskender >Priority: Major > Attachments: PHOENIX-6055.master.001.patch, > PHOENIX-6055.master.002.patch > > Time Spent: 1h > Remaining Estimate: 0h > > In the IndexRebuildRegionScanner, in the following line we return the > timestamp of the latest expected mutation rather than the expected mutation > itself. > > byte[] dataKey = indexMaintainer.buildDataRowKey(new > ImmutableBytesWritable(indexRow.getRow()), viewConstants); > String errorMsg = "Not matching index row"; > logToIndexToolOutputTable(dataKey, indexRow.getRow(), > getTimestamp(expectedMutationList.get(0)), 0L, errorMsg, isBeforeRebuild, > INVALID_ROW); > > Instead of expectedMutationList.get(0), expected should be used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6142) Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
[ https://issues.apache.org/jira/browse/PHOENIX-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated PHOENIX-6142: -- Description: We are targeting PHOENIX-6141 for 4.17. Until we have it, we should aim at making DDL operations resilient to orphan parent->child linking rows. DDL operations identified which can fail due to orphan rows are: # Any ALTER TABLE ADD/DROP/SET calls on the base table T will fail if there are orphan links from T to some already dropped view. This happens because the call to [MetaDataEndpointImpl.findAllChildViews()|https://github.com/apache/phoenix/blob/fece8e69b9c03c80db7a0801d99e5de31fe15ffa/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2142] from [MetaDataEndpointImpl.mutateColumn()|https://github.com/apache/phoenix/blob/fece8e69b9c03c80db7a0801d99e5de31fe15ffa/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L259] fails with a TableNotFoundException. # Any DROP TABLE/VIEW call without CASCADE will fail even though there are actually no child views since the orphan rows wrongly indicate that there are child views. # During the upgrade path for UpgradeUtil.syncUpdateCacheFreqAllIndexes(), we will just ignore any orphan views (for ex, see [this|https://github.com/apache/phoenix/blob/fece8e69b9c03c80db7a0801d99e5de31fe15ffa/phoenix-core/src/main/java/org/apache/phoenix/util/UpgradeUtil.java#L1368-L1374]), but the call to UpgradeUtil.upgradeTable() will fail with a TableNotFoundException for each orphan view. # During a CREATE TABLE/VIEW, we try to drop any views from the previous life of that table/view, however we might end up dropping a legitimate view (with the same name) which is on another table/view because of this. Before dropping any views that we see from a parent->child link, we need to ensure that the view is in fact a child view of the same table/view we think it is an orphan of. was: We are targeting PHOENIX-6141 for 4.17. Until we have it, we should aim at making DDL operations resilient to orphan parent->child linking rows. DDL operations identified which can fail due to orphan rows are: # Any ALTER TABLE ADD/DROP/SET calls on the base table T will fail if there are orphan links from T to some already dropped view. This happens because the call to [MetaDataEndpointImpl.findAllChildViews()|https://github.com/apache/phoenix/blob/fece8e69b9c03c80db7a0801d99e5de31fe15ffa/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2142] from [MetaDataEndpointImpl.mutateColumn()|https://github.com/apache/phoenix/blob/fece8e69b9c03c80db7a0801d99e5de31fe15ffa/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L259] fails with a TableNotFoundException. # Any DROP TABLE/VIEW call without CASCADE will fail even though there are actually no child views since the orphan rows wrongly indicate that there are child views. # During the upgrade path for UpgradeUtil.syncUpdateCacheFreqAllIndexes(), we will just ignore any orphan views (for ex, see [this|https://github.com/apache/phoenix/blob/fece8e69b9c03c80db7a0801d99e5de31fe15ffa/phoenix-core/src/main/java/org/apache/phoenix/util/UpgradeUtil.java#L1368-L1374]), but the call to UpgradeUtil.upgradeTable() will fail with a TableNotFoundException for each orphan view. > Make DDL operations resilient to orphan parent->child linking rows in > SYSTEM.CHILD_LINK > --- > > Key: PHOENIX-6142 > URL: https://issues.apache.org/jira/browse/PHOENIX-6142 > Project: Phoenix > Issue Type: Bug >Affects Versions: 5.0.0, 4.15.0 >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni >Priority: Blocker > Fix For: 5.1.0, 4.16.0 > > > We are targeting PHOENIX-6141 for 4.17. Until we have it, we should aim at > making DDL operations resilient to orphan parent->child linking rows. DDL > operations identified which can fail due to orphan rows are: > # Any ALTER TABLE ADD/DROP/SET calls on the base table T will fail if there > are orphan links from T to some already dropped view. This happens because > the call to > [MetaDataEndpointImpl.findAllChildViews()|https://github.com/apache/phoenix/blob/fece8e69b9c03c80db7a0801d99e5de31fe15ffa/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2142] > from > [MetaDataEndpointImpl.mutateColumn()|https://github.com/apache/phoenix/blob/fece8e69b9c03c80db7a0801d99e5de31fe15ffa/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L259] > fails with a TableNotFoundException. > # Any DROP TABLE/VIEW call without CASCADE will fail even though there are > actually no child views since the orphan rows
[jira] [Resolved] (PHOENIX-6014) Add documentation for PHOENIX-2715
[ https://issues.apache.org/jira/browse/PHOENIX-6014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved PHOENIX-6014. -- Resolution: Fixed Added properties to [tuning.md.|https://phoenix.apache.org/tuning.html] [~ankit] please check if the description is correct. > Add documentation for PHOENIX-2715 > -- > > Key: PHOENIX-6014 > URL: https://issues.apache.org/jira/browse/PHOENIX-6014 > Project: Phoenix > Issue Type: Improvement >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > PHOENIX-2715 added a major and useful feature, but there is no reference to > it. > Add the config properties to [https://phoenix.apache.org/tuning.html] , and > preferably add a section to Features on using it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (PHOENIX-6014) Add documentation for PHOENIX-2715
[ https://issues.apache.org/jira/browse/PHOENIX-6014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth reassigned PHOENIX-6014: Assignee: Istvan Toth > Add documentation for PHOENIX-2715 > -- > > Key: PHOENIX-6014 > URL: https://issues.apache.org/jira/browse/PHOENIX-6014 > Project: Phoenix > Issue Type: Improvement >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > PHOENIX-2715 added a major and useful feature, but there is no reference to > it. > Add the config properties to [https://phoenix.apache.org/tuning.html] , and > preferably add a section to Features on using it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [phoenix-thirdparty] richardantal opened a new pull request #1: PHOENIX-6152 Add phoenix-shaded-protobuf to phoenix-thirdpart.
richardantal opened a new pull request #1: URL: https://github.com/apache/phoenix-thirdparty/pull/1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [DISCUSS] Changing/unifying Github project integration settings
Right after sent this out, I have found the announcement that this can be set right from the repo now: https://cwiki.apache.org/confluence/display/INFRA/git+-+.asf.yaml+features So I'm going to go ahead and experiment with this, as it's as easy as changing a file in a repo to turn the features on or off. On Wed, Sep 23, 2020 at 11:19 AM Istvan Toth wrote: > Hi! > > ASF Infra provides GitHub integration features, among them the ability to > copy comments on PRs to the JIRA tickets, and sending notifications about > PR activity to the development list (here). > > > https://blogs.apache.org/infra/entry/improved_integration_between_apache_and > > The latter is enabled for some repos (Omid), but the former is not. > > I propose asking infra to enable all JIRA related GutHUB PR integrations > for all six of our repositories. > > I am not sure about the mailing list notifications, I plan not to ask to > enable them now, unless some of you think that we should. > > I have opened https://issues.apache.org/jira/browse/PHOENIX-6147 for > this. > > If you have any input on this, or any objection,please let me know here, > or in PHOENIX-6147. > > If there are no objections, I will file an INFRA ticket for this early > next week. > > regards > Istvan >
[jira] [Updated] (PHOENIX-6156) IndexOutOfBoundsException, Expression 'AND' with 'RVC'
[
https://issues.apache.org/jira/browse/PHOENIX-6156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JUIL CHO updated PHOENIX-6156:
--
Description:
Bug in WhereOptimizer$KeyExpressionVisitor.intersectTrailing included in
PHOENIX-3383(https://issues.apache.org/jira/browse/PHOENIX-3383) patch
Using AndExpression with RVCExpression in a query results in an exception.
- Test Query
{code:java}
CREATE TABLE TEST.TABLE1 (CREATE TABLE TEST.TABLE1 ( ID1 BIGINT NOT NULL,
MMDD INTEGER NOT NULL, ID2 VARCHAR NOT NULL, ID3 VARCHAR NOT NULL,
CONSTRAINT PK PRIMARY KEY (ID1,MMDD,ID2,ID3));
SELECT ID1, MMDD, ID2, ID3FROM TEST.TABLE1WHERE (ID1 = 100200300 AND
MMDD = 20200101 AND ID2 = 'LENGTH_8_BYTE_OVER' AND ID3 = 'ABCDEFGHIJK')
AND (ID1, MMDD, ID2, ID3) IN ((100200300, 20200101,'LENGTH_8_BYTE_OVER',
'ABCDEFGHIJK'),(200200300, 20200201,'ABC', 'ABCDEFGHIJK'));
{code}
- raise IndexOutOfExceptin
{code:java}
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.intersectTrailing(WhereOptimizer.java:1362)
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.andKeySlots(WhereOptimizer.java:974)
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.visitLeave(WhereOptimizer.java:1501)
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.visitLeave(WhereOptimizer.java:512)
at
org.apache.phoenix.expression.AndExpression.accept(AndExpression.java:100)
at
org.apache.phoenix.compile.WhereOptimizer.pushKeyExpressionsToScan(WhereOptimizer.java:142)
at org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:157)
at org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:232)
at
org.apache.phoenix.execute.DelegateQueryPlan.iterator(DelegateQueryPlan.java:144)
at
org.apache.phoenix.execute.DelegateQueryPlan.iterator(DelegateQueryPlan.java:139)
at
org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:312)
at
org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:291)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at
org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:290)
at
org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:283)
at
org.apache.phoenix.jdbc.PhoenixPreparedStatement.executeQuery(PhoenixPreparedStatement.java:194)
{code}
Exception occurs when the value of the ID2 column is 8 bytes or more.
This is because of the way bytes are read in the function below.
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.intersectTrailing
- My log
{code:java}
andExpression=((ID1 = 100200300 AND MMDD = 20200101 AND ID2 =
'LENGTH_8_BYTE_OVER' AND ID3 = 'ABCDEFGHIJK') AND (ID1, MMDD, ID2, ID3) IN
([128,0,0,0,5,248,239,108,129,52,58,165,76,69,78,71,84,72,95,56,95,66,89,84,69,95,79,86,69,82,0,65,66,67,68,69,70,71,72,73,74,75],[128,0,0,0,11,238,208,108,129,52,59,9,65,66,67,0,65,66,67,68,69,70,71,72,73,74,75],...)){code}
- bytes, 8byte(bigint) + 4byte(int) + LENGTH_8_BYTE_OVER + delimeter(\0) +
ABCDEFGHIJK
{code:java}
128,0,0,0,5,248,239,108,129,52,58,165,76,69,78,71,84,72,95,56,95,66,89,84,69,95,79,86,69,82,0,65,66,67,68,69,70,71,72,73,74,75{code}
intersectTrailing function reads byte : 8byte > 8byte > 4byte > amount
If the value of the ID2 column is more than 8 bytes, the delimeter is
encountered when reading the'amount' byte, and the loop is executed once more.
{code:java}
2020-09-23 17:27:36,907 compile.WhereOptimizer$KeyExpressionVisitor(1358) -
MYLOG intersectTrailing :
result=\x80\x00\x00\x00\x05\xF8\xEFl\x814:\xA5LENGTH_8_BYTE_OVER\x00ABCDEFGHIJK,
pkPos=0, otherRange=\x814:\xA5, otherPKPos=1
2020-09-23 17:27:36,907 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=1, getOffset=8, getLength=4, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56, 95, 66, 89, 84, 69, 95, 79, 86, 69, 82, 0,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75],
str=�4:�LENGTH_8_BYTE_OVERABCDEFGHIJK
2020-09-23 17:27:36,908 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=2, getOffset=12, getLength=10, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56, 95, 66, 89, 84, 69, 95, 79, 86, 69, 82, 0,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75],
str=�4:�LENGTH_8_BYTE_OVERABCDEFGHIJK
2020-09-23 17:27:36,908 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=3, getOffset=23, getLength=11, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56, 95, 66, 89, 84, 69, 95, 79, 86, 69, 82, 0,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75],
str=�4:�LENGTH_8_BYTE_OVERABCDEFGHIJK
2020-09-23 17:27:36,908 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=4, getOffset=34, getLength=0, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56
[jira] [Created] (PHOENIX-6156) IndexOutOfBoundsException, Expression 'AND' with 'RVC'
JUIL CHO created PHOENIX-6156:
-
Summary: IndexOutOfBoundsException, Expression 'AND' with 'RVC'
Key: PHOENIX-6156
URL: https://issues.apache.org/jira/browse/PHOENIX-6156
Project: Phoenix
Issue Type: Bug
Components: core
Affects Versions: 4.14.1
Reporter: JUIL CHO
Bug in WhereOptimizer$KeyExpressionVisitor.intersectTrailing included in
[PHOENIX-3383](https://issues.apache.org/jira/browse/PHOENIX-3383) patch
Using AndExpression with RVCExpression in a query results in an exception.
Exception occurs when the value of the ID2 column is 8 bytes or more.
This is because of the way bytes are read in the function below.
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.intersectTrailing
- My log
{code:java}
andExpression=((ID1 = 100200300 AND MMDD = 20200101 AND ID2 =
'LENGTH_8_BYTE_OVER' AND ID3 = 'ABCDEFGHIJK') AND (ID1, MMDD, ID2, ID3) IN
([128,0,0,0,5,248,239,108,129,52,58,165,76,69,78,71,84,72,95,56,95,66,89,84,69,95,79,86,69,82,0,65,66,67,68,69,70,71,72,73,74,75],[128,0,0,0,11,238,208,108,129,52,59,9,65,66,67,0,65,66,67,68,69,70,71,72,73,74,75],...)){code}
- bytes, 8byte(bigint) + 4byte(int) + LENGTH_8_BYTE_OVER + delimeter(\0) +
ABCDEFGHIJK
{code:java}
128,0,0,0,5,248,239,108,129,52,58,165,76,69,78,71,84,72,95,56,95,66,89,84,69,95,79,86,69,82,0,65,66,67,68,69,70,71,72,73,74,75{code}
intersectTrailing function reads byte : 8byte > 8byte > 4byte > amount
If the value of the ID2 column is more than 8 bytes, the delimeter is
encountered when reading the'amount' byte, and the loop is executed once more.
{code:java}
2020-09-23 17:27:36,907 compile.WhereOptimizer$KeyExpressionVisitor(1358) -
MYLOG intersectTrailing :
result=\x80\x00\x00\x00\x05\xF8\xEFl\x814:\xA5LENGTH_8_BYTE_OVER\x00ABCDEFGHIJK,
pkPos=0, otherRange=\x814:\xA5, otherPKPos=1
2020-09-23 17:27:36,907 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=1, getOffset=8, getLength=4, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56, 95, 66, 89, 84, 69, 95, 79, 86, 69, 82, 0,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75],
str=�4:�LENGTH_8_BYTE_OVERABCDEFGHIJK
2020-09-23 17:27:36,908 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=2, getOffset=12, getLength=10, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56, 95, 66, 89, 84, 69, 95, 79, 86, 69, 82, 0,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75],
str=�4:�LENGTH_8_BYTE_OVERABCDEFGHIJK
2020-09-23 17:27:36,908 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=3, getOffset=23, getLength=11, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56, 95, 66, 89, 84, 69, 95, 79, 86, 69, 82, 0,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75],
str=�4:�LENGTH_8_BYTE_OVERABCDEFGHIJK
2020-09-23 17:27:36,908 compile.WhereOptimizer$KeyExpressionVisitor(1429) -
MYLOG intersectTrailing : pos=4, getOffset=34, getLength=0, ptr=[-127, 52, 58,
-91, 76, 69, 78, 71, 84, 72, 95, 56, 95, 66, 89, 84, 69, 95, 79, 86, 69, 82, 0,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75],
str=�4:�LENGTH_8_BYTE_OVERABCDEFGHIJK
2020-09-23 17:27:36,908 compile.WhereOptimizer$KeyExpressionVisitor(1445) -
MYLOG intersectTrailing : error=index (4) must be less than size (4)
{code}
- Test Query
{code:java}
CREATE TABLE TEST.TABLE1 (CREATE TABLE TEST.TABLE1 ( ID1 BIGINT NOT NULL,
MMDD INTEGER NOT NULL, ID2 VARCHAR NOT NULL, ID3 VARCHAR NOT NULL,
CONSTRAINT PK PRIMARY KEY (ID1,MMDD,ID2,ID3));
SELECT ID1, MMDD, ID2, ID3FROM TEST.TABLE1WHERE (ID1 = 100200300 AND
MMDD = 20200101 AND ID2 = 'LENGTH_8_BYTE_OVER' AND ID3 = 'ABCDEFGHIJK')
AND (ID1, MMDD, ID2, ID3) IN ((100200300, 20200101,'LENGTH_8_BYTE_OVER',
'ABCDEFGHIJK'),(200200300, 20200201,'ABC', 'ABCDEFGHIJK'));
{code}
- raise IndexOutOfExceptin
{code:java}
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.intersectTrailing(WhereOptimizer.java:1362)
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.andKeySlots(WhereOptimizer.java:974)
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.visitLeave(WhereOptimizer.java:1501)
at
org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.visitLeave(WhereOptimizer.java:512)
at
org.apache.phoenix.expression.AndExpression.accept(AndExpression.java:100)
at
org.apache.phoenix.compile.WhereOptimizer.pushKeyExpressionsToScan(WhereOptimizer.java:142)
at org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:157)
at org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:232)
at
org.apache.phoenix.execute.DelegateQueryPlan.iterator(DelegateQueryPlan.java:144)
at
org.apache.phoenix.execute.DelegateQueryPlan.iterator(DelegateQueryPlan.java:139)
at
org.apache.phoenix.jdbc.PhoenixStatement$1.call
[DISCUSS] Changing/unifying Github project integration settings
Hi! ASF Infra provides GitHub integration features, among them the ability to copy comments on PRs to the JIRA tickets, and sending notifications about PR activity to the development list (here). https://blogs.apache.org/infra/entry/improved_integration_between_apache_and The latter is enabled for some repos (Omid), but the former is not. I propose asking infra to enable all JIRA related GutHUB PR integrations for all six of our repositories. I am not sure about the mailing list notifications, I plan not to ask to enable them now, unless some of you think that we should. I have opened https://issues.apache.org/jira/browse/PHOENIX-6147 for this. If you have any input on this, or any objection,please let me know here, or in PHOENIX-6147. If there are no objections, I will file an INFRA ticket for this early next week. regards Istvan
[jira] [Assigned] (PHOENIX-6147) Copy Github PR discussions to JIRA
[ https://issues.apache.org/jira/browse/PHOENIX-6147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth reassigned PHOENIX-6147: Assignee: Istvan Toth > Copy Github PR discussions to JIRA > -- > > Key: PHOENIX-6147 > URL: https://issues.apache.org/jira/browse/PHOENIX-6147 > Project: Phoenix > Issue Type: Wish >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > A lot of discussion on patched happens on GitHub PRs. > While the GitHub PR interface is superior to the Jira for this purpose, it > means that we are missing a lot of information from the JIRA. > Try to set up a link, so that GitHub PR conversations are copied back to the > corresponding JIRA. -- This message was sent by Atlassian Jira (v8.3.4#803005)
