[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated PHOENIX-5753:
---
Fix Version/s: 5.1.0
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch,
> PHOENIX-5753_v5-4.x.patch, PHOENIX-5753_v5-master.patch,
> PHOENIX-5753_v6-master.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: PHOENIX-5753_v6-master.patch
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch,
> PHOENIX-5753_v5-4.x.patch, PHOENIX-5753_v5-master.patch,
> PHOENIX-5753_v6-master.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: PHOENIX-5753_v5-master.patch
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch,
> PHOENIX-5753_v5-4.x.patch, PHOENIX-5753_v5-master.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: (was: PHOENIX-5753_v5-4.x-HBase-1.4.patch)
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch,
> PHOENIX-5753_v5-4.x.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: PHOENIX-5753_v5-4.x.patch
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch,
> PHOENIX-5753_v5-4.x-HBase-1.4.patch, PHOENIX-5753_v5-4.x.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: PHOENIX-5753_v5-4.x-HBase-1.4.patch
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch,
> PHOENIX-5753_v5-4.x-HBase-1.4.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: (was: PHOENIX-5753_v4-4.x-HBase-1.4.patch)
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: PHOENIX-5753_v4-4.x-HBase-1.4.patch
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch,
> PHOENIX-5753_v4-4.x-HBase-1.4.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: PHOENIX-5753_v4-4.x-HBase-1.4.patch
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: (was: PHOENIX-5753_v4-4.x-HBase-1.4.patch)
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: (was: PHOENIX-5753_v3-4.x-HBase-1.4.patch)
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: PHOENIX-5753_v3-4.x-HBase-1.4.patch
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch,
> PHOENIX-5753_v3-4.x-HBase-1.4.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: (was: PHOENIX-5753_v2-4.x-HBase-1.4.patch)
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: PHOENIX-5753_v2-4.x-HBase-1.4.patch
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch,
> PHOENIX-5753_v2-4.x-HBase-1.4.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: PHOENIX-5753_v2-4.x-HBase-1.4.patch
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v2-4.x-HBase-1.4.patch
>
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: (was: PHOENIX-5753_v1-4.x-HBase-1.4.patch)
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Attachment: PHOENIX-5753_v1-4.x-HBase-1.4.patch
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
> Attachments: PHOENIX-5753_v1-4.x-HBase-1.4.patch
>
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
> clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
> 260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we could simply skip remaining columns of this
> RVC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Description:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty, but obviously, the above inserted row
(1,3,4,10,1) should be returned.
I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272// WHERE (A,B) < ('a','b')
273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
274// so we still need to filter on A,B
275stopExtracting = true;
276}
{code}
Eventually after we completed the {{WhereOptimizer.pushKeyExpressionsToScan}},
the result
{{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
{{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
[[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
(1,3,4,10,1) could not be retrieved.
But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
5}} and {{pk4 < 7}} , we could only have
{{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) < (5,7)}}
to {{pk3 <= 5}} , we could simply skip remaining columns of this RVC.
was:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty, but obviously, the above inserted row
(1,3,4,10,1) should be returned.
I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Description:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty, but obviously, the above inserted row
(1,3,4,10,1) should be returned.
I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
clipped to {{pk3 <= 5}} and {{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272// WHERE (A,B) < ('a','b')
273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
274// so we still need to filter on A,B
275stopExtracting = true;
276}
{code}
Eventually after we completed the {{WhereOptimizer.pushKeyExpressionsToScan}},
the result
{{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
{{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
[[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
(1,3,4,10,1) could not be retrieved.
But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
5}} and {{pk4 < 7}} , we could only have
{{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) < (5,7)}}
to {{pk3 <= 5}} , we could simply skip this RVC.
was:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty, but obviously, the above inserted row
(1,3,4,10,1) should be returned.
I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
clipped to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Description:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty, but obviously, the above inserted row
(1,3,4,10,1) should be returned.
I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
260 in {{WhereOptimizer.pushKeyExpressionsToScan}} , {{(pk3,pk4) < (5,7)}} is
clipped to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272// WHERE (A,B) < ('a','b')
273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
274// so we still need to filter on A,B
275stopExtracting = true;
276}
{code}
Eventually after we completed the {{WhereOptimizer.pushKeyExpressionsToScan}},
the result
{{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
{{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
[[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
(1,3,4,10,1) could not be retrieved.
But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
5}} and {{pk4 < 7}} , we could only have
{{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) < (5,7)}}
to {{pk3 <= 5}} , we could simply skip this RVC.
was:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty, but obviously, the above inserted row
(1,3,4,10,1) should be returned.
I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
260 in {{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is
clipped to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Description:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty, but obviously, the above inserted row
(1,3,4,10,1) should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272// WHERE (A,B) < ('a','b')
273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
274// so we still need to filter on A,B
275stopExtracting = true;
276}
{code}
Eventually after we completed the {{WhereOptimizer.pushKeyExpressionsToScan}},
the result
{{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
{{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
[[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
(1,3,4,10,1) could not be retrieved.
But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
5}} and {{pk4 < 7}} , we could only have
{{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) < (5,7)}}
to {{pk3 <= 5}} , we could simply skip this RVC.
was:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty, but obviously, the above inserted row
(1,3,4,10,1) should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Description:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty, but obviously, the above inserted row
(1,3,4,10,1) should be returned.
I think this problem is introduced by PHOENIX-3383 and PHOENIX-4841, when we
clip the {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line
260 in {{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is
clipped to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272// WHERE (A,B) < ('a','b')
273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
274// so we still need to filter on A,B
275stopExtracting = true;
276}
{code}
Eventually after we completed the {{WhereOptimizer.pushKeyExpressionsToScan}},
the result
{{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
{{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
[[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
(1,3,4,10,1) could not be retrieved.
But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
5}} and {{pk4 < 7}} , we could only have
{{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) < (5,7)}}
to {{pk3 <= 5}} , we could simply skip this RVC.
was:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty, but obviously, the above inserted row
(1,3,4,10,1) should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Fix Version/s: 4.16.0
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Priority: Major
> Labels: DESC
> Fix For: 4.16.0
>
>
> Given following table and data:
> {code:java}
>CREATE TABLE test
>(
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
>)
> {code}
> Noticed pk3 is DESC.
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code}
> the returned result is empty, but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383, when we clip the
> {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
> {{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and
> {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263leftRanges = invertKeyRanges(leftRanges);
> 264}
> 265slotSpanArray[cnf.size()] = clipLeftSpan-1;
> 266cnf.add(leftRanges);
> 267clipLeftSpan = 0;
> 268prevSortOrder = sortOrder;
> 269// since we have to clip the portion with the same
> sort order, we can no longer
> 270// extract the nodes from the where clause
> 271// for eg. for the schema A VARCHAR DESC, B VARCHAR
> ASC and query
> 272// WHERE (A,B) < ('a','b')
> 273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
> 274// so we still need to filter on A,B
> 275stopExtracting = true;
> 276}
> {code}
> Eventually after we completed the
> {{WhereOptimizer.pushKeyExpressionsToScan}}, the result
> {{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
> {{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
> [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
> (1,3,4,10,1) could not be retrieved.
> But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
> 5}} and {{pk4 < 7}} , we could only have
> {{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) <
> (5,7)}} to {{pk3 <= 5}} , we should stop traverse the rowKey column further.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Description:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty, but obviously, the above inserted row
(1,3,4,10,1) should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272// WHERE (A,B) < ('a','b')
273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
274// so we still need to filter on A,B
275stopExtracting = true;
276}
{code}
Eventually after we completed the {{WhereOptimizer.pushKeyExpressionsToScan}},
the result
{{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
{{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
[[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
(1,3,4,10,1) could not be retrieved.
But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
5}} and {{pk4 < 7}} , we could only have
{{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) < (5,7)}}
to {{pk3 <= 5}} , we should stop traverse the rowKey column further.
was:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty,but obviously, the above inserted row (1,3,4,10,1)
should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Description:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is DESC.
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty,but obviously, the above inserted row (1,3,4,10,1)
should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272// WHERE (A,B) < ('a','b')
273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
274// so we still need to filter on A,B
275stopExtracting = true;
276}
{code}
Eventually after we completed the {{WhereOptimizer.pushKeyExpressionsToScan}},
the result
{{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
{{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
[[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
(1,3,4,10,1) could not be retrieved.
But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
5}} and {{pk4 < 7}} , we could only have
{{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) < (5,7)}}
to {{pk3 <= 5}} , we should stop traverse the rowKey column further.
was:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is desc,
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty,but obviously, the above inserted row (1,3,4,10,1)
should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Description:
Given following table and data:
{code:java}
CREATE TABLE test (
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
}
{code}
Noticed pk3 is desc,
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty,but obviously, the above inserted row (1,3,4,10,1)
should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272// WHERE (A,B) < ('a','b')
273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
274// so we still need to filter on A,B
275stopExtracting = true;
276}
{code}
Eventually after we completed the {{WhereOptimizer.pushKeyExpressionsToScan}},
the result
{{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
{{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
[[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
(1,3,4,10,1) could not be retrieved.
But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
5}} and {{pk4 < 7}} , we could only have
{{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) < (5,7)}}
to {{pk3 <= 5}} , we should stop traverse the rowKey column further.
was:
Given following table and data:
{code:java}
CREATE TABLE test (
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
}
{code:java}
Noticed pk3 is desc,
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code:java}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code:java}
the returned result is empty,but obviously, the above inserted row (1,3,4,10,1)
should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Description:
Given following table and data:
{code:java}
CREATE TABLE test
(
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
)
{code}
Noticed pk3 is desc,
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty,but obviously, the above inserted row (1,3,4,10,1)
should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272// WHERE (A,B) < ('a','b')
273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
274// so we still need to filter on A,B
275stopExtracting = true;
276}
{code}
Eventually after we completed the {{WhereOptimizer.pushKeyExpressionsToScan}},
the result
{{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
{{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
[[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
(1,3,4,10,1) could not be retrieved.
But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
5}} and {{pk4 < 7}} , we could only have
{{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) < (5,7)}}
to {{pk3 <= 5}} , we should stop traverse the rowKey column further.
was:
Given following table and data:
{code:java}
CREATE TABLE test (
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
}
{code}
Noticed pk3 is desc,
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code}
the returned result is empty,but obviously, the above inserted row (1,3,4,10,1)
should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[
https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-5753:
--
Description:
Given following table and data:
{code:java}
CREATE TABLE test (
pk1 INTEGER NOT NULL ,
pk2 INTEGER NOT NULL,
pk3 INTEGER NOT NULL,
pk4 INTEGER NOT NULL,
v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
}
{code:java}
Noticed pk3 is desc,
{code:java}
UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
{code:java}
If we execute the following sql:
{code:java}
select * from test
where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
{code:java}
the returned result is empty,but obviously, the above inserted row (1,3,4,10,1)
should be returned.
I think this problem is introduced by PHOENIX-3383, when we clip the
{{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
{{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is clipped
to {{pk3 <= 5}} and
{{pk4 < 7}} .
{code:java}
257List leftRanges = clipLeft(schema,
slot.getPKPosition()
258+ slotOffset - clipLeftSpan, clipLeftSpan,
keyRanges, ptr);
259keyRanges =
260clipRight(schema, slot.getPKPosition() +
slotOffset - 1, keyRanges,
261leftRanges, ptr);
262if (prevSortOrder == SortOrder.DESC) {
263leftRanges = invertKeyRanges(leftRanges);
264}
265slotSpanArray[cnf.size()] = clipLeftSpan-1;
266cnf.add(leftRanges);
267clipLeftSpan = 0;
268prevSortOrder = sortOrder;
269// since we have to clip the portion with the same sort
order, we can no longer
270// extract the nodes from the where clause
271// for eg. for the schema A VARCHAR DESC, B VARCHAR ASC
and query
272// WHERE (A,B) < ('a','b')
273// the range (* - a\xFFb) is converted to (~a-*)(*-b)
274// so we still need to filter on A,B
275stopExtracting = true;
276}
{code:java}
Eventually after we completed the {{WhereOptimizer.pushKeyExpressionsToScan}},
the result
{{ScanRanges.ranges}} is [[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]],
{{ScanRanges.useSkipScanFilter}} is {{true}} and {{SkipScanFilter}} is also
[[[1 - 2]], [[3 - 4]], [[~5 - *)], [(* - 7)]], so the the above inserted row
(1,3,4,10,1) could not be retrieved.
But as we know, {{(pk3,pk4) < (5,7)}} is not semantically equals to {{pk3 <=
5}} and {{pk4 < 7}} , we could only have
{{pk3 <= 5}} but not {{pk4 < 7}}, so when we clipped {{(pk3,pk4) < (5,7)}}
to {{pk3 <= 5}} , we should stop traverse the rowKey column further.
was:Consider
> Fix erroneous query result when RVC is clipped with desc column
> ---
>
> Key: PHOENIX-5753
> URL: https://issues.apache.org/jira/browse/PHOENIX-5753
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: chenglei
>Priority: Major
> Labels: DESC
>
> Given following table and data:
> {code:java}
> CREATE TABLE test (
> pk1 INTEGER NOT NULL ,
> pk2 INTEGER NOT NULL,
> pk3 INTEGER NOT NULL,
> pk4 INTEGER NOT NULL,
> v INTEGER, CONSTRAINT PK PRIMARY KEY(pk1,pk2,pk3 desc,pk4))
> }
> {code:java}
> Noticed pk3 is desc,
> {code:java}
>UPSERT INTO test (pk1, pk2, pk3, pk4, v) VALUES (1,3,4,10,1)
> {code:java}
> If we execute the following sql:
> {code:java}
> select * from test
> where (pk1 >=1 and pk1<=2) and (pk2>=3 and pk2<=4) and (pk3,pk4) < (5,7)
> {code:java}
> the returned result is empty,but obviously, the above inserted row
> (1,3,4,10,1) should be returned.
> I think this problem is introduced by PHOENIX-3383, when we clip the
> {{(pk3,pk4) < (5,7)}} because {{pk3}} is {{DESC}} by following line 260 in
> {{WhereOptimizer.pushKeyExpressionsToScan}}, {{(pk3,pk4) < (5,7)}} is
> clipped to {{pk3 <= 5}} and
> {{pk4 < 7}} .
> {code:java}
> 257List leftRanges = clipLeft(schema,
> slot.getPKPosition()
> 258+ slotOffset - clipLeftSpan, clipLeftSpan,
> keyRanges, ptr);
> 259keyRanges =
> 260clipRight(schema, slot.getPKPosition() +
> slotOffset - 1, keyRanges,
> 261leftRanges, ptr);
> 262if (prevSortOrder == SortOrder.DESC) {
> 263
[jira] [Updated] (PHOENIX-5753) Fix erroneous query result when RVC is clipped with desc column
[ https://issues.apache.org/jira/browse/PHOENIX-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenglei updated PHOENIX-5753: -- Description: Consider > Fix erroneous query result when RVC is clipped with desc column > --- > > Key: PHOENIX-5753 > URL: https://issues.apache.org/jira/browse/PHOENIX-5753 > Project: Phoenix > Issue Type: Bug >Affects Versions: 5.0.0, 4.15.0 >Reporter: chenglei >Priority: Major > Labels: DESC > > Consider -- This message was sent by Atlassian Jira (v8.3.4#803005)
