[jira] [Updated] (PHOENIX-6528) When view index pk has a variable length column, read repair doesn't work correctly
[ https://issues.apache.org/jira/browse/PHOENIX-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated PHOENIX-6528: - Fix Version/s: 5.1.3 5.2.0 > When view index pk has a variable length column, read repair doesn't work > correctly > --- > > Key: PHOENIX-6528 > URL: https://issues.apache.org/jira/browse/PHOENIX-6528 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.14.3 >Reporter: Gokcen Iskender >Assignee: Gokcen Iskender >Priority: Critical > Fix For: 5.2.0, 5.1.3 > > > If a view index rowkey has a variable length column, read repair might work > incorrectly. We fixed PHOENIX-6266 and same fix should be applied to the case > where the index row is UNVERIFIED so that read repair happens. > This only happens for view indexes (mutitenant or not) > This is the reason > org.apache.phoenix.end2end.index.ViewIndexIT.testRowKeyComposition[ViewIndexIT_isNamespaceMapped=false] > sometimes flaps -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6498) Fix incorrect Correlated Exists Subquery rewrite when Subquery is aggregate
[
https://issues.apache.org/jira/browse/PHOENIX-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated PHOENIX-6498:
-
Fix Version/s: 5.1.3
> Fix incorrect Correlated Exists Subquery rewrite when Subquery is aggregate
> ---
>
> Key: PHOENIX-6498
> URL: https://issues.apache.org/jira/browse/PHOENIX-6498
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.16.1, 5.1.2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 4.17.0, 5.2.0, 5.1.3
>
>
> Given following tables :
> {code:java}
> create table item
>(item_id varchar not null primary key,
> name varchar,
> price integer,
> discount1 integer,
> discount2 integer,
> supplier_id varchar,
>description varchar)
>
> create table order
> (order_id varchar not null primary key,
>customer_id varchar,
>item_id varchar,
>price integer,
>quantity integer,
>date timestamp)
> {code}
> for the correlated exists subquery:
> {code:java}
> SELECT item_id, name FROM item i WHERE exists
> (SELECT 1 FROM order o where o.item_id = i.item_id
> group by customer_id having count(order_id) > 1) ORDER BY name
> {code}
> Phoenix would throw following exception:
> {code:java}
> java.sql.SQLException: ERROR 1018 (42Y27): Aggregate may not contain columns
> not in GROUP BY. ITEM_ID
> at
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:606)
> at
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:217)
> at
> org.apache.phoenix.compile.ExpressionCompiler.throwNonAggExpressionInAggException(ExpressionCompiler.java:1090)
> at
> org.apache.phoenix.compile.ProjectionCompiler.compile(ProjectionCompiler.java:445)
> at
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:755)
> at
> org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:674)
> at
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:251)
> at
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:178)
> at
> org.apache.phoenix.compile.QueryCompiler.compileSubquery(QueryCompiler.java:661)
> at
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:289)
> at
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:370)
> at
> org.apache.phoenix.compile.QueryCompiler.compileJoinQuery(QueryCompiler.java:302)
> at
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:249)
> at
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:178)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:672)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:2011)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:2004)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1998)
> {code}
> That is because Phoenix incorrectly rewrite the subquery as:
> {code:java}
>SELECT DISTINCT 1 $3,O.ITEM_ID $2 FROM ORDER_TABLE O GROUP BY CUSTOMER_ID
> HAVING COUNT(ORDER_ID) > 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-6507) DistinctAggregatingResultIterator should keep original tuple order of the AggregatingResultIterator
[
https://issues.apache.org/jira/browse/PHOENIX-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated PHOENIX-6507:
-
Fix Version/s: 5.1.3
> DistinctAggregatingResultIterator should keep original tuple order of the
> AggregatingResultIterator
> ---
>
> Key: PHOENIX-6507
> URL: https://issues.apache.org/jira/browse/PHOENIX-6507
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.16.1, 5.1.2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 4.17.0, 5.2.0, 5.1.3
>
> Attachments: PHOENIX-6507_v2-4.x.patch, PHOENIX-6507_v2-master.patch
>
>
> Given following tables :
> {code:java}
>create table test (
> pk1 varchar not null ,
> pk2 varchar not null,
> pk3 varchar not null,
>v1 varchar,
>v2 varchar,
> CONSTRAINT TEST_PK PRIMARY KEY ( pk1,pk2,pk3))
> {code}
> The result of the following sql may be incorrect:
> {code:java}
>select distinct pk1,max(v1) from test group by pk1,pk2,pk3 order by pk1
> {code}
> The problem is the {{order by}} is complied out because it is the prefix of
> the {{group by}}, but for {{distinct}},
> {{DistinctAggregatingResultIterator}} using {{HashSet}} to rearrange the
> aggregated tuples.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-6472) In case of region inconsistency phoenix should stop gracefully
[ https://issues.apache.org/jira/browse/PHOENIX-6472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated PHOENIX-6472: - Fix Version/s: 5.1.3 > In case of region inconsistency phoenix should stop gracefully > -- > > Key: PHOENIX-6472 > URL: https://issues.apache.org/jira/browse/PHOENIX-6472 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.16.0 >Reporter: Neha Gupta >Assignee: Xinyi Yan >Priority: Major > Fix For: 4.17.0, 5.2.0, 5.1.3 > > > When Phoenix fetches HRegionLocation data via hbase client, in case of region > overlap/inconsistencies it should throw an exception and stop gracefully > instead of getting stuck in an infinite loop and create millions of > HRegionLocation objects. Phoenix CQSI code for [getting > AllTableRegions|https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java#L664] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6344) CASCADE on ALTER should NOOP when there are no secondary indexes
[ https://issues.apache.org/jira/browse/PHOENIX-6344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated PHOENIX-6344: - Fix Version/s: 5.1.3 > CASCADE on ALTER should NOOP when there are no secondary indexes > > > Key: PHOENIX-6344 > URL: https://issues.apache.org/jira/browse/PHOENIX-6344 > Project: Phoenix > Issue Type: Improvement >Reporter: Swaroopa Kadam >Assignee: Swaroopa Kadam >Priority: Minor > Fix For: 4.16.0, 5.2.0, 5.1.3 > > > When a table/view does not have a secondary index, using cascade in the ALTER > TABLE/VIEW .. ADD should continue with default behavior (of only > adding the column to table/view like a regular alter statement) -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] Releasing 5.1.3 / 4.16.2
4.16 is in even worse shape: PHOENIX-6511 +addendums PHOENIX-6338 PHOENIX-6346 PHOENIX-6347 PHOENIX-6351 PHOENIX-6356 PHOENIX-6420 PHOENIX-6421 PHOENIX-6423 PHOENIX-6424 PHOENIX-6436 PHOENIX-6444 PHOENIX-6451 PHOENIX-6456 PHOENIX-6462 PHOENIX-6472 PHOENIX-6476 PHOENIX-6498 PHOENIX-6507 PHOENIX-6511 PHOENIX-6528 PHOENIX-6543 I'm not committing to fixing this now, please reply here if you start working on this. On Fri, Oct 29, 2021 at 5:10 PM Istvan Toth wrote: > I've compared master and 5.1, and found the following missing fixes from > 5.1: > PHOENIX-6462 > PHOENIX-6472 > PHOENIX-6498 > PHOENIX-6507 > PHOENIX-6528 > PHOENIX-6577 (this one was me) > > I'm running mvn verify on them, then I will commit them to 5.1. > > Istvan > > On Wed, Oct 27, 2021 at 7:47 AM Istvan Toth wrote: > >> Thank you Viraj. >> >> >> On Sat, Oct 23, 2021 at 8:22 AM Viraj Jasani wrote: >> >>> Thanks Luoc. 2.4.8 is also about to be released soon (most likely next >>> week). We might also want to consider upgrading the dependency for 2.4 >>> release line after 2.4.8 is released. >>> >>> Thanks Istvan for starting this thread. +1 for new 5.1 patch release. If >>> not done till 15th Nov, I can volunteer to be RM to complete it before >>> start of Dec. >>> >>> >>> On Fri, 22 Oct 2021 at 9:03 PM, luoc wrote: >>> >>> > Hello, Istvan. I am going to create a ticket to upgrade the hbase >>> > dependency with the latest 2.3.7. And before that happens, I >>> will >>> > test them in our test cluster. >>> > >>> > >>> > -- luoc >>> > >>> > >>> > >>> > >>> > -- 原始邮件 -- >>> > 发件人: >>> > "dev" >>> > < >>> > st...@apache.org>; >>> > 发送时间: 2021年10月22日(星期五) 下午3:00 >>> > 收件人: "dev">> > >>> > 主题: [DISCUSS] Releasing 5.1.3 / 4.16.2 >>> > >>> > >>> > >>> > Hi! >>> > >>> > The last 5.1 and 4.16 releases were respectively made in June and May. >>> > We've accumulated a number of good fixes in the meantime, I think now >>> may >>> > be a good time to do >>> > patch releases, before the holiday season comes, which would make it >>> more >>> > difficult to do it. >>> > >>> > I also get the impression that quite a few low-risk fixes have not been >>> > backported to the maintenance branches, we should do a pass on >>> backporting >>> > those patches as well. >>> > >>> > Do you think that this is a good time for this, or should we wait until >>> > next year ? >>> > Do we have some specific fix that we want to make sure gets into the >>> next >>> > maintenance release ? >>> > If we agree to do this does anyone volunteer to be the RM for either? >>> > >>> > regards >>> > Istvan >>> >>
Re: [DISCUSS] Releasing 5.1.3 / 4.16.2
I've compared master and 5.1, and found the following missing fixes from 5.1: PHOENIX-6462 PHOENIX-6472 PHOENIX-6498 PHOENIX-6507 PHOENIX-6528 PHOENIX-6577 (this one was me) I'm running mvn verify on them, then I will commit them to 5.1. Istvan On Wed, Oct 27, 2021 at 7:47 AM Istvan Toth wrote: > Thank you Viraj. > > > On Sat, Oct 23, 2021 at 8:22 AM Viraj Jasani wrote: > >> Thanks Luoc. 2.4.8 is also about to be released soon (most likely next >> week). We might also want to consider upgrading the dependency for 2.4 >> release line after 2.4.8 is released. >> >> Thanks Istvan for starting this thread. +1 for new 5.1 patch release. If >> not done till 15th Nov, I can volunteer to be RM to complete it before >> start of Dec. >> >> >> On Fri, 22 Oct 2021 at 9:03 PM, luoc wrote: >> >> > Hello, Istvan. I am going to create a ticket to upgrade the hbase >> > dependency with the latest 2.3.7. And before that happens, I >> will >> > test them in our test cluster. >> > >> > >> > -- luoc >> > >> > >> > >> > >> > -- 原始邮件 -- >> > 发件人: >> > "dev" >> > < >> > st...@apache.org>; >> > 发送时间: 2021年10月22日(星期五) 下午3:00 >> > 收件人: "dev"> > >> > 主题: [DISCUSS] Releasing 5.1.3 / 4.16.2 >> > >> > >> > >> > Hi! >> > >> > The last 5.1 and 4.16 releases were respectively made in June and May. >> > We've accumulated a number of good fixes in the meantime, I think now >> may >> > be a good time to do >> > patch releases, before the holiday season comes, which would make it >> more >> > difficult to do it. >> > >> > I also get the impression that quite a few low-risk fixes have not been >> > backported to the maintenance branches, we should do a pass on >> backporting >> > those patches as well. >> > >> > Do you think that this is a good time for this, or should we wait until >> > next year ? >> > Do we have some specific fix that we want to make sure gets into the >> next >> > maintenance release ? >> > If we agree to do this does anyone volunteer to be the RM for either? >> > >> > regards >> > Istvan >> >
[jira] [Updated] (PHOENIX-6472) In case of region inconsistency phoenix should stop gracefully
[ https://issues.apache.org/jira/browse/PHOENIX-6472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated PHOENIX-6472: - Fix Version/s: 5.2.0 > In case of region inconsistency phoenix should stop gracefully > -- > > Key: PHOENIX-6472 > URL: https://issues.apache.org/jira/browse/PHOENIX-6472 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.16.0 >Reporter: Neha Gupta >Assignee: Xinyi Yan >Priority: Major > Fix For: 4.17.0, 5.2.0 > > > When Phoenix fetches HRegionLocation data via hbase client, in case of region > overlap/inconsistencies it should throw an exception and stop gracefully > instead of getting stuck in an infinite loop and create millions of > HRegionLocation objects. Phoenix CQSI code for [getting > AllTableRegions|https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java#L664] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6584) Annotate HBase WALs with Schema Id
[ https://issues.apache.org/jira/browse/PHOENIX-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated PHOENIX-6584: - Fix Version/s: 5.2.0 4.17.0 > Annotate HBase WALs with Schema Id > -- > > Key: PHOENIX-6584 > URL: https://issues.apache.org/jira/browse/PHOENIX-6584 > Project: Phoenix > Issue Type: Sub-task >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 4.17.0, 5.2.0 > > > PHOENIX-5435 introduced WAL annotation with a tuple of fields (tenant, > Phoenix schema, table, and last DDL timestamp) to identify the Phoenix object > which had created an HBase edit, for use in change detection systems. > PHOENIX-6227 introduces the ability to store schema information about Phoenix > metadata in an external schema registry and save a lookup id to get the > metadata back. > Switch the WAL annotations to use this schema id rather than the existing > tuple. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (PHOENIX-6584) Annotate HBase WALs with Schema Id
Geoffrey Jacoby created PHOENIX-6584: Summary: Annotate HBase WALs with Schema Id Key: PHOENIX-6584 URL: https://issues.apache.org/jira/browse/PHOENIX-6584 Project: Phoenix Issue Type: Sub-task Reporter: Geoffrey Jacoby Assignee: Geoffrey Jacoby PHOENIX-5435 introduced WAL annotation with a tuple of fields (tenant, Phoenix schema, table, and last DDL timestamp) to identify the Phoenix object which had created an HBase edit, for use in change detection systems. PHOENIX-6227 introduces the ability to store schema information about Phoenix metadata in an external schema registry and save a lookup id to get the metadata back. Switch the WAL annotations to use this schema id rather than the existing tuple. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5448) ReplicationSink Message Producer
[ https://issues.apache.org/jira/browse/PHOENIX-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated PHOENIX-5448: - Summary: ReplicationSink Message Producer (was: ReplicationEndpoint Message Producer) > ReplicationSink Message Producer > > > Key: PHOENIX-5448 > URL: https://issues.apache.org/jira/browse/PHOENIX-5448 > Project: Phoenix > Issue Type: Sub-task >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > > This endpoint will read WAL Entries from the replication stream, and output > to message bus topics (initially Kafka, but intended to be pluggable) using > the generic Avro schema implemented in PHOENIX-5447. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6227) Option for DDL changes to export to external schema repository
[ https://issues.apache.org/jira/browse/PHOENIX-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated PHOENIX-6227: - Description: When a user creates or drops a table or view, or adds/removes a column from one, there should be the option for Phoenix to notify an external schema repository. This should be a configurable plugin so that core Phoenix is not coupled to any particular repository implementation. This will also store a schema id generated by the external schema registry in a new field in System.Catalog so that a future JIRA can switch to using the schema id in change detection WAL annotations. Because of this this JIRA will not be able to be backported to 5.1.x or 4.16.x was:When a user creates or drops a table or view, or adds/removes a column from one, there should be the option for Phoenix to notify an external schema repository. This should be a configurable plugin so that core Phoenix is not coupled to any particular repository implementation. > Option for DDL changes to export to external schema repository > -- > > Key: PHOENIX-6227 > URL: https://issues.apache.org/jira/browse/PHOENIX-6227 > Project: Phoenix > Issue Type: Sub-task >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 4.17.0, 5.2.0 > > > When a user creates or drops a table or view, or adds/removes a column from > one, there should be the option for Phoenix to notify an external schema > repository. This should be a configurable plugin so that core Phoenix is not > coupled to any particular repository implementation. > This will also store a schema id generated by the external schema registry in > a new field in System.Catalog so that a future JIRA can switch to using the > schema id in change detection WAL annotations. Because of this this JIRA will > not be able to be backported to 5.1.x or 4.16.x -- This message was sent by Atlassian Jira (v8.3.4#803005)
