I appreciate the drive to get 5.2.0 out of the door, but I would prefer to have a few more days to fix the registry issues, and run some tests on them before cutting the branch, Viraj. The non-ZK registry support is one of the bigger new features, and I'd prefer not to have known breaking bugs in the release. Can we target Friday or the next Monday for the cut ?
Istvan On Sat, Feb 10, 2024 at 10:07 AM rajeshb...@apache.org < chrajeshbab...@gmail.com> wrote: > Yes Viraj, I can release 5.1.4 > > Thanks, > Rajeshbabu. > > On Sat, Feb 10, 2024, 10:28 AM Viraj Jasani <vjas...@apache.org> wrote: > >> I think we can also target 5.2.1 very soon, perhaps just next month, with >> more CVE fixes and any other fixes if ready. >> >> >> On Fri, Feb 9, 2024 at 6:39 PM Viraj Jasani <vjas...@apache.org> wrote: >> >> > For 5.2.0, it would be great to focus on the known data integrity >> issues. >> > We can fix non-zk registry, cover a few more CVEs by upgrading third >> party >> > dependencies and stabilize tests. As for the tests, they don’t seem >> broken, >> > but are flaky. I have got multiple builds without any test failures on >> PR >> > for PHOENIX-7106. >> > >> > If this looks good to you, I can start release preparation next week. >> What >> > do you think, Istvan? >> > >> > In the meantime, I have 5.1 backport PR open, awaiting good build >> results >> > before committing it. >> > Rajeshbabu, would you like to be RM for 5.1.4 once the PR is merged? >> > >> > >> > >> > On Fri, Feb 9, 2024 at 11:13 AM Istvan Toth <st...@apache.org> wrote: >> > >> >> Yes, they basically make the non-ZK registries unusable. >> >> (at least the connectionless problems should be fixed.) >> >> >> >> I hope to have the final fix for those sometime next week. >> >> >> >> Also have we looked at potential CVE issues on master recently ? >> >> >> >> I think we should also look at the most flakey tests I linked above, >> >> and fix them or at least make sure that they are test issues and not >> real >> >> bugs. >> >> >> >> Istvan >> >> >> >> >> >> >> >> On Fri, Feb 9, 2024 at 6:55 PM Viraj Jasani <vjas...@apache.org> >> wrote: >> >> >> >> > Thanks Istvan. >> >> > I would like to cut 5.2 branch from master. Do you see non-ZK >> registry >> >> for >> >> > MapReduce jobs as blocker for 5.2.0? >> >> > >> >> > >> >> > On Fri, Feb 9, 2024 at 12:24 AM Istvan Toth <st...@apache.org> >> wrote: >> >> > >> >> > > ParallelPhoenixConnectionFailureTest.testExecuteQueryChainFailure >> also >> >> > > fails too often, especially when the test host is slow and/or the >> >> load is >> >> > > high. >> >> > > On my fast laptop, I can semi-reliably break it by running >> >> > > mvn clean verify -am -pl phoenix-core -DnumForkedUT=20 >> >> > > >> >> > > On Fri, Feb 9, 2024 at 9:19 AM Istvan Toth <st...@apache.org> >> wrote: >> >> > > >> >> > > > We're making progress. >> >> > > > I can see that Viraj has just landed PHOENIX-7601, and Rajeshbabu >> >> has >> >> > > > released Omid 1.1.1. >> >> > > > Thank you! >> >> > > > >> >> > > > At the moment, the following outstanding issues are on my radar: >> >> > > > >> >> > > > https://issues.apache.org/jira/browse/PHOENIX-7191 >> >> > > > https://issues.apache.org/jira/browse/PHOENIX-7193 >> >> > > > >> >> > > > These are bugs in my non-ZK registry implementation, which were >> >> found >> >> > > > during HBase 3 work. >> >> > > > I have some PRs up, but they may not be complete. I will push for >> >> > reviews >> >> > > > once I have the HBase 3 tests passing, and possibly updated them >> >> based >> >> > on >> >> > > > that. >> >> > > > >> >> > > > We also have a number of very flakey tests, see: >> >> > > > >> >> > > > >> >> > > > >> >> > > >> >> > >> >> >> https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/test_results_analyzer/ >> >> > > > >> >> > > > >> >> > > > >> >> > > > On Tue, Jan 30, 2024 at 7:09 AM Istvan Toth <st...@cloudera.com> >> >> > wrote: >> >> > > > >> >> > > >> As Viraj wrote, those are just plans. >> >> > > >> If HBase 3 won't be released by the time the other features are >> >> ready, >> >> > > >> then it won't make it into 5.3. >> >> > > >> If other major features are ready by that time, then they will >> be >> >> > > >> included. (though we are not aware of any now) >> >> > > >> >> >> > > >> As for the new major version, in the past Phoenix didn't have a >> >> > > >> compatibility module system, >> >> > > >> so a new branch was required, which didn't support older >> HBases. >> >> > Also, >> >> > > >> the API changes between HBase 1.x and 2.x were much larger, >> >> > > >> The HBase 2 and 3 API are pretty similar, apart from the >> removal of >> >> > > >> deprecated 1.x APIs. (and the protobuf/protocol thing, which >> >> requires >> >> > a >> >> > > >> rather ugly hack). >> >> > > >> >> >> > > >> I will start the discussion on how we can add HBase 3 support as >> >> soon >> >> > as >> >> > > >> I have a working POC patch. >> >> > > >> >> >> > > >> We could call 5.3 6.0 instead, after all Phoenix isn't using a >> >> strict >> >> > > >> semantic versioning, but then 6.0 would also support HBase 2. >> >> > > >> If we do not come to a consensus on the version name, we can >> always >> >> > have >> >> > > >> a vote on it. >> >> > > >> >> >> > > >> I think that the main motivation is that the community wants to >> >> > maintain >> >> > > >> as few branches as possible. >> >> > > >> >> >> > > >> Istvan >> >> > > >> >> >> > > >> On Mon, Jan 29, 2024 at 10:02 PM Stephen Jiang < >> >> > syuanjiang...@gmail.com >> >> > > > >> >> > > >> wrote: >> >> > > >> >> >> > > >>> I am not sure how close HBase 3.0 is. Even if it is only less >> >> than >> >> > one >> >> > > >>> year away, the adoption would be low at the beginning. I don't >> >> think >> >> > > 5.3 >> >> > > >>> should wait for that. And traditionally, Phoenix would have a >> >> major >> >> > > >>> release to support the HBase major release (4.x for HBase 1.x >> and >> >> 5.x >> >> > > for >> >> > > >>> HBase 2.x), in this case, we are talking about Phoenix 6.0 for >> >> HBase >> >> > > 3.0. >> >> > > >>> >> >> > > >>> Maybe we should adopt the HBase release model: master branch >> for >> >> next >> >> > > >>> major >> >> > > >>> release (6.0) and branch-5.x branch for next 5.x minor release >> and >> >> > > >>> branch-5.2 for 5.2 minor release. >> >> > > >>> >> >> > > >>> Thanks >> >> > > >>> Stephen >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> On Thu, Jan 25, 2024 at 11:50 PM Viraj Jasani < >> vjas...@apache.org >> >> > >> >> > > >>> wrote: >> >> > > >>> >> >> > > >>> > Sounds good. >> >> > > >>> > >> >> > > >>> > Planned major changes for 5.3.0: >> >> > > >>> > >> >> > > >>> > 1. JSON support. >> >> > > >>> > 2. HBase 3.0 support. >> >> > > >>> > 3. CDC feature (leveraging uncovered global index framework >> and >> >> > JSON >> >> > > >>> > support). >> >> > > >>> > >> >> > > >>> > >> >> > > >>> > On Wed, Jan 24, 2024 at 8:22 PM Kadir Ozdemir >> >> > > >>> > <kozde...@salesforce.com.invalid> wrote: >> >> > > >>> > >> >> > > >>> > > I suggest including another major change for 5.3, Phoenix >> CDC, >> >> > > >>> > > https://issues.apache.org/jira/browse/PHOENIX-7001. The PR >> >> for >> >> > it >> >> > > >>> will >> >> > > >>> > be >> >> > > >>> > > posted soon. >> >> > > >>> > > >> >> > > >>> > > On Thu, Jan 25, 2024 at 5:35 AM Viraj Jasani < >> >> vjas...@apache.org >> >> > > >> >> > > >>> wrote: >> >> > > >>> > > >> >> > > >>> > > > Thanks Istvan! I agree with your points. It’s really >> been a >> >> > while >> >> > > >>> we >> >> > > >>> > are >> >> > > >>> > > > talking about releasing 5.2.0 and yet due to bandwidth >> >> issues, >> >> > > >>> unable >> >> > > >>> > to >> >> > > >>> > > do >> >> > > >>> > > > so. >> >> > > >>> > > > >> >> > > >>> > > > I agree to getting these fixes out, cut 5.2 and start the >> >> > release >> >> > > >>> work >> >> > > >>> > > and >> >> > > >>> > > > in the meantime I also need to prepare 5.1 backport. >> >> > > >>> > > > >> >> > > >>> > > > For 5.3, let's plan HBase 3.0 support and JSON as major >> >> > changes. >> >> > > >>> > > > >> >> > > >>> > > > >> >> > > >>> > > > On Mon, Jan 22, 2024 at 9:17 PM Istvan Toth < >> >> st...@apache.org> >> >> > > >>> wrote: >> >> > > >>> > > > >> >> > > >>> > > > > Hi! >> >> > > >>> > > > > >> >> > > >>> > > > > In my opinion cutting 5.2 now only makes sense IF we DO >> >> NOT >> >> > > plan >> >> > > >>> to >> >> > > >>> > > > release >> >> > > >>> > > > > the outstanding big features (like JSON) in 5.2. , >> >> otherwise >> >> > > it's >> >> > > >>> > just >> >> > > >>> > > > > extra work to maintain more branches. >> >> > > >>> > > > > >> >> > > >>> > > > > Having said that, releasing a 5.2 and 5.1.4 with the >> data >> >> > > >>> integrity >> >> > > >>> > > fixes >> >> > > >>> > > > > real soon, and then releasing 5.3 in a few months with >> >> JSON, >> >> > > and >> >> > > >>> any >> >> > > >>> > > > other >> >> > > >>> > > > > outstanding big features >> >> > > >>> > > > > that are close to being finished (and HBase 3.0 >> support, >> >> if >> >> > > it's >> >> > > >>> > ready >> >> > > >>> > > by >> >> > > >>> > > > > then) would not be a bad idea. >> >> > > >>> > > > > >> >> > > >>> > > > > On the CLDR side the only outstanding big feature which >> >> could >> >> > > >>> impact >> >> > > >>> > > > > Viraj's integrity work is HBase 3.0 support, and even >> >> that is >> >> > > >>> only >> >> > > >>> > > > because >> >> > > >>> > > > > it may require some larger refactors of existing code, >> not >> >> > > >>> because it >> >> > > >>> > > > would >> >> > > >>> > > > > change the actual behaviour or algorithms. >> >> > > >>> > > > > >> >> > > >>> > > > > Phoenix used to have several minor releases per year, >> the >> >> > > current >> >> > > >>> > state >> >> > > >>> > > > of >> >> > > >>> > > > > extreme longevity of 5.1 and several big new features >> >> being >> >> > > >>> added to >> >> > > >>> > it >> >> > > >>> > > > > (like uncovered indexes) is not ideal. >> >> > > >>> > > > > Releasing 5.2 and 5.3 relatively close together could >> be a >> >> > > >>> return to >> >> > > >>> > a >> >> > > >>> > > > > quicker cadence for minor releases, which could also >> help >> >> > with >> >> > > >>> the >> >> > > >>> > > public >> >> > > >>> > > > > image and adoption of Phoenix. >> >> > > >>> > > > > >> >> > > >>> > > > > We were talking about releasing 5.2 at least a year >> ago, >> >> and >> >> > I >> >> > > >>> have >> >> > > >>> > > > started >> >> > > >>> > > > > working on that then, but then emergencies have come >> up at >> >> > > >>> $dayjob, >> >> > > >>> > > and I >> >> > > >>> > > > > could not see that through. >> >> > > >>> > > > > (So I am in part responsible for the lack of minor >> >> releases) >> >> > > >>> > > > > >> >> > > >>> > > > > regards >> >> > > >>> > > > > Istvan >> >> > > >>> > > > > >> >> > > >>> > > > > >> >> > > >>> > > > > On Mon, Jan 22, 2024 at 11:35 PM Viraj Jasani < >> >> > > >>> vjas...@apache.org> >> >> > > >>> > > > wrote: >> >> > > >>> > > > > >> >> > > >>> > > > > > Sorry for the late reply. >> >> > > >>> > > > > > >> >> > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense? >> >> > > >>> > > > > > >> >> > > >>> > > > > > No problem with that. I can cut 5.2 branch by the >> end of >> >> > this >> >> > > >>> week >> >> > > >>> > or >> >> > > >>> > > > at >> >> > > >>> > > > > > the start of next week. >> >> > > >>> > > > > > >> >> > > >>> > > > > > If there is any very big change or feature ready for >> >> merge >> >> > to >> >> > > >>> > master >> >> > > >>> > > > > branch >> >> > > >>> > > > > > with PR approvals already in place, please do let me >> >> know >> >> > so >> >> > > >>> that I >> >> > > >>> > > can >> >> > > >>> > > > > > help collaborate on how best we can get it merged >> >> without >> >> > > >>> impacting >> >> > > >>> > > 5.2 >> >> > > >>> > > > > > release if required. My main motivation was for any >> big >> >> > > change >> >> > > >>> to >> >> > > >>> > go >> >> > > >>> > > > > > through newly introduced tests so that we know that >> >> > anything >> >> > > >>> > > additional >> >> > > >>> > > > > is >> >> > > >>> > > > > > not broken, and also to prioritize for upcoming 5.2.0 >> >> and >> >> > > 5.1.4 >> >> > > >>> > > > releases. >> >> > > >>> > > > > > Moreover, there are several PRs getting merged on the >> >> > master >> >> > > >>> > branch, >> >> > > >>> > > we >> >> > > >>> > > > > can >> >> > > >>> > > > > > continue that as long as they are not very big >> changes, >> >> > which >> >> > > >>> might >> >> > > >>> > > > > require >> >> > > >>> > > > > > significant time to understand any correlation with >> data >> >> > > >>> integrity >> >> > > >>> > > > > issues. >> >> > > >>> > > > > > >> >> > > >>> > > > > > The PR is also ready for review with some additional >> >> cases >> >> > > >>> fixed >> >> > > >>> > last >> >> > > >>> > > > > week: >> >> > > >>> > > > > > >> >> > > >>> > > > >> >> > > >>> > > >> >> > > >>> > >> >> > > >>> >> >> > > >> >> > >> >> >> https://urldefense.com/v3/__https://github.com/apache/phoenix/pull/1736__;!!DCbAVzZNrAf4!D4OVjUp2EWW2BqhGnBxsapDX_AHsibRphIpoFBWfgRsd3dsAikrFLo6PGxdTzGbSXJJ2fJ0j9mcz3asXMXo$ >> >> > > >>> > > > > > >> >> > > >>> > > > > > Depending on the review bandwidth, I am hopeful we >> >> should >> >> > be >> >> > > >>> good >> >> > > >>> > to >> >> > > >>> > > > land >> >> > > >>> > > > > > them sooner. >> >> > > >>> > > > > > >> >> > > >>> > > > > > >> >> > > >>> > > > > > On Wed, Jan 17, 2024 at 11:31 AM Rushabh Shah >> >> > > >>> > > > > > <rushabh.s...@salesforce.com.invalid> wrote: >> >> > > >>> > > > > > >> >> > > >>> > > > > > > Thank you Viraj for initiating this thread. >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > > Given the critical nature of these issues, I >> would >> >> like >> >> > > to >> >> > > >>> > > propose >> >> > > >>> > > > > that >> >> > > >>> > > > > > > we >> >> > > >>> > > > > > > treat this as a high priority for the upcoming >> 5.2.0 >> >> > > >>> release, and >> >> > > >>> > > not >> >> > > >>> > > > > > > include any other feature or big change to master >> >> branch >> >> > > >>> until we >> >> > > >>> > > > merge >> >> > > >>> > > > > > > this. >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense? This >> will >> >> > > enable >> >> > > >>> > other >> >> > > >>> > > > > > > developers to merge features into master branch >> >> (5.3.0) >> >> > and >> >> > > >>> you >> >> > > >>> > can >> >> > > >>> > > > > take >> >> > > >>> > > > > > > some more time to make sure we cover all the corner >> >> cases >> >> > > >>> for the >> >> > > >>> > > > data >> >> > > >>> > > > > > > integrity issues that you uncovered. >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > On Fri, Jan 5, 2024 at 6:38 PM Viraj Jasani < >> >> > > >>> vjas...@apache.org> >> >> > > >>> > > > > wrote: >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > > Sounds good, thanks Rajeshbabu. I will try to get >> >> the >> >> > > >>> first PR >> >> > > >>> > > out >> >> > > >>> > > > > next >> >> > > >>> > > > > > > > week and while reviews happen in parallel, will >> try >> >> to >> >> > > get >> >> > > >>> 5.1 >> >> > > >>> > PR >> >> > > >>> > > > > soon. >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > > On Wed, Jan 3, 2024 at 8:49 PM >> >> rajeshb...@apache.org < >> >> > > >>> > > > > > > > chrajeshbab...@gmail.com> wrote: >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > > > Hi Viraj, >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > > Would be better to include the changes in 5.1.4 >> >> as >> >> > in >> >> > > >>> any >> >> > > >>> > way >> >> > > >>> > > it >> >> > > >>> > > > > > will >> >> > > >>> > > > > > > > take >> >> > > >>> > > > > > > > > at least 3-4 days to complete the omid release. >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > > Thanks, >> >> > > >>> > > > > > > > > Rajeshbabu. >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > > On Thu, Jan 4, 2024 at 5:06 AM Viraj Jasani < >> >> > > >>> > > vjas...@apache.org> >> >> > > >>> > > > > > > wrote: >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > > > Thank you Kadir and Geoffrey for your >> replies!! >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > > > How does this affect 5.1.4, which is also >> >> listed >> >> > as >> >> > > >>> a Fix >> >> > > >>> > > > > Version >> >> > > >>> > > > > > > for >> >> > > >>> > > > > > > > > > > PHOENIX-7106? >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > > Yes, it also needs to be ported to 5.1. Once >> the >> >> > > >>> master PR >> >> > > >>> > is >> >> > > >>> > > > up >> >> > > >>> > > > > > for >> >> > > >>> > > > > > > > > final >> >> > > >>> > > > > > > > > > review, I would start working on the backport >> >> PR. >> >> > > >>> > > > > > > > > > We just need some more additional testing to >> >> ensure >> >> > > old >> >> > > >>> > > client >> >> > > >>> > > > > > (e.g. >> >> > > >>> > > > > > > > > 5.1.3) >> >> > > >>> > > > > > > > > > is compatible with the new server with the >> >> changes. >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > > Hence, yes it is now a blocker for upcoming >> >> 5.1.4 >> >> > as >> >> > > >>> well >> >> > > >>> > > since >> >> > > >>> > > > > > 5.1.4 >> >> > > >>> > > > > > > > RC >> >> > > >>> > > > > > > > > > preparation is still pending (while Omid >> >> release is >> >> > > in >> >> > > >>> > > > progress). >> >> > > >>> > > > > > > > > > Otherwise if 5.1.4 was ready for release, I >> >> would >> >> > > have >> >> > > >>> > > proposed >> >> > > >>> > > > > > > > immediate >> >> > > >>> > > > > > > > > > 5.1.5 release to include the changes proposed >> >> with >> >> > > >>> > > > PHOENIX-7106. >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > > On Wed, Jan 3, 2024 at 3:08 PM Geoffrey >> Jacoby < >> >> > > >>> > > > > gjac...@apache.org >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > > > wrote: >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > > > I agree that data integrity issues are a >> >> higher >> >> > > >>> priority >> >> > > >>> > > than >> >> > > >>> > > > > > > feature >> >> > > >>> > > > > > > > > > > development, so I also support the >> decision. >> >> The >> >> > > fact >> >> > > >>> > that >> >> > > >>> > > > > > several >> >> > > >>> > > > > > > of >> >> > > >>> > > > > > > > > the >> >> > > >>> > > > > > > > > > > major remaining 5.2 features are currently >> >> being >> >> > > >>> > developed >> >> > > >>> > > in >> >> > > >>> > > > > > > > > > long-running >> >> > > >>> > > > > > > > > > > feature branches also helps, as work can >> >> continue >> >> > > >>> there >> >> > > >>> > at >> >> > > >>> > > > the >> >> > > >>> > > > > > cost >> >> > > >>> > > > > > > > of >> >> > > >>> > > > > > > > > a >> >> > > >>> > > > > > > > > > > rebase later. >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > > How does this affect 5.1.4, which is also >> >> listed >> >> > as >> >> > > >>> a Fix >> >> > > >>> > > > > Version >> >> > > >>> > > > > > > for >> >> > > >>> > > > > > > > > > > PHOENIX-7106? From the bug description it >> also >> >> > > sounds >> >> > > >>> > like >> >> > > >>> > > > > 5.1.3 >> >> > > >>> > > > > > > and >> >> > > >>> > > > > > > > > the >> >> > > >>> > > > > > > > > > > forthcoming .4 are affected, since we have >> >> > > >>> server-side >> >> > > >>> > > paging >> >> > > >>> > > > > in >> >> > > >>> > > > > > > 5.1. >> >> > > >>> > > > > > > > > > (Feel >> >> > > >>> > > > > > > > > > > free to move that to a separate thread if >> you >> >> > feel >> >> > > it >> >> > > >>> > > should >> >> > > >>> > > > > be a >> >> > > >>> > > > > > > > > > separate >> >> > > >>> > > > > > > > > > > discussion.) Should this be a blocker for >> >> > releasing >> >> > > >>> > 5.1.4? >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > > Geoffrey >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > > On Wed, Jan 3, 2024 at 5:06 PM Kadir >> Ozdemir < >> >> > > >>> > > > > > > > > > > ka...@gsuite.cloud.apache.org> >> >> > > >>> > > > > > > > > > > wrote: >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > Being a database, Phoenix has to make >> sure >> >> that >> >> > > the >> >> > > >>> > data >> >> > > >>> > > > > stays >> >> > > >>> > > > > > on >> >> > > >>> > > > > > > > > disk >> >> > > >>> > > > > > > > > > > > intact and its queries return correct >> data. >> >> In >> >> > > this >> >> > > >>> > case, >> >> > > >>> > > > > > Phoenix >> >> > > >>> > > > > > > > > fails >> >> > > >>> > > > > > > > > > > to >> >> > > >>> > > > > > > > > > > > return correct data for some queries if >> >> their >> >> > > scans >> >> > > >>> > > > > experience >> >> > > >>> > > > > > > > region >> >> > > >>> > > > > > > > > > > > movement. Now that we know these data >> >> integrity >> >> > > >>> issues >> >> > > >>> > > and >> >> > > >>> > > > > how >> >> > > >>> > > > > > to >> >> > > >>> > > > > > > > > > > reproduce >> >> > > >>> > > > > > > > > > > > them, fixing them should be our first >> >> priority. >> >> > > >>> So, I >> >> > > >>> > > fully >> >> > > >>> > > > > > > support >> >> > > >>> > > > > > > > > > this >> >> > > >>> > > > > > > > > > > > proposal. >> >> > > >>> > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > On Wed, Jan 3, 2024 at 10:58 PM Viraj >> >> Jasani < >> >> > > >>> > > > > > vjas...@apache.org >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > > > > wrote: >> >> > > >>> > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > > Hello, >> >> > > >>> > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > > I would like to bring PHOENIX-7106 >> >> > > >>> > > > > > > > > > > > > < >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > >> >> > > >>> > > > > >> >> > > >>> > > > >> >> > > >>> > > >> >> > > >>> > >> >> > > >>> >> >> > > >> >> > >> >> >> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/PHOENIX-7106__;!!DCbAVzZNrAf4!FZG5sv55IC1NqItQLY7GKWgUG2Do0gSta01gOiSdd36Dx3XHGtQx4M3c9visVXIt9DctPQzS-orob9vhzrCfVA$ >> >> > > >>> > > > > > > > > to everyone's >> >> > > >>> > > > > > > > > > > > > attention here and brief about the data >> >> > > integrity >> >> > > >>> > > issues >> >> > > >>> > > > > that >> >> > > >>> > > > > > > we >> >> > > >>> > > > > > > > > have >> >> > > >>> > > > > > > > > > > in >> >> > > >>> > > > > > > > > > > > > various coprocessors. Majority of the >> >> issues >> >> > > are >> >> > > >>> > > related >> >> > > >>> > > > to >> >> > > >>> > > > > > the >> >> > > >>> > > > > > > > > fact >> >> > > >>> > > > > > > > > > > that >> >> > > >>> > > > > > > > > > > > > we do not return valid rowkey for >> certain >> >> > > >>> queries. If >> >> > > >>> > > any >> >> > > >>> > > > > > > region >> >> > > >>> > > > > > > > > > moves >> >> > > >>> > > > > > > > > > > in >> >> > > >>> > > > > > > > > > > > > the middle of the scan, the HBase >> client >> >> > relies >> >> > > >>> on >> >> > > >>> > the >> >> > > >>> > > > last >> >> > > >>> > > > > > > > > returned >> >> > > >>> > > > > > > > > > > > rowkey >> >> > > >>> > > > > > > > > > > > > and accordingly changes the scan >> >> boundaries >> >> > > >>> while the >> >> > > >>> > > > > scanner >> >> > > >>> > > > > > > is >> >> > > >>> > > > > > > > > > > getting >> >> > > >>> > > > > > > > > > > > > reset to continue the scan operation. >> If >> >> the >> >> > > >>> region >> >> > > >>> > > does >> >> > > >>> > > > > not >> >> > > >>> > > > > > > > move, >> >> > > >>> > > > > > > > > > scan >> >> > > >>> > > > > > > > > > > > is >> >> > > >>> > > > > > > > > > > > > not expected to return invalid data, >> >> however >> >> > if >> >> > > >>> the >> >> > > >>> > > > region >> >> > > >>> > > > > > > moves >> >> > > >>> > > > > > > > in >> >> > > >>> > > > > > > > > > the >> >> > > >>> > > > > > > > > > > > > middle of ongoing scan operation, scan >> >> would >> >> > > >>> return >> >> > > >>> > > > > > > > > invalid/incorrect >> >> > > >>> > > > > > > > > > > > data >> >> > > >>> > > > > > > > > > > > > causing data integrity issues. >> >> > > >>> > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > > Given the critical nature of these >> >> issues, I >> >> > > >>> would >> >> > > >>> > like >> >> > > >>> > > > to >> >> > > >>> > > > > > > > propose >> >> > > >>> > > > > > > > > > that >> >> > > >>> > > > > > > > > > > > we >> >> > > >>> > > > > > > > > > > > > treat this as a high priority for the >> >> > upcoming >> >> > > >>> 5.2.0 >> >> > > >>> > > > > release, >> >> > > >>> > > > > > > and >> >> > > >>> > > > > > > > > not >> >> > > >>> > > > > > > > > > > > > include any other feature or big >> change to >> >> > > master >> >> > > >>> > > branch >> >> > > >>> > > > > > until >> >> > > >>> > > > > > > we >> >> > > >>> > > > > > > > > > merge >> >> > > >>> > > > > > > > > > > > > this. The PR is still not ready as >> >> additional >> >> > > >>> changes >> >> > > >>> > > are >> >> > > >>> > > > > > still >> >> > > >>> > > > > > > > in >> >> > > >>> > > > > > > > > my >> >> > > >>> > > > > > > > > > > > > local, requiring rebase with the >> current >> >> > > master. >> >> > > >>> > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > > I would get back to this discuss >> thread as >> >> > soon >> >> > > >>> as >> >> > > >>> > the >> >> > > >>> > > PR >> >> > > >>> > > > > and >> >> > > >>> > > > > > > the >> >> > > >>> > > > > > > > > doc >> >> > > >>> > > > > > > > > > > are >> >> > > >>> > > > > > > > > > > > > updated with the latest findings so >> far. >> >> The >> >> > > >>> changes >> >> > > >>> > > > > include >> >> > > >>> > > > > > > many >> >> > > >>> > > > > > > > > of >> >> > > >>> > > > > > > > > > > our >> >> > > >>> > > > > > > > > > > > > coproc scanner implementations and >> hence >> >> it >> >> > > would >> >> > > >>> > > require >> >> > > >>> > > > > > > > > significant >> >> > > >>> > > > > > > > > > > > > review as well. >> >> > > >>> > > > > > > > > > > > > It would be great if we can hold on to >> >> > merging >> >> > > >>> any >> >> > > >>> > > > feature >> >> > > >>> > > > > or >> >> > > >>> > > > > > > big >> >> > > >>> > > > > > > > > > > change >> >> > > >>> > > > > > > > > > > > to >> >> > > >>> > > > > > > > > > > > > master branch until this gets in so as >> to >> >> not >> >> > > >>> > > complicate >> >> > > >>> > > > > > > > > > > > merging/rebasing. >> >> > > >>> > > > > > > > > > > > > Once this is merged to the master >> branch, >> >> I >> >> > > would >> >> > > >>> > like >> >> > > >>> > > to >> >> > > >>> > > > > cut >> >> > > >>> > > > > > > 5.2 >> >> > > >>> > > > > > > > > > > branch >> >> > > >>> > > > > > > > > > > > > from master and we can move forward >> with >> >> > 5.2.0 >> >> > > >>> > release. >> >> > > >>> > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > > Please let me know if this looks good >> or >> >> if >> >> > you >> >> > > >>> have >> >> > > >>> > > any >> >> > > >>> > > > > > other >> >> > > >>> > > > > > > > high >> >> > > >>> > > > > > > > > > > > > priority work for 5.2.0. >> >> > > >>> > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > >> >> > > >>> > > > > >> >> > > >>> > > > >> >> > > >>> > > >> >> > > >>> > >> >> > > >>> >> >> > > >> >> >> > > >> >> >> > > >> -- >> >> > > >> *István Tóth* | Sr. Staff Software Engineer >> >> > > >> *Email*: st...@cloudera.com >> >> > > >> cloudera.com <https://www.cloudera.com> >> >> > > >> [image: Cloudera] <https://www.cloudera.com/> >> >> > > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> >> >> [image: >> >> > > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> >> [image: >> >> > > >> Cloudera on LinkedIn] < >> https://www.linkedin.com/company/cloudera> >> >> > > >> ------------------------------ >> >> > > >> ------------------------------ >> >> > > >> >> >> > > > >> >> > > >> >> > >> >> >> > >> >
