ParallelPhoenixConnectionFailureTest.testExecuteQueryChainFailure also fails too often, especially when the test host is slow and/or the load is high. On my fast laptop, I can semi-reliably break it by running mvn clean verify -am -pl phoenix-core -DnumForkedUT=20
On Fri, Feb 9, 2024 at 9:19 AM Istvan Toth <st...@apache.org> wrote: > We're making progress. > I can see that Viraj has just landed PHOENIX-7601, and Rajeshbabu has > released Omid 1.1.1. > Thank you! > > At the moment, the following outstanding issues are on my radar: > > https://issues.apache.org/jira/browse/PHOENIX-7191 > https://issues.apache.org/jira/browse/PHOENIX-7193 > > These are bugs in my non-ZK registry implementation, which were found > during HBase 3 work. > I have some PRs up, but they may not be complete. I will push for reviews > once I have the HBase 3 tests passing, and possibly updated them based on > that. > > We also have a number of very flakey tests, see: > > > https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/test_results_analyzer/ > > > > On Tue, Jan 30, 2024 at 7:09 AM Istvan Toth <st...@cloudera.com> wrote: > >> As Viraj wrote, those are just plans. >> If HBase 3 won't be released by the time the other features are ready, >> then it won't make it into 5.3. >> If other major features are ready by that time, then they will be >> included. (though we are not aware of any now) >> >> As for the new major version, in the past Phoenix didn't have a >> compatibility module system, >> so a new branch was required, which didn't support older HBases. Also, >> the API changes between HBase 1.x and 2.x were much larger, >> The HBase 2 and 3 API are pretty similar, apart from the removal of >> deprecated 1.x APIs. (and the protobuf/protocol thing, which requires a >> rather ugly hack). >> >> I will start the discussion on how we can add HBase 3 support as soon as >> I have a working POC patch. >> >> We could call 5.3 6.0 instead, after all Phoenix isn't using a strict >> semantic versioning, but then 6.0 would also support HBase 2. >> If we do not come to a consensus on the version name, we can always have >> a vote on it. >> >> I think that the main motivation is that the community wants to maintain >> as few branches as possible. >> >> Istvan >> >> On Mon, Jan 29, 2024 at 10:02 PM Stephen Jiang <syuanjiang...@gmail.com> >> wrote: >> >>> I am not sure how close HBase 3.0 is. Even if it is only less than one >>> year away, the adoption would be low at the beginning. I don't think 5.3 >>> should wait for that. And traditionally, Phoenix would have a major >>> release to support the HBase major release (4.x for HBase 1.x and 5.x for >>> HBase 2.x), in this case, we are talking about Phoenix 6.0 for HBase 3.0. >>> >>> Maybe we should adopt the HBase release model: master branch for next >>> major >>> release (6.0) and branch-5.x branch for next 5.x minor release and >>> branch-5.2 for 5.2 minor release. >>> >>> Thanks >>> Stephen >>> >>> >>> >>> On Thu, Jan 25, 2024 at 11:50 PM Viraj Jasani <vjas...@apache.org> >>> wrote: >>> >>> > Sounds good. >>> > >>> > Planned major changes for 5.3.0: >>> > >>> > 1. JSON support. >>> > 2. HBase 3.0 support. >>> > 3. CDC feature (leveraging uncovered global index framework and JSON >>> > support). >>> > >>> > >>> > On Wed, Jan 24, 2024 at 8:22 PM Kadir Ozdemir >>> > <kozde...@salesforce.com.invalid> wrote: >>> > >>> > > I suggest including another major change for 5.3, Phoenix CDC, >>> > > https://issues.apache.org/jira/browse/PHOENIX-7001. The PR for it >>> will >>> > be >>> > > posted soon. >>> > > >>> > > On Thu, Jan 25, 2024 at 5:35 AM Viraj Jasani <vjas...@apache.org> >>> wrote: >>> > > >>> > > > Thanks Istvan! I agree with your points. It’s really been a while >>> we >>> > are >>> > > > talking about releasing 5.2.0 and yet due to bandwidth issues, >>> unable >>> > to >>> > > do >>> > > > so. >>> > > > >>> > > > I agree to getting these fixes out, cut 5.2 and start the release >>> work >>> > > and >>> > > > in the meantime I also need to prepare 5.1 backport. >>> > > > >>> > > > For 5.3, let's plan HBase 3.0 support and JSON as major changes. >>> > > > >>> > > > >>> > > > On Mon, Jan 22, 2024 at 9:17 PM Istvan Toth <st...@apache.org> >>> wrote: >>> > > > >>> > > > > Hi! >>> > > > > >>> > > > > In my opinion cutting 5.2 now only makes sense IF we DO NOT plan >>> to >>> > > > release >>> > > > > the outstanding big features (like JSON) in 5.2. , otherwise it's >>> > just >>> > > > > extra work to maintain more branches. >>> > > > > >>> > > > > Having said that, releasing a 5.2 and 5.1.4 with the data >>> integrity >>> > > fixes >>> > > > > real soon, and then releasing 5.3 in a few months with JSON, and >>> any >>> > > > other >>> > > > > outstanding big features >>> > > > > that are close to being finished (and HBase 3.0 support, if it's >>> > ready >>> > > by >>> > > > > then) would not be a bad idea. >>> > > > > >>> > > > > On the CLDR side the only outstanding big feature which could >>> impact >>> > > > > Viraj's integrity work is HBase 3.0 support, and even that is >>> only >>> > > > because >>> > > > > it may require some larger refactors of existing code, not >>> because it >>> > > > would >>> > > > > change the actual behaviour or algorithms. >>> > > > > >>> > > > > Phoenix used to have several minor releases per year, the current >>> > state >>> > > > of >>> > > > > extreme longevity of 5.1 and several big new features being >>> added to >>> > it >>> > > > > (like uncovered indexes) is not ideal. >>> > > > > Releasing 5.2 and 5.3 relatively close together could be a >>> return to >>> > a >>> > > > > quicker cadence for minor releases, which could also help with >>> the >>> > > public >>> > > > > image and adoption of Phoenix. >>> > > > > >>> > > > > We were talking about releasing 5.2 at least a year ago, and I >>> have >>> > > > started >>> > > > > working on that then, but then emergencies have come up at >>> $dayjob, >>> > > and I >>> > > > > could not see that through. >>> > > > > (So I am in part responsible for the lack of minor releases) >>> > > > > >>> > > > > regards >>> > > > > Istvan >>> > > > > >>> > > > > >>> > > > > On Mon, Jan 22, 2024 at 11:35 PM Viraj Jasani < >>> vjas...@apache.org> >>> > > > wrote: >>> > > > > >>> > > > > > Sorry for the late reply. >>> > > > > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense? >>> > > > > > >>> > > > > > No problem with that. I can cut 5.2 branch by the end of this >>> week >>> > or >>> > > > at >>> > > > > > the start of next week. >>> > > > > > >>> > > > > > If there is any very big change or feature ready for merge to >>> > master >>> > > > > branch >>> > > > > > with PR approvals already in place, please do let me know so >>> that I >>> > > can >>> > > > > > help collaborate on how best we can get it merged without >>> impacting >>> > > 5.2 >>> > > > > > release if required. My main motivation was for any big change >>> to >>> > go >>> > > > > > through newly introduced tests so that we know that anything >>> > > additional >>> > > > > is >>> > > > > > not broken, and also to prioritize for upcoming 5.2.0 and 5.1.4 >>> > > > releases. >>> > > > > > Moreover, there are several PRs getting merged on the master >>> > branch, >>> > > we >>> > > > > can >>> > > > > > continue that as long as they are not very big changes, which >>> might >>> > > > > require >>> > > > > > significant time to understand any correlation with data >>> integrity >>> > > > > issues. >>> > > > > > >>> > > > > > The PR is also ready for review with some additional cases >>> fixed >>> > last >>> > > > > week: >>> > > > > > >>> > > > >>> > > >>> > >>> https://urldefense.com/v3/__https://github.com/apache/phoenix/pull/1736__;!!DCbAVzZNrAf4!D4OVjUp2EWW2BqhGnBxsapDX_AHsibRphIpoFBWfgRsd3dsAikrFLo6PGxdTzGbSXJJ2fJ0j9mcz3asXMXo$ >>> > > > > > >>> > > > > > Depending on the review bandwidth, I am hopeful we should be >>> good >>> > to >>> > > > land >>> > > > > > them sooner. >>> > > > > > >>> > > > > > >>> > > > > > On Wed, Jan 17, 2024 at 11:31 AM Rushabh Shah >>> > > > > > <rushabh.s...@salesforce.com.invalid> wrote: >>> > > > > > >>> > > > > > > Thank you Viraj for initiating this thread. >>> > > > > > > >>> > > > > > > > Given the critical nature of these issues, I would like to >>> > > propose >>> > > > > that >>> > > > > > > we >>> > > > > > > treat this as a high priority for the upcoming 5.2.0 >>> release, and >>> > > not >>> > > > > > > include any other feature or big change to master branch >>> until we >>> > > > merge >>> > > > > > > this. >>> > > > > > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense? This will enable >>> > other >>> > > > > > > developers to merge features into master branch (5.3.0) and >>> you >>> > can >>> > > > > take >>> > > > > > > some more time to make sure we cover all the corner cases >>> for the >>> > > > data >>> > > > > > > integrity issues that you uncovered. >>> > > > > > > >>> > > > > > > >>> > > > > > > On Fri, Jan 5, 2024 at 6:38 PM Viraj Jasani < >>> vjas...@apache.org> >>> > > > > wrote: >>> > > > > > > >>> > > > > > > > Sounds good, thanks Rajeshbabu. I will try to get the >>> first PR >>> > > out >>> > > > > next >>> > > > > > > > week and while reviews happen in parallel, will try to get >>> 5.1 >>> > PR >>> > > > > soon. >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > On Wed, Jan 3, 2024 at 8:49 PM rajeshb...@apache.org < >>> > > > > > > > chrajeshbab...@gmail.com> wrote: >>> > > > > > > > >>> > > > > > > > > Hi Viraj, >>> > > > > > > > > >>> > > > > > > > > Would be better to include the changes in 5.1.4 as in >>> any >>> > way >>> > > it >>> > > > > > will >>> > > > > > > > take >>> > > > > > > > > at least 3-4 days to complete the omid release. >>> > > > > > > > > >>> > > > > > > > > Thanks, >>> > > > > > > > > Rajeshbabu. >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > On Thu, Jan 4, 2024 at 5:06 AM Viraj Jasani < >>> > > vjas...@apache.org> >>> > > > > > > wrote: >>> > > > > > > > > >>> > > > > > > > > > Thank you Kadir and Geoffrey for your replies!! >>> > > > > > > > > > >>> > > > > > > > > > > How does this affect 5.1.4, which is also listed as >>> a Fix >>> > > > > Version >>> > > > > > > for >>> > > > > > > > > > > PHOENIX-7106? >>> > > > > > > > > > >>> > > > > > > > > > Yes, it also needs to be ported to 5.1. Once the >>> master PR >>> > is >>> > > > up >>> > > > > > for >>> > > > > > > > > final >>> > > > > > > > > > review, I would start working on the backport PR. >>> > > > > > > > > > We just need some more additional testing to ensure old >>> > > client >>> > > > > > (e.g. >>> > > > > > > > > 5.1.3) >>> > > > > > > > > > is compatible with the new server with the changes. >>> > > > > > > > > > >>> > > > > > > > > > Hence, yes it is now a blocker for upcoming 5.1.4 as >>> well >>> > > since >>> > > > > > 5.1.4 >>> > > > > > > > RC >>> > > > > > > > > > preparation is still pending (while Omid release is in >>> > > > progress). >>> > > > > > > > > > Otherwise if 5.1.4 was ready for release, I would have >>> > > proposed >>> > > > > > > > immediate >>> > > > > > > > > > 5.1.5 release to include the changes proposed with >>> > > > PHOENIX-7106. >>> > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > > On Wed, Jan 3, 2024 at 3:08 PM Geoffrey Jacoby < >>> > > > > gjac...@apache.org >>> > > > > > > >>> > > > > > > > > wrote: >>> > > > > > > > > > >>> > > > > > > > > > > I agree that data integrity issues are a higher >>> priority >>> > > than >>> > > > > > > feature >>> > > > > > > > > > > development, so I also support the decision. The fact >>> > that >>> > > > > > several >>> > > > > > > of >>> > > > > > > > > the >>> > > > > > > > > > > major remaining 5.2 features are currently being >>> > developed >>> > > in >>> > > > > > > > > > long-running >>> > > > > > > > > > > feature branches also helps, as work can continue >>> there >>> > at >>> > > > the >>> > > > > > cost >>> > > > > > > > of >>> > > > > > > > > a >>> > > > > > > > > > > rebase later. >>> > > > > > > > > > > >>> > > > > > > > > > > How does this affect 5.1.4, which is also listed as >>> a Fix >>> > > > > Version >>> > > > > > > for >>> > > > > > > > > > > PHOENIX-7106? From the bug description it also sounds >>> > like >>> > > > > 5.1.3 >>> > > > > > > and >>> > > > > > > > > the >>> > > > > > > > > > > forthcoming .4 are affected, since we have >>> server-side >>> > > paging >>> > > > > in >>> > > > > > > 5.1. >>> > > > > > > > > > (Feel >>> > > > > > > > > > > free to move that to a separate thread if you feel it >>> > > should >>> > > > > be a >>> > > > > > > > > > separate >>> > > > > > > > > > > discussion.) Should this be a blocker for releasing >>> > 5.1.4? >>> > > > > > > > > > > >>> > > > > > > > > > > Geoffrey >>> > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > > On Wed, Jan 3, 2024 at 5:06 PM Kadir Ozdemir < >>> > > > > > > > > > > ka...@gsuite.cloud.apache.org> >>> > > > > > > > > > > wrote: >>> > > > > > > > > > > >>> > > > > > > > > > > > Being a database, Phoenix has to make sure that the >>> > data >>> > > > > stays >>> > > > > > on >>> > > > > > > > > disk >>> > > > > > > > > > > > intact and its queries return correct data. In this >>> > case, >>> > > > > > Phoenix >>> > > > > > > > > fails >>> > > > > > > > > > > to >>> > > > > > > > > > > > return correct data for some queries if their scans >>> > > > > experience >>> > > > > > > > region >>> > > > > > > > > > > > movement. Now that we know these data integrity >>> issues >>> > > and >>> > > > > how >>> > > > > > to >>> > > > > > > > > > > reproduce >>> > > > > > > > > > > > them, fixing them should be our first priority. >>> So, I >>> > > fully >>> > > > > > > support >>> > > > > > > > > > this >>> > > > > > > > > > > > proposal. >>> > > > > > > > > > > > >>> > > > > > > > > > > > On Wed, Jan 3, 2024 at 10:58 PM Viraj Jasani < >>> > > > > > vjas...@apache.org >>> > > > > > > > >>> > > > > > > > > > wrote: >>> > > > > > > > > > > > >>> > > > > > > > > > > > > Hello, >>> > > > > > > > > > > > > >>> > > > > > > > > > > > > I would like to bring PHOENIX-7106 >>> > > > > > > > > > > > > < >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >>> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/PHOENIX-7106__;!!DCbAVzZNrAf4!FZG5sv55IC1NqItQLY7GKWgUG2Do0gSta01gOiSdd36Dx3XHGtQx4M3c9visVXIt9DctPQzS-orob9vhzrCfVA$ >>> > > > > > > > > to everyone's >>> > > > > > > > > > > > > attention here and brief about the data integrity >>> > > issues >>> > > > > that >>> > > > > > > we >>> > > > > > > > > have >>> > > > > > > > > > > in >>> > > > > > > > > > > > > various coprocessors. Majority of the issues are >>> > > related >>> > > > to >>> > > > > > the >>> > > > > > > > > fact >>> > > > > > > > > > > that >>> > > > > > > > > > > > > we do not return valid rowkey for certain >>> queries. If >>> > > any >>> > > > > > > region >>> > > > > > > > > > moves >>> > > > > > > > > > > in >>> > > > > > > > > > > > > the middle of the scan, the HBase client relies >>> on >>> > the >>> > > > last >>> > > > > > > > > returned >>> > > > > > > > > > > > rowkey >>> > > > > > > > > > > > > and accordingly changes the scan boundaries >>> while the >>> > > > > scanner >>> > > > > > > is >>> > > > > > > > > > > getting >>> > > > > > > > > > > > > reset to continue the scan operation. If the >>> region >>> > > does >>> > > > > not >>> > > > > > > > move, >>> > > > > > > > > > scan >>> > > > > > > > > > > > is >>> > > > > > > > > > > > > not expected to return invalid data, however if >>> the >>> > > > region >>> > > > > > > moves >>> > > > > > > > in >>> > > > > > > > > > the >>> > > > > > > > > > > > > middle of ongoing scan operation, scan would >>> return >>> > > > > > > > > invalid/incorrect >>> > > > > > > > > > > > data >>> > > > > > > > > > > > > causing data integrity issues. >>> > > > > > > > > > > > > >>> > > > > > > > > > > > > Given the critical nature of these issues, I >>> would >>> > like >>> > > > to >>> > > > > > > > propose >>> > > > > > > > > > that >>> > > > > > > > > > > > we >>> > > > > > > > > > > > > treat this as a high priority for the upcoming >>> 5.2.0 >>> > > > > release, >>> > > > > > > and >>> > > > > > > > > not >>> > > > > > > > > > > > > include any other feature or big change to master >>> > > branch >>> > > > > > until >>> > > > > > > we >>> > > > > > > > > > merge >>> > > > > > > > > > > > > this. The PR is still not ready as additional >>> changes >>> > > are >>> > > > > > still >>> > > > > > > > in >>> > > > > > > > > my >>> > > > > > > > > > > > > local, requiring rebase with the current master. >>> > > > > > > > > > > > > >>> > > > > > > > > > > > > I would get back to this discuss thread as soon >>> as >>> > the >>> > > PR >>> > > > > and >>> > > > > > > the >>> > > > > > > > > doc >>> > > > > > > > > > > are >>> > > > > > > > > > > > > updated with the latest findings so far. The >>> changes >>> > > > > include >>> > > > > > > many >>> > > > > > > > > of >>> > > > > > > > > > > our >>> > > > > > > > > > > > > coproc scanner implementations and hence it would >>> > > require >>> > > > > > > > > significant >>> > > > > > > > > > > > > review as well. >>> > > > > > > > > > > > > It would be great if we can hold on to merging >>> any >>> > > > feature >>> > > > > or >>> > > > > > > big >>> > > > > > > > > > > change >>> > > > > > > > > > > > to >>> > > > > > > > > > > > > master branch until this gets in so as to not >>> > > complicate >>> > > > > > > > > > > > merging/rebasing. >>> > > > > > > > > > > > > Once this is merged to the master branch, I would >>> > like >>> > > to >>> > > > > cut >>> > > > > > > 5.2 >>> > > > > > > > > > > branch >>> > > > > > > > > > > > > from master and we can move forward with 5.2.0 >>> > release. >>> > > > > > > > > > > > > >>> > > > > > > > > > > > > Please let me know if this looks good or if you >>> have >>> > > any >>> > > > > > other >>> > > > > > > > high >>> > > > > > > > > > > > > priority work for 5.2.0. >>> > > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >>> >> >> >> -- >> *István Tóth* | Sr. Staff Software Engineer >> *Email*: st...@cloudera.com >> cloudera.com <https://www.cloudera.com> >> [image: Cloudera] <https://www.cloudera.com/> >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> >> ------------------------------ >> ------------------------------ >> >
