I agree that data integrity issues are a higher priority than feature development, so I also support the decision. The fact that several of the major remaining 5.2 features are currently being developed in long-running feature branches also helps, as work can continue there at the cost of a rebase later.
How does this affect 5.1.4, which is also listed as a Fix Version for PHOENIX-7106? From the bug description it also sounds like 5.1.3 and the forthcoming .4 are affected, since we have server-side paging in 5.1. (Feel free to move that to a separate thread if you feel it should be a separate discussion.) Should this be a blocker for releasing 5.1.4? Geoffrey On Wed, Jan 3, 2024 at 5:06 PM Kadir Ozdemir <ka...@gsuite.cloud.apache.org> wrote: > Being a database, Phoenix has to make sure that the data stays on disk > intact and its queries return correct data. In this case, Phoenix fails to > return correct data for some queries if their scans experience region > movement. Now that we know these data integrity issues and how to reproduce > them, fixing them should be our first priority. So, I fully support this > proposal. > > On Wed, Jan 3, 2024 at 10:58 PM Viraj Jasani <vjas...@apache.org> wrote: > > > Hello, > > > > I would like to bring PHOENIX-7106 > > <https://issues.apache.org/jira/browse/PHOENIX-7106> to everyone's > > attention here and brief about the data integrity issues that we have in > > various coprocessors. Majority of the issues are related to the fact that > > we do not return valid rowkey for certain queries. If any region moves in > > the middle of the scan, the HBase client relies on the last returned > rowkey > > and accordingly changes the scan boundaries while the scanner is getting > > reset to continue the scan operation. If the region does not move, scan > is > > not expected to return invalid data, however if the region moves in the > > middle of ongoing scan operation, scan would return invalid/incorrect > data > > causing data integrity issues. > > > > Given the critical nature of these issues, I would like to propose that > we > > treat this as a high priority for the upcoming 5.2.0 release, and not > > include any other feature or big change to master branch until we merge > > this. The PR is still not ready as additional changes are still in my > > local, requiring rebase with the current master. > > > > I would get back to this discuss thread as soon as the PR and the doc are > > updated with the latest findings so far. The changes include many of our > > coproc scanner implementations and hence it would require significant > > review as well. > > It would be great if we can hold on to merging any feature or big change > to > > master branch until this gets in so as to not complicate > merging/rebasing. > > Once this is merged to the master branch, I would like to cut 5.2 branch > > from master and we can move forward with 5.2.0 release. > > > > Please let me know if this looks good or if you have any other high > > priority work for 5.2.0. > > >
