Lars, I agree 100% that we should have smaller, more frequent releases going forward. As for this release, I have two concerns.
The first is indexes. I've added several JIRAs that had been incorrectly not marked with a Fix Version to 4.15 / 5.1. These are all part of the Self-Repairing Index project, which spans several JIRAs and whose first major one (PHOENIX-5156, allowing newly created mutable indexes to self-repair inconsistencies at read time) is already in 4.15 and 5.1. Outstanding JIRAs include PHOENIX-5211 to extend the logic to immutable indexes, and PHOENIX-5333, to give users a tool to convert their legacy indexes to the new model. These are all under review and should land very soon. Especially given the multiple reports on the user list of operators encountering index consistency problems (which I have also seen in my own environments), I think it's important that our next release include these fixes, and that they go out in a unified way. The second concern is testing, particularly upgrade, perf and chaos testing. In addition to the large index changes (for which I know some perf work and live-cluster testing has been done, with more planned), there are other major changes in 4.15 such as the splittable system catalog. If all the issues on the current list were fixed, I'd still be reluctant to put the bits into production without more due diligence. We've released binaries with significant regressions in them that were missed in our test suites before, and it's important to avoid that this time. Yet Lars's point that we've waited far too long to release is of course correct. Perhaps the solution is to do what the HBase community did when the 2.x branch dragged out too long, and after the listed issues are Fixed, we release an explicit beta, closed to new features, from which a final release can graduate. In parallel, we could release a 4.14.3 with just the index changes and the current diff from 4.14.2 so users get those faster. Or maybe our testing's advanced further than I know about, and we're closer to green than I think. Happy to hear everyone's thoughts. Geoffrey On Thu, Jun 27, 2019 at 10:26 AM [email protected] <[email protected]> wrote: > Hi all, > we're getting close. The test suite is passing fairly reliably now.(minus > some strange failure to archive the artifact in -1.4 and PartialCommitIT > failing in -1.3 only). > I put a lot of effort into speeding up the tests and making them pass. > Let's please (pretty please :) ) keep it that way.A passing, comprehensive > test suite is key to frequent releases. > > I also committed and push some issues to 4.15.1 and 5.1.1 already. But I > can't do it alone. > > There are 14 items to go for 4.15.0. Some of those are potentially serious. > https://issues.apache.org/jira/issues/?jql=project%20%3D%20PHOENIX%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20%22Patch%20Available%22)%20AND%20fixVersion%20%3D%204.15.0 > > And 26 items for 5.1.0 > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20PHOENIX%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)%20AND%20fixVersion%20%3D%205.1.0 > > Let's make a final push and get these done (or moved to 4.15.1/5.1.1, > resp)If you have any issues open, please either get them committed to move > them to the next release. > > And then let's try to never get into this situation again where we have a > huge unreleased (and unreleasable) code base with 100's or 1000's of > unreleased changes. > Thanks! > -- Lars >
