Hi! A question of process: Should we backport fixes to the 4.16 branch at our discretion, or is backporting those handled by the release manager (Xinyi) ?
On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com> wrote: > Hi! > > Thanks for everyone's effort on fixing the flakey tests. > However, our ASF Jenkins runs are still very unstable, and I am afraid > that the single clean 4.x run was down more to luck than to fixing the > underlying problem. > While I do not consider this a blocker, fixing this would make the pre- > and post commit tests far more useful, and let us start with a clean slate > for the next release. > I have created https://issues.apache.org/jira/browse/PHOENIX-6288 to > track the generic cluster setup issue, but my attempts to fix this, or at > least figure out the root cause have been unsuccessful so far. > > I ask for your help in fixing this issue. I have added some inconclusive > analysis to the ticket, and the full failsafe output directory is > downloadable as artifacts from the failed multibranch runs (stored for a > few days), which should hold the answer to this riddle. > > regards, > Istvan > > On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <yanxi...@apache.org> wrote: > >> Hi, >> >> We finally have a stable 4.x branch build after many flapper test fixing >> contributions from the community. I'm going to fork a new branch(4.16) >> from >> the current 4.x branch for the 4.16 release. As a cutoff, I will not >> include any new features other than PHOENIX-6211 and bug fix. Please let >> me >> know if you have any questions or concerns. >> >> Thanks, >> Xinyi >> >> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vjas...@apache.org> wrote: >> >> > An update on test stability: >> > >> > As per recent 4.x build results, we are left with very few flappers and >> > specifically >> > with HBase profile 1.6, I can see recent 7 build (multibranch + PR >> > precommit results) >> > results without any test failure. >> > >> > >> > On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <yanxi...@apache.org> wrote: >> > >> > > Yes, we are currently working on fixing the 4.x branch test flappers >> > while >> > > waiting for the PHOENIX-5435. After that, I will try to have an RC >> ASAP. >> > > >> > > On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal < >> ankitsingha...@gmail.com> >> > > wrote: >> > > >> > > > I see that both the blockers listed here PHOENIX-5712 and >> PHOENIX-6241 >> > > > have been resolved(Thanks to Xinyi), and also as per the JIRA query >> > there >> > > > is no Jira marked as a blocker for 4.16 except the one related to >> > > > documentation >> > > > for "splittable catalog table". >> > > > >> > > > Xinyi, so are we good to start the release process now? >> > > > >> > > > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <yanxi...@apache.org> >> wrote: >> > > > >> > > > > Thanks for replying and providing suggestions. I looked at the >> wrong >> > > > result >> > > > > Jira list that Daniel provided and did some local testing, and >> here >> > is >> > > > the >> > > > > result: >> > > > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642: cannot >> > > reproduce >> > > > > it. >> > > > > [More information is required] PHOENIX-4504 cannot reproduce it >> but >> > > > someone >> > > > > claimed that he had a similar issue. >> > > > > [unusual query] PHOENIX-4540 and PHOENIX-6217 >> > > > > >> > > > > Based on my finding, I think it's better to have more frequent >> > > > housekeeping >> > > > > and resolve unreproducible bugs especially since many of them are >> > > > > considering out of date (phoenix-4.11 or even phoenix-4.6). Since >> I >> > > still >> > > > > need time to work on the blocker Jira(PHOENIX-5712, PHOENIX-6241) >> and >> > > fix >> > > > > test flappers, if you want to fix "unusual query" bugs, feel free >> to >> > do >> > > > so. >> > > > > >> > > > > >> > > > > Sincerely, >> > > > > Xinyi >> > > > > >> > > > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <an...@apache.org> >> > > wrote: >> > > > > >> > > > > > Thanks Daniel and appreciate the effort you put in getting the >> list >> > > > ready >> > > > > > for bugs producing wrong results >> > > > > > but none of them seems to be a blocker to me for 4.16 as they >> are >> > not >> > > > the >> > > > > > regression and doesn't break the general functionality >> > > > > > except for specific features, RVC/desc as Chenglei also pointed >> out >> > > > > (though >> > > > > > I'll defer the assessment to RM "Xinyi"). >> > > > > > Probably these can be a part of 4.16.1 or we can do 4.17.0 soon >> > maybe >> > > > > after >> > > > > > a few weeks/month? >> > > > > > >> > > > > > Considering that we have already fixed 137 bugs and done 85+ >> > > > > > improvements/features in 4.16, >> > > > > > it will not be a good idea to deprive the user from such fixes. >> > > > > > It's been a year since our last 4.15 release, having no release >> > > brings >> > > > > more >> > > > > > questions on the project >> > > > > > rather than the bugs which affect a certain % of feature/users, >> > would >> > > > the >> > > > > > release notes >> > > > > > explaining the stability of certain features set the right >> > > expectation >> > > > > for >> > > > > > those users who rely on these features to wait for a future >> > release? >> > > > > > >> > > > > > Regards, >> > > > > > Ankit Singhal >> > > > > > >> > > > > > On Tue, Dec 1, 2020 at 8:21 PM cheng...@apache.org < >> > > > cheng...@apache.org> >> > > > > > wrote: >> > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > In my opinion, we should keep releases light and frequent, >> and >> > for >> > > > > some >> > > > > > > unusual query bugs like RVC and DESC >> > > > > > > we could delay fix to next release . I think we should release >> > > 4.16.0 >> > > > > and >> > > > > > > 5.1.0 as quickly as possible. In China, many users >> > > > > > > in HBase&Phoenix User Group thought that Phoenix was dead >> > because >> > > > our >> > > > > > too >> > > > > > > long interval release and stopped using >> > > > > > > Phoenix. >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" < >> > > > chinmayskulka...@gmail.com >> > > > > > >> > > > > > > wrote: >> > > > > > > >I agree. These are all major bugs and we should aim at >> solving >> > > them >> > > > > > after >> > > > > > > >checking that they are still issues. I am +1 on 5833 and I >> think >> > > > 5484 >> > > > > > > would >> > > > > > > >be a great addition to 4.16 as well. We should aim at >> resolving >> > > high >> > > > > > > >priority bugs like this in every release. >> > > > > > > > >> > > > > > > >Sometimes we let these bugs slip without a resolution before >> a >> > > > > release, >> > > > > > > >citing that these are "known issues" or "not regressions from >> > the >> > > > last >> > > > > > > >release". In some cases this may be fine since we want to >> keep >> > > > > releases >> > > > > > > >light and frequent, but perhaps we can track such issues and >> aim >> > > at >> > > > > > > >reducing the number of bugs by x% in each release? This will >> > also >> > > > keep >> > > > > > old >> > > > > > > >Jiras alive since we will potentially periodically review >> them. >> > > > > > > > >> > > > > > > > >> > > > > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby < >> > > gjac...@apache.org> >> > > > > > > wrote: >> > > > > > > > >> > > > > > > >> I've got PHOENIX-5435 in review right now, and would like >> to >> > get >> > > > it >> > > > > in >> > > > > > > 4.16 >> > > > > > > >> / 5.1. >> > > > > > > >> >> > > > > > > >> It's allowing the annotation of Phoenix metadata into HBase >> > WALs >> > > > as >> > > > > a >> > > > > > > >> pre-req for the Phoenix Change Detection Capture framework >> > > > > > > (PHOENIX-5442). >> > > > > > > >> Since it has both client/server logic, and adds a field to >> > > > > > > System.Catalog, >> > > > > > > >> it can't go in a patch release. >> > > > > > > >> >> > > > > > > >> Depending on timing, I'd _like_ to get PHOENIX-6227, which >> is >> > > the >> > > > > last >> > > > > > > part >> > > > > > > >> of CDC that will go into core Phoenix, into 4.16, but since >> > that >> > > > > _can_ >> > > > > > > go >> > > > > > > >> in a patch release and I haven't started it yet, if the >> > release >> > > > gets >> > > > > > cut >> > > > > > > >> before it's ready, no big deal. (The rest of CDC will go >> into >> > > > > > > >> phoenix-connectors for a future release of that project.) >> > > > > > > >> >> > > > > > > >> As for the correctness problems that Daniel points out, I >> > think >> > > we >> > > > > > > should >> > > > > > > >> fix the ones that were detected with a recent version >> (4.14 or >> > > > > 4.15?), >> > > > > > > and >> > > > > > > >> test to see which of the older ones can still be >> reproduced. >> > > Once >> > > > we >> > > > > > > know >> > > > > > > >> which bugs are real and which are just historical, we can >> > better >> > > > > judge >> > > > > > > >> scope. And hopefully close a bunch of obsolete bugs. >> (Thanks, >> > > > > Daniel, >> > > > > > > for >> > > > > > > >> collecting that list!) >> > > > > > > >> >> > > > > > > >> Geoffrey >> > > > > > > >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong >> > > > > > > >> <daniel.w...@salesforce.com.invalid> wrote: >> > > > > > > >> >> > > > > > > >> > Hi, I wanted to bring up wrong results in Phoenix and >> some >> > > JIRAs >> > > > > > > around >> > > > > > > >> > them that I think we should fix as the wrong result >> lessens >> > > the >> > > > > end >> > > > > > > >> user's >> > > > > > > >> > trust in Phoenix. Releasing a new version without >> > addressing >> > > > > these >> > > > > > > in a >> > > > > > > >> > minor release hurts our visibility in that these critical >> > > issues >> > > > > are >> > > > > > > not >> > > > > > > >> > addressed. >> > > > > > > >> > >> > > > > > > >> > Jira's that I'm involved with for example: I've already >> > given >> > > a >> > > > > > patch >> > > > > > > >> > several months ago for 5833 and there is a chance it may >> fix >> > > > 5484. >> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5833 >> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5484 >> > > > > > > >> > >> > > > > > > >> > In addition, inspecting apache JIRA i see several other >> > wrong >> > > > > result >> > > > > > > >> JIRAs >> > > > > > > >> > from the community. Some of these certainly are probably >> > old >> > > > > issues >> > > > > > > or >> > > > > > > >> > incorrect understanding but some of these are opened by >> our >> > > own >> > > > > dev >> > > > > > > >> > community and are likely real problems. >> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-6217 >> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5571 >> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4642 >> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4540 >> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4504 >> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4419 >> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4116 >> > > > > > > >> > >> > > > > > > >> > What is our stance on this type of issue? Are we going >> to >> > say >> > > > > these >> > > > > > > were >> > > > > > > >> > issues prior to 4.15 and not address them? Should we >> have >> > > > > > > requirements >> > > > > > > >> for >> > > > > > > >> > our releases to fix wrong results? >> > > > > > > >> > >> > > > > > > >> > Daniel Wong >> > > > > > > >> > >> > > > > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan < >> > > yanxi...@apache.org> >> > > > > > > wrote: >> > > > > > > >> > >> > > > > > > >> > > Hi all, >> > > > > > > >> > > >> > > > > > > >> > > It's time to discuss the Phoenix 4.16 release. After >> many >> > > > > people's >> > > > > > > >> > > contributions on the bug fixes, new features, and other >> > > works >> > > > in >> > > > > > the >> > > > > > > >> past >> > > > > > > >> > > few months, we are kind of close to the point to have >> a RC >> > > > > (still >> > > > > > > need >> > > > > > > >> to >> > > > > > > >> > > fix test flappers). Please let me know if you think any >> > JIRA >> > > > > must >> > > > > > be >> > > > > > > >> part >> > > > > > > >> > > of the Phoenix 4.16 release other than major blocker >> > > > > PHOENIX-5712. >> > > > > > > >> > > >> > > > > > > >> > > If no surprise comes up, I will not wait for any new >> major >> > > > > > features >> > > > > > > and >> > > > > > > >> > > focus on the RC as soon as possible. >> > > > > > > >> > > >> > > > > > > >> > > Sincerely, >> > > > > > > >> > > Xinyi >> > > > > > > >> > > >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > -- >> > > > > > > >> > Daniel Wong >> > > > > > > >> > Salesforce >> > > > > > > >> > Mobile: 628.217.1808 >> > > > > > > >> > >> > > > > > > >> >> > > > > > > > >> > > > > > > > >> > > > > > > >-- >> > > > > > > >Chinmay Kulkarni >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > > > -- > *István Tóth* | Staff Software Engineer > st...@cloudera.com <https://www.cloudera.com> > [image: Cloudera] <https://www.cloudera.com/> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> > <https://www.cloudera.com/> > ------------------------------ >
