Hi Joel, LUCENE-7791 is actually quite a bad bug, and the patch looks low risk and committable; maybe we should merge it for 6.5.1?
Mike McCandless http://blog.mikemccandless.com On Thu, Apr 20, 2017 at 3:25 PM, Steve Rowe <[email protected]> wrote: > Joel, > > I’ve created <https://issues.apache.org/jira/browse/LUCENE-7793> and < > https://issues.apache.org/jira/browse/LUCENE-7794> to add precommit > checking (or the equivalent) to buildAndPushRelease.py and > smokeTestRelease.py, respectively. > > -- > Steve > www.lucidworks.com > > > On Apr 20, 2017, at 10:29 AM, Steve Rowe <[email protected]> wrote: > > > > I was thinking that if buildAndPushRelease.py can identify problems > early, the RM won’t have to later redo work when a problem is found, e.g. > by the smoke tester. IMHO the extra 10 minutes (on my laptop anyway) from > auto-running precommit would be worth it. > > > > But I agree that the smoke tester should be heavy, since it’s possible > that the RM had weird local conditions that weren’t caught by the release > production tools. > > > > -- > > Steve > > www.lucidworks.com > > > >> On Apr 20, 2017, at 10:23 AM, Joel Bernstein <[email protected]> > wrote: > >> > >> They are both quite long processes. Perhaps we should look at > eliminating some of the overlap? > >> > >> Since the smokeTesters is run by those voting on the release maybe this > should be the heavier of the two. > >> > >> Joel Bernstein > >> http://joelsolr.blogspot.com/ > >> > >> On Thu, Apr 20, 2017 at 10:00 AM, Steve Rowe <[email protected]> wrote: > >> Also, I think buildAndPushRelease.py should invoke “precommit" too, > (near) where it runs 'ant clean test'. > >> > >> -- > >> Steve > >> www.lucidworks.com > >> > >>> On Apr 20, 2017, at 9:51 AM, Steve Rowe <[email protected]> wrote: > >>> > >>> That’s bizarre. > >>> > >>> The smoke tester runs ‘ant validate’, which does *not* include > "documentation-lint” (which includes "-ecj-javadoc-lint”, where unused > imports are looked for). > >>> > >>> By contrast, ‘ant precommit’ runs “check-working-copy”, “validate”, > and “documentation-lint”. > >>> > >>> Jenkins caught this not because of the smoke tester, but rather > because the “nightly-smoke” task it uses runs “-jenkins-base”, which, like > “precommit”, runs “check-working-copy”, “validate”, and > “documentation-lint” (among others). > >>> > >>> I think we should change the smoke tester to look at the same things > as precommit, > >>> > >>> -- > >>> Steve > >>> www.lucidworks.com > >>> > >>>> On Apr 20, 2017, at 9:29 AM, Joel Bernstein <[email protected]> > wrote: > >>>> > >>>> Smoke testes passed as well > >>>> > >>>> > >>>> Joel Bernstein > >>>> http://joelsolr.blogspot.com/ > >>>> > >>>> On Thu, Apr 20, 2017 at 9:21 AM, Steve Rowe <[email protected]> wrote: > >>>> I guess precommit is only run as part of the smoke tester. > >>>> > >>>>> On Apr 20, 2017, at 9:20 AM, Joel Bernstein <[email protected]> > wrote: > >>>>> > >>>>> Interesting the entire release process completed without complaining. > >>>>> > >>>>> Joel Bernstein > >>>>> http://joelsolr.blogspot.com/ > >>>>> > >>>>> On Thu, Apr 20, 2017 at 9:10 AM, Steve Rowe <[email protected]> > wrote: > >>>>> The unused import was the only problem; I’ve removed it on > branch_6_5. > >>>>> > >>>>> Sorry, hope I didn’t cause a delay in the release process. > >>>>> > >>>>> -- > >>>>> Steve > >>>>> www.lucidworks.com > >>>>> > >>>>>> On Apr 20, 2017, at 8:57 AM, Steve Rowe <[email protected]> wrote: > >>>>>> > >>>>>> Crap, Jenkins found an unused import in my commit on branch_6_5 > :(. I forgot to run precommit. Doing so now. > >>>>>> > >>>>>> -- > >>>>>> Steve > >>>>>> www.lucidworks.com > >>>>>> > >>>>>>> On Apr 19, 2017, at 8:07 PM, Steve Rowe <[email protected]> wrote: > >>>>>>> > >>>>>>> Done. > >>>>>>> > >>>>>>> -- > >>>>>>> Steve > >>>>>>> www.lucidworks.com > >>>>>>> > >>>>>>>> On Apr 19, 2017, at 7:49 PM, Steve Rowe <[email protected]> wrote: > >>>>>>>> > >>>>>>>> Woohoo! Thanks Joel. Running all tests on branch_6_5 now. > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Steve > >>>>>>>> www.lucidworks.com > >>>>>>>> > >>>>>>>>> On Apr 19, 2017, at 7:20 PM, Joel Bernstein <[email protected]> > wrote: > >>>>>>>>> > >>>>>>>>> I'm OK with restarting RC2. Let me know when you finish the back > port. > >>>>>>>>> > >>>>>>>>> Joel Bernstein > >>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>> > >>>>>>>>> On Wed, Apr 19, 2017 at 7:11 PM, Steve Rowe <[email protected]> > wrote: > >>>>>>>>> Joel, > >>>>>>>>> > >>>>>>>>> I know you’re in-process with RC2, but if you have to restart > for some reason, I’d like to include SOLR-10527. > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Steve > >>>>>>>>> www.lucidworks.com > >>>>>>>>> > >>>>>>>>>> On Apr 19, 2017, at 6:57 PM, Joel Bernstein <[email protected]> > wrote: > >>>>>>>>>> > >>>>>>>>>> I also had to change the smoke tester because of the SSL issue. > Which in theory means everyone else will have to as well. I'm pushing out > my changes to branch_6_5, so people can just update their branch before > running the smoke tester. > >>>>>>>>>> > >>>>>>>>>> Joel Bernstein > >>>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>>> > >>>>>>>>>> On Wed, Apr 19, 2017 at 5:02 PM, Joel Bernstein < > [email protected]> wrote: > >>>>>>>>>> Ok, I had to push out three changes to the branch but I seemed > to have moved passed this issue. > >>>>>>>>>> > >>>>>>>>>> Joel Bernstein > >>>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>>> > >>>>>>>>>> On Wed, Apr 19, 2017 at 4:50 PM, Joel Bernstein < > [email protected]> wrote: > >>>>>>>>>> I'll push out the changes and then revert. Hopefully the > changes will do the trick after they've been pushed. > >>>>>>>>>> > >>>>>>>>>> Joel Bernstein > >>>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>>> > >>>>>>>>>> On Wed, Apr 19, 2017 at 4:37 PM, Steve Rowe <[email protected]> > wrote: > >>>>>>>>>> Yeah, that is a sucky part of this: you want to make > modifications on the release branch, but you can’t test unless you commit > (and push). > >>>>>>>>>> > >>>>>>>>>> Another alternative: make modifications in another checkout, > then invoke the script with from an unmodified source tree. > >>>>>>>>>> > >>>>>>>>>> e.g.: > >>>>>>>>>> > >>>>>>>>>> cd .. > >>>>>>>>>> cp -r lucene-solr lucene-solr-2 # assuming lucene-solr is your > checkout dir > >>>>>>>>>> # edit lucene-solr-2/dev-tools/scripts/buildAndPushRelease.py > >>>>>>>>>> cd lucene-solr > >>>>>>>>>> python3 -u > >>>>>>>>>> ../lucene-solr-2/dev-tools/scripts/buildAndPushRelease.py > … > >>>>>>>>>> > >>>>>>>>>> (a shorter form of that is to make a copy the script alone and > invoke the modified version from your checkout) > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Steve > >>>>>>>>>> www.lucidworks.com > >>>>>>>>>> > >>>>>>>>>>> On Apr 19, 2017, at 4:32 PM, Joel Bernstein < > [email protected]> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Ha, that doesn't work either. Now I get this error: > >>>>>>>>>>> > >>>>>>>>>>> RuntimeError: There are unpushed commits - "git log > origin/branch_6_5.." output is: > >>>>>>>>>>> > >>>>>>>>>>> So, I'm going to have to push this out. > >>>>>>>>>>> > >>>>>>>>>>> Let's first decide if this makes sense as the way forward. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Joel Bernstein > >>>>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>>>> > >>>>>>>>>>> On Wed, Apr 19, 2017 at 4:28 PM, Joel Bernstein < > [email protected]> wrote: > >>>>>>>>>>> Steve, when I change the script to turn off verification I get > the following error: > >>>>>>>>>>> RuntimeError: git clone is dirty: > >>>>>>>>>>> > >>>>>>>>>>> So as part of the work around I think I'm going to have to > commit locally and then revert locally. Does that make sense to you as the > way forward? > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Joel Bernstein > >>>>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>>>> > >>>>>>>>>>> On Wed, Apr 19, 2017 at 4:24 PM, Alexandre Rafalovitch < > [email protected]> wrote: > >>>>>>>>>>> Could it be anything to do with the fact that the signer is > Semantic > >>>>>>>>>>> and they are being actively distrusted by - at least - Google: > >>>>>>>>>>> http://www.securityweek.com/google-stops-trusting- > symantec-issued-certificates > >>>>>>>>>>> > >>>>>>>>>>> Regards, > >>>>>>>>>>> Alex. > >>>>>>>>>>> ---- > >>>>>>>>>>> http://www.solr-start.com/ - Resources for Solr users, new > and experienced > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> On 19 April 2017 at 16:22, Joel Bernstein <[email protected]> > wrote: > >>>>>>>>>>>> Ok, I'll turn off the cert verification. I wasn't sure if > cert verification > >>>>>>>>>>>> was something that was integral to the process. > >>>>>>>>>>>> > >>>>>>>>>>>> Joel Bernstein > >>>>>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>>>>> > >>>>>>>>>>>> On Wed, Apr 19, 2017 at 4:18 PM, Steve Rowe <[email protected]> > wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Hi Joel, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Not sure why this is suddenly an issue - I guess “SNI” > >>>>>>>>>>>>> <https://en.wikipedia.org/wiki/Server_Name_Indication> has > been enabled on > >>>>>>>>>>>>> archive.apache.org? > >>>>>>>>>>>>> > >>>>>>>>>>>>> Some useful info here (about a “requests” lib, AFAICT an > alternative to > >>>>>>>>>>>>> urllib): > >>>>>>>>>>>>> <http://docs.python-requests.org/en/master/community/faq/# > what-are-hostname-doesn-t-match-errors> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Short term, you could turn off certificate verification. > The bottom > >>>>>>>>>>>>> answer here uses the same lib as the script (urllib instead > of urllib2, > >>>>>>>>>>>>> which is assumed in the other answers on the page), to turn > off certificate > >>>>>>>>>>>>> verification: > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> <http://stackoverflow.com/questions/19268548/python- > ignore-certicate-validation-urllib2> > >>>>>>>>>>>>> > >>>>>>>>>>>>> -- > >>>>>>>>>>>>> Steve > >>>>>>>>>>>>> www.lucidworks.com > >>>>>>>>>>>>> > >>>>>>>>>>>>>> On Apr 19, 2017, at 3:43 PM, Joel Bernstein < > [email protected]> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> It appears that it's failing on verifying the SSL cert for > >>>>>>>>>>>>>> https://archive.apache.org/dist/lucene/java/. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Joel Bernstein > >>>>>>>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Wed, Apr 19, 2017 at 3:36 PM, Joel Bernstein < > [email protected]> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> I started working on 6.5.1 RC2. I ran the following: > >>>>>>>>>>>>>> python3 -u dev-tools/scripts/buildAndPushRelease.py > --push-local > >>>>>>>>>>>>>> /tmp/releases/6.5.1 --rc-num 2 --sign EE64CB1E > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I believe this is same basic command I used for RC1. But > this time I got > >>>>>>>>>>>>>> a new error. At first I thought it might be incorrect > keystore password but > >>>>>>>>>>>>>> I changed it and it is correct. Any thoughts what the issue > is? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Traceback (most recent call last): > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 313, > in <module> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> main() > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 294, > in main > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> rev = prepare(c.root, c.version, c.key_id, c.key_password) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 98, > in prepare > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> checkDOAPfiles(version) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 143, > in > >>>>>>>>>>>>>> checkDOAPfiles > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> distpage = load(url) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 67, > in load > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> content = urllib.request.urlopen( > urlString).read().decode('utf-8') > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> File > >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/ > python3.6/urllib/request.py", > >>>>>>>>>>>>>> line 223, in urlopen > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> return opener.open(url, data, timeout) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> File > >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/ > python3.6/urllib/request.py", > >>>>>>>>>>>>>> line 526, in open > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> response = self._open(req, data) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> File > >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/ > python3.6/urllib/request.py", > >>>>>>>>>>>>>> line 544, in _open > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> '_open', req) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> File > >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/ > python3.6/urllib/request.py", > >>>>>>>>>>>>>> line 504, in _call_chain > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> result = func(*args) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> File > >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/ > python3.6/urllib/request.py", > >>>>>>>>>>>>>> line 1361, in https_open > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> context=self._context, check_hostname=self._check_hostname) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> File > >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/ > python3.6/urllib/request.py", > >>>>>>>>>>>>>> line 1320, in do_open > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> raise URLError(err) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> urllib.error.URLError: <urlopen error [SSL: > CERTIFICATE_VERIFY_FAILED] > >>>>>>>>>>>>>> certificate verify failed (_ssl.c:749)> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Joel Bernstein > >>>>>>>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> ------------------------------------------------------------ > --------- > >>>>>>>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> ------------------------------------------------------------ > --------- > >>>>>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> ------------------------------------------------------------ > --------- > >>>>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ------------------------------------------------------------ > --------- > >>>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> ------------------------------------------------------------ > --------- > >>>>> To unsubscribe, e-mail: [email protected] > >>>>> For additional commands, e-mail: [email protected] > >>>>> > >>>>> > >>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: [email protected] > >>>> For additional commands, e-mail: [email protected] > >>>> > >>>> > >>> > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > >> > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
