Ok Jim, it's all yours. Thanks for stepping in. I'll delete RC2 from svn when it finishes uploading.
Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Apr 20, 2017 at 4:46 PM, jim ferenczi <[email protected]> wrote: > I can do it Joel. As Mike said, this is a bad bug that can lead to data > lost... > > 2017-04-20 22:43 GMT+02:00 Joel Bernstein <[email protected]>: > >> I'm just uploading RC2 now. We can respin, but I won't be available for a >> respin this evening or tomorrow. If someone else wants to step in thats ok. >> >> Joel Bernstein >> http://joelsolr.blogspot.com/ >> >> On Thu, Apr 20, 2017 at 4:28 PM, Michael McCandless < >> [email protected]> wrote: >> >>> Hi Joel, >>> >>> LUCENE-7791 is actually quite a bad bug, and the patch looks low risk >>> and committable; maybe we should merge it for 6.5.1? >>> >>> Mike McCandless >>> >>> http://blog.mikemccandless.com >>> >>> On Thu, Apr 20, 2017 at 3:25 PM, Steve Rowe <[email protected]> wrote: >>> >>>> Joel, >>>> >>>> I’ve created <https://issues.apache.org/jira/browse/LUCENE-7793> and < >>>> https://issues.apache.org/jira/browse/LUCENE-7794> to add precommit >>>> checking (or the equivalent) to buildAndPushRelease.py and >>>> smokeTestRelease.py, respectively. >>>> >>>> -- >>>> Steve >>>> www.lucidworks.com >>>> >>>> > On Apr 20, 2017, at 10:29 AM, Steve Rowe <[email protected]> wrote: >>>> > >>>> > I was thinking that if buildAndPushRelease.py can identify problems >>>> early, the RM won’t have to later redo work when a problem is found, e.g. >>>> by the smoke tester. IMHO the extra 10 minutes (on my laptop anyway) from >>>> auto-running precommit would be worth it. >>>> > >>>> > But I agree that the smoke tester should be heavy, since it’s >>>> possible that the RM had weird local conditions that weren’t caught by the >>>> release production tools. >>>> > >>>> > -- >>>> > Steve >>>> > www.lucidworks.com >>>> > >>>> >> On Apr 20, 2017, at 10:23 AM, Joel Bernstein <[email protected]> >>>> wrote: >>>> >> >>>> >> They are both quite long processes. Perhaps we should look at >>>> eliminating some of the overlap? >>>> >> >>>> >> Since the smokeTesters is run by those voting on the release maybe >>>> this should be the heavier of the two. >>>> >> >>>> >> Joel Bernstein >>>> >> http://joelsolr.blogspot.com/ >>>> >> >>>> >> On Thu, Apr 20, 2017 at 10:00 AM, Steve Rowe <[email protected]> >>>> wrote: >>>> >> Also, I think buildAndPushRelease.py should invoke “precommit" too, >>>> (near) where it runs 'ant clean test'. >>>> >> >>>> >> -- >>>> >> Steve >>>> >> www.lucidworks.com >>>> >> >>>> >>> On Apr 20, 2017, at 9:51 AM, Steve Rowe <[email protected]> wrote: >>>> >>> >>>> >>> That’s bizarre. >>>> >>> >>>> >>> The smoke tester runs ‘ant validate’, which does *not* include >>>> "documentation-lint” (which includes "-ecj-javadoc-lint”, where unused >>>> imports are looked for). >>>> >>> >>>> >>> By contrast, ‘ant precommit’ runs “check-working-copy”, “validate”, >>>> and “documentation-lint”. >>>> >>> >>>> >>> Jenkins caught this not because of the smoke tester, but rather >>>> because the “nightly-smoke” task it uses runs “-jenkins-base”, which, like >>>> “precommit”, runs “check-working-copy”, “validate”, and >>>> “documentation-lint” (among others). >>>> >>> >>>> >>> I think we should change the smoke tester to look at the same >>>> things as precommit, >>>> >>> >>>> >>> -- >>>> >>> Steve >>>> >>> www.lucidworks.com >>>> >>> >>>> >>>> On Apr 20, 2017, at 9:29 AM, Joel Bernstein <[email protected]> >>>> wrote: >>>> >>>> >>>> >>>> Smoke testes passed as well >>>> >>>> >>>> >>>> >>>> >>>> Joel Bernstein >>>> >>>> http://joelsolr.blogspot.com/ >>>> >>>> >>>> >>>> On Thu, Apr 20, 2017 at 9:21 AM, Steve Rowe <[email protected]> >>>> wrote: >>>> >>>> I guess precommit is only run as part of the smoke tester. >>>> >>>> >>>> >>>>> On Apr 20, 2017, at 9:20 AM, Joel Bernstein <[email protected]> >>>> wrote: >>>> >>>>> >>>> >>>>> Interesting the entire release process completed without >>>> complaining. >>>> >>>>> >>>> >>>>> Joel Bernstein >>>> >>>>> http://joelsolr.blogspot.com/ >>>> >>>>> >>>> >>>>> On Thu, Apr 20, 2017 at 9:10 AM, Steve Rowe <[email protected]> >>>> wrote: >>>> >>>>> The unused import was the only problem; I’ve removed it on >>>> branch_6_5. >>>> >>>>> >>>> >>>>> Sorry, hope I didn’t cause a delay in the release process. >>>> >>>>> >>>> >>>>> -- >>>> >>>>> Steve >>>> >>>>> www.lucidworks.com >>>> >>>>> >>>> >>>>>> On Apr 20, 2017, at 8:57 AM, Steve Rowe <[email protected]> >>>> wrote: >>>> >>>>>> >>>> >>>>>> Crap, Jenkins found an unused import in my commit on branch_6_5 >>>> :(. I forgot to run precommit. Doing so now. >>>> >>>>>> >>>> >>>>>> -- >>>> >>>>>> Steve >>>> >>>>>> www.lucidworks.com >>>> >>>>>> >>>> >>>>>>> On Apr 19, 2017, at 8:07 PM, Steve Rowe <[email protected]> >>>> wrote: >>>> >>>>>>> >>>> >>>>>>> Done. >>>> >>>>>>> >>>> >>>>>>> -- >>>> >>>>>>> Steve >>>> >>>>>>> www.lucidworks.com >>>> >>>>>>> >>>> >>>>>>>> On Apr 19, 2017, at 7:49 PM, Steve Rowe <[email protected]> >>>> wrote: >>>> >>>>>>>> >>>> >>>>>>>> Woohoo! Thanks Joel. Running all tests on branch_6_5 now. >>>> >>>>>>>> >>>> >>>>>>>> -- >>>> >>>>>>>> Steve >>>> >>>>>>>> www.lucidworks.com >>>> >>>>>>>> >>>> >>>>>>>>> On Apr 19, 2017, at 7:20 PM, Joel Bernstein < >>>> [email protected]> wrote: >>>> >>>>>>>>> >>>> >>>>>>>>> I'm OK with restarting RC2. Let me know when you finish the >>>> back port. >>>> >>>>>>>>> >>>> >>>>>>>>> Joel Bernstein >>>> >>>>>>>>> http://joelsolr.blogspot.com/ >>>> >>>>>>>>> >>>> >>>>>>>>> On Wed, Apr 19, 2017 at 7:11 PM, Steve Rowe <[email protected]> >>>> wrote: >>>> >>>>>>>>> Joel, >>>> >>>>>>>>> >>>> >>>>>>>>> I know you’re in-process with RC2, but if you have to restart >>>> for some reason, I’d like to include SOLR-10527. >>>> >>>>>>>>> >>>> >>>>>>>>> -- >>>> >>>>>>>>> Steve >>>> >>>>>>>>> www.lucidworks.com >>>> >>>>>>>>> >>>> >>>>>>>>>> On Apr 19, 2017, at 6:57 PM, Joel Bernstein < >>>> [email protected]> wrote: >>>> >>>>>>>>>> >>>> >>>>>>>>>> I also had to change the smoke tester because of the SSL >>>> issue. Which in theory means everyone else will have to as well. I'm >>>> pushing out my changes to branch_6_5, so people can just update their >>>> branch before running the smoke tester. >>>> >>>>>>>>>> >>>> >>>>>>>>>> Joel Bernstein >>>> >>>>>>>>>> http://joelsolr.blogspot.com/ >>>> >>>>>>>>>> >>>> >>>>>>>>>> On Wed, Apr 19, 2017 at 5:02 PM, Joel Bernstein < >>>> [email protected]> wrote: >>>> >>>>>>>>>> Ok, I had to push out three changes to the branch but I >>>> seemed to have moved passed this issue. >>>> >>>>>>>>>> >>>> >>>>>>>>>> Joel Bernstein >>>> >>>>>>>>>> http://joelsolr.blogspot.com/ >>>> >>>>>>>>>> >>>> >>>>>>>>>> On Wed, Apr 19, 2017 at 4:50 PM, Joel Bernstein < >>>> [email protected]> wrote: >>>> >>>>>>>>>> I'll push out the changes and then revert. Hopefully the >>>> changes will do the trick after they've been pushed. >>>> >>>>>>>>>> >>>> >>>>>>>>>> Joel Bernstein >>>> >>>>>>>>>> http://joelsolr.blogspot.com/ >>>> >>>>>>>>>> >>>> >>>>>>>>>> On Wed, Apr 19, 2017 at 4:37 PM, Steve Rowe < >>>> [email protected]> wrote: >>>> >>>>>>>>>> Yeah, that is a sucky part of this: you want to make >>>> modifications on the release branch, but you can’t test unless you commit >>>> (and push). >>>> >>>>>>>>>> >>>> >>>>>>>>>> Another alternative: make modifications in another checkout, >>>> then invoke the script with from an unmodified source tree. >>>> >>>>>>>>>> >>>> >>>>>>>>>> e.g.: >>>> >>>>>>>>>> >>>> >>>>>>>>>> cd .. >>>> >>>>>>>>>> cp -r lucene-solr lucene-solr-2 # assuming lucene-solr is >>>> your checkout dir >>>> >>>>>>>>>> # edit lucene-solr-2/dev-tools/script >>>> s/buildAndPushRelease.py >>>> >>>>>>>>>> cd lucene-solr >>>> >>>>>>>>>> python3 -u >>>> >>>>>>>>>> ../lucene-solr-2/dev-tools/scripts/buildAndPushRelease.py >>>> … >>>> >>>>>>>>>> >>>> >>>>>>>>>> (a shorter form of that is to make a copy the script alone >>>> and invoke the modified version from your checkout) >>>> >>>>>>>>>> >>>> >>>>>>>>>> -- >>>> >>>>>>>>>> Steve >>>> >>>>>>>>>> www.lucidworks.com >>>> >>>>>>>>>> >>>> >>>>>>>>>>> On Apr 19, 2017, at 4:32 PM, Joel Bernstein < >>>> [email protected]> wrote: >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> Ha, that doesn't work either. Now I get this error: >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> RuntimeError: There are unpushed commits - "git log >>>> origin/branch_6_5.." output is: >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> So, I'm going to have to push this out. >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> Let's first decide if this makes sense as the way forward. >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> Joel Bernstein >>>> >>>>>>>>>>> http://joelsolr.blogspot.com/ >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> On Wed, Apr 19, 2017 at 4:28 PM, Joel Bernstein < >>>> [email protected]> wrote: >>>> >>>>>>>>>>> Steve, when I change the script to turn off verification I >>>> get the following error: >>>> >>>>>>>>>>> RuntimeError: git clone is dirty: >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> So as part of the work around I think I'm going to have to >>>> commit locally and then revert locally. Does that make sense to you as the >>>> way forward? >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> Joel Bernstein >>>> >>>>>>>>>>> http://joelsolr.blogspot.com/ >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> On Wed, Apr 19, 2017 at 4:24 PM, Alexandre Rafalovitch < >>>> [email protected]> wrote: >>>> >>>>>>>>>>> Could it be anything to do with the fact that the signer is >>>> Semantic >>>> >>>>>>>>>>> and they are being actively distrusted by - at least - >>>> Google: >>>> >>>>>>>>>>> http://www.securityweek.com/go >>>> ogle-stops-trusting-symantec-issued-certificates >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> Regards, >>>> >>>>>>>>>>> Alex. >>>> >>>>>>>>>>> ---- >>>> >>>>>>>>>>> http://www.solr-start.com/ - Resources for Solr users, new >>>> and experienced >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> On 19 April 2017 at 16:22, Joel Bernstein < >>>> [email protected]> wrote: >>>> >>>>>>>>>>>> Ok, I'll turn off the cert verification. I wasn't sure if >>>> cert verification >>>> >>>>>>>>>>>> was something that was integral to the process. >>>> >>>>>>>>>>>> >>>> >>>>>>>>>>>> Joel Bernstein >>>> >>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>> >>>>>>>>>>>> >>>> >>>>>>>>>>>> On Wed, Apr 19, 2017 at 4:18 PM, Steve Rowe < >>>> [email protected]> wrote: >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> Hi Joel, >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> Not sure why this is suddenly an issue - I guess “SNI” >>>> >>>>>>>>>>>>> <https://en.wikipedia.org/wiki/Server_Name_Indication> >>>> has been enabled on >>>> >>>>>>>>>>>>> archive.apache.org? >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> Some useful info here (about a “requests” lib, AFAICT an >>>> alternative to >>>> >>>>>>>>>>>>> urllib): >>>> >>>>>>>>>>>>> <http://docs.python-requests.o >>>> rg/en/master/community/faq/#what-are-hostname-doesn-t-match-errors> >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> Short term, you could turn off certificate verification. >>>> The bottom >>>> >>>>>>>>>>>>> answer here uses the same lib as the script (urllib >>>> instead of urllib2, >>>> >>>>>>>>>>>>> which is assumed in the other answers on the page), to >>>> turn off certificate >>>> >>>>>>>>>>>>> verification: >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> <http://stackoverflow.com/ques >>>> tions/19268548/python-ignore-certicate-validation-urllib2> >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> -- >>>> >>>>>>>>>>>>> Steve >>>> >>>>>>>>>>>>> www.lucidworks.com >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> On Apr 19, 2017, at 3:43 PM, Joel Bernstein < >>>> [email protected]> wrote: >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> It appears that it's failing on verifying the SSL cert >>>> for >>>> >>>>>>>>>>>>>> https://archive.apache.org/dist/lucene/java/. >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> Joel Bernstein >>>> >>>>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> On Wed, Apr 19, 2017 at 3:36 PM, Joel Bernstein < >>>> [email protected]> >>>> >>>>>>>>>>>>>> wrote: >>>> >>>>>>>>>>>>>> I started working on 6.5.1 RC2. I ran the following: >>>> >>>>>>>>>>>>>> python3 -u dev-tools/scripts/buildAndPushRelease.py >>>> --push-local >>>> >>>>>>>>>>>>>> /tmp/releases/6.5.1 --rc-num 2 --sign EE64CB1E >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> I believe this is same basic command I used for RC1. But >>>> this time I got >>>> >>>>>>>>>>>>>> a new error. At first I thought it might be incorrect >>>> keystore password but >>>> >>>>>>>>>>>>>> I changed it and it is correct. Any thoughts what the >>>> issue is? >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> Traceback (most recent call last): >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line >>>> 313, in <module> >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> main() >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line >>>> 294, in main >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> rev = prepare(c.root, c.version, c.key_id, >>>> c.key_password) >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line >>>> 98, in prepare >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> checkDOAPfiles(version) >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line >>>> 143, in >>>> >>>>>>>>>>>>>> checkDOAPfiles >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> distpage = load(url) >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line >>>> 67, in load >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> content = urllib.request.urlopen(urlStri >>>> ng).read().decode('utf-8') >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> File >>>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>>> >>>>>>>>>>>>>> line 223, in urlopen >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> return opener.open(url, data, timeout) >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> File >>>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>>> >>>>>>>>>>>>>> line 526, in open >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> response = self._open(req, data) >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> File >>>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>>> >>>>>>>>>>>>>> line 544, in _open >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> '_open', req) >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> File >>>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>>> >>>>>>>>>>>>>> line 504, in _call_chain >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> result = func(*args) >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> File >>>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>>> >>>>>>>>>>>>>> line 1361, in https_open >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> context=self._context, check_hostname=self._check_hos >>>> tname) >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> File >>>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>>> >>>>>>>>>>>>>> line 1320, in do_open >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> raise URLError(err) >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> urllib.error.URLError: <urlopen error [SSL: >>>> CERTIFICATE_VERIFY_FAILED] >>>> >>>>>>>>>>>>>> certificate verify failed (_ssl.c:749)> >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>>> Joel Bernstein >>>> >>>>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>> >>>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> ------------------------------ >>>> --------------------------------------- >>>> >>>>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>> >>>>>>>>>>>>> For additional commands, e-mail: >>>> [email protected] >>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> ------------------------------ >>>> --------------------------------------- >>>> >>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>> >>>>>>>>>>> For additional commands, e-mail: [email protected] >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> >>>>>>>>>> >>>> >>>>>>>>>> >>>> >>>>>>>>>> ------------------------------------------------------------ >>>> --------- >>>> >>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>> >>>>>>>>>> For additional commands, e-mail: [email protected] >>>> >>>>>>>>>> >>>> >>>>>>>>>> >>>> >>>>>>>>>> >>>> >>>>>>>>>> >>>> >>>>>>>>> >>>> >>>>>>>>> >>>> >>>>>>>>> ------------------------------------------------------------ >>>> --------- >>>> >>>>>>>>> To unsubscribe, e-mail: [email protected] >>>> >>>>>>>>> For additional commands, e-mail: [email protected] >>>> >>>>>>>>> >>>> >>>>>>>>> >>>> >>>>>>>> >>>> >>>>>>> >>>> >>>>>> >>>> >>>>> >>>> >>>>> >>>> >>>>> ------------------------------------------------------------ >>>> --------- >>>> >>>>> To unsubscribe, e-mail: [email protected] >>>> >>>>> For additional commands, e-mail: [email protected] >>>> >>>>> >>>> >>>>> >>>> >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------------ >>>> --------- >>>> >>>> To unsubscribe, e-mail: [email protected] >>>> >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>>> >>>> >>>> >>> >>>> >> >>>> >> >>>> >> ------------------------------------------------------------ >>>> --------- >>>> >> To unsubscribe, e-mail: [email protected] >>>> >> For additional commands, e-mail: [email protected] >>>> >> >>>> >> >>>> > >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>> >> >
