I can do it Joel. As Mike said, this is a bad bug that can lead to data lost...
2017-04-20 22:43 GMT+02:00 Joel Bernstein <[email protected]>: > I'm just uploading RC2 now. We can respin, but I won't be available for a > respin this evening or tomorrow. If someone else wants to step in thats ok. > > Joel Bernstein > http://joelsolr.blogspot.com/ > > On Thu, Apr 20, 2017 at 4:28 PM, Michael McCandless < > [email protected]> wrote: > >> Hi Joel, >> >> LUCENE-7791 is actually quite a bad bug, and the patch looks low risk and >> committable; maybe we should merge it for 6.5.1? >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> On Thu, Apr 20, 2017 at 3:25 PM, Steve Rowe <[email protected]> wrote: >> >>> Joel, >>> >>> I’ve created <https://issues.apache.org/jira/browse/LUCENE-7793> and < >>> https://issues.apache.org/jira/browse/LUCENE-7794> to add precommit >>> checking (or the equivalent) to buildAndPushRelease.py and >>> smokeTestRelease.py, respectively. >>> >>> -- >>> Steve >>> www.lucidworks.com >>> >>> > On Apr 20, 2017, at 10:29 AM, Steve Rowe <[email protected]> wrote: >>> > >>> > I was thinking that if buildAndPushRelease.py can identify problems >>> early, the RM won’t have to later redo work when a problem is found, e.g. >>> by the smoke tester. IMHO the extra 10 minutes (on my laptop anyway) from >>> auto-running precommit would be worth it. >>> > >>> > But I agree that the smoke tester should be heavy, since it’s possible >>> that the RM had weird local conditions that weren’t caught by the release >>> production tools. >>> > >>> > -- >>> > Steve >>> > www.lucidworks.com >>> > >>> >> On Apr 20, 2017, at 10:23 AM, Joel Bernstein <[email protected]> >>> wrote: >>> >> >>> >> They are both quite long processes. Perhaps we should look at >>> eliminating some of the overlap? >>> >> >>> >> Since the smokeTesters is run by those voting on the release maybe >>> this should be the heavier of the two. >>> >> >>> >> Joel Bernstein >>> >> http://joelsolr.blogspot.com/ >>> >> >>> >> On Thu, Apr 20, 2017 at 10:00 AM, Steve Rowe <[email protected]> >>> wrote: >>> >> Also, I think buildAndPushRelease.py should invoke “precommit" too, >>> (near) where it runs 'ant clean test'. >>> >> >>> >> -- >>> >> Steve >>> >> www.lucidworks.com >>> >> >>> >>> On Apr 20, 2017, at 9:51 AM, Steve Rowe <[email protected]> wrote: >>> >>> >>> >>> That’s bizarre. >>> >>> >>> >>> The smoke tester runs ‘ant validate’, which does *not* include >>> "documentation-lint” (which includes "-ecj-javadoc-lint”, where unused >>> imports are looked for). >>> >>> >>> >>> By contrast, ‘ant precommit’ runs “check-working-copy”, “validate”, >>> and “documentation-lint”. >>> >>> >>> >>> Jenkins caught this not because of the smoke tester, but rather >>> because the “nightly-smoke” task it uses runs “-jenkins-base”, which, like >>> “precommit”, runs “check-working-copy”, “validate”, and >>> “documentation-lint” (among others). >>> >>> >>> >>> I think we should change the smoke tester to look at the same things >>> as precommit, >>> >>> >>> >>> -- >>> >>> Steve >>> >>> www.lucidworks.com >>> >>> >>> >>>> On Apr 20, 2017, at 9:29 AM, Joel Bernstein <[email protected]> >>> wrote: >>> >>>> >>> >>>> Smoke testes passed as well >>> >>>> >>> >>>> >>> >>>> Joel Bernstein >>> >>>> http://joelsolr.blogspot.com/ >>> >>>> >>> >>>> On Thu, Apr 20, 2017 at 9:21 AM, Steve Rowe <[email protected]> >>> wrote: >>> >>>> I guess precommit is only run as part of the smoke tester. >>> >>>> >>> >>>>> On Apr 20, 2017, at 9:20 AM, Joel Bernstein <[email protected]> >>> wrote: >>> >>>>> >>> >>>>> Interesting the entire release process completed without >>> complaining. >>> >>>>> >>> >>>>> Joel Bernstein >>> >>>>> http://joelsolr.blogspot.com/ >>> >>>>> >>> >>>>> On Thu, Apr 20, 2017 at 9:10 AM, Steve Rowe <[email protected]> >>> wrote: >>> >>>>> The unused import was the only problem; I’ve removed it on >>> branch_6_5. >>> >>>>> >>> >>>>> Sorry, hope I didn’t cause a delay in the release process. >>> >>>>> >>> >>>>> -- >>> >>>>> Steve >>> >>>>> www.lucidworks.com >>> >>>>> >>> >>>>>> On Apr 20, 2017, at 8:57 AM, Steve Rowe <[email protected]> wrote: >>> >>>>>> >>> >>>>>> Crap, Jenkins found an unused import in my commit on branch_6_5 >>> :(. I forgot to run precommit. Doing so now. >>> >>>>>> >>> >>>>>> -- >>> >>>>>> Steve >>> >>>>>> www.lucidworks.com >>> >>>>>> >>> >>>>>>> On Apr 19, 2017, at 8:07 PM, Steve Rowe <[email protected]> >>> wrote: >>> >>>>>>> >>> >>>>>>> Done. >>> >>>>>>> >>> >>>>>>> -- >>> >>>>>>> Steve >>> >>>>>>> www.lucidworks.com >>> >>>>>>> >>> >>>>>>>> On Apr 19, 2017, at 7:49 PM, Steve Rowe <[email protected]> >>> wrote: >>> >>>>>>>> >>> >>>>>>>> Woohoo! Thanks Joel. Running all tests on branch_6_5 now. >>> >>>>>>>> >>> >>>>>>>> -- >>> >>>>>>>> Steve >>> >>>>>>>> www.lucidworks.com >>> >>>>>>>> >>> >>>>>>>>> On Apr 19, 2017, at 7:20 PM, Joel Bernstein < >>> [email protected]> wrote: >>> >>>>>>>>> >>> >>>>>>>>> I'm OK with restarting RC2. Let me know when you finish the >>> back port. >>> >>>>>>>>> >>> >>>>>>>>> Joel Bernstein >>> >>>>>>>>> http://joelsolr.blogspot.com/ >>> >>>>>>>>> >>> >>>>>>>>> On Wed, Apr 19, 2017 at 7:11 PM, Steve Rowe <[email protected]> >>> wrote: >>> >>>>>>>>> Joel, >>> >>>>>>>>> >>> >>>>>>>>> I know you’re in-process with RC2, but if you have to restart >>> for some reason, I’d like to include SOLR-10527. >>> >>>>>>>>> >>> >>>>>>>>> -- >>> >>>>>>>>> Steve >>> >>>>>>>>> www.lucidworks.com >>> >>>>>>>>> >>> >>>>>>>>>> On Apr 19, 2017, at 6:57 PM, Joel Bernstein < >>> [email protected]> wrote: >>> >>>>>>>>>> >>> >>>>>>>>>> I also had to change the smoke tester because of the SSL >>> issue. Which in theory means everyone else will have to as well. I'm >>> pushing out my changes to branch_6_5, so people can just update their >>> branch before running the smoke tester. >>> >>>>>>>>>> >>> >>>>>>>>>> Joel Bernstein >>> >>>>>>>>>> http://joelsolr.blogspot.com/ >>> >>>>>>>>>> >>> >>>>>>>>>> On Wed, Apr 19, 2017 at 5:02 PM, Joel Bernstein < >>> [email protected]> wrote: >>> >>>>>>>>>> Ok, I had to push out three changes to the branch but I >>> seemed to have moved passed this issue. >>> >>>>>>>>>> >>> >>>>>>>>>> Joel Bernstein >>> >>>>>>>>>> http://joelsolr.blogspot.com/ >>> >>>>>>>>>> >>> >>>>>>>>>> On Wed, Apr 19, 2017 at 4:50 PM, Joel Bernstein < >>> [email protected]> wrote: >>> >>>>>>>>>> I'll push out the changes and then revert. Hopefully the >>> changes will do the trick after they've been pushed. >>> >>>>>>>>>> >>> >>>>>>>>>> Joel Bernstein >>> >>>>>>>>>> http://joelsolr.blogspot.com/ >>> >>>>>>>>>> >>> >>>>>>>>>> On Wed, Apr 19, 2017 at 4:37 PM, Steve Rowe <[email protected]> >>> wrote: >>> >>>>>>>>>> Yeah, that is a sucky part of this: you want to make >>> modifications on the release branch, but you can’t test unless you commit >>> (and push). >>> >>>>>>>>>> >>> >>>>>>>>>> Another alternative: make modifications in another checkout, >>> then invoke the script with from an unmodified source tree. >>> >>>>>>>>>> >>> >>>>>>>>>> e.g.: >>> >>>>>>>>>> >>> >>>>>>>>>> cd .. >>> >>>>>>>>>> cp -r lucene-solr lucene-solr-2 # assuming lucene-solr is >>> your checkout dir >>> >>>>>>>>>> # edit lucene-solr-2/dev-tools/scripts/buildAndPushRelease.py >>> >>>>>>>>>> cd lucene-solr >>> >>>>>>>>>> python3 -u >>> >>>>>>>>>> ../lucene-solr-2/dev-tools/scripts/buildAndPushRelease.py >>> … >>> >>>>>>>>>> >>> >>>>>>>>>> (a shorter form of that is to make a copy the script alone >>> and invoke the modified version from your checkout) >>> >>>>>>>>>> >>> >>>>>>>>>> -- >>> >>>>>>>>>> Steve >>> >>>>>>>>>> www.lucidworks.com >>> >>>>>>>>>> >>> >>>>>>>>>>> On Apr 19, 2017, at 4:32 PM, Joel Bernstein < >>> [email protected]> wrote: >>> >>>>>>>>>>> >>> >>>>>>>>>>> Ha, that doesn't work either. Now I get this error: >>> >>>>>>>>>>> >>> >>>>>>>>>>> RuntimeError: There are unpushed commits - "git log >>> origin/branch_6_5.." output is: >>> >>>>>>>>>>> >>> >>>>>>>>>>> So, I'm going to have to push this out. >>> >>>>>>>>>>> >>> >>>>>>>>>>> Let's first decide if this makes sense as the way forward. >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> Joel Bernstein >>> >>>>>>>>>>> http://joelsolr.blogspot.com/ >>> >>>>>>>>>>> >>> >>>>>>>>>>> On Wed, Apr 19, 2017 at 4:28 PM, Joel Bernstein < >>> [email protected]> wrote: >>> >>>>>>>>>>> Steve, when I change the script to turn off verification I >>> get the following error: >>> >>>>>>>>>>> RuntimeError: git clone is dirty: >>> >>>>>>>>>>> >>> >>>>>>>>>>> So as part of the work around I think I'm going to have to >>> commit locally and then revert locally. Does that make sense to you as the >>> way forward? >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> Joel Bernstein >>> >>>>>>>>>>> http://joelsolr.blogspot.com/ >>> >>>>>>>>>>> >>> >>>>>>>>>>> On Wed, Apr 19, 2017 at 4:24 PM, Alexandre Rafalovitch < >>> [email protected]> wrote: >>> >>>>>>>>>>> Could it be anything to do with the fact that the signer is >>> Semantic >>> >>>>>>>>>>> and they are being actively distrusted by - at least - >>> Google: >>> >>>>>>>>>>> http://www.securityweek.com/google-stops-trusting-symantec-i >>> ssued-certificates >>> >>>>>>>>>>> >>> >>>>>>>>>>> Regards, >>> >>>>>>>>>>> Alex. >>> >>>>>>>>>>> ---- >>> >>>>>>>>>>> http://www.solr-start.com/ - Resources for Solr users, new >>> and experienced >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> On 19 April 2017 at 16:22, Joel Bernstein < >>> [email protected]> wrote: >>> >>>>>>>>>>>> Ok, I'll turn off the cert verification. I wasn't sure if >>> cert verification >>> >>>>>>>>>>>> was something that was integral to the process. >>> >>>>>>>>>>>> >>> >>>>>>>>>>>> Joel Bernstein >>> >>>>>>>>>>>> http://joelsolr.blogspot.com/ >>> >>>>>>>>>>>> >>> >>>>>>>>>>>> On Wed, Apr 19, 2017 at 4:18 PM, Steve Rowe < >>> [email protected]> wrote: >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> Hi Joel, >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> Not sure why this is suddenly an issue - I guess “SNI” >>> >>>>>>>>>>>>> <https://en.wikipedia.org/wiki/Server_Name_Indication> >>> has been enabled on >>> >>>>>>>>>>>>> archive.apache.org? >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> Some useful info here (about a “requests” lib, AFAICT an >>> alternative to >>> >>>>>>>>>>>>> urllib): >>> >>>>>>>>>>>>> <http://docs.python-requests.o >>> rg/en/master/community/faq/#what-are-hostname-doesn-t-match-errors> >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> Short term, you could turn off certificate verification. >>> The bottom >>> >>>>>>>>>>>>> answer here uses the same lib as the script (urllib >>> instead of urllib2, >>> >>>>>>>>>>>>> which is assumed in the other answers on the page), to >>> turn off certificate >>> >>>>>>>>>>>>> verification: >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> <http://stackoverflow.com/ques >>> tions/19268548/python-ignore-certicate-validation-urllib2> >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> -- >>> >>>>>>>>>>>>> Steve >>> >>>>>>>>>>>>> www.lucidworks.com >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>>> On Apr 19, 2017, at 3:43 PM, Joel Bernstein < >>> [email protected]> wrote: >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> It appears that it's failing on verifying the SSL cert for >>> >>>>>>>>>>>>>> https://archive.apache.org/dist/lucene/java/. >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> Joel Bernstein >>> >>>>>>>>>>>>>> http://joelsolr.blogspot.com/ >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> On Wed, Apr 19, 2017 at 3:36 PM, Joel Bernstein < >>> [email protected]> >>> >>>>>>>>>>>>>> wrote: >>> >>>>>>>>>>>>>> I started working on 6.5.1 RC2. I ran the following: >>> >>>>>>>>>>>>>> python3 -u dev-tools/scripts/buildAndPushRelease.py >>> --push-local >>> >>>>>>>>>>>>>> /tmp/releases/6.5.1 --rc-num 2 --sign EE64CB1E >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> I believe this is same basic command I used for RC1. But >>> this time I got >>> >>>>>>>>>>>>>> a new error. At first I thought it might be incorrect >>> keystore password but >>> >>>>>>>>>>>>>> I changed it and it is correct. Any thoughts what the >>> issue is? >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> Traceback (most recent call last): >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line >>> 313, in <module> >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> main() >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line >>> 294, in main >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> rev = prepare(c.root, c.version, c.key_id, c.key_password) >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line >>> 98, in prepare >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> checkDOAPfiles(version) >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line >>> 143, in >>> >>>>>>>>>>>>>> checkDOAPfiles >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> distpage = load(url) >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line >>> 67, in load >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> content = urllib.request.urlopen(urlStri >>> ng).read().decode('utf-8') >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> File >>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>> >>>>>>>>>>>>>> line 223, in urlopen >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> return opener.open(url, data, timeout) >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> File >>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>> >>>>>>>>>>>>>> line 526, in open >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> response = self._open(req, data) >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> File >>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>> >>>>>>>>>>>>>> line 544, in _open >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> '_open', req) >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> File >>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>> >>>>>>>>>>>>>> line 504, in _call_chain >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> result = func(*args) >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> File >>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>> >>>>>>>>>>>>>> line 1361, in https_open >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> context=self._context, check_hostname=self._check_hos >>> tname) >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> File >>> >>>>>>>>>>>>>> "/Library/Frameworks/Python.fr >>> amework/Versions/3.6/lib/python3.6/urllib/request.py", >>> >>>>>>>>>>>>>> line 1320, in do_open >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> raise URLError(err) >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> urllib.error.URLError: <urlopen error [SSL: >>> CERTIFICATE_VERIFY_FAILED] >>> >>>>>>>>>>>>>> certificate verify failed (_ssl.c:749)> >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> Joel Bernstein >>> >>>>>>>>>>>>>> http://joelsolr.blogspot.com/ >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> ------------------------------ >>> --------------------------------------- >>> >>>>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>> >>>>>>>>>>>>> For additional commands, e-mail: >>> [email protected] >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> ------------------------------------------------------------ >>> --------- >>> >>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>> >>>>>>>>>>> For additional commands, e-mail: [email protected] >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> ------------------------------------------------------------ >>> --------- >>> >>>>>>>>>> To unsubscribe, e-mail: [email protected] >>> >>>>>>>>>> For additional commands, e-mail: [email protected] >>> >>>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> ------------------------------------------------------------ >>> --------- >>> >>>>>>>>> To unsubscribe, e-mail: [email protected] >>> >>>>>>>>> For additional commands, e-mail: [email protected] >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>> >>> >>>>>>> >>> >>>>>> >>> >>>>> >>> >>>>> >>> >>>>> ------------------------------------------------------------ >>> --------- >>> >>>>> To unsubscribe, e-mail: [email protected] >>> >>>>> For additional commands, e-mail: [email protected] >>> >>>>> >>> >>>>> >>> >>>> >>> >>>> >>> >>>> ------------------------------------------------------------ >>> --------- >>> >>>> To unsubscribe, e-mail: [email protected] >>> >>>> For additional commands, e-mail: [email protected] >>> >>>> >>> >>>> >>> >>> >>> >> >>> >> >>> >> --------------------------------------------------------------------- >>> >> To unsubscribe, e-mail: [email protected] >>> >> For additional commands, e-mail: [email protected] >>> >> >>> >> >>> > >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >> >
