Re: [VOTE] Release Apache CouchDB 2.3.1 RC2
Dug into this a bit more with Nick's help on IRC. These are the two new tests for validating the fix to mochiweb for small buffers that are failing. It *appears* that the function Nick built to restart chttpd is not working right on Windows. To wit, I can repeat the test without a chttpd/mochiweb restart using curl and get the right results for buffer_too_small_url_fails: https://gist.github.com/wohali/050621d5c5cdf874529bf094545bcca1 The same curl commands produce the same output on Windows as Linux, though of course the semantics inside of eunit are different, as they seem to expect a 400 in the former case of a 1024 byte buffer which is not possible to send over the wire if the buffer is too small (this is the crashing parser bug we're trying to work around.) In short, I think we need to improve the test case for Windows, but it is the weekend and I need Nick's help here to resolve it. -Joan - Original Message - > From: "Joan Touzet" > To: dev@couchdb.apache.org > Sent: Friday, March 1, 2019 3:48:29 PM > Subject: Re: [VOTE] Release Apache CouchDB 2.3.1 RC2 > > Tested just now on Windows 10. > > Signature: passes > Checksums: pass > make check: fails repeatedly, times out on these two tests: > > chttpd_socket_buffer_size_test:71: > buffer_too_small_url_fails... > chttpd_socket_buffer_size_test:83: > buffer_too_small_header_fails... > > Full log of the failures is at: > > https://gist.github.com/wohali/94c22ca493f60ba6353c13e6c09b4f35 > > NOTE that Windows logging always has SASL errors on, I've never been > able to suppress those log lines, so just search on *timed out* and > you'll find the two failures. > > I can make a release with this, but it's probably not right to ignore > these tests. > > -1 :( > > -Joan > > > - Original Message - > > From: "Jan Lehnardt" > > To: dev@couchdb.apache.org > > Sent: Friday, March 1, 2019 2:49:38 PM > > Subject: Re: [VOTE] Release Apache CouchDB 2.3.1 RC2 > > > > Russell confirmed on IRC. The vote can proceed. > > > > For extra safety: if anyone runs make check on this tarball and > > find > > test fails in the log, please post them here, that would block the > > release. > > > > It's all fine on Travis and I think Jenkins. > > > > Cheers > > Jan > > — > > > > > On 1. Mar 2019, at 11:17, Jan Lehnardt wrote: > > > > > > I have a hard time reconciling those statements with what I’m > > > seeing in the logs: > > > > > > Consider this snippet from the latest 2.3.x build log[1]: > > > https://gist.github.com/janl/3c7db5f3ff466f9985306253d61abc3b > > > > > > It is the output for couchdb_views_tests[2], which has 5 test > > > groups defined, and all test groups with all their inside them > > > get an `...ok` line. > > > > > > 4/5 groups get a `erl_child_setup: failed with error 32 on line > > > 253` message *after* the tests in each group ran to successful > > > completion. > > > > > > As such, I’d suggest we do NOT need to abort this vote just yet. > > > > > > It’d be great to figure out what causes those messages to avoid > > > confusion, and we’ve addressed an unrelated but related issue > > > about hard failing the test suite if a sub-group fails, but the > > > 2.3.x log does not exhibit this. > > > > > > Obligatory statement that votes don’t happen on IRC. > > > > > > Best > > > Jan > > > — > > > > > > [1]: full log here: > > > https://api.travis-ci.org/v3/job/494557908/log.txt > > > [2]: > > > https://github.com/apache/couchdb/blob/2.3.x/src/couch/test/couchdb_views_tests.erl > > > > > > > > > > > > > > >> On 27. Feb 2019, at 22:26, Joan Touzet > > >> wrote: > > >> > > >> Based on discussion with Russell Branca (chewbranca) in IRC, we > > >> need to > > >> abort this RC vote as he is effectively voting -1. Here's the > > >> full > > >> transcript of our discussion: > > >> > > >> > > >> > > >> 16:06 <+Wohali> chewbranca: you there? are you seeing these > > >> eunit > > >> context setup errors in 2.3.0 as well as the 2.3.1 > > >> RC > > >> and master? > > >> 16:06 <+Wohali> I don't want to hold up 2.3.1 over something > > >> that > > >> was a > > >> pre-existing condition, but if it's something that > > >> changed between 2.3.0 and 2.3.1/master, we need to > > >> fix > > >> it > > >> 16:07 Wohali: well the fundamental issue right now > > >> is > > >> test > > >> suite failures don't fail the build, which IMO > > >> should > > >> be fixed before any further builds > > >> 16:08 I've been using this diff locally, which > > >> fails > > >> the > > >> `make eunit` check upon an eunit failure: > > >> > > >> https://gist.github.com/chewbranca/65d2969ac191a5dfaf87172ace18d2ee > > >> 16:08 not sure that's the best approach, but we > > >> need > > >> something like that
Re: [VOTE] Release Apache CouchDB 2.3.1 RC2
Tested just now on Windows 10. Signature: passes Checksums: pass make check: fails repeatedly, times out on these two tests: chttpd_socket_buffer_size_test:71: buffer_too_small_url_fails... chttpd_socket_buffer_size_test:83: buffer_too_small_header_fails... Full log of the failures is at: https://gist.github.com/wohali/94c22ca493f60ba6353c13e6c09b4f35 NOTE that Windows logging always has SASL errors on, I've never been able to suppress those log lines, so just search on *timed out* and you'll find the two failures. I can make a release with this, but it's probably not right to ignore these tests. -1 :( -Joan - Original Message - > From: "Jan Lehnardt" > To: dev@couchdb.apache.org > Sent: Friday, March 1, 2019 2:49:38 PM > Subject: Re: [VOTE] Release Apache CouchDB 2.3.1 RC2 > > Russell confirmed on IRC. The vote can proceed. > > For extra safety: if anyone runs make check on this tarball and find > test fails in the log, please post them here, that would block the > release. > > It's all fine on Travis and I think Jenkins. > > Cheers > Jan > — > > > On 1. Mar 2019, at 11:17, Jan Lehnardt wrote: > > > > I have a hard time reconciling those statements with what I’m > > seeing in the logs: > > > > Consider this snippet from the latest 2.3.x build log[1]: > > https://gist.github.com/janl/3c7db5f3ff466f9985306253d61abc3b > > > > It is the output for couchdb_views_tests[2], which has 5 test > > groups defined, and all test groups with all their inside them > > get an `...ok` line. > > > > 4/5 groups get a `erl_child_setup: failed with error 32 on line > > 253` message *after* the tests in each group ran to successful > > completion. > > > > As such, I’d suggest we do NOT need to abort this vote just yet. > > > > It’d be great to figure out what causes those messages to avoid > > confusion, and we’ve addressed an unrelated but related issue > > about hard failing the test suite if a sub-group fails, but the > > 2.3.x log does not exhibit this. > > > > Obligatory statement that votes don’t happen on IRC. > > > > Best > > Jan > > — > > > > [1]: full log here: > > https://api.travis-ci.org/v3/job/494557908/log.txt > > [2]: > > https://github.com/apache/couchdb/blob/2.3.x/src/couch/test/couchdb_views_tests.erl > > > > > > > > > >> On 27. Feb 2019, at 22:26, Joan Touzet wrote: > >> > >> Based on discussion with Russell Branca (chewbranca) in IRC, we > >> need to > >> abort this RC vote as he is effectively voting -1. Here's the full > >> transcript of our discussion: > >> > >> > >> > >> 16:06 <+Wohali> chewbranca: you there? are you seeing these eunit > >> context setup errors in 2.3.0 as well as the 2.3.1 > >> RC > >> and master? > >> 16:06 <+Wohali> I don't want to hold up 2.3.1 over something that > >> was a > >> pre-existing condition, but if it's something that > >> changed between 2.3.0 and 2.3.1/master, we need to > >> fix > >> it > >> 16:07 Wohali: well the fundamental issue right now is > >> test > >> suite failures don't fail the build, which IMO > >> should > >> be fixed before any further builds > >> 16:08 I've been using this diff locally, which fails > >> the > >> `make eunit` check upon an eunit failure: > >> > >> https://gist.github.com/chewbranca/65d2969ac191a5dfaf87172ace18d2ee > >> 16:08 not sure that's the best approach, but we need > >> something like that > >> 16:08 <+Wohali> What I'm asking is: do you think this should block > >> the > >> release of 2.3.1? > >> 16:08 <+Wohali> By all means PR that to master and let's get shit > >> in > >> gear > >> 16:08 <+Wohali> I'm trying to work out when this problem started > >> occurring, though. > >> 16:09 yes, should definitely block any further > >> releases, > >> because unless someone is manually inspecting the > >> eunit output, then we could have test failures > >> bubbling through > >> 16:11 in theory this particular issue was introduced > >> 26 > >> days ago with the change to running individual > >> eunit > >> tests: > >> > >> https://github.com/apache/couchdb/commit/20bbfbf972ad1f822e2ef1edfb3d47f2cec3f639 > >> 16:11 so this is probably a new thing, but we've > >> definitely > >> had issues with eunit over the years > >> 16:12 Wohali: I can make a quick PR with the diff I > >> pasted > >> above and then we should be good to go IMO, but > >> it > >> wouldn't hurt to see if there's a more proper way > >> to > >> do that in a Makefile than just `|| exit 1` > >> 16:16 <+Wohali> chewbranca: are you 100% s
Re: [VOTE] Release Apache CouchDB 2.3.1 RC2
Russell confirmed on IRC. The vote can proceed. For extra safety: if anyone runs make check on this tarball and find test fails in the log, please post them here, that would block the release. It's all fine on Travis and I think Jenkins. Cheers Jan — > On 1. Mar 2019, at 11:17, Jan Lehnardt wrote: > > I have a hard time reconciling those statements with what I’m seeing in the > logs: > > Consider this snippet from the latest 2.3.x build log[1]: > https://gist.github.com/janl/3c7db5f3ff466f9985306253d61abc3b > > It is the output for couchdb_views_tests[2], which has 5 test groups defined, > and all test groups with all their inside them get an `...ok` line. > > 4/5 groups get a `erl_child_setup: failed with error 32 on line 253` message > *after* the tests in each group ran to successful completion. > > As such, I’d suggest we do NOT need to abort this vote just yet. > > It’d be great to figure out what causes those messages to avoid confusion, > and we’ve addressed an unrelated but related issue about hard failing the > test suite if a sub-group fails, but the 2.3.x log does not exhibit this. > > Obligatory statement that votes don’t happen on IRC. > > Best > Jan > — > > [1]: full log here: https://api.travis-ci.org/v3/job/494557908/log.txt > [2]: > https://github.com/apache/couchdb/blob/2.3.x/src/couch/test/couchdb_views_tests.erl > > > > >> On 27. Feb 2019, at 22:26, Joan Touzet wrote: >> >> Based on discussion with Russell Branca (chewbranca) in IRC, we need to >> abort this RC vote as he is effectively voting -1. Here's the full >> transcript of our discussion: >> >> >> >> 16:06 <+Wohali> chewbranca: you there? are you seeing these eunit >> context setup errors in 2.3.0 as well as the 2.3.1 RC >> and master? >> 16:06 <+Wohali> I don't want to hold up 2.3.1 over something that was a >> pre-existing condition, but if it's something that >> changed between 2.3.0 and 2.3.1/master, we need to fix >> it >> 16:07 Wohali: well the fundamental issue right now is test >> suite failures don't fail the build, which IMO should >> be fixed before any further builds >> 16:08 I've been using this diff locally, which fails the >> `make eunit` check upon an eunit failure: >> https://gist.github.com/chewbranca/65d2969ac191a5dfaf87172ace18d2ee >> 16:08 not sure that's the best approach, but we need >> something like that >> 16:08 <+Wohali> What I'm asking is: do you think this should block the >> release of 2.3.1? >> 16:08 <+Wohali> By all means PR that to master and let's get shit in >> gear >> 16:08 <+Wohali> I'm trying to work out when this problem started >> occurring, though. >> 16:09 yes, should definitely block any further releases, >> because unless someone is manually inspecting the >> eunit output, then we could have test failures >> bubbling through >> 16:11 in theory this particular issue was introduced 26 >> days ago with the change to running individual eunit >> tests: >> https://github.com/apache/couchdb/commit/20bbfbf972ad1f822e2ef1edfb3d47f2cec3f639 >> 16:11 so this is probably a new thing, but we've definitely >> had issues with eunit over the years >> 16:12 Wohali: I can make a quick PR with the diff I pasted >> above and then we should be good to go IMO, but it >> wouldn't hurt to see if there's a more proper way to >> do that in a Makefile than just `|| exit 1` >> 16:16 <+Wohali> chewbranca: are you 100% sure that context setup >> failures mean the tests are actually failing? They seem >> to be running and passing even after that. I'm too >> unfamiliar to know for sure. >> 16:17 <+Wohali> chewbranca: that change you linked isn't in 2.3.1. >> 16:17 context setup failure means that setting up a series >> of eunit test generators failed and those tests >> aren't being executed >> 16:17 <+Wohali> ok. >> 16:18 those will fail if you do `|| exit 1`, but they >> continue running today because we don't exit on the >> individual eunit runs >> 16:18 <+Wohali> 2.3.1 has a critical fix for buffer sizes that we need >> to get out there. WOuld you accept me manually reviewing >> the output of 2.3.1's test suite to ensure no context >> setup failures? >> 16:18 <+Wohali> then we make this a blocker for 2.4.0? >> 16:18 what I linked above is just a diff that I've been >> using locally because I wanted the suite to fail, and >> it works >> 16:19 Wohali: IMO let's just add that diff and then if >> folks know a more proper Mak
Re: Maintainig two codebases
> On 1. Mar 2019, at 15:26, Robert Newson wrote: > > Hi, > > The first decision is to what extent we are supporting the "old" version > (once the new one exists, of course). I think it would be limited to security > fixes. Eventually yes, but I can imagine a grace period for bugfix release before go to security-only. Reality might not require it, but I don’t want to close the door on this just yet. Either way tho, this doesn’t change the discussion at hand. > If so, this is exactly the same as when we released 1.2, 1.3, and so on. > Master is always latest (FDB, in this case) and there are branches like 1.2.x > or 2.0.x if we need to backport security fixes to make a release there. One special case is that we are now effectively doing 3.0 and 4.0 in parallel for a bit until 3.0 is out. We just need to decide which version stays in-branch until 3.x is out. I don’t think there is a clear winner either way. To deal with the PR awkwardness, we could fork current master into a new repo, make that FDB CouchDB, and when 3.x ships and gets its 3.x.x branch we merge the other repo back as master, maybe via a special case where FDB CouchDB master straight out replaces then current couchdb.git master, and that master is moved to 3.x.x. Will need some ASF Infra help obvs. I would want to avoid having to merge things back and forth for anything that isn’t kept around (chttpd, replicator, etc), as nice as Nebraska is, I wouldn’t want to put anyone through a big merge bonanza again. Best Jan — > > B. > > -- > Robert Samuel Newson > rnew...@apache.org > > On Fri, 1 Mar 2019, at 13:49, Ilya Khlopotov wrote: >> There are ongoing discussions about rebasing CouchDB on top of >> FoundationDB. It seems like the community is in agreement that it is >> worth it to try. This would mean that we would be supporting and >> extending two quite separate codebases. How are we going to do it? >> >> Possible options are: >> - brand new repository >> - separate branch which we would treat as master for FDB rebase project >> >> I think that separate branch approach has a number of disadvantages: >> - CI might require different dependencies >> - It would be awkward to open GH issues since we would have to always >> refer to the project we are talking about >> - There would be little friction when we open PR since the correct base >> branch would need to be selected >> >> Best regards, >> iilyak >> -- Professional Support for Apache CouchDB: https://neighbourhood.ie/couchdb-support/
Re: Maintainig two codebases
Hi, The first decision is to what extent we are supporting the "old" version (once the new one exists, of course). I think it would be limited to security fixes. If so, this is exactly the same as when we released 1.2, 1.3, and so on. Master is always latest (FDB, in this case) and there are branches like 1.2.x or 2.0.x if we need to backport security fixes to make a release there. B. -- Robert Samuel Newson rnew...@apache.org On Fri, 1 Mar 2019, at 13:49, Ilya Khlopotov wrote: > There are ongoing discussions about rebasing CouchDB on top of > FoundationDB. It seems like the community is in agreement that it is > worth it to try. This would mean that we would be supporting and > extending two quite separate codebases. How are we going to do it? > > Possible options are: > - brand new repository > - separate branch which we would treat as master for FDB rebase project > > I think that separate branch approach has a number of disadvantages: > - CI might require different dependencies > - It would be awkward to open GH issues since we would have to always > refer to the project we are talking about > - There would be little friction when we open PR since the correct base > branch would need to be selected > > Best regards, > iilyak >
Maintainig two codebases
There are ongoing discussions about rebasing CouchDB on top of FoundationDB. It seems like the community is in agreement that it is worth it to try. This would mean that we would be supporting and extending two quite separate codebases. How are we going to do it? Possible options are: - brand new repository - separate branch which we would treat as master for FDB rebase project I think that separate branch approach has a number of disadvantages: - CI might require different dependencies - It would be awkward to open GH issues since we would have to always refer to the project we are talking about - There would be little friction when we open PR since the correct base branch would need to be selected Best regards, iilyak
Re: [VOTE] Release Apache CouchDB 2.3.1 RC2
I have a hard time reconciling those statements with what I’m seeing in the logs: Consider this snippet from the latest 2.3.x build log[1]: https://gist.github.com/janl/3c7db5f3ff466f9985306253d61abc3b It is the output for couchdb_views_tests[2], which has 5 test groups defined, and all test groups with all their inside them get an `...ok` line. 4/5 groups get a `erl_child_setup: failed with error 32 on line 253` message *after* the tests in each group ran to successful completion. As such, I’d suggest we do NOT need to abort this vote just yet. It’d be great to figure out what causes those messages to avoid confusion, and we’ve addressed an unrelated but related issue about hard failing the test suite if a sub-group fails, but the 2.3.x log does not exhibit this. Obligatory statement that votes don’t happen on IRC. Best Jan — [1]: full log here: https://api.travis-ci.org/v3/job/494557908/log.txt [2]: https://github.com/apache/couchdb/blob/2.3.x/src/couch/test/couchdb_views_tests.erl > On 27. Feb 2019, at 22:26, Joan Touzet wrote: > > Based on discussion with Russell Branca (chewbranca) in IRC, we need to > abort this RC vote as he is effectively voting -1. Here's the full > transcript of our discussion: > > > > 16:06 <+Wohali> chewbranca: you there? are you seeing these eunit >context setup errors in 2.3.0 as well as the 2.3.1 RC >and master? > 16:06 <+Wohali> I don't want to hold up 2.3.1 over something that was a >pre-existing condition, but if it's something that >changed between 2.3.0 and 2.3.1/master, we need to fix >it > 16:07 Wohali: well the fundamental issue right now is test > suite failures don't fail the build, which IMO should > be fixed before any further builds > 16:08 I've been using this diff locally, which fails the > `make eunit` check upon an eunit failure: > https://gist.github.com/chewbranca/65d2969ac191a5dfaf87172ace18d2ee > 16:08 not sure that's the best approach, but we need > something like that > 16:08 <+Wohali> What I'm asking is: do you think this should block the >release of 2.3.1? > 16:08 <+Wohali> By all means PR that to master and let's get shit in >gear > 16:08 <+Wohali> I'm trying to work out when this problem started >occurring, though. > 16:09 yes, should definitely block any further releases, > because unless someone is manually inspecting the > eunit output, then we could have test failures > bubbling through > 16:11 in theory this particular issue was introduced 26 > days ago with the change to running individual eunit > tests: > https://github.com/apache/couchdb/commit/20bbfbf972ad1f822e2ef1edfb3d47f2cec3f639 > 16:11 so this is probably a new thing, but we've definitely > had issues with eunit over the years > 16:12 Wohali: I can make a quick PR with the diff I pasted > above and then we should be good to go IMO, but it > wouldn't hurt to see if there's a more proper way to > do that in a Makefile than just `|| exit 1` > 16:16 <+Wohali> chewbranca: are you 100% sure that context setup >failures mean the tests are actually failing? They seem >to be running and passing even after that. I'm too >unfamiliar to know for sure. > 16:17 <+Wohali> chewbranca: that change you linked isn't in 2.3.1. > 16:17 context setup failure means that setting up a series > of eunit test generators failed and those tests > aren't being executed > 16:17 <+Wohali> ok. > 16:18 those will fail if you do `|| exit 1`, but they > continue running today because we don't exit on the > individual eunit runs > 16:18 <+Wohali> 2.3.1 has a critical fix for buffer sizes that we need >to get out there. WOuld you accept me manually reviewing >the output of 2.3.1's test suite to ensure no context >setup failures? > 16:18 <+Wohali> then we make this a blocker for 2.4.0? > 16:18 what I linked above is just a diff that I've been > using locally because I wanted the suite to fail, and > it works > 16:19 Wohali: IMO let's just add that diff and then if > folks know a more proper Makefile approach to doing > that type of thing then they can fix it later > 16:19 <+Wohali> to both 2.3.1 and master? And to Makefile.Win I presume? >;) Then we'll have to cancel the current RC and re-spin. > ... > 16:25 https://github.com/apache/couchdb/pull/1951 > > > > > - Original Message - >> From: "Dave Cottlehuber" >> To
