Re: [VOTE] Release Apache CouchDB 3.4.1

2024-09-24 Thread Jay Doane
+1

erlang 25.3.2
elixir 1.17.2
python 3.12.3
java zulu-7.48.0.11
macos 13.7

On Mon, Sep 23, 2024 at 8:34 AM Nick Vatamaniuc  wrote:

> After tagging the new 3.4.0 release, but before announcing it and
> publishing the binaries, we discovered an issue where the new password
> auto-upgrade setting could lock out users from a _users db during
> downgrade. We discussed it on dev slack channel and deemed that it's
> not ideal as we normally strive to allow safely downgrading back to a
> previous version.
>
> Since we already tagged 3.4.0 we decided to create a new 3.4.1 release
> and will end up announcing 3.4.1 and publish binaries for it instead
> of 3.4.0.
>
> So this is the RC1 proposal for the new 3.4.1 release:
>
> Candidate release notes:
> https://docs.couchdb.org/en/latest/whatsnew/3.4.html#version-3-4-1
>
> We encourage the whole community to download and test these release
> artefacts so that any critical issues can be resolved before the
> release is made. Everyone is free to vote on this release, so dig
> right in! (Only PMC members have binding votes, but they depend on
> community feedback to gauge if an official release is ready to be
> made.)
>
> The release artefacts we are voting on are available here:
>
> https://dist.apache.org/repos/dist/dev/couchdb/source/3.4.1/rc1/
>
> There, you will find a tarball, a GPG signature, and the SHA256 checksum.
>
> Please follow the test procedure here:
>
> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
>
> Please remember that "RC1" is an annotation. If the vote passes, these
> artefacts will be released as Apache CouchDB 3.4.1.
>
> Please cast your votes now.
>
> Thanks,
> -Nick
>


Re: [VOTE] Release Apache CouchDB 3.4.0 (RC4)

2024-09-17 Thread Jay Doane
+1 using:
erlang 27.0.1
elixir 1.17.2
python 3.12.3
java zulu-7.48.0.11
macos 13.6.9


On Tue, Sep 17, 2024 at 11:06 AM Nick Vatamaniuc 
wrote:

> I would like to propose that we release Apache CouchDB 3.4.0.
>
> This is RC4.
>
> We discovered an issue with raising the pbkdf2 iteration count too
> high (by a 10x factor) and we decided to make another RC to fix it.
>
> Along the way we brought some minor test changes that should help with
> eunit tests on Windows.
>
> Changes since the last round:
>
> https://github.com/apache/couchdb/compare/3.4.0-RC3...3.4.0-RC4
>
> Candidate release notes:
>
> https://docs.couchdb.org/en/latest/whatsnew/3.4.html
>
> We encourage the whole community to download and test these release
> artefacts so that any critical issues can be resolved before the
> release is made. Everyone is free to vote on this release, so dig
> right in! (Only PMC members have binding votes, but they depend on
> community feedback to gauge if an official release is ready to be
> made.)
>
> The release artefacts we are voting on are available here:
>
> https://dist.apache.org/repos/dist/dev/couchdb/source/3.4.0/rc4/
>
> There, you will find a tarball, a GPG signature, and a SHA256 checksum.
>
> Please follow the test procedure here:
>
>
> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
>
> Please remember that "RC4" is an annotation. If the vote passes, these
> artefacts will be released as Apache CouchDB 3.4.0.
>
> Please cast your votes now.
>
> Thank you,
> -Nick
>


Re: [VOTE] Release Apache CouchDB 3.4.0 (RC3)

2024-09-16 Thread Jay Doane
+1 with:
macos 13.6.9
erlang 27.0.1
elixir 1.17.2
python 3.12.3
java zulu-7.48.0.11
and the following script in case anyone else wants to try it:
```
#!/bin/bash -ex

#
https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release

VERSION=3.4.0
RC=rc3

ARTIFACT_URL=
https://dist.apache.org/repos/dist/dev/couchdb/source/$VERSION/$RC/

TMP_DIR=/tmp/couchdb

rm -rf $TMP_DIR

mkdir -p $TMP_DIR/dist
cd $TMP_DIR/dist

erl -eval 'erlang:display(erlang:system_info(otp_release)), halt().'
 -noshell
elixir --version
python --version
java -version

wget --no-parent --no-directories -r $ARTIFACT_URL

gpg --verify apache-couchdb-*.tar.gz.asc

sha256sum --check apache-couchdb-*.tar.gz.sha256

tar -xvzf apache-couchdb-*.tar.gz

cd apache-couchdb-$VERSION

./configure --js-engine=quickjs --with-clouseau

make check

make release

echo 'adm = pass' >>  rel/couchdb/etc/local.ini

./rel/couchdb/bin/couchdb &

open http://127.0.0.1:5984/_utils/
```


On Sun, Sep 15, 2024 at 4:02 PM Nick Vatamaniuc  wrote:

> I would like to propose that we release Apache CouchDB 3.4.0.
>
> This is RC3.
>
> Thanks to Ronny for discovering and reporting the issue with the RC2
> configure script on Windows.
>
> Changes since the last round:
>
> https://github.com/apache/couchdb/compare/3.4.0-RC2...3.4.0-RC3
>
> Candidate release notes:
>
> https://docs.couchdb.org/en/latest/whatsnew/3.4.html
>
> We encourage the whole community to download and test these release
> artefacts so that any critical issues can be resolved before the
> release is made. Everyone is free to vote on this release, so dig
> right in! (Only PMC members have binding votes, but they depend on
> community feedback to gauge if an official release is ready to be
> made.)
>
> The release artefacts we are voting on are available here:
>
> https://dist.apache.org/repos/dist/dev/couchdb/source/3.4.0/rc3/
>
> There, you will find a tarball, a GPG signature, and a SHA256 checksum.
>
> Please follow the test procedure here:
>
>
> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
>
> Please remember that "RC3" is an annotation. If the vote passes, these
> artefacts will be released as Apache CouchDB 3.4.0.
>
> Please cast your votes now.
>
> Thank you,
> -Nick
>


Re: [VOTE] Release Apache CouchDB 3.3.3

2023-12-01 Thread Jay Doane
signature: ok
checksum: match
make check: pass
make release: ok
fauxton verify install: pass

env:
macOS 13.6.2
erlang 25.3.2
elixir 1.13.4-otp-25
python 3.12.0
spidermonkey 91

+1

On Fri, Dec 1, 2023 at 10:09 AM Jiahui Li  wrote:

> Hi,
>
> +1
>
> Signature: ok
> Checksums: match
> make check: passed
> make release: ok
>
> env:
>   - macOS/x86_64 14.1.1, Erlang 24.3.4.14, Elixir 1.15.7, Python 3.11.6,
> SpiderMonkey 91
>   - macOS/x86_64 14.1.1, Erlang 25.3.2.7, Elixir 1.15.7, Python 3.11.6,
> SpiderMonkey 91
>
> Thanks for preparing the release!
>
>
> Jiahui Li (Jessica)
> 
> From: Nick Vatamaniuc 
> Sent: Wednesday, November 29, 2023 1:22 PM
> To: dev@couchdb.apache.org 
> Subject: [EXTERNAL] [VOTE] Release Apache CouchDB 3.3.3
>
> Dear community,
>
> I would like to propose that we release Apache CouchDB 3.3.3.
>
> Candidate release notes:
> https://github.com/apache/couchdb/blob/3.3.x/src/docs/src/whatsnew/3.3.rst
>
> We encourage the whole community to download and test these release
> artefacts so that any critical issues can be resolved before the
> release is made. Everyone is free to vote on this release, so dig
> right in! (Only PMC members have binding votes, but they depend on
> community feedback to gauge if an official release is ready to be
> made.)
>
> The release artefacts we are voting on are available here:
> https://dist.apache.org/repos/dist/dev/couchdb/source/3.3.3/rc.1/
>
> There, you will find a tarball, a GPG signature, and the SHA256 checksum.
>
> Please follow the test procedure here:
>
> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
>
> Please remember that "RC1" is an annotation. If the vote passes, these
> artefacts will be released as Apache CouchDB 3.3.3.
>
> Please cast your votes now.
>
> Thanks,
> -Nick
>


Re: [VOTE] Release Apache CouchDB 3.2.3

2023-04-18 Thread Jay Doane
+1

Signature: ok
Checksums: match
make check: all tests pass
make release, fauxton installation: work as expected

environment: MacOS 13.3.1 on M1 Max, Erlang 23.3.4.18, Elixir 1.12.3,
Python 3.9.13

Note that I first tried using OTP 24 and Elixir 1.13, but `make
elixir-suite` failed in those cases.

On Sun, Apr 16, 2023 at 1:14 AM Nick Vatamaniuc  wrote:

> Dear community,
>
> I would like to propose that we release Apache CouchDB 3.2.3.
>
> Candidate release notes:
>
> https://github.com/apache/couchdb/blob/3.2.3-docs/src/docs/src/whatsnew/3.2.rst
>
> We encourage the whole community to download and test these release
> artefacts so that any critical issues can be resolved before the
> release is made. Everyone is free to vote on this release, so dig
> right in! (Only PMC members have binding votes, but they depend on
> community feedback to gauge if an official release is ready to be
> made.)
>
> The release artefacts we are voting on are available here:
> https://dist.apache.org/repos/dist/dev/couchdb/source/3.2.3/rc.1/
>
> There, you will find a tarball, a GPG signature, and SHA256/SHA512
> checksums.
>
> Please follow the test procedure here:
>
> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
>
> Since 3.2.3 is based on an older release, there is a Python
> compatibility issue where some tests may not work with the later
> version. I used Python 3.8, Erlang 23 and Elixir 1.12.3
>
> Please remember that "RC1" is an annotation. If the vote passes, these
> artefacts will be released as Apache CouchDB 3.2.3.
>
> Please cast your votes now.
>
> Thanks,
> -Nick
>


Re: [VOTE] Release Apache CouchDB 3.3.2

2023-04-18 Thread Jay Doane
+1

Signature: ok
Checksums: match
make check: all tests pass
make release, fauxton installation: work as expected

environment: MacOS 13.3.1 on M1 Max, Erlang 25.3, Elixir 1.13.4, Python
3.9.13


On Sun, Apr 16, 2023 at 1:07 AM Nick Vatamaniuc  wrote:

> Dear community,
>
> I would like to propose that we release Apache CouchDB 3.3.0.
>
> Candidate release notes:
> https://github.com/apache/couchdb/blob/3.3.x/src/docs/src/whatsnew/3.3.rst
>
> We encourage the whole community to download and test these release
> artefacts so that any critical issues can be resolved before the
> release is made. Everyone is free to vote on this release, so dig
> right in! (Only PMC members have binding votes, but they depend on
> community feedback to gauge if an official release is ready to be
> made.)
>
> The release artefacts we are voting on are available here:
> https://dist.apache.org/repos/dist/dev/couchdb/source/3.3.2/rc.1/
>
> There, you will find a tarball, a GPG signature, and SHA256/SHA512
> checksums.
>
> Please follow the test procedure here:
>
> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
>
> Please remember that "RC1" is an annotation. If the vote passes, these
> artefacts will be released as Apache CouchDB 3.3.2.
>
> Please cast your votes now.
>
> Thanks,
> -Nick
>


Re: Subject: [ANNOUNCE] Ronny Berndt joins the PMC

2023-03-04 Thread Jay Doane
Congratulations, Ronny!

Great work on the project.

On Fri, Mar 3, 2023 at 3:22 PM Nick Vatamaniuc  wrote:

> Dear community,
>
> I am delighted to announce that Ronny Berndt joins the Apache CouchDB
> Project Management Committee today.
>
> Ronny has made outstanding, sustained contributions to the project.
> This appointment is an official acknowledgement of their position
> within the community, and our trust in their ability to provide
> oversight for the project.
>
> Everybody, please join me in congratulating Ronny!
>
> On behalf of the CouchDB PMC,
> -Nick Vatamaniuc
>


Re: [VOTE] Release Apache CouchDB 3.3.0 (RC2)

2022-12-22 Thread Jay Doane
+1

macos 13.1 (22C65)
erlang 25.1.2
elixir 1.13.4-otp-24
spidermonkey 91

sig: ok
checksums: ok
make check: ok with elixir 1.13, but failed with 1.14
make release: ok
fauxton db & doc creation: ok

Nice work everyone!

On Thu, Dec 22, 2022 at 12:11 AM Nick Vatamaniuc 
wrote:

> Dear community,
>
> I would like to propose that we release Apache CouchDB 3.3.0.
>
> Candidate release notes:
> https://docs.couchdb.org/en/latest/whatsnew/3.3.html
>
> Changes since RC1:
> https://github.com/apache/couchdb/compare/3.3.0-RC1...3.3.0-RC2
>
> We encourage the whole community to download and test these release
> artefacts so that any critical issues can be resolved before the
> release is made. Everyone is free to vote on this release, so dig
> right in! (Only PMC members have binding votes, but they depend on
> community feedback to gauge if an official release is ready to be
> made.)
>
> The release artefacts we are voting on are available here:
> https://dist.apache.org/repos/dist/dev/couchdb/source/3.3.0/rc.2/
>
> There, you will find a tarball, a GPG signature, and SHA256/SHA512
> checksums.
>
> Please follow the test procedure here:
>
> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
>
> Please remember that "RC2" is an annotation. If the vote passes, these
> artefacts will be released as Apache CouchDB 3.3.0.
>
> Please cast your votes now.
>
> Thanks,
> -Nick
>


Re: [DISCUSS] Integrate couchdb-config into the main repo

2022-09-28 Thread Jay Doane
+1

On Wed, Sep 28, 2022 at 12:33 PM Nick Vatamaniuc 
wrote:

> Hi everyone,
>
> What do we think about integrating couchdb-config into the main couchdb
> repo?
>
> Currently it's a separate repo, with a different CI (Travis), which
> takes quite a bit of time waiting to schedule runs, has a different
> configuration file etc. Any updates to config needs a new tag and a
> corresponding bump and a separate PR in the main repo.
>
> Over the years it has been intermingled with couch_log and other
> CouchDB bits and that wouldn't make it as appealing to use as a
> separate application, so let's at least make life easier for us? What
> does everyone think?
>
> Thanks,
> -Nick
>


Re: [DISCUSSION] Make Erlang 23 the minimum supported distribution

2022-06-14 Thread Jay Doane
+1

On Mon, Jun 13, 2022 at 1:21 PM Nick Vatamaniuc  wrote:

> Hello everyone,
>
> I'd like to propose making Erlang 23 the minimum supported Erlang
> distribution. We have accumulated a lot of ifdefs and other cruft
> supporting various APIs and syntactic constructs since Erlang 20. With
> Erlang 25 just released it could be a good time to do some cleanup.
>
> We could remove ifdefs for crypto, time functions, the non-syntactic
> get_stacktrace() macro, as well as have access to nifty new
> functionality like persistent terms, atomic counters and a few other
> things.
>
> If we want to switch to rebar3, that would also be easier with Erlang
> 23 being the minimum.
>
> What do we think?
>
> Thanks,
> -Nick
>


Re: [ANNOUNCE] Ronny Berndt elected as CouchDB committer

2022-04-09 Thread Jay Doane
Congrats, Ronny, and welcome!

On Fri, Apr 8, 2022 at 12:37 AM Jan Lehnardt  wrote:

> Dear community,
>
> I am pleased to announce that the CouchDB Project Management Committee has
> elected Ronny Berndt as a CouchDB committer.
>
> Apache ID: ronny
>
> Slack nick: big-r
>
> Committers are given a binding vote in certain project decisions, as well
> as write access to public project infrastructure.
>
> This election was made in recognition of Ronny's commitment to the
> project. We mean this in the sense of being loyal to the project and its
> interests.
>
> Please join me in extending a warm welcome to Ronny!
>
> On behalf of the CouchDB PMC,
> Jan
> —


Re: [DISCUSS] erlfmt the 3.x branch

2021-11-18 Thread Jay Doane
Hi Nick,

As with main, my objections to a mass reformatting of hundreds of files
boils down to:

1. Making it more difficult to understand the logical changes of the code
(via blame) after reformatting
2. I don't agree with all the opinions of the formatter, nor does my
editor/mode

Is this something that could be done piecemeal for the purpose of
backporting/diffing, or do you feel it's necessary to reformat everything
at once?

Thanks,
Jay

On Wed, Nov 17, 2021 at 11:24 PM Nick Vatamaniuc  wrote:

> Hi all,
>
> I had noticed we had agreed to use erlfmt but only ran it on main and
> not 3.x. That makes it harder to port changes or diff-ing modules
> sometimes. Would there be any objections to a pull request
> reformatting 3.x?
>
> The only issue is that we may have to avoid running erlfmt if we
> detect Erlang 20, but that's just a few if / else checks. The CI
> checks on 21+ would still check and fail if erlfmt check fails.
>
> Thanks,
> -Nick
>


Re: [VOTE] Release Apache CouchDB 3.2.1

2021-11-03 Thread Jay Doane
+1

GPG signature, checksums, `make check`, fauxton: all good

macOS 11.6.1, OTP 23.3.4.8, SpiderMonkey 60

Small nit on release notes:
> there are less corner cases
s/less/fewer/

Very nice work, Nick!


On Tue, Nov 2, 2021 at 7:00 PM Nick Vatamaniuc  wrote:

> Dear community,
>
> I would like to propose that we release Apache CouchDB 3.2.1
>
> Candidate release notes:
>
> https://docs.couchdb.org/en/latest/whatsnew/3.2.html#version-3-2-1
>
> We encourage the whole community to download and test these release
> artefacts so that any critical issues can be resolved before the
> release is made. Everyone is free to vote on this release, so dig
> right in! (Only PMC members have binding votes, but they depend on
> community feedback to gauge if an official release is ready to be
> made.)
>
> The release artefacts we are voting on are available here:
>
> https://dist.apache.org/repos/dist/dev/couchdb/source/3.2.1/rc.1/
>
> There, you will find a tarball, a GPG signature, and SHA256/SHA512
> checksums.
>
> Please follow the test procedure here:
>
>
> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
>
> Please remember that "RC1" is an annotation. If the vote passes, these
> artefacts will be released as Apache CouchDB 3.2.1
>
> Please cast your votes now.
>
> Thanks,
> -Nick
>


Re: [VOTE] Release CouchDB 3.2.0 (RC2 round)

2021-10-05 Thread Jay Doane
+1

Tested on MacOS 10.15.7, OTP 22.3.4.15, Spidermonkey 60

Signature: ok
Checksums: ok
make check: ok
make release and starting bin/couchdb: ok
http://localhost:5984/_utils (Fauxton): ok
 - Created 2 DBs
 - Setup a replication job between them
 - Docs from first db replicated to the second db

Documentation on release testing seems a bit stale.
- No longer a `-c` configure option
- Is `etap` still a thing?

On Tue, Oct 5, 2021 at 12:30 PM Nick Vatamaniuc  wrote:

> Dear community,
>
> I would like to propose that we release Apache CouchDB 3.2.0.
>
> Candidate release notes:
>
> https://docs.couchdb.org/en/latest/whatsnew/3.2.html
>
> This is the RC2 round. Changes since last round:
> https://github.com/apache/couchdb/compare/3.2.0-RC1...3.2.0-RC2
>
> We encourage the whole community to download and test these release
> artefacts so that any critical issues can be resolved before the
> release is made. Everyone is free to vote on this release, so dig
> right in! (Only PMC members have binding votes, but they depend on
> community feedback to gauge if an official release is ready to be
> made.)
>
> The release artefacts we are voting on are available here:
>
> https://dist.apache.org/repos/dist/dev/couchdb/source/3.2.0/rc.2/
>
> There, you will find a tarball, a GPG signature, and SHA256/SHA512
> checksums.
>
> Please follow the test procedure here:
>
>
> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
>
> Please remember that "RC2" is an annotation. If the vote passes, these
> artefacts will be released as Apache CouchDB 3.2.0.
>
> Please cast your votes now.
>
> Thanks,
> -Nick
>


Re: [ANNOUNCE] Glynn Bird joins the PMC

2020-09-14 Thread Jay Doane
Congratulations, Glynn!

On Mon, Sep 14, 2020 at 9:22 AM Michelle P  wrote:

>
> Dear community,
>
> I am delighted to announce that Glynn Bird joins the Apache CouchDB
> Project Management Committee today.
>
> Glynn has made outstanding, sustained contributions to the project. This
> appointment is an official acknowledgement of their position within the
> community, and our trust in their ability to provide oversight for the
> project.
>
> Everybody, please join me in congratulating Glynn!
>
> On behalf of the CouchDB PMC,
> Michelle
>
>


Re: [DISCUSS] ldap_auth donation

2020-09-01 Thread Jay Doane
Hi Joan,

Yes, I'm planning to maintain it, and hopefully it's readable enough for
others to do so if necessary. In any case, since current Cloudant Local
customers are being encouraged (with few on-prem alternatives) to migrate
to CouchDB, there should be a built-in user base for as well. This is
definitely not abandonware!

And, yes, I'm still planning to get weatherreport into 3.2 as well. Thanks
for the encouragement :)

Jay

On Tue, Sep 1, 2020 at 1:42 PM Joan Touzet  wrote:

> I remember this code!
>
> Sure, let's get it out there, *as long as* someone is going to maintain
> it going forward.
>
> I had been hoping to see more rallying behind how to use the JWT
> integration for OAuth and SAML workflows, but no one's done any
> walkthroughs / blogposts that I've seen. Putting those things in front
> of LDAP may be easier (but would be outside the scope of CouchDB...)
>
> While we're on the topic of donation, any chance of weatherreport
> getting donated? You had brought this up a few months ago, Jay, and I'd
> love to see that in 3.2 as well.
>
> -Joan
>
>
>
> On 2020-09-01 3:02 p.m., Jay Doane wrote:
> > Greetings,
> >
> > In 2015 IBM Cloudant developed an LDAP based authentication handler for
> > its CouchDB 2.x-based Cloudant Local offering. Since then, it has been
> used
> > in production on several large Cloudant Local deployments, accruing many
> > bug fixes and enhancements in the process.
> >
> > Over the years, there has clearly been interest in using LDAP with
> CouchDB
> > [1], so it seems like the ldap_auth functionality might be something the
> > greater community could benefit from. If there are no objections from the
> > community or PMC, I'd be happy to open a PR in the hopes of getting it
> > included in CouchDB 3.2. To give an idea of what it's all about, I've
> > included the README.md contents below.
> >
> > Thanks,
> > Jay
> >
> > [1] https://couchdb.markmail.org/search/?q=LDAP
> >
> > README.md:
> >
> > # Delegating Basic and Cookie authentication and authorization to LDAP
> >
> > CouchDB includes a built-in security model, with self-contained
> > authentication and authorization which rely on .ini files and/or user
> > databases to store user data, including usernames (uids), password
> > hashes, and user roles for accessing database resources.
> >
> > For an organization which already uses an LDAP service for access
> > control, it may make more sense to delegate authentication and
> > authorization services to LDAP when accessing CouchDB. This is where
> > `ldap_auth` comes in.
> >
> > At a high level, when `ldap_auth` is configured, it completely
> > replaces the default Basic and Cookie authentication and
> > authorization. It ignores .ini files and user databases altogether,
> > and attempts to validate user credentials using the configured LDAP
> > service. Once credentials have been authentication by the LDAP
> > service, `ldap_auth` determines the user's roles (which in turn
> > determine what the user is authorized to access) based on the LDAP
> > groups of which the user is a member.
> >
> > ## ldap_interface
> >
> > This low-level interface module uses
> > [eldap](http://www.erlang.org/doc/man/eldap.html) to connect to,
> > authenticate, and search LDAP server(s) which are configured to model
> > user accounts and their associated CouchDB roles.
> >
> > ### Configuration
> >
> > All configuration is done via `.ini` file, mostly in the `[ldap_auth]`
> > section. The following parameters can be modified, and have the
> > associated defaults in parentheses:
> >
> > - `servers (127.0.0.1)` one or more LDAP servers; defaults to a single
> > host, but could be a comma separated list (note that all servers must
> > use the same port, a limitation of the underlying eldap library)
> >
> > - `port (389)` LDAP server port for un-encrypted communication
> >
> > - `ssl_port (636)` LDAP server port for encrypted communication
> >
> > - `use_ssl (true)` if `true`, use TLS to encrypt traffic to LDAP
> > servers
> >
> > - `timeout (5000)` milliseconds to wait for a response from an LDAP
> > server before throwing an error
> >
> > - `user_base_dn (ou=users,dc=example,dc=com)` defines a directory
> > location to start searching for users
> >
> > - `user_classes (person)` defines which `objectClass`es indicate
> > a particular entry as a user during search
> >
> > - `user_uid_attribute (uid)` defines which attribute ma

[DISCUSS] ldap_auth donation

2020-09-01 Thread Jay Doane
Greetings,

In 2015 IBM Cloudant developed an LDAP based authentication handler for
its CouchDB 2.x-based Cloudant Local offering. Since then, it has been used
in production on several large Cloudant Local deployments, accruing many
bug fixes and enhancements in the process.

Over the years, there has clearly been interest in using LDAP with CouchDB
[1], so it seems like the ldap_auth functionality might be something the
greater community could benefit from. If there are no objections from the
community or PMC, I'd be happy to open a PR in the hopes of getting it
included in CouchDB 3.2. To give an idea of what it's all about, I've
included the README.md contents below.

Thanks,
Jay

[1] https://couchdb.markmail.org/search/?q=LDAP

README.md:

# Delegating Basic and Cookie authentication and authorization to LDAP

CouchDB includes a built-in security model, with self-contained
authentication and authorization which rely on .ini files and/or user
databases to store user data, including usernames (uids), password
hashes, and user roles for accessing database resources.

For an organization which already uses an LDAP service for access
control, it may make more sense to delegate authentication and
authorization services to LDAP when accessing CouchDB. This is where
`ldap_auth` comes in.

At a high level, when `ldap_auth` is configured, it completely
replaces the default Basic and Cookie authentication and
authorization. It ignores .ini files and user databases altogether,
and attempts to validate user credentials using the configured LDAP
service. Once credentials have been authentication by the LDAP
service, `ldap_auth` determines the user's roles (which in turn
determine what the user is authorized to access) based on the LDAP
groups of which the user is a member.

## ldap_interface

This low-level interface module uses
[eldap](http://www.erlang.org/doc/man/eldap.html) to connect to,
authenticate, and search LDAP server(s) which are configured to model
user accounts and their associated CouchDB roles.

### Configuration

All configuration is done via `.ini` file, mostly in the `[ldap_auth]`
section. The following parameters can be modified, and have the
associated defaults in parentheses:

- `servers (127.0.0.1)` one or more LDAP servers; defaults to a single
host, but could be a comma separated list (note that all servers must
use the same port, a limitation of the underlying eldap library)

- `port (389)` LDAP server port for un-encrypted communication

- `ssl_port (636)` LDAP server port for encrypted communication

- `use_ssl (true)` if `true`, use TLS to encrypt traffic to LDAP
servers

- `timeout (5000)` milliseconds to wait for a response from an LDAP
server before throwing an error

- `user_base_dn (ou=users,dc=example,dc=com)` defines a directory
location to start searching for users

- `user_classes (person)` defines which `objectClass`es indicate
a particular entry as a user during search

- `user_uid_attribute (uid)` defines which attribute maps to
username

- `group_base_dn (ou=groups,dc=example,dc=com)` defines directory
location to start searching for groups

- `group_classes (posixGroup)` defines which `objectClass`es indicate
a particular entry as a group during search

- `group_member_attribute (memberUid)` defines which group attribute
maps to user Uid

- `group_role_attribute (description)` defines which group attribute
maps to a particular role

- `searcher_dn (uid=ldapsearch,ou=users,dc=example,dc=com)` defines
the DN to use when searching for users and groups

- `searcher_password (secret)` defines the password for the
`searcher_dn` above

- `user_bind_dns ([])` defines one or more base DNs into which the
authenticating user's username can be inserted as the
`user_uid_attribute`, and used to bind directly. See the "Efficiency
Considerations" section below for details

**Please note** that at least one of the above parameters must be set
inside the `[ldap_auth]` section of a .ini configuration file in order
for ldap_auth to be considered "configured", and to function as an
authentication/authorization handler. Failure to explicitly set at
least one `[ldap_auth]` parameter will result in the system using the
default basic and cookie authentication/authorization handlers instead.

### Efficiency Considerations

  `ldap_auth` effectively has 2 modes of operation, depending on
whether `user_bind_dns` has been defined or not. Both modes return a
list of roles associated with a user upon authentication success.

  By default, `ldap_auth` opens a connection to an LDAP server and
binds the connection handle with the searcher DN and password. It then
uses that connection to search for a user record with the
authenticating username. If a matching user DN is found, it opens a
second connection to the server and attempts to bind that connection
using the username and password credentials. If the bind is
successful, it closes that connection, and uses the original
"searcher" bound connection to sea

Re: [ANNOUNCE] Juan José Rodriguez elected as CouchDB committer

2020-02-07 Thread Jay Doane
Congrats Juanjo! Thanks for all your hard work, and welcome aboard!

On Fri, Feb 7, 2020 at 11:29 AM Joan Touzet  wrote:

> Dear community,
>
> I am pleased to announce that the CouchDB Project Management Committee
> has elected Juan José Rodriguez as a CouchDB committer.
>
>  Apache ID: juanjo
>
>  Github ID: jjrodrig
>
> Committers are given a binding vote in certain project decisions, as
> well as write access to public project infrastructure.
>
> This election was made in recognition of Juanjo's commitment to the
> project. We mean this in the sense of being loyal to the project and its
> interests.
>
> Please join me in extending a warm welcome to Juanjo!
>
> On behalf of the CouchDB PMC,
> Joan Touzet
>


Re: [PROPOSAL] Remove `make elixir` from `make check`

2019-12-13 Thread Jay Doane
On Thu, Dec 12, 2019 at 11:27 PM Joan Touzet  wrote:

> Jay, consider this a +1 to remove those if you can get to them before I do.
>

Joan, I don't think I was clear, but when I said
> I'm +1 to removing these tests temporarily.
by "these tests" I actually meant all the elixir tests, not just the two I
mentioned. I don't think we'd get very far by just disabling those two, as
there are many others that fail intermittently as well. I'm +1 to remove
them, and here's a PR in case someone wants it to happen sooner rather than
later:

https://github.com/apache/couchdb/pull/2362

Thanks,
Jay

>
> If this makes the Elixir tests useful again, then it's a good compromise.
>
> -Joan
>
> On 2019-12-13 2:18 a.m., Jay Doane wrote:
> > I've been trying to merge into master for the last 24+ hours, and have
> > restarted the build about 10 times so far, but it has yet to pass.
> >
> > I kept track of failures, and these came up numerous times:
> >
> >   1) test GET /dbname/_design_docs (PartitionDDocTest)
> >   test/elixir/test/partition_ddoc_test.exs:160
> >   ** (RuntimeError) timed out after 10294 ms
> >   code: retry_until(
> >   stacktrace:
> > (couchdbtest) test/elixir/lib/couch/db_test.ex:301:
> > Couch.DBTest.retry_until/4
> > test/elixir/test/partition_ddoc_test.exs:163: (test)
> >
> >   1) test All Docs tests (AllDocsTest)
> >   test/elixir/test/all_docs_test.exs:15
> >   ** (RuntimeError) timed out after 30097 ms
> >   code: retry_until(fn ->
> >   stacktrace:
> > (couchdbtest) test/elixir/lib/couch/db_test.ex:301:
> > Couch.DBTest.retry_until/4
> > test/elixir/test/all_docs_test.exs:75: (test)
> >
> > which both seem to be timing out.
> >
> > I'm +1 to removing these tests temporarily.
> >
> > Jay
> >
> > On Thu, Dec 12, 2019 at 1:39 PM Joan Touzet  wrote:
> >
> >> Hi again,
> >>
> >> As I've been looking more closely at the CI suite for the Jenkins
> >> transition, I've noticed that our Elixir test cases are actually the
> >> most likely to fail. In 6 consecutive CI runs, 5 runs failed due to
> >> failures in the Elixir suite. (The 6th failed due to a JS test failure.)
> >>
> >> We started the Elixir effort to retire the JS suite. We reached a
> >> decision some months ago to put it into `make check` so that people
> >> would pay attention to its output, and work to fix those tests,
> >> accelerating our chances to get rid of the JS suite.
> >>
> >> Unfortunately, that's not materialised. Our Elixir test porters seem to
> >> have stopped their work for a while now, and no one is systematically
> >> addressing the failures in that suite. I've also heard other developers
> >> mention (via IRC) that some of the test cases hold invalid assumptions
> >> about how CouchDB works, especially with the Erlang-based clustering
> >> code. It sounds to me like the effort needs a full code review.
> >>
> >> With 3.0 around the corner, I want people to be able to trust the output
> >> of `make check` when downloading the tarball. If there is no objection,
> >> when I merge the Erlang version / CI changes on Monday, I will also
> >> comment out the call to `make elixir` as part of make check.
> >>
> >> When the Elixir porting team is more confident in the reliability and
> >> completeness of their work, and we can successfully retire the JS suite,
> >> we can reconsider.
> >>
> >> -Joan "really wanting to see green, but only seeing red" Touzet
> >>
> >
>


Re: [PROPOSAL] Remove `make elixir` from `make check`

2019-12-12 Thread Jay Doane
I've been trying to merge into master for the last 24+ hours, and have
restarted the build about 10 times so far, but it has yet to pass.

I kept track of failures, and these came up numerous times:

 1) test GET /dbname/_design_docs (PartitionDDocTest)
 test/elixir/test/partition_ddoc_test.exs:160
 ** (RuntimeError) timed out after 10294 ms
 code: retry_until(
 stacktrace:
   (couchdbtest) test/elixir/lib/couch/db_test.ex:301:
Couch.DBTest.retry_until/4
   test/elixir/test/partition_ddoc_test.exs:163: (test)

 1) test All Docs tests (AllDocsTest)
 test/elixir/test/all_docs_test.exs:15
 ** (RuntimeError) timed out after 30097 ms
 code: retry_until(fn ->
 stacktrace:
   (couchdbtest) test/elixir/lib/couch/db_test.ex:301:
Couch.DBTest.retry_until/4
   test/elixir/test/all_docs_test.exs:75: (test)

which both seem to be timing out.

I'm +1 to removing these tests temporarily.

Jay

On Thu, Dec 12, 2019 at 1:39 PM Joan Touzet  wrote:

> Hi again,
>
> As I've been looking more closely at the CI suite for the Jenkins
> transition, I've noticed that our Elixir test cases are actually the
> most likely to fail. In 6 consecutive CI runs, 5 runs failed due to
> failures in the Elixir suite. (The 6th failed due to a JS test failure.)
>
> We started the Elixir effort to retire the JS suite. We reached a
> decision some months ago to put it into `make check` so that people
> would pay attention to its output, and work to fix those tests,
> accelerating our chances to get rid of the JS suite.
>
> Unfortunately, that's not materialised. Our Elixir test porters seem to
> have stopped their work for a while now, and no one is systematically
> addressing the failures in that suite. I've also heard other developers
> mention (via IRC) that some of the test cases hold invalid assumptions
> about how CouchDB works, especially with the Erlang-based clustering
> code. It sounds to me like the effort needs a full code review.
>
> With 3.0 around the corner, I want people to be able to trust the output
> of `make check` when downloading the tarball. If there is no objection,
> when I merge the Erlang version / CI changes on Monday, I will also
> comment out the call to `make elixir` as part of make check.
>
> When the Elixir porting team is more confident in the reliability and
> completeness of their work, and we can successfully retire the JS suite,
> we can reconsider.
>
> -Joan "really wanting to see green, but only seeing red" Touzet
>


Re: [PROPOSAL] Drop Erlang 19 support in CouchDB 3.0

2019-12-11 Thread Jay Doane
+1

On Wed, Dec 11, 2019 at 5:35 PM Joan Touzet  wrote:

> Hello everyone,
>
> I'm working this week with Paul Davis on our new Jenkins CI
> infrastructure, which is coming along nicely. One of the changes I'm
> planning to make is that our PR tests will run against only 3 versions
> of Erlang:
>
> 1. The oldest we support (right now, 19.3.6.latest)
> 2. The version we currently ship with our binary distros & Docker
>(right now, 20.3.8.latest)
> 3. The very latest version we support (right now, 22.2)
>
> In preparing the containers for CI testing, it's turning out to be very
> difficult to build Erlang 19.* anymore on modern Linuxes. This is
> because they ship with OpenSSL 1.1+, and 19.* cannot build against
> anything newer than OpenSSL 1.0.
>
> I can jump through a huge number of hoops for this...or we can just drop
> Erlang 19 support for CouchDB 3.0 and require Erlang 20. (Note we
> blacklist a number of versions of Erlang 20.) I would then replace
> 19.3.6.latest with 20.3.8.11 [1].
>
> I think this is the right thing to do for the project, and I don't
> expect any vetos, but because this is a deprecation, I'm posting it here
> before merging the change to rebar.config.script (and Jenkinsfile).
>
> Let me know if you object.
>
> -Joan
>
> [1]: https://github.com/apache/couchdb/blob/master/rebar.config.script#L44
>
>


Re: [VOTE] Couch replicator _scheduler/jobs and _scheduler/docs API change

2019-11-01 Thread Jay Doane
Makes sense to me. +1

On Thu, Oct 31, 2019 at 12:46 PM Nick Vatamaniuc 
wrote:

> Hi everyone,
>
> For CouchDB 3.0 I am proposing to add all the replication status
> details from _active_tasks like "docs_written", "docs_read", etc to
> _scheduler/jobs and _scheduler/docs endpoints.
>
> Couch replicator scheduler has been out for a few years already. It
> added flexibility, performance and other features. One of the goals of
> the scheduler was to provide a better set of APIs for monitoring
> replication job progress. Those APIs are _scheduler/jobs and
> _scheduler/docs on top of the already existing _active_tasks and just
> polling _replicator db documents.
>
> _scheduler/jobs is focused on replication jobs, their history, how
> many times they started, stopped, their state. It returns replications
> jobs indexed by their job ids, and includes jobs started from
> _replicator dbs as well as from the _replicate HTTP endpoint.
>
> _scheduler/docs in turn, is focused on showing the state of
> replication documents. It returns replication jobs indexed by
> _replicator db and doc id which started the jobs.
>
> _scheduler/docs API was returning an "info" field for each job. That
> info field was designed to contain details pertaining to the current
> state of the job. So if there is an error it will have the error
> message. For a completed job, it was showing the replication stats
> persisted to the replication document, so that users do not have to
> poll their status.
>
> However, when the jobs were running or were in a pending state that
> field was "null". The idea was to eventually replace "null" with stats
> from _active_tasks such that users don't need to ever look at
> _active_tasks to monitor their replication progress. So the proposal
> here is to do exactly that.
>
> To be specific:
>
> 1) In the "pending" or "running" state _scheduler/docs will return
> active_tasks status values in the "info" field. ("completed" is
> already doing that btw).
>
> 2) _scheduler/jobs will have a new "info" field which also show the
> same status values
>
> There is a draft PR for it
>
> https://github.com/apache/couchdb/pull/2292#issuecomment-548526613
>
> Some examples of what it looks like:
>
> https://github.com/apache/couchdb/pull/2292#issuecomment-548526346
> https://github.com/apache/couchdb/pull/2292#issuecomment-548526613
>
> What does everyone think?
>
> Let's do a lazy consensus since we are replacing a place-holder null
> value and adding a new field to _scheduler jobs. Let me know if this
> is not appropriate and we need a different voting / discussion
> procedure.
>
> Cheers,
> -Nick
>


[DISCUSS] [PROPOSAL] Accept donation of the IBM Cloudant Weather Report diagnostic tool?

2019-08-13 Thread Jay Doane
In the interest of making CouchDB 3.0 "the best CouchDB Classic possible",
I'd like to discuss whether to accept a donation from Cloudant of the
"Weather Report" diagnostic tool. This tool (and dependencies) are OTP
applications, and it is typically run from an escript which connects to a
running cluster, gathers numerous diagnostics, and emits various warning
and errors when it finds something to complain about. It was originally
ported from a fork of Riaknostic (the Automated diagnostic tools for Riak)
[1] by Mike Wallace.

The checks it makes are represented by the following modules:

weatherreport_check_custodian.erl
weatherreport_check_disk.erl
weatherreport_check_internal_replication.erl
weatherreport_check_ioq.erl
weatherreport_check_mem3_sync.erl
weatherreport_check_membership.erl
weatherreport_check_memory_use.erl
weatherreport_check_message_queues.erl
weatherreport_check_node_stats.erl
weatherreport_check_nodes_connected.erl
weatherreport_check_process_calls.erl
weatherreport_check_process_memory.erl
weatherreport_check_safe_to_rebuild.erl
weatherreport_check_search.erl
weatherreport_check_tcp_queues.erl

While some of these checks are self-contained, check_node_stats,
check_process_calls, check_process_memory, and check_message_queues all use
recon [2] under the hood. Similarly, check_custodian
and check_safe_to_rebuild use another Cloudant OTP application called
Custodian, which periodically scans the "dbs" database to track the
location of every shard of every database and can integrate with sensu [3]
to ensure that operators are aware of any shard that is under-replicated.

I have created a POC branch [4] that adds Weather Report, Custodian, and
Recon to CouchDB, and when I ran it in my dev environment (without search
running), got the following diagnostic output:

$ ./weatherreport --etc ~/proj/couchdb/dev/lib/node1/etc/ -a
['node1@127.0.0.1'] [error] Local search node at 'clouseau@127.0.0.1' not
responding: pang
['node2@127.0.0.1'] [error] Local search node at 'clouseau@127.0.0.1' not
responding: pang
['node3@127.0.0.1'] [error] Local search node at 'clouseau@127.0.0.1' not
responding: pang
['node1@127.0.0.1'] [notice] Data directory
/Users/jay/proj/couchdb/dev/lib/node1/data is not mounted with 'noatime'.
Please remount its disk with the 'noatime' flag to improve performance.
['node2@127.0.0.1'] [notice] Data directory
/Users/jay/proj/couchdb/dev/lib/node2/data is not mounted with 'noatime'.
Please remount its disk with the 'noatime' flag to improve performance.
['node3@127.0.0.1'] [notice] Data directory
/Users/jay/proj/couchdb/dev/lib/node3/data is not mounted with 'noatime'.
Please remount its disk with the 'noatime' flag to improve performance.
returned 1

There is still a little cleanup to be done before these tools would be
ready to donate, but it seems that overall they already integrate tolerably
well with CouchDB.

As far as licenses go, Riaknostic is Apache 2.0. Recon is not [5], but it
seems like it should be ok to include in CouchDB based on my possibly naive
reading. Currently Custodian has no license (just Copyright 2013 Cloudant),
but I assume it would get an Apache license, just like all other donated
code.

Would this be a welcome addition to CouchDB? Please let me know what you
think.

Thanks,
Jay

[1] https://github.com/basho/riaknostic
[2] http://ferd.github.io/recon/
[3] https://sensu.io
[4]
https://github.com/apache/couchdb/compare/master...cloudant:weatherreport?expand=1
[5] https://github.com/ferd/recon/blob/master/LICENSE


Re: [VOTE] Adopt FoundationDB

2019-07-30 Thread Jay Doane
+1

On Tue, Jul 30, 2019 at 1:27 AM Jan Lehnardt  wrote:

> Dear CouchDB developers,
>
> This vote decides whether the CouchDB project accepts the proposal[1]
> to switch our underlying storage and distributed systems technology out
> for FoundationDB[2].
>
> At the outset, we said that we wanted to cover these topic areas before
> making a vote:
>
> - Bylaw changes
> - RFC process: done, passed
> - Add qualified vote option: done, changes proposed were not
>   ratified
>
> - Roadmap: proposal done, detailed discussions TBD, includes
>   deprecations
>
> - Onboarding: ASF onboarding links shared, CouchDB specific onboarding
>   TBD.
>
> - (Re-)Branding: tentatively: 3.0 is the last release before FDB
>   CouchDB and 4.0 is the FDB CouchDB. If we need nicknames, we can
>   decide on those later.
>
> - FoundationDB Governance: FoundationDB is currently loosely organised
>   between Apple and a few key stakeholder companies invested in the
>   technology. Apple contributions are trending downwards relatively,
>   approaching 50%, which means in the future, more non-Apple than Apple
>   contributions are likely.
>
>   In addition, the CouchDB PMC has requested addition to the current
>   organisational FDB weekly meeting, which is where any more formal
>   governance decisions are going to be made and the CouchDB PMC can be
>   a part of the surrounding discussions.
>
> - FoundationDB Operations knowledge: IBM has intends to share this
>   knowledge as they acquire it in conjunction with Apache CouchDB in
>   terms of general ops knowledge, best practices and tooling.
>
> - Proj. Mgmt.: RFC process + outline list of TBD RFCs allow for enough
>   visibility and collaboration opportunities, everyone on dev@ list is
>   encouraged to participate.
>
> - Tech deep dives: DISCUSS threads and RFCs are covering this, current
>   list of TBD DISCUSS/RFCs, for the proposal. Most of which were
>   already discussed on dev@ or RFC’d in our documentation repo:
>
> * JSON doc storage and storage of edit conflicts
> * revision management
> * _changes feed
> * _db_updates
> * _all_docs
> * database creation and deletion
> * attachments
> * mango indexes (including collation)
> * map-only views / search / geo
> * reduces
> * aggregate metrics (data_size, etc.)
> * release engineering
> * local/desktop/dev install security
>
> * * *
>
> As shown above, all topics we wanted to have clarity on have been
> advanced to a point where we are now ready to make a decision:
>
>   Should Apache CouchDB adopt FoundationDB?
>
> Since this is a big decision, I suggest we make this a Lazy 2/3
> Majority Vote with PMC Binding Votes, and a 7 day duration (as per our
> bylaws[3]).
>
> You can cast your votes now.
>
> Best
> Jan
> —
> [1]:
> https://lists.apache.org/thread.html/04e7889354c077a6beb91fd1292b6d38b7a3f2c6a5dc7d20f5b87c44@%3Cdev.couchdb.apache.org%3E
> [2]: https://www.foundationdb.org
> [3]: https://couchdb.apache.org/bylaws.html
>
>
>


Re: [ANNOUNCE] Jay Doane elected as CouchDB committer

2019-01-14 Thread Jay Doane
Thanks for the warm welcome everybody. I'm honored to work with such
talented folks on this great project!

Best regards,
Jay

On Thu, Jan 10, 2019 at 5:56 AM Andy Wenk  wrote:

> Hey Jay welcome ;-)
>
> Cheers
>
> Andy
> --
> Andy Wenk
> Hamburg - Germany
> RockIt!
>
> GPG public key:
> https://goo.gl/mhw74A
>
>
>
> > On 9. Jan 2019, at 16:16, Garren Smith  wrote:
> >
> > Woohoo. Congrats Jay
> >
> > On Wed, Jan 9, 2019 at 10:07 AM Alessio 'Blaster' Biancalana <
> > dottorblas...@gmail.com> wrote:
> >
> >> Totally deserved!
> >> Kudos!
> >>
> >> Alessio
> >>
> >> On Wed, Jan 9, 2019 at 8:20 AM Jan Lehnardt  wrote:
> >>
> >>> Welcome aboard, Jay! :)
> >>>
> >>> Best
> >>> Jan
> >>> —
> >>>
> >>>> On 9. Jan 2019, at 00:11, Joan Touzet  wrote:
> >>>>
> >>>> Dear community,
> >>>>
> >>>> I am pleased to announce that the CouchDB Project Management Committee
> >>> has elected Jay Doane as a CouchDB committer.
> >>>>
> >>>>   Apache ID: jaydoane
> >>>>
> >>>>   IRC nick: jaydoane
> >>>>
> >>>>   Twitter: jaydoane
> >>>>
> >>>> Committers are given a binding vote in certain project decisions, as
> >>> well as write access to public project infrastructure.
> >>>>
> >>>> This election was made in recognition of Jay's commitment to the
> >>> project. We mean this in the sense of being loyal to the project and
> its
> >>> interests.
> >>>>
> >>>> Please join me in extending a warm welcome to Jay!
> >>>>
> >>>> On behalf of the CouchDB PMC,
> >>>> Joan "more committers!" Touzet
> >>>
> >>>
> >>
>
>


[PROPOSAL] Change the minimum supported Erlang version to OTP 19

2018-12-20 Thread Jay Doane
Currently, CouchDB requires at least OTP 17 or later to build and run
[1][2]. However, recent work undertaken to eliminate compiler warnings
[3][4] has highlighted the additional effort needed to continue to support
older Erlang versions. Some of the issues that have come up are:
1. erlang:now/0 deprecated in OTP 18 [5]
2. crypto:rand_uniform/2 deprecated in OTP 20 [6], but no rand module
pre-OTP 18
which both require using rebar platform defines [7] and ifdefs [8] to work
around compiler warnings.

Joan raised the idea that maybe it's time to move to a more recent minimum
version to simplify the code, and also because there a many compelling new
features in later versions that we currently cannot use:
1. maps introduced in OTP 17, but only became performant for large number
of entries in OTP 18 [9]
2. off heap messages introduced in OTP 19 [10]

Since CouchDB now ships with it's own OTP 19.6.3 Erlang binaries [9], it's
not clear whether we need to continue supporting OTP 17 and 18. As a bonus,
removing those versions will also speed up travis builds.

Any thoughts either for or against this proposal?

Best regards,
Jay

[1] https://github.com/apache/couchdb/blob/master/rebar.config.script#L94
[2] https://github.com/apache/couchdb/blob/master/.travis.yml#L10
[3] https://github.com/apache/couchdb-ets-lru/pull/7
[4] https://github.com/apache/couchdb/pull/1798
[5] http://erlang.org/doc/apps/erts/time_correction.html
[6] http://erlang.org/pipermail/erlang-questions/2017-May/092435.html
[7]
https://github.com/apache/couchdb/blob/master/src/couch/rebar.config.script#L148-L154
[8]
https://github.com/apache/couchdb/blob/master/src/couch/src/couch_rand.erl#L22-L57
[9] http://erlang.org/download/otp_src_18.0.readme
[10]
https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html
[9] https://github.com/apache/couchdb-ci/blob/master/README.md


Re: [VOTE] Release Apache CouchDB 2.3.0-RC1

2018-12-03 Thread Jay Doane
Hi Joan,

I tested on MacOS 10.13.6 and the only untoward thing I've seen seen so far
is in the logs, this is repeated every 5s:

[notice] 2018-11-30T22:43:54.596760Z couchdb@localhost <0.346.0> 
chttpd_auth_cache changes listener died database_does_not_exist at
mem3_shards:load_shards_from_db/6(line:395) <=
mem3_shards:load_shards_from_disk/1(line:370) <=
mem3_shards:load_shards_from_disk/2(line:399) <=
mem3_shards:for_docid/3(line:86) <= fabric_doc_open:go/3(line:39) <=
chttpd_auth_cache:ensure_auth_ddoc_exists/2(line:195) <=
chttpd_auth_cache:listen_for_changes/1(line:142)
[error] 2018-11-30T22:43:54.596823Z couchdb@localhost emulator 
Error in process <0.2569.0> on node couchdb@localhost with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,395}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,370}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,399}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,86}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,39}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,195}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,142}]}]}

until I create a _users db, after which it stops. It also stops as soon as
setup completes, so I don't think this error is necessarily a problem for
the release per se.

I notice there's a closed ticket already for these errors at
https://github.com/apache/couchdb/issues/1354 but I have also been
tinkering with a solution that will prevent the change listener from
starting if the auth db doesn't exist.

BR,
Jay

On Thu, Nov 29, 2018 at 11:22 AM Joan Touzet  wrote:

> Dear community,
>
> I would like to release Apache CouchDB 2.3.0-RC1.
>
> Changes since last round:
>
>  * https://github.com/apache/couchdb/compare/2.2.0...2.3.0-RC1
>
> Candidate release notes:
>
>  * http://docs.couchdb.org/en/latest/whatsnew/2.3.html
>
> We encourage the whole community to download and test these release
> artefacts so that any critical issues can be resolved before the release is
> made. Everyone is free to vote on this release, so dig right in!
>
> The release artefacts we are voting on are available here:
>
>
> https://dist.apache.org/repos/dist/dev/couchdb/source/2.3.0/rc.1/apache-couchdb-2.3.0-RC1.tar.gz
>
> https://dist.apache.org/repos/dist/dev/couchdb/source/2.3.0/rc.1/apache-couchdb-2.3.0-RC1.tar.gz.asc
>
> https://dist.apache.org/repos/dist/dev/couchdb/source/2.3.0/rc.1/apache-couchdb-2.3.0-RC1.tar.gz.sha256
>
> https://dist.apache.org/repos/dist/dev/couchdb/source/2.3.0/rc.1/apache-couchdb-2.3.0-RC1.tar.gz.sha512
>
> Please follow the test procedure here:
>
>
> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
>
> Please remember that "RC1" is an annotation. If the vote passes, these
> artefacts will be released as Apache CouchDB 2.3.0.
>
> Please cast your votes now.
>
> Thanks,
> Joan "we promised and delivered 2 releases a year" Touzet
>


Re: [RFC] On the Testing of CouchDB

2018-01-07 Thread Jay Doane
Elixir's focus on positive developer experience seems like a big selling
point (among many others), so count me in.

I'd like to work on stats.js, provided nobody else has started it yet.

Jay

On Thu, Dec 14, 2017 at 5:03 PM, Russell Branca 
wrote:

> Howdy folks!
>
> The testing of CouchDB is something that has seen focus and improvements
> for the last several years, for instance migrating the etap suite to eunit,
> and updating the JS suite to run against clusters in 2.x. There's still
> improvements to be made, and that was one of the topics of the CouchDB dev
> summit early in the year [1].
>
> Before we go further, I want to clarify some nomenclature. I'm by no means
> going to try and define unit testing vs integration testing vs quantum
> phase shift testing, but instead I want to focus on the distinction of
> where the testing takes place. Fundamentally, we have two places we test
> CouchDB: 1) at the Erlang VM level where we conduct assertions against
> module functions or process states; 2) at the HTTP level where we test the
> behavior of CouchDB at the user level API. This post focuses entirely on
> the latter; that's not to say the former doesn't also merit attention, just
> that the two are different enough that we can focus on them in isolation.
>
> So with that, let's chat about the current HTTP test suite in CouchDB. This
> is the "JS suite" I referred to above, which is a custom built test suite
> written in Javascript and executed in the aging SpiderMonkey. The JS suite
> has put in work for years, but it's showing it's age, and is a bit awkward
> to work with and improve. However, I think the biggest issue with the JS
> suite is that it's utilized far less than it should be, and folks seem to
> avoid extending it or adding additional tests to it. There's been
> discussion for years about replacing said suite, but the discussions
> invariably got blocked on the bike shed of whether to rewrite the suite in
> Javascript or Python. This thread provides a third option, with code!
>
> I started hacking on a replacement for the JS suite, this time written in
> Elixir. Overall I'm quite impressed with how it's come along, and have some
> good examples to show. This is basically an Elixir app that has an HTTP
> client and then runs a series of tests that conduct tests against the
> CouchDB HTTP API and make assertions therein.
>
> You can find the current code in [2], and a comparison of the changes in
> [3]. The core HTTP client is only a handful of lines of codes and works
> quite well [4]. The utility functions used across all tests are located in
> [5], and the tests themselves are in [6]. The existing test modules have a
> 1:1 correspondence with the associated JS suite test modules, and in
> general are as direct of a port as possible.
>
> The test modules ported in their entirety or most of the way are:
>
>   * all_docs.js
>   * basics.js
>   * config.js
>   * reduce.js
>   * rewrite.js
>   * uuids.js
>   * view_collation.js
>
> Paul has dove in and is responsible for a few of those test modules and
> he's almost completed porting the replication.js suite as well. We started
> with the hard ones first, so for the most part the rest of the ports should
> be fairly smooth sailing.
>
> Here's an example of a very basic test:
>
> ```erlang
> defmodule WelcomeTest do
>   use CouchTestCase
>
>   test "Welcome endpoint" do
> assert Couch.get("/").body["couchdb"] == "Welcome", "Should say
> welcome"
>   end
>
> end
>
> ```
>
>
> As you can see, the `Couch` client is very simple HTTP client with
> easy HTTP verb based methods. Let's look at a more complicated test
> for asserting we can create documents in a database:
>
>
> ```erlang
>
>   @tag :with_db
>   test "Create a document and save it to the database", context do
> resp = Couch.post("/#{context[:db_name]}", [body: %{:_id => "0",
> :a => 1, :b => 1}])
> assert resp.status_code == 201, "Should be 201 created"
> assert resp.body["id"], "Id should be present"
> assert resp.body["rev"], "Rev should be present"
>
> resp2 = Couch.get("/#{context[:db_name]}/#{resp.body["id"]}")
> assert resp2.body["_id"] == resp.body["id"], "Ids should match"
> assert resp2.body["_rev"] == resp.body["rev"], "Revs should match"
>   end
>
> ```
>
>
> This is fairly straightforward code to POST a new doc, make assertions
> on the response, and then fetch the doc to make sure everything
> matches up. What I really wanted to highlight here is the `@tag
> :with_db` decorator. We can easily add custom "tags" to the tests to
> simplify setup and teardown. That `:with_db` tag does two things, it
> dynamically generates a random database name, and then takes care of
> setup/teardown for creating and deleting said database for that
> particular test. This is really useful and has been very nice to work
> with so far. We also have tag functionality in place for executing a
> test with a particular set of config options:
>
>
> ```erlang
>
>   

Re: Stabilizing our automated builds - help needed!

2017-03-19 Thread Jay Doane
On Sun, Mar 19, 2017 at 10:50 PM, Joan Touzet  wrote:

>
> --
> module 'couch_log_writer_syslog_test'
>   couch_log_writer_syslog_test: couch_log_writer_syslog_test_...*timed
> out*
>   undefined
>
> Not sure why this times out, can we increase the timeout maybe?
>

Another data point: I always get this same error on my mac laptop, but have
so far been unable to identify the issue. Since nobody else (until now) had
seen it, I assumed it was just a problem with my dev environment.

Jay