Re: [DISCUSS] CouchDB Roadmap

[Garren Smith]

Nice summary Jan thanks.

Just to also comment on Mango aggregations. This is a bit of a
stumbling block for FDB. Some of the work I did recently was to move the
Mango selector matching from the co-ordinator down to the actual node that
has the document. The next step would be to then be to do in-memory
aggregations on the node and send the result back to the co-ordinator.
Unfortunately FDB doesn't support doing that. So any mango aggregation work
that happens now we need to make sure it is possible in FDB as well.

On Fri, Jan 25, 2019 at 1:57 PM Jan Lehnardt  wrote:

> Hi everyone,
>
> this is one of the threads we promised. This one covers the roadmap
> discussion.
>
> This thread aims to answer:
>
> - which version would the FDB work land in?
> - what happens with non-FDB CouchDB until then?
>   - specifically, how are the current roadmap items affected?
> - how are the two different versions being managed in the future.
>
> Out of scope:
>
> - whether how how we name things publicly.
>
> The below contains my current thinking for an ideal way to handle this and
> it is presented as a strawperson argument to be picked apart and refined,
> designed to solicit input form everybody.
>
>
> Please comment on the details of The Proposal (below) as well as the
> categorisation and comments in The Tickets (also below). There are a lot of
> details in here, feel free to comment on just one aspect of this rather
> than feeling compelled to make a comment on everything.
>
> * * *
>
>
> Naming prelude: to make this thread easier to follow, I’ll call the
> current `master` version of CouchDB “Erlang CouchDB” (despite there being
> non-Erlang bits in there) and the version that would be based on
> FoundationDB “FBD CouchDB”.
>
> Note: these aren’t suggestions for marketing names or anything, just
> shorthand for this particular discussion.
>
> * * *
>
>
> ## Guiding Principles:
>
> - Getting to a position where FDB CouchDB is production ready is going
> take a while. In the meantime, Erlang CouchDB will continue to receive
> feature-, bugfix-, and security-updates. This is *at least* until an FDB
> CouchDB verion is available, plus a grace period for a transition that is
> to be discussed (gut feel, this is anything between 1, 2 and 3 years).
>
> - As a result of a multi-year dev-lifespan of Erlang CouchDB, and
> extrapolating from experience in existing CouchDB production timespans, we
> can easily imagine folks running Erlang CouchDB in production for *at
> least* the next 10 years (At Neighbourhoodie, we are still supporting
> CouchDB 1.2.1 customers e.g.)
>
> - A direct consequence of this, and in light of major feature work *at
> some point* transitioning to FDB CouchDB, we should take this opportunity
> and rally to make Erlang CouchDB “The Best Erlang CouchDB we can make”™.
> That means we’ll re-evaluate the current roadmap against what’s essential
> to make that happen, while leaving things that might be easier to do on FDB
> CouchDB for the new setup.
>
>
> * * *
>
>
> ## The Proposal
>
> In very broad strokes, I imagine a CouchDB 3.x series that is “The Best
> Erlang CouchDB we can make”™ that has all the big ticket items we planned
> on doing modulo some that we leave for FDB CouchDB because they become
> possible or significantly easier. The main reason for calling this 3.x
> would be a set of deprecations as well as the addition of the _access field
> to docs (where we would want a forward compatible version of the 2.x series
> as well, so upgrades become possible) and the imposing of additional limits
> to document, attachment and HTTP request sizes to be within recommended
> best practices and to be forward compatible with FDB CouchDB, but which can
> be brought back to 2.x settings for folks who need it.
>
> FDB CouchDB would then become the 4.x series. Even though we make major
> changes under the hood, we would want for most CouchDB 2.x/3.x users to be
> able to upgrade as seamlessly as possible, so we’ll aim for a high degree
> of API compatibility like we’ve done in the 1.x -> 2.x transition and which
> worked very well for us.
>
> * * *
>
>
> ## The Annotated Roadmap
>
> The rest of this email is a list of all tickets that currently have
> `roadmap` label with a comment on how to proceed with it plus a few bits
> and bobs that are relevant in the context of an FDB future.
>
> Broadly, there are these categories:
>
> 1. we should add this to “The Best Erlang CouchDB we can make”™, category
> YES
>
> 2. this is a ticket that can be worked on independently of either CouchDB
> variant (say improvements to Mango, which can happen at any time), category
> INDIE
>
> 3. we should hold off unti

Re: [DISCUSS] CouchDB Roadmap

[Robert Samuel Newson]

Hi,

An excellent summary, I agree on almost all points.

Aggregate functions in Mango will need a plan that we can carry forward to fdb 
couchdb. As noted in my opening email, the current aggregate functionality 
(“reduce”) in couchdb is the main piece that we don’t have a solution to with 
fdb (because we do not intimately control the b+tree updates).

The “cluster aware” client ticket I think we should close as “will not do”. I 
don’t think we should remove that level of abstraction.

An extra round of applause for redesigning the couchdb security system, though 
the three bullet points in 1504 add up to much less than a redesign.

B.

> On 25 Jan 2019, at 11:56, Jan Lehnardt  wrote:
> 
> Hi everyone,
> 
> this is one of the threads we promised. This one covers the roadmap 
> discussion.
> 
> This thread aims to answer:
> 
> - which version would the FDB work land in?
> - what happens with non-FDB CouchDB until then?
>  - specifically, how are the current roadmap items affected?
> - how are the two different versions being managed in the future.
> 
> Out of scope:
> 
> - whether how how we name things publicly.
> 
> The below contains my current thinking for an ideal way to handle this and it 
> is presented as a strawperson argument to be picked apart and refined, 
> designed to solicit input form everybody.
> 
> 
> Please comment on the details of The Proposal (below) as well as the 
> categorisation and comments in The Tickets (also below). There are a lot of 
> details in here, feel free to comment on just one aspect of this rather than 
> feeling compelled to make a comment on everything.
> 
> * * *
> 
> 
> Naming prelude: to make this thread easier to follow, I’ll call the current 
> `master` version of CouchDB “Erlang CouchDB” (despite there being non-Erlang 
> bits in there) and the version that would be based on FoundationDB “FBD 
> CouchDB”.
> 
> Note: these aren’t suggestions for marketing names or anything, just 
> shorthand for this particular discussion.
> 
> * * *
> 
> 
> ## Guiding Principles:
> 
> - Getting to a position where FDB CouchDB is production ready is going take a 
> while. In the meantime, Erlang CouchDB will continue to receive feature-, 
> bugfix-, and security-updates. This is *at least* until an FDB CouchDB verion 
> is available, plus a grace period for a transition that is to be discussed 
> (gut feel, this is anything between 1, 2 and 3 years).
> 
> - As a result of a multi-year dev-lifespan of Erlang CouchDB, and 
> extrapolating from experience in existing CouchDB production timespans, we 
> can easily imagine folks running Erlang CouchDB in production for *at least* 
> the next 10 years (At Neighbourhoodie, we are still supporting CouchDB 1.2.1 
> customers e.g.)
> 
> - A direct consequence of this, and in light of major feature work *at some 
> point* transitioning to FDB CouchDB, we should take this opportunity and 
> rally to make Erlang CouchDB “The Best Erlang CouchDB we can make”™. That 
> means we’ll re-evaluate the current roadmap against what’s essential to make 
> that happen, while leaving things that might be easier to do on FDB CouchDB 
> for the new setup.
> 
> 
> * * *
> 
> 
> ## The Proposal
> 
> In very broad strokes, I imagine a CouchDB 3.x series that is “The Best 
> Erlang CouchDB we can make”™ that has all the big ticket items we planned on 
> doing modulo some that we leave for FDB CouchDB because they become possible 
> or significantly easier. The main reason for calling this 3.x would be a set 
> of deprecations as well as the addition of the _access field to docs (where 
> we would want a forward compatible version of the 2.x series as well, so 
> upgrades become possible) and the imposing of additional limits to document, 
> attachment and HTTP request sizes to be within recommended best practices and 
> to be forward compatible with FDB CouchDB, but which can be brought back to 
> 2.x settings for folks who need it.
> 
> FDB CouchDB would then become the 4.x series. Even though we make major 
> changes under the hood, we would want for most CouchDB 2.x/3.x users to be 
> able to upgrade as seamlessly as possible, so we’ll aim for a high degree of 
> API compatibility like we’ve done in the 1.x -> 2.x transition and which 
> worked very well for us.
> 
> * * *
> 
> 
> ## The Annotated Roadmap
> 
> The rest of this email is a list of all tickets that currently have `roadmap` 
> label with a comment on how to proceed with it plus a few bits and bobs that 
> are relevant in the context of an FDB future.
> 
> Broadly, there are these categories:
> 
> 1. we should add this to “The Best Erlang CouchDB we can make”™, category YES
> 
> 2. this is a

[DISCUSS] CouchDB Roadmap

[Jan Lehnardt]

Hi everyone,

this is one of the threads we promised. This one covers the roadmap discussion.

This thread aims to answer:

- which version would the FDB work land in?
- what happens with non-FDB CouchDB until then?
  - specifically, how are the current roadmap items affected?
- how are the two different versions being managed in the future.

Out of scope:

- whether how how we name things publicly.

The below contains my current thinking for an ideal way to handle this and it 
is presented as a strawperson argument to be picked apart and refined, designed 
to solicit input form everybody.


Please comment on the details of The Proposal (below) as well as the 
categorisation and comments in The Tickets (also below). There are a lot of 
details in here, feel free to comment on just one aspect of this rather than 
feeling compelled to make a comment on everything.

* * *


Naming prelude: to make this thread easier to follow, I’ll call the current 
`master` version of CouchDB “Erlang CouchDB” (despite there being non-Erlang 
bits in there) and the version that would be based on FoundationDB “FBD 
CouchDB”.

Note: these aren’t suggestions for marketing names or anything, just shorthand 
for this particular discussion.

* * *


## Guiding Principles:

- Getting to a position where FDB CouchDB is production ready is going take a 
while. In the meantime, Erlang CouchDB will continue to receive feature-, 
bugfix-, and security-updates. This is *at least* until an FDB CouchDB verion 
is available, plus a grace period for a transition that is to be discussed (gut 
feel, this is anything between 1, 2 and 3 years).

- As a result of a multi-year dev-lifespan of Erlang CouchDB, and extrapolating 
from experience in existing CouchDB production timespans, we can easily imagine 
folks running Erlang CouchDB in production for *at least* the next 10 years (At 
Neighbourhoodie, we are still supporting CouchDB 1.2.1 customers e.g.)

- A direct consequence of this, and in light of major feature work *at some 
point* transitioning to FDB CouchDB, we should take this opportunity and rally 
to make Erlang CouchDB “The Best Erlang CouchDB we can make”™. That means we’ll 
re-evaluate the current roadmap against what’s essential to make that happen, 
while leaving things that might be easier to do on FDB CouchDB for the new 
setup.


* * *


## The Proposal

In very broad strokes, I imagine a CouchDB 3.x series that is “The Best Erlang 
CouchDB we can make”™ that has all the big ticket items we planned on doing 
modulo some that we leave for FDB CouchDB because they become possible or 
significantly easier. The main reason for calling this 3.x would be a set of 
deprecations as well as the addition of the _access field to docs (where we 
would want a forward compatible version of the 2.x series as well, so upgrades 
become possible) and the imposing of additional limits to document, attachment 
and HTTP request sizes to be within recommended best practices and to be 
forward compatible with FDB CouchDB, but which can be brought back to 2.x 
settings for folks who need it.

FDB CouchDB would then become the 4.x series. Even though we make major changes 
under the hood, we would want for most CouchDB 2.x/3.x users to be able to 
upgrade as seamlessly as possible, so we’ll aim for a high degree of API 
compatibility like we’ve done in the 1.x -> 2.x transition and which worked 
very well for us.

* * *


## The Annotated Roadmap

The rest of this email is a list of all tickets that currently have `roadmap` 
label with a comment on how to proceed with it plus a few bits and bobs that 
are relevant in the context of an FDB future.

Broadly, there are these categories:

1. we should add this to “The Best Erlang CouchDB we can make”™, category YES

2. this is a ticket that can be worked on independently of either CouchDB 
variant (say improvements to Mango, which can happen at any time), category 
INDIE

3. we should hold off until FDB CouchDB to tackle this feature, category WAIT


## The Tickets

### Category: YES

Build and ship view/secondary index warmer 
https://github.com/apache/couchdb/issues/1825

Comment: n/a


Update SpiderMonkey version (was: move to Chakra Core) 
https://github.com/apache/couchdb/issues/1875

Comment: we should get on this, really.


Migrate to rebar3: https://github.com/apache/couchdb/issues/1428

Comment: n/a


Improve plugin architecture / Erlang release building  
https://github.com/apache/couchdb/issues/1515

Comment: I believe there is a separate proposal for this coming.


1.x/2.x Deprecations https://github.com/apache/couchdb/issues/1534

Comment: to make upgrading to FDB CouchDB easier, we should be actively 
addressing deprecations as well as system limitations.


Retire the node-local interface (port 5986) 
https://github.com/apache/couchdb/issues/1523

Comment: Joan is already working on this.


Automate incrementally growing a cluster 
https://github.com/apache/couchdb/issues/1517

Comment: N

CouchDB Roadmap uploaded into GitHub Issues

[Joan Touzet]

Hi everyone,

>From the last CouchDB Summit (February 2017), 42 issues have been
entered into GitHub describing the foreseeable future of CouchDB. The
list included some items which have already been completed, like CI and
packaging work, pluggable storage engines, and Fauxton information
density improvements.  Those issues were not carried over.

However, what remains is some interesting work, and we'd love it if
people helped us build out the solutions to these open issues.

Some feature-focused highlights include:

* Redesign CouchDB security system - #1504
* Better support for db-per-user - #1524
* Support sub-document operations - #1498
* Support GraphQL - #1499
* Mobile First replication protocol - #1503
* First-class conflict handling - #1507
* HTTP/2 support - #1497

There's a bunch of nuts-and-bolts issues in the list, too.

You can view the whole list here:

https://s.apache.org/couchdb_roadmap

I hope this long-overdue issue filing sparks not just solid discussion,
but implementations that help drive CouchDB 3.0, 4.0 and beyond.

All the best,
Joan "We can do it!" Touzet

Re: CouchDB 1.7.0 Roadmap

[Giovanni Lenzi]

Hi Johs,

Yes, I confirm that proxy feature is already working for us.

--Giovanni

2015-11-13 8:29 GMT+01:00 Johs Ensby :

> Giovanni,
> you said earlier in a conversation about proxy
>
> > On 21. okt. 2015, at 17.23, Giovanni Lenzi  > wrote:
> > I think CouchDB already has a forward proxy feature for #1 use case (
> http://docs.couchdb.org/en/1.6.1/config/proxying.html <
> http://docs.couchdb.org/en/1.6.1/config/proxying.html>)
> > At Smileupps we use this proxying feature to give hoodie access through
> the same couchdb instance host and port.
>
> I never managed set this up, is working as describes in 1.6.1 docs and
> thus not nessecary to put on the 1.7.0 wish list to Alex?
>
> Johs
>
>
>

Re: CouchDB 1.7.0 Roadmap

[Oskar Maria Grande]

Hi awesome CouchDB folks,

my friend Peter and I would very much enjoy evaluating the mentioned auth 
topics, JWT is especially intriguing to us.

> May be we can also include else experimental features, like JWT and/or
> Delegated auth. Personally, I would like to see them, but it's all up
> to you Klaus and Jan (;

Peter is studying at Stockholm University and we hope to be able to use an 
interesting (security as well as Erlang related) topic for his thesis and maybe 
to work on together in general :)

Suffice to say we’ve both been huge CouchDB fans and this would mean quite a 
lot to us in terms of finally getting our hands dirty with Couch internals!

Any personal 2c would be very much welcome!

Thanks and cheers from Vienna,

Oskar & Peter

# Oskar Maria Grande | @musha68k
# http://daruma.io
# +43 676 955 3646

> On 12 Nov 2015, at 16:05, Alexander Shorin  wrote:
> 
> Dear CouchDB team,
> 
> While we're all working on 2.0 is in progress, I fear that we'll end
> this year without a single release. Technically, there is only one
> month left till 2016 excluding holidays, but let's be honest - that's
> not enough for 2.0. So I propose the plan for 1.7 release to not end
> this year with empty list.
> 
> There are a couple of important changes that we have for it and users
> are waiting for. Primary is the Erlang 18 compatibility, but not only.
> 
> What we already have on 1.x.x branch:
> 
> - COUCHDB-1011: replicate by document ids from futon
> - COUCHDB-1275: decode database names in recent used list
> - COUCHDB-2225 Enforce that shared libraries can be built by the system
> - COUCHDB-2430: Disable Nagle's algorithm
> - COUCHDB-2583: fix connection dropping by the resources which doesn't
> require any payload
> - COUCHDB-2761: Support glibc >= 2.20
> - COUCHDB-2783: Bind both to IPv4 and IPv6
> - Futon: Fixed potential XSS issue in jquery.ui
> - jquery.couch: Fixed document copying
> - sslv3 support is deprecated
> - Support for user configurable SSL ciphers
> - Multiple minor documentation fixes
> - Support Erlang 18
> 
> What we can backport without worry:
> 
> - COUCHDB-1356: Return username on POST to /_session
> - COUCHDB-1447: X-Couch-* headers missed if custom headers were returned
> - COUCHDB-1964: eunit test suite
> - COUCHDB-2310: /db/_bulk_get
> - COUCHDB-2375: Respond with HTTP 400 Bad Request on invalid revision number
> - COUCHDB-2534: db security should respect authed users
> - COUCHDB-2732: Use thread local storage for couch_ejson_compare NIF
> - COUCHDB-2752: Validate Host header
> - COUCHDB-2873: Update snappy to 1.1.3
> - Multiple improvements that we have for replicator
> 
> What I would like to add:
> 
> - COUCHDB-2722: Keys from rewrited query params should be blank when
> not specified in the URI
> - COUCHDB-2874: Rewrites via query server
> - COUCHDB-2877: Return nicer error for bad Authorization header
> - Deprecation of /_log
> - Deprecation of OAuth auth
> - Enable CORS by default:
> https://fetch.spec.whatwg.org/#basic-safe-cors-protocol-setup
> - Remove Fauxton - AFAIK, it supports 1.x no more and current version
> in 1.x.x branch is heavily outdated.
> - Mark this release as LTS with short (really) cycle of bug fixes ship
> 
> Questionalbe:
> - Add systemd notification support.
> 
> May be we can also include else experimental features, like JWT and/or
> Delegated auth. Personally, I would like to see them, but it's all up
> to you Klaus and Jan (;
> 
> But even without these experimental features, we have quite long list
> of changes to ship.
> 
> The plan is simple: for November get all from backport and add lists
> into 1.x.x branch and ship 1.7 in first half of December. Quite good
> Christmas Eve present for everyone. Personal deadlines 30th November
> and 20th December respectively.
> 
> Since "everyone is busy on 2.0" I'll take care of this.
> 
> P.S. If someone has else important bugfixes on mind to include, please
> drop a notice. For 2.0 we have ETOOMANY useful changes, but I would
> like to stop only on really important ones. Like replicator ones as I
> mentioned.
> 
> --
> ,,,^..^,,,
> 

Re: CouchDB 1.7.0 Roadmap

[Johs Ensby]

Giovanni,
you said earlier in a conversation about proxy

> On 21. okt. 2015, at 17.23, Giovanni Lenzi  > wrote:
> I think CouchDB already has a forward proxy feature for #1 use case 
> (http://docs.couchdb.org/en/1.6.1/config/proxying.html 
> )
> At Smileupps we use this proxying feature to give hoodie access through the 
> same couchdb instance host and port. 

I never managed set this up, is working as describes in 1.6.1 docs and thus not 
nessecary to put on the 1.7.0 wish list to Alex?

Johs

Re: CouchDB 1.7.0 Roadmap

[Giovanni Lenzi]

Ooppss, sent to public ML. Sorry for the partial off-topic. You can reply
privately if you want.

Thanks

--Giovanni

2015-11-12 16:54 GMT+01:00 Giovanni Lenzi <g.le...@smileupps.com>:

> Hi Alexander,
>
> I'm writing you privately because I don't want the main thread to go
> off-topic
>
> Thanks for your roadmap to 1.7.0.. seems very very juicy!
>
> About the "COUCHDB-2752: Validate Host header" in your list. As
> documented here (https://issues.apache.org/jira/browse/COUCHDB-2752), to
> me it seems that flag can perfectly be used in the proposed way here:
> http://couchdb.markmail.org/message/q2623pzw7lt73lcg?q=CouchDB+secure+even+withouth+a+proxy
>
> Do you confirm?
>
>
> --Giovanni
>
> 2015-11-12 16:05 GMT+01:00 Alexander Shorin <kxe...@gmail.com>:
>
>> Dear CouchDB team,
>>
>> While we're all working on 2.0 is in progress, I fear that we'll end
>> this year without a single release. Technically, there is only one
>> month left till 2016 excluding holidays, but let's be honest - that's
>> not enough for 2.0. So I propose the plan for 1.7 release to not end
>> this year with empty list.
>>
>> There are a couple of important changes that we have for it and users
>> are waiting for. Primary is the Erlang 18 compatibility, but not only.
>>
>> What we already have on 1.x.x branch:
>>
>> - COUCHDB-1011: replicate by document ids from futon
>> - COUCHDB-1275: decode database names in recent used list
>> - COUCHDB-2225 Enforce that shared libraries can be built by the system
>> - COUCHDB-2430: Disable Nagle's algorithm
>> - COUCHDB-2583: fix connection dropping by the resources which doesn't
>> require any payload
>> - COUCHDB-2761: Support glibc >= 2.20
>> - COUCHDB-2783: Bind both to IPv4 and IPv6
>> - Futon: Fixed potential XSS issue in jquery.ui
>> - jquery.couch: Fixed document copying
>> - sslv3 support is deprecated
>> - Support for user configurable SSL ciphers
>> - Multiple minor documentation fixes
>> - Support Erlang 18
>>
>> What we can backport without worry:
>>
>> - COUCHDB-1356: Return username on POST to /_session
>> - COUCHDB-1447: X-Couch-* headers missed if custom headers were returned
>> - COUCHDB-1964: eunit test suite
>> - COUCHDB-2310: /db/_bulk_get
>> - COUCHDB-2375: Respond with HTTP 400 Bad Request on invalid revision
>> number
>> - COUCHDB-2534: db security should respect authed users
>> - COUCHDB-2732: Use thread local storage for couch_ejson_compare NIF
>> - COUCHDB-2752: Validate Host header
>> - COUCHDB-2873: Update snappy to 1.1.3
>> - Multiple improvements that we have for replicator
>>
>> What I would like to add:
>>
>> - COUCHDB-2722: Keys from rewrited query params should be blank when
>> not specified in the URI
>> - COUCHDB-2874: Rewrites via query server
>> - COUCHDB-2877: Return nicer error for bad Authorization header
>> - Deprecation of /_log
>> - Deprecation of OAuth auth
>> - Enable CORS by default:
>> https://fetch.spec.whatwg.org/#basic-safe-cors-protocol-setup
>> - Remove Fauxton - AFAIK, it supports 1.x no more and current version
>> in 1.x.x branch is heavily outdated.
>> - Mark this release as LTS with short (really) cycle of bug fixes ship
>>
>> Questionalbe:
>> - Add systemd notification support.
>>
>> May be we can also include else experimental features, like JWT and/or
>> Delegated auth. Personally, I would like to see them, but it's all up
>> to you Klaus and Jan (;
>>
>> But even without these experimental features, we have quite long list
>> of changes to ship.
>>
>> The plan is simple: for November get all from backport and add lists
>> into 1.x.x branch and ship 1.7 in first half of December. Quite good
>> Christmas Eve present for everyone. Personal deadlines 30th November
>> and 20th December respectively.
>>
>> Since "everyone is busy on 2.0" I'll take care of this.
>>
>> P.S. If someone has else important bugfixes on mind to include, please
>> drop a notice. For 2.0 we have ETOOMANY useful changes, but I would
>> like to stop only on really important ones. Like replicator ones as I
>> mentioned.
>>
>> --
>> ,,,^..^,,,
>>
>
>

Re: CouchDB 1.7.0 Roadmap

[Alexander Shorin]

On Thu, Nov 12, 2015 at 6:54 PM, Giovanni Lenzi <g.le...@smileupps.com> wrote:
> I'm writing you privately because I don't want the main thread to go
> off-topic

Well, no (:

> Thanks for your roadmap to 1.7.0.. seems very very juicy!
>
> About the "COUCHDB-2752: Validate Host header" in your list. As documented
> here (https://issues.apache.org/jira/browse/COUCHDB-2752), to me it seems
> that flag can perfectly be used in the proposed way here:
> http://couchdb.markmail.org/message/q2623pzw7lt73lcg?q=CouchDB+secure+even+withouth+a+proxy
>
> Do you confirm?

After quick look, it seems so.

--
,,,^..^,,,

CouchDB 1.7.0 Roadmap

[Alexander Shorin]

Dear CouchDB team,

While we're all working on 2.0 is in progress, I fear that we'll end
this year without a single release. Technically, there is only one
month left till 2016 excluding holidays, but let's be honest - that's
not enough for 2.0. So I propose the plan for 1.7 release to not end
this year with empty list.

There are a couple of important changes that we have for it and users
are waiting for. Primary is the Erlang 18 compatibility, but not only.

What we already have on 1.x.x branch:

- COUCHDB-1011: replicate by document ids from futon
- COUCHDB-1275: decode database names in recent used list
- COUCHDB-2225 Enforce that shared libraries can be built by the system
- COUCHDB-2430: Disable Nagle's algorithm
- COUCHDB-2583: fix connection dropping by the resources which doesn't
require any payload
- COUCHDB-2761: Support glibc >= 2.20
- COUCHDB-2783: Bind both to IPv4 and IPv6
- Futon: Fixed potential XSS issue in jquery.ui
- jquery.couch: Fixed document copying
- sslv3 support is deprecated
- Support for user configurable SSL ciphers
- Multiple minor documentation fixes
- Support Erlang 18

What we can backport without worry:

- COUCHDB-1356: Return username on POST to /_session
- COUCHDB-1447: X-Couch-* headers missed if custom headers were returned
- COUCHDB-1964: eunit test suite
- COUCHDB-2310: /db/_bulk_get
- COUCHDB-2375: Respond with HTTP 400 Bad Request on invalid revision number
- COUCHDB-2534: db security should respect authed users
- COUCHDB-2732: Use thread local storage for couch_ejson_compare NIF
- COUCHDB-2752: Validate Host header
- COUCHDB-2873: Update snappy to 1.1.3
- Multiple improvements that we have for replicator

What I would like to add:

- COUCHDB-2722: Keys from rewrited query params should be blank when
not specified in the URI
- COUCHDB-2874: Rewrites via query server
- COUCHDB-2877: Return nicer error for bad Authorization header
- Deprecation of /_log
- Deprecation of OAuth auth
- Enable CORS by default:
https://fetch.spec.whatwg.org/#basic-safe-cors-protocol-setup
- Remove Fauxton - AFAIK, it supports 1.x no more and current version
in 1.x.x branch is heavily outdated.
- Mark this release as LTS with short (really) cycle of bug fixes ship

Questionalbe:
- Add systemd notification support.

May be we can also include else experimental features, like JWT and/or
Delegated auth. Personally, I would like to see them, but it's all up
to you Klaus and Jan (;

But even without these experimental features, we have quite long list
of changes to ship.

The plan is simple: for November get all from backport and add lists
into 1.x.x branch and ship 1.7 in first half of December. Quite good
Christmas Eve present for everyone. Personal deadlines 30th November
and 20th December respectively.

Since "everyone is busy on 2.0" I'll take care of this.

P.S. If someone has else important bugfixes on mind to include, please
drop a notice. For 2.0 we have ETOOMANY useful changes, but I would
like to stop only on really important ones. Like replicator ones as I
mentioned.

--
,,,^..^,,,

Re: CouchDB 1.7.0 Roadmap

[Giovanni Lenzi]

Hi Alexander,

I'm writing you privately because I don't want the main thread to go
off-topic

Thanks for your roadmap to 1.7.0.. seems very very juicy!

About the "COUCHDB-2752: Validate Host header" in your list. As documented
here (https://issues.apache.org/jira/browse/COUCHDB-2752), to me it seems
that flag can perfectly be used in the proposed way here:
http://couchdb.markmail.org/message/q2623pzw7lt73lcg?q=CouchDB+secure+even+withouth+a+proxy

Do you confirm?


--Giovanni

2015-11-12 16:05 GMT+01:00 Alexander Shorin <kxe...@gmail.com>:

> Dear CouchDB team,
>
> While we're all working on 2.0 is in progress, I fear that we'll end
> this year without a single release. Technically, there is only one
> month left till 2016 excluding holidays, but let's be honest - that's
> not enough for 2.0. So I propose the plan for 1.7 release to not end
> this year with empty list.
>
> There are a couple of important changes that we have for it and users
> are waiting for. Primary is the Erlang 18 compatibility, but not only.
>
> What we already have on 1.x.x branch:
>
> - COUCHDB-1011: replicate by document ids from futon
> - COUCHDB-1275: decode database names in recent used list
> - COUCHDB-2225 Enforce that shared libraries can be built by the system
> - COUCHDB-2430: Disable Nagle's algorithm
> - COUCHDB-2583: fix connection dropping by the resources which doesn't
> require any payload
> - COUCHDB-2761: Support glibc >= 2.20
> - COUCHDB-2783: Bind both to IPv4 and IPv6
> - Futon: Fixed potential XSS issue in jquery.ui
> - jquery.couch: Fixed document copying
> - sslv3 support is deprecated
> - Support for user configurable SSL ciphers
> - Multiple minor documentation fixes
> - Support Erlang 18
>
> What we can backport without worry:
>
> - COUCHDB-1356: Return username on POST to /_session
> - COUCHDB-1447: X-Couch-* headers missed if custom headers were returned
> - COUCHDB-1964: eunit test suite
> - COUCHDB-2310: /db/_bulk_get
> - COUCHDB-2375: Respond with HTTP 400 Bad Request on invalid revision
> number
> - COUCHDB-2534: db security should respect authed users
> - COUCHDB-2732: Use thread local storage for couch_ejson_compare NIF
> - COUCHDB-2752: Validate Host header
> - COUCHDB-2873: Update snappy to 1.1.3
> - Multiple improvements that we have for replicator
>
> What I would like to add:
>
> - COUCHDB-2722: Keys from rewrited query params should be blank when
> not specified in the URI
> - COUCHDB-2874: Rewrites via query server
> - COUCHDB-2877: Return nicer error for bad Authorization header
> - Deprecation of /_log
> - Deprecation of OAuth auth
> - Enable CORS by default:
> https://fetch.spec.whatwg.org/#basic-safe-cors-protocol-setup
> - Remove Fauxton - AFAIK, it supports 1.x no more and current version
> in 1.x.x branch is heavily outdated.
> - Mark this release as LTS with short (really) cycle of bug fixes ship
>
> Questionalbe:
> - Add systemd notification support.
>
> May be we can also include else experimental features, like JWT and/or
> Delegated auth. Personally, I would like to see them, but it's all up
> to you Klaus and Jan (;
>
> But even without these experimental features, we have quite long list
> of changes to ship.
>
> The plan is simple: for November get all from backport and add lists
> into 1.x.x branch and ship 1.7 in first half of December. Quite good
> Christmas Eve present for everyone. Personal deadlines 30th November
> and 20th December respectively.
>
> Since "everyone is busy on 2.0" I'll take care of this.
>
> P.S. If someone has else important bugfixes on mind to include, please
> drop a notice. For 2.0 we have ETOOMANY useful changes, but I would
> like to stop only on really important ones. Like replicator ones as I
> mentioned.
>
> --
> ,,,^..^,,,
>

Re: CouchDB 1.7.0 Roadmap

[Alexander Shorin]

On Thu, Nov 12, 2015 at 7:20 PM, Tomas Novysedlak
 wrote:
>
>   Just a quick one. How about long promised
> https://issues.apache.org/jira/browse/COUCHDB-1415 ?

I'll take a look, but no promises. I'm also not quite sure which exact
changes fixed this for 2.0 and will it be safe to backport them.

--
,,,^..^,,,

Re: CouchDB 1.7.0 Roadmap

[Tomas Novysedlak]

Hi Alexander,

  Just a quick one. How about long promised
https://issues.apache.org/jira/browse/COUCHDB-1415 ?

Thank you,
  Tomas


On Thu, Nov 12, 2015 at 4:05 PM, Alexander Shorin  wrote:

> Dear CouchDB team,
>
> While we're all working on 2.0 is in progress, I fear that we'll end
> this year without a single release. Technically, there is only one
> month left till 2016 excluding holidays, but let's be honest - that's
> not enough for 2.0. So I propose the plan for 1.7 release to not end
> this year with empty list.
>
> There are a couple of important changes that we have for it and users
> are waiting for. Primary is the Erlang 18 compatibility, but not only.
>
> What we already have on 1.x.x branch:
>
> - COUCHDB-1011: replicate by document ids from futon
> - COUCHDB-1275: decode database names in recent used list
> - COUCHDB-2225 Enforce that shared libraries can be built by the system
> - COUCHDB-2430: Disable Nagle's algorithm
> - COUCHDB-2583: fix connection dropping by the resources which doesn't
> require any payload
> - COUCHDB-2761: Support glibc >= 2.20
> - COUCHDB-2783: Bind both to IPv4 and IPv6
> - Futon: Fixed potential XSS issue in jquery.ui
> - jquery.couch: Fixed document copying
> - sslv3 support is deprecated
> - Support for user configurable SSL ciphers
> - Multiple minor documentation fixes
> - Support Erlang 18
>
> What we can backport without worry:
>
> - COUCHDB-1356: Return username on POST to /_session
> - COUCHDB-1447: X-Couch-* headers missed if custom headers were returned
> - COUCHDB-1964: eunit test suite
> - COUCHDB-2310: /db/_bulk_get
> - COUCHDB-2375: Respond with HTTP 400 Bad Request on invalid revision
> number
> - COUCHDB-2534: db security should respect authed users
> - COUCHDB-2732: Use thread local storage for couch_ejson_compare NIF
> - COUCHDB-2752: Validate Host header
> - COUCHDB-2873: Update snappy to 1.1.3
> - Multiple improvements that we have for replicator
>
> What I would like to add:
>
> - COUCHDB-2722: Keys from rewrited query params should be blank when
> not specified in the URI
> - COUCHDB-2874: Rewrites via query server
> - COUCHDB-2877: Return nicer error for bad Authorization header
> - Deprecation of /_log
> - Deprecation of OAuth auth
> - Enable CORS by default:
> https://fetch.spec.whatwg.org/#basic-safe-cors-protocol-setup
> - Remove Fauxton - AFAIK, it supports 1.x no more and current version
> in 1.x.x branch is heavily outdated.
> - Mark this release as LTS with short (really) cycle of bug fixes ship
>
> Questionalbe:
> - Add systemd notification support.
>
> May be we can also include else experimental features, like JWT and/or
> Delegated auth. Personally, I would like to see them, but it's all up
> to you Klaus and Jan (;
>
> But even without these experimental features, we have quite long list
> of changes to ship.
>
> The plan is simple: for November get all from backport and add lists
> into 1.x.x branch and ship 1.7 in first half of December. Quite good
> Christmas Eve present for everyone. Personal deadlines 30th November
> and 20th December respectively.
>
> Since "everyone is busy on 2.0" I'll take care of this.
>
> P.S. If someone has else important bugfixes on mind to include, please
> drop a notice. For 2.0 we have ETOOMANY useful changes, but I would
> like to stop only on really important ones. Like replicator ones as I
> mentioned.
>
> --
> ,,,^..^,,,
>

Re: Roadmap process and merge procedure

[Noah Slater]

Following up on this. We are two weeks away from the next release.

Bob, you mentioned on IRC that you could help me with this. What time is
convenient for you?


On 17 April 2013 16:01, Noah Slater nsla...@apache.org wrote:

 I'll let someone more familiar with Git, and the conversations we had
 around this, answer.

 To be honest, I am less interested in debating the specifics of the
 proposal than I am about actually getting a proposal agreed upon, and
 putting it into practice. We are a little over a week away from the next
 bugfix release window, and a little over two months away from the next
 feature release window.

 I need someone to help me put this stuff into motion. We keep talking
 about the new release process / merge procedure, but it's been almost a
 year now, and nothing has happened. (And I am as much if not more so to
 blame for that as anyone else.)

 What would be great is if someone who was involved in fleshing out the
 original proposal (Bob, Paul, Jan, etc) could lend a hand and pair up with
 me for an evening to get all this put into motion.


 On 17 April 2013 15:40, Dirkjan Ochtman dirk...@ochtman.nl wrote:

 How does that impact master vs. 1.4.x?

 On Wed, Apr 17, 2013 at 4:36 PM, Noah Slater nsla...@apache.org wrote:
  The goal was that you only merge in features when they are ready, and
 come
  with tests, and docs, and what have you. And that you actually call a
 lazy
  consensus merge request on dev@ before you can merge in.
 
 
 
 
  On 17 April 2013 15:18, Dirkjan Ochtman dirk...@ochtman.nl wrote:
 
  On Wed, Apr 17, 2013 at 4:02 PM, Noah Slater nsla...@apache.org
 wrote:
   So that features can be merged into it as they become ready.
  
   Check out:
  
   http://wiki.apache.org/couchdb/Merge_Procedure
  
   Thoughts?
 
  Seems like you could call the next-feature-release branch master,
  and not have to start a new branch every time.
 
  Cheers,
 
  Dirkjan
 
 
 
 
  --
  NS




 --
 NS




-- 
NS

Re: Roadmap process and merge procedure

[Noah Slater]

Hey guys,

Just following up on this, as it's been five days. And I can't move ahead
without help.

At a minimum, we need to create the 1.4.x branch. But I'd really like for
us to properly document our merge procedure (which has languished because I
don't know Git well enough to fill out the details) and then for us to call
a vote on it, so that we can move forward as we originally planned to, with
features only landing on a release branch with tests, docs, etc.

If someone wants to volunteer to pair with me, perhaps we can knock this
out?

Thanks,


On 12 April 2013 13:31, Noah Slater nsla...@apache.org wrote:

 Devs,

 With 1.3.0 out, it is time we revisit these docs:

  http://wiki.apache.org/couchdb/Roadmap_Process

 http://wiki.apache.org/couchdb/Merge_Procedure

 Our next bugfix release target is 1st May.

 Our next feature release target is 1st July.

 What do we need to do *now* to prepare for these?

 Do we need to create release branches?

 If yes, can we do it, and can we document it, so I can bake it into the
 release procedure.

 Do we need to communicate how to merge in work for the releases?

 Remember we discussed (and sort of half documented in that second wiki
 page) that to merge in code for a release, you have to do a VOTE on a
 merge request to dev, and you have to include tests, and docs, and what
 have you, and we do lazy consensus on it.

 If we're serious about this release cadence (and I think we all are) then
 we need to sort this stuff out ASAP, and communicate it, and actually put
 it into practice.

 I need your help though, because I am still learning Git, etc. So I am
 hoping that a few of you can weigh in on the current state of the merge
 procedure proposal, and maybe fix it up. Then, once we're happy with it, I
 can do the DISCUSS/VOTE dance on dev@ and make this official.

 I am super excited about this.

 Thanks,

 --
 NS




-- 
NS

Re: Roadmap process and merge procedure

[Dirkjan Ochtman]

On Wed, Apr 17, 2013 at 3:45 PM, Noah Slater nsla...@apache.org wrote:
 At a minimum, we need to create the 1.4.x branch. But I'd really like for
 us to properly document our merge procedure (which has languished because I
 don't know Git well enough to fill out the details) and then for us to call
 a vote on it, so that we can move forward as we originally planned to, with
 features only landing on a release branch with tests, docs, etc.

Sorry, this might just be me, but why would you want to branch for
1.4.x now instead of, like, 2 (or 4) weeks before the intended release
date?

Cheers,

Dirkjan

Re: Roadmap process and merge procedure

[Noah Slater]

So that features can be merged into it as they become ready.

Check out:

http://wiki.apache.org/couchdb/Merge_Procedure

Thoughts?


On 17 April 2013 14:47, Dirkjan Ochtman dirk...@ochtman.nl wrote:

 On Wed, Apr 17, 2013 at 3:45 PM, Noah Slater nsla...@apache.org wrote:
  At a minimum, we need to create the 1.4.x branch. But I'd really like for
  us to properly document our merge procedure (which has languished
 because I
  don't know Git well enough to fill out the details) and then for us to
 call
  a vote on it, so that we can move forward as we originally planned to,
 with
  features only landing on a release branch with tests, docs, etc.

 Sorry, this might just be me, but why would you want to branch for
 1.4.x now instead of, like, 2 (or 4) weeks before the intended release
 date?

 Cheers,

 Dirkjan




-- 
NS

Re: Roadmap process and merge procedure

[Dirkjan Ochtman]

On Wed, Apr 17, 2013 at 4:02 PM, Noah Slater nsla...@apache.org wrote:
 So that features can be merged into it as they become ready.

 Check out:

 http://wiki.apache.org/couchdb/Merge_Procedure

 Thoughts?

Seems like you could call the next-feature-release branch master,
and not have to start a new branch every time.

Cheers,

Dirkjan

Re: Roadmap process and merge procedure

[Noah Slater]

The goal was that you only merge in features when they are ready, and come
with tests, and docs, and what have you. And that you actually call a lazy
consensus merge request on dev@ before you can merge in.




On 17 April 2013 15:18, Dirkjan Ochtman dirk...@ochtman.nl wrote:

 On Wed, Apr 17, 2013 at 4:02 PM, Noah Slater nsla...@apache.org wrote:
  So that features can be merged into it as they become ready.
 
  Check out:
 
  http://wiki.apache.org/couchdb/Merge_Procedure
 
  Thoughts?

 Seems like you could call the next-feature-release branch master,
 and not have to start a new branch every time.

 Cheers,

 Dirkjan




-- 
NS

Re: Roadmap process and merge procedure

[Noah Slater]

I'll let someone more familiar with Git, and the conversations we had
around this, answer.

To be honest, I am less interested in debating the specifics of the
proposal than I am about actually getting a proposal agreed upon, and
putting it into practice. We are a little over a week away from the next
bugfix release window, and a little over two months away from the next
feature release window.

I need someone to help me put this stuff into motion. We keep talking about
the new release process / merge procedure, but it's been almost a year now,
and nothing has happened. (And I am as much if not more so to blame for
that as anyone else.)

What would be great is if someone who was involved in fleshing out the
original proposal (Bob, Paul, Jan, etc) could lend a hand and pair up with
me for an evening to get all this put into motion.


On 17 April 2013 15:40, Dirkjan Ochtman dirk...@ochtman.nl wrote:

 How does that impact master vs. 1.4.x?

 On Wed, Apr 17, 2013 at 4:36 PM, Noah Slater nsla...@apache.org wrote:
  The goal was that you only merge in features when they are ready, and
 come
  with tests, and docs, and what have you. And that you actually call a
 lazy
  consensus merge request on dev@ before you can merge in.
 
 
 
 
  On 17 April 2013 15:18, Dirkjan Ochtman dirk...@ochtman.nl wrote:
 
  On Wed, Apr 17, 2013 at 4:02 PM, Noah Slater nsla...@apache.org
 wrote:
   So that features can be merged into it as they become ready.
  
   Check out:
  
   http://wiki.apache.org/couchdb/Merge_Procedure
  
   Thoughts?
 
  Seems like you could call the next-feature-release branch master,
  and not have to start a new branch every time.
 
  Cheers,
 
  Dirkjan
 
 
 
 
  --
  NS




-- 
NS

Roadmap process and merge procedure

[Noah Slater]

Devs,

With 1.3.0 out, it is time we revisit these docs:

 http://wiki.apache.org/couchdb/Roadmap_Process

http://wiki.apache.org/couchdb/Merge_Procedure

Our next bugfix release target is 1st May.

Our next feature release target is 1st July.

What do we need to do *now* to prepare for these?

Do we need to create release branches?

If yes, can we do it, and can we document it, so I can bake it into the
release procedure.

Do we need to communicate how to merge in work for the releases?

Remember we discussed (and sort of half documented in that second wiki
page) that to merge in code for a release, you have to do a VOTE on a
merge request to dev, and you have to include tests, and docs, and what
have you, and we do lazy consensus on it.

If we're serious about this release cadence (and I think we all are) then
we need to sort this stuff out ASAP, and communicate it, and actually put
it into practice.

I need your help though, because I am still learning Git, etc. So I am
hoping that a few of you can weigh in on the current state of the merge
procedure proposal, and maybe fix it up. Then, once we're happy with it, I
can do the DISCUSS/VOTE dance on dev@ and make this official.

I am super excited about this.

Thanks,

-- 
NS

Re: [PROPOSAL] Official roadmap, and merge procedure

[Jan Lehnardt]

On Jun 17, 2012, at 20:48 , Paul Davis wrote:

 The only thing I see an issue with is the merge commit policy.
 
 You'll find quite quickly that you'll have situations that require
 merge commits with diff's to work properly. A traditional example is
 bug fixes that reach back through many release branches. If the RM
 rejects anything that's not a clean merge then what the committer is
 required to do is either write three different commits tailored to
 each release branch (which defeats the purpose of the merge) or merge
 work back to a release branch and then submit the merge commit as a
 commit to be merged into the release branch (which makes my brain hurt
 just thinking about it).
 
 If you really want the RM to be a gate keeper/reviewer what I'd
 suggest is that we take a hard line against working on master at all.
 When we want to submit work we do the email thing that'll include a
 list of commits, one per branch. Ie, something like:
 
1234-master-cool
1234-1.4.x-cool
 
 And then we do our review dance and CI checks. (Also, if one commit
 works on all branches, that's fine, this just allows a degree of
 freedom for when things don't apply the same everywhere). Along with
 the rational for tests and docs rational for why things are back
 ported vs are etc etc should be included in the email to dev@. Also,
 we'll want to get a template/example/something up on the net so people
 know what to submit when they do submit.
 
 And the actual push to branches happens with one of the RM's reviewing
 it and then doing a rebase onto those branches (and rejecting as
 failed).
 
 This way we have single commits per feature/fix/whatever on all of our
 release branches but we allow the committer to produce the commits
 that will inevitably have to be tailored to each branch in some cases.

I think we sorta went through this in Dublin, but were a bit handwavey
with details. Thanks for writing this up. Could you add a summary or
run-book type thing to the Merge_procedure wiki page?

Cheers
Jan
-- 



 
 
 
 On Sun, Jun 17, 2012 at 11:48 AM, Noah Slater nsla...@tumbolia.org wrote:
 Thanks for the comments guys.
 
 Bob was in the room when this proposal was drafted (literally) but Paul is
 our other active release manager, so I want to wait for his review before I
 comment on anything in the thread. I am also interested to hear what Jason
 Smith thinks, and indeed, anyone else from Cloudant, or other downstream
 CouchDB individuals this may effect. If you're interested in helping the
 CouchDB community test CouchDB, through QA, CI, or any other method, then
 your should also chime in here.
 
 On Sun, Jun 17, 2012 at 4:28 PM, Jan Lehnardt j...@apache.org wrote:
 
 
 On Jun 16, 2012, at 18:45 , Noah Slater wrote:
 
 Devs,
 
 A few of us met in Dublin recently, and we discussed the project roadmap.
 
 Key takeaways from that meeting:
 
 1. We'd like to proposed formal time-based releases
 
 
 2. Branch and hack all you like, but if you want to ship something, you
 have to submit a merge request to an active release branch. Before you do
 this, you should follow a test procedure. And before your request will be
 accepted, it will be put through QA by the community.
 
 
 Details of these proposals can be found here:
 
 http://wiki.apache.org/couchdb/Roadmap_Process
 
 
 http://wiki.apache.org/couchdb/Merge_Procedure
 
 
 Please reply back to this thread with your comments on the proposals.
 
 (The last one needs to be fleshed out, a little...)
 
 In particular this bit:
 
 Feature branches are merged to release branches using 'git merge --no-ff
 FEATURE BRANCH'. This guarantees a merge commit, which are the only kinds
 of commit that will appear on release branches. If the merge results in
 conflicts, the release manager rejects the merge commit with an reply to
 the dev@ thread. If the merge is successful, the release manager should
 run 'make distcheck' and push the merge upstream if the tests pass.
 
 Merges are currently not allowed on release branches and master. IIRC we
 came up with git merge --no-ff being safe to enable, but this currently
 is still disabled.
 
 Cheers
 Jan
 --
 
 

Re: [PROPOSAL] Official roadmap, and merge procedure

[Benoit Chesneau]

bump.

On Mon, Jun 18, 2012 at 10:29 AM, Benoit Chesneau bchesn...@gmail.com wrote:
 On Sat, Jun 16, 2012 at 6:45 PM, Noah Slater nsla...@tumbolia.org wrote:
 Devs,

 A few of us met in Dublin recently, and we discussed the project roadmap.

 Key takeaways from that meeting:

 1. We'd like to proposed formal time-based releases


 2. Branch and hack all you like, but if you want to ship something, you
 have to submit a merge request to an active release branch. Before you do
 this, you should follow a test procedure. And before your request will be
 accepted, it will be put through QA by the community.


 Details of these proposals can be found here:

 http://wiki.apache.org/couchdb/Roadmap_Process


 http://wiki.apache.org/couchdb/Merge_Procedure


 Please reply back to this thread with your comments on the proposals.

 (The last one needs to be fleshed out, a little...)

 Thanks,

 N

 Thanks for these wikis.  Roadmap process is good but the merge
 procedure is a little obscure.

 *) What happen in master during the release procedure? Are you
 freezing it ? Imo we should freeze it except if we want to reedit the
 current nightmare. Freezing the master give the following advantage:

 - focus developers on the release
 - make sure anything done for the release will be easily merged in the master.

 Imo this freeze shouldn't be a problem since we have the
 topic/features branches to continue devs on other things. or remote
 branch. Anyway this should be clear on the wiki.

 *) You're speaking about merge, but what if a bugfix only goes for a
 release and doesn't apply to master and other releases branches? I'm
 thinking to  2 scenari there:

 - bug only happen in a release branch and has only be raised here.
 Where should the bugfix should be applied first? Do we care to try to
 merge it in other branches (painful and open the door to other bugs)
 - bug is found in the master: what is the procedure to check if this
 bu g apply to other branch

 *) related to above: release manager: whos is (s)he ? Only one guy? Do
 we have like in perl or debian a release manager / major versions ?

 Having one release manager / version would do the trick, since he
 would be supposed to know the state of the last version of his release
 (1.x, 2.x) ... And how bugs can be applied in.

 Anyway hope we can answer to these questions.

 - benoît

Re: [PROPOSAL] Official roadmap, and merge procedure

[Benoit Chesneau]

On Sat, Jun 16, 2012 at 6:45 PM, Noah Slater nsla...@tumbolia.org wrote:
 Devs,

 A few of us met in Dublin recently, and we discussed the project roadmap.

 Key takeaways from that meeting:

 1. We'd like to proposed formal time-based releases


 2. Branch and hack all you like, but if you want to ship something, you
 have to submit a merge request to an active release branch. Before you do
 this, you should follow a test procedure. And before your request will be
 accepted, it will be put through QA by the community.


 Details of these proposals can be found here:

 http://wiki.apache.org/couchdb/Roadmap_Process


 http://wiki.apache.org/couchdb/Merge_Procedure


 Please reply back to this thread with your comments on the proposals.

 (The last one needs to be fleshed out, a little...)

 Thanks,

 N

Thanks for these wikis.  Roadmap process is good but the merge
procedure is a little obscure.

*) What happen in master during the release procedure? Are you
freezing it ? Imo we should freeze it except if we want to reedit the
current nightmare. Freezing the master give the following advantage:

- focus developers on the release
- make sure anything done for the release will be easily merged in the master.

Imo this freeze shouldn't be a problem since we have the
topic/features branches to continue devs on other things. or remote
branch. Anyway this should be clear on the wiki.

*) You're speaking about merge, but what if a bugfix only goes for a
release and doesn't apply to master and other releases branches? I'm
thinking to  2 scenari there:

- bug only happen in a release branch and has only be raised here.
Where should the bugfix should be applied first? Do we care to try to
merge it in other branches (painful and open the door to other bugs)
- bug is found in the master: what is the procedure to check if this
bu g apply to other branch

*) related to above: release manager: whos is (s)he ? Only one guy? Do
we have like in perl or debian a release manager / major versions ?

Having one release manager / version would do the trick, since he
would be supposed to know the state of the last version of his release
(1.x, 2.x) ... And how bugs can be applied in.

Anyway hope we can answer to these questions.

- benoît

Re: [PROPOSAL] Official roadmap, and merge procedure

[Jan Lehnardt]

Heya,

I'm in favour of the proposal. I recognise that there are a few things
that need to be refined, but I think what we have is good enough to get
going and refine as we go along.

With QA for example, it is up to the release master to judge whether a
release is good to go. If the community QA doesn't provide the required
results, the release master just won't do a release and thus, we hope,
encourage more QA :) Down the road, we might be able to formalise QA
more, so we have better strict criteria when a release can go. Ultimately
though, this is the release master's call.

In addition, I'd like to spend some time on automating some of the tasks
so that e.g. feature branches can be run through CI, so we have more
transparency into the state of a particular branch during development,
but I don't think that is a prerequisite to get this going :)

Cheers
Jan
-- 




On Jun 16, 2012, at 21:49 , Marco Monteiro wrote:

 On 16 June 2012 17:45, Noah Slater nsla...@tumbolia.org wrote:
 
 1. We'd like to proposed formal time-based releases
 
 
 
 http://wiki.apache.org/couchdb/Roadmap_Process
 
 
 This is a big improvement, but... :)
 
 If the QA is good enough, it would be a little better to have feature
 releases every month or two, support only the latest or two latest
 feature releases before a breaking change (but not support anything
 older than a year) and have a new release between feature releases for
 every bugfix or two.
 
 Every feature merge would be done on the first one or two weeks, the
 remaining time being used to test before the release.
 
 If QA is good enough, there will be almost no bugfix release. Anyone
 wanting to upgrade to fix any issue would always be using the latest
 feature release (with any bugfix) for it's major version number.
 
 This would be much easier to support leaving much more time to make
 QA good enough.
 
 Cheers,
 Marco

Re: [PROPOSAL] Official roadmap, and merge procedure

[Jan Lehnardt]

On Jun 16, 2012, at 18:45 , Noah Slater wrote:

 Devs,
 
 A few of us met in Dublin recently, and we discussed the project roadmap.
 
 Key takeaways from that meeting:
 
 1. We'd like to proposed formal time-based releases
 
 
 2. Branch and hack all you like, but if you want to ship something, you
 have to submit a merge request to an active release branch. Before you do
 this, you should follow a test procedure. And before your request will be
 accepted, it will be put through QA by the community.
 
 
 Details of these proposals can be found here:
 
 http://wiki.apache.org/couchdb/Roadmap_Process
 
 
 http://wiki.apache.org/couchdb/Merge_Procedure
 
 
 Please reply back to this thread with your comments on the proposals.
 
 (The last one needs to be fleshed out, a little...)

In particular this bit:

 Feature branches are merged to release branches using 'git merge --no-ff 
 FEATURE BRANCH'. This guarantees a merge commit, which are the only kinds 
 of commit that will appear on release branches. If the merge results in 
 conflicts, the release manager rejects the merge commit with an reply to the 
 dev@ thread. If the merge is successful, the release manager should run 'make 
 distcheck' and push the merge upstream if the tests pass.

Merges are currently not allowed on release branches and master. IIRC we came 
up with git merge --no-ff being safe to enable, but this currently is still 
disabled.

Cheers
Jan
-- 

Re: [PROPOSAL] Official roadmap, and merge procedure

[Noah Slater]

Thanks for the comments guys.

Bob was in the room when this proposal was drafted (literally) but Paul is
our other active release manager, so I want to wait for his review before I
comment on anything in the thread. I am also interested to hear what Jason
Smith thinks, and indeed, anyone else from Cloudant, or other downstream
CouchDB individuals this may effect. If you're interested in helping the
CouchDB community test CouchDB, through QA, CI, or any other method, then
your should also chime in here.

On Sun, Jun 17, 2012 at 4:28 PM, Jan Lehnardt j...@apache.org wrote:


 On Jun 16, 2012, at 18:45 , Noah Slater wrote:

  Devs,
 
  A few of us met in Dublin recently, and we discussed the project roadmap.
 
  Key takeaways from that meeting:
 
  1. We'd like to proposed formal time-based releases
 
 
  2. Branch and hack all you like, but if you want to ship something, you
  have to submit a merge request to an active release branch. Before you do
  this, you should follow a test procedure. And before your request will be
  accepted, it will be put through QA by the community.
 
 
  Details of these proposals can be found here:
 
  http://wiki.apache.org/couchdb/Roadmap_Process
 
 
  http://wiki.apache.org/couchdb/Merge_Procedure
 
 
  Please reply back to this thread with your comments on the proposals.
 
  (The last one needs to be fleshed out, a little...)

 In particular this bit:

  Feature branches are merged to release branches using 'git merge --no-ff
 FEATURE BRANCH'. This guarantees a merge commit, which are the only kinds
 of commit that will appear on release branches. If the merge results in
 conflicts, the release manager rejects the merge commit with an reply to
 the dev@ thread. If the merge is successful, the release manager should
 run 'make distcheck' and push the merge upstream if the tests pass.

 Merges are currently not allowed on release branches and master. IIRC we
 came up with git merge --no-ff being safe to enable, but this currently
 is still disabled.

 Cheers
 Jan
 --

Re: [PROPOSAL] Official roadmap, and merge procedure

[Paul Davis]

The only thing I see an issue with is the merge commit policy.

You'll find quite quickly that you'll have situations that require
merge commits with diff's to work properly. A traditional example is
bug fixes that reach back through many release branches. If the RM
rejects anything that's not a clean merge then what the committer is
required to do is either write three different commits tailored to
each release branch (which defeats the purpose of the merge) or merge
work back to a release branch and then submit the merge commit as a
commit to be merged into the release branch (which makes my brain hurt
just thinking about it).

If you really want the RM to be a gate keeper/reviewer what I'd
suggest is that we take a hard line against working on master at all.
When we want to submit work we do the email thing that'll include a
list of commits, one per branch. Ie, something like:

1234-master-cool
1234-1.4.x-cool

And then we do our review dance and CI checks. (Also, if one commit
works on all branches, that's fine, this just allows a degree of
freedom for when things don't apply the same everywhere). Along with
the rational for tests and docs rational for why things are back
ported vs are etc etc should be included in the email to dev@. Also,
we'll want to get a template/example/something up on the net so people
know what to submit when they do submit.

And the actual push to branches happens with one of the RM's reviewing
it and then doing a rebase onto those branches (and rejecting as
failed).

This way we have single commits per feature/fix/whatever on all of our
release branches but we allow the committer to produce the commits
that will inevitably have to be tailored to each branch in some cases.



On Sun, Jun 17, 2012 at 11:48 AM, Noah Slater nsla...@tumbolia.org wrote:
 Thanks for the comments guys.

 Bob was in the room when this proposal was drafted (literally) but Paul is
 our other active release manager, so I want to wait for his review before I
 comment on anything in the thread. I am also interested to hear what Jason
 Smith thinks, and indeed, anyone else from Cloudant, or other downstream
 CouchDB individuals this may effect. If you're interested in helping the
 CouchDB community test CouchDB, through QA, CI, or any other method, then
 your should also chime in here.

 On Sun, Jun 17, 2012 at 4:28 PM, Jan Lehnardt j...@apache.org wrote:


 On Jun 16, 2012, at 18:45 , Noah Slater wrote:

  Devs,
 
  A few of us met in Dublin recently, and we discussed the project roadmap.
 
  Key takeaways from that meeting:
 
  1. We'd like to proposed formal time-based releases
 
 
  2. Branch and hack all you like, but if you want to ship something, you
  have to submit a merge request to an active release branch. Before you do
  this, you should follow a test procedure. And before your request will be
  accepted, it will be put through QA by the community.
 
 
  Details of these proposals can be found here:
 
  http://wiki.apache.org/couchdb/Roadmap_Process
 
 
  http://wiki.apache.org/couchdb/Merge_Procedure
 
 
  Please reply back to this thread with your comments on the proposals.
 
  (The last one needs to be fleshed out, a little...)

 In particular this bit:

  Feature branches are merged to release branches using 'git merge --no-ff
 FEATURE BRANCH'. This guarantees a merge commit, which are the only kinds
 of commit that will appear on release branches. If the merge results in
 conflicts, the release manager rejects the merge commit with an reply to
 the dev@ thread. If the merge is successful, the release manager should
 run 'make distcheck' and push the merge upstream if the tests pass.

 Merges are currently not allowed on release branches and master. IIRC we
 came up with git merge --no-ff being safe to enable, but this currently
 is still disabled.

 Cheers
 Jan
 --

[PROPOSAL] Official roadmap, and merge procedure

[Noah Slater]

Devs,

A few of us met in Dublin recently, and we discussed the project roadmap.

Key takeaways from that meeting:

1. We'd like to proposed formal time-based releases


2. Branch and hack all you like, but if you want to ship something, you
have to submit a merge request to an active release branch. Before you do
this, you should follow a test procedure. And before your request will be
accepted, it will be put through QA by the community.


Details of these proposals can be found here:

http://wiki.apache.org/couchdb/Roadmap_Process


http://wiki.apache.org/couchdb/Merge_Procedure


Please reply back to this thread with your comments on the proposals.

(The last one needs to be fleshed out, a little...)

Thanks,

N

Re: [PROPOSAL] Official roadmap, and merge procedure

[Dirkjan Ochtman]

On Sat, Jun 16, 2012 at 6:45 PM, Noah Slater nsla...@tumbolia.org wrote:
 1. We'd like to proposed formal time-based releases

Love it. In fact, I think I proposed it before. I think this will be a
much better way of making CouchDB administration easy.

 Details of these proposals can be found here:

 http://wiki.apache.org/couchdb/Roadmap_Process

Looks good, but supporting 4 feature releases at the same time sounds
like a lot of work, particularly given the current amounts of
developer participation. ISTM supporting two, maybe three releases
would probably be Good Enough.

Cheers,

Dirkjan

Re: [PROPOSAL] Official roadmap, and merge procedure

[Martin Hewitt]

This is an excellent idea, more predictability is always a Good Thing, 
regardless of how big each release is. Being able to schedule downtime for 
upgrades on a predictable schedule is a massive tick. 

Martin
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Saturday, 16 June 2012 at 18:30, Dirkjan Ochtman wrote:

 On Sat, Jun 16, 2012 at 6:45 PM, Noah Slater nsla...@tumbolia.org 
 (mailto:nsla...@tumbolia.org) wrote:
  1. We'd like to proposed formal time-based releases
 
 
 Love it. In fact, I think I proposed it before. I think this will be a
 much better way of making CouchDB administration easy.
 
  Details of these proposals can be found here:
  
  http://wiki.apache.org/couchdb/Roadmap_Process
 
 Looks good, but supporting 4 feature releases at the same time sounds
 like a lot of work, particularly given the current amounts of
 developer participation. ISTM supporting two, maybe three releases
 would probably be Good Enough.
 
 Cheers,
 
 Dirkjan 

Re: [PROPOSAL] Official roadmap, and merge procedure

[Marco Monteiro]

On 16 June 2012 17:45, Noah Slater nsla...@tumbolia.org wrote:

 1. We'd like to proposed formal time-based releases



 http://wiki.apache.org/couchdb/Roadmap_Process


This is a big improvement, but... :)

If the QA is good enough, it would be a little better to have feature
releases every month or two, support only the latest or two latest
feature releases before a breaking change (but not support anything
older than a year) and have a new release between feature releases for
every bugfix or two.

Every feature merge would be done on the first one or two weeks, the
remaining time being used to test before the release.

If QA is good enough, there will be almost no bugfix release. Anyone
wanting to upgrade to fix any issue would always be using the latest
feature release (with any bugfix) for it's major version number.

This would be much easier to support leaving much more time to make
QA good enough.

Cheers,
Marco

Re: Roadmap for the 1.3.0 release

[Jan Lehnardt]

On Feb 15, 2012, at 03:22 , Brian Mitchell wrote:

 Has there been any discussion around BigCouch integration strategies? It 
 seems like it would fit the bill for the next undertaking on the general 
 couch side. Does anyone from Cloudant have a suggestion for the timeline here?

Cloudant havent't yet approached Apache CouchDB with anything 
regarding their announcement.


 Any other work from mobile builds and the like might be interesting to 
 support. Were there any interesting changes to pull in from the mobile and 
 embedded device ports? Also, I'm not sure where the view engine refactoring 
 work ended up… I'll look at JIRA but maybe there are specific follow-ups to 
 be made here.

The biggest one IIRC was the use of emonk instead of an external Spidermonkey 
for iOS. Maybe we can make that a compile option like ./configure 
--enable-emonk-view-server so people can choose which one to run.


 Finally, for our included javascript libraries and Futon runtime, are we 
 going to replace or update anything here? I'd imagine a newer jQuery could be 
 an advantage for those doing CouchApps.

Good that you bring this up. There's also other upstream dependencies that we 
should look at updating and we don't have a process for that, it usually 
happens whenever a developer feels like it. I haven't thought about this much 
yet, but maybe update all dependencies could be a todo before a release 
branch is created?

Cheers
Jan
-- 

Re: Roadmap for the 1.3.0 release

[Bob Dionne]

On Feb 14, 2012, at 9:22 PM, Brian Mitchell wrote:

 Has there been any discussion around BigCouch integration strategies? It 
 seems like it would fit the bill for the next undertaking on the general 
 couch side. Does anyone from Cloudant have a suggestion for the timeline here?

There's been a lot of discussion in #couchdb and #couchdb-dev but little on the 
ml. I'm not sure about timeline. There seems to be a lot of issues, most of 
them minor technical ones (the type that readily bog down once more than 3 
people get involved). BigCouch embeds couchdb and was architected to be the 
clustering layer that couchdb lacks, so in that sense I think we're in pretty 
good shape. 

Ideally we'd have one common code base but it may be that some configuration of 
components is done at build time, perhaps driven by 3 types, mobile, single 
instance, and clustered

Does it make sense to anyone to think of this in the opposite direction, .i.e. 
upgrade/enhance BigCouch to the latest couchdb and then call that couchdb 2.0?



 
 Any other work from mobile builds and the like might be interesting to 
 support. Were there any interesting changes to pull in from the mobile and 
 embedded device ports? Also, I'm not sure where the view engine refactoring 
 work ended up… I'll look at JIRA but maybe there are specific follow-ups to 
 be made here.
 
 Finally, for our included javascript libraries and Futon runtime, are we 
 going to replace or update anything here? I'd imagine a newer jQuery could be 
 an advantage for those doing CouchApps.
 
 Brian.
 
 On Tuesday, February 14, 2012 at 7:28 , Noah Slater wrote:  
 So far we have:
 
 https://issues.apache.org/jira/browse/COUCHDB-1410
 
 

Re: Roadmap for the 1.3.0 release

[Benoit Chesneau]

On Wed, Feb 15, 2012 at 2:13 PM, Bob Dionne
dio...@dionne-associates.com wrote:

 On Feb 14, 2012, at 9:22 PM, Brian Mitchell wrote:

 Has there been any discussion around BigCouch integration strategies? It 
 seems like it would fit the bill for the next undertaking on the general 
 couch side. Does anyone from Cloudant have a suggestion for the timeline 
 here?

 There's been a lot of discussion in #couchdb and #couchdb-dev but little on 
 the ml. I'm not sure about timeline. There seems to be a lot of issues, most 
 of them minor technical ones (the type that readily bog down once more than 3 
 people get involved). BigCouch embeds couchdb and was architected to be the 
 clustering layer that couchdb lacks, so in that sense I think we're in pretty 
 good shape.

 Ideally we'd have one common code base but it may be that some configuration 
 of components is done at build time, perhaps driven by 3 types, mobile, 
 single instance, and clustered

 Does it make sense to anyone to think of this in the opposite direction, 
 .i.e. upgrade/enhance BigCouch to the latest couchdb and then call that 
 couchdb 2.0?

I was thinking that having a single instance that you could upgrade as
a cluster member with just a configuration swicth would be a better
plan. With smart rebalancing I could create cluster really
dynamically. I understand  that currently it will be hard technically
to do that since couch embedded in bigcouch has been modified to get
some infos from the cluster (like the design doc, validate func ...)
but it's still possible. Not sure if it should happen first, but I
really wish we follow this way rather than creating different
instances types.


- benoît

Re: Roadmap for the 1.3.0 release

[Benoit Chesneau]

On Wed, Feb 15, 2012 at 11:48 AM, Jan Lehnardt j...@apache.org wrote:

 On Feb 15, 2012, at 03:22 , Brian Mitchell wrote:

 Any other work from mobile builds and the like might be interesting to 
 support. Were there any interesting changes to pull in from the mobile and 
 embedded device ports? Also, I'm not sure where the view engine refactoring 
 work ended up… I'll look at JIRA but maybe there are specific follow-ups to 
 be made here.

 The biggest one IIRC was the use of emonk instead of an external Spidermonkey 
 for iOS. Maybe we can make that a compile option like ./configure 
 --enable-emonk-view-server so people can choose which one to run.


There is some work started about that in the refuge project using old
work done by @davisp used in couchbase and my emonk_helper. I will
provide patches asap.


 Finally, for our included javascript libraries and Futon runtime, are we 
 going to replace or update anything here? I'd imagine a newer jQuery could 
 be an advantage for those doing CouchApps.

 Good that you bring this up. There's also other upstream dependencies that we 
 should look at updating and we don't have a process for that, it usually 
 happens whenever a developer feels like it. I haven't thought about this much 
 yet, but maybe update all dependencies could be a todo before a release 
 branch is created?

+1 for a ticket :)

Also include support for mobile support in futon would be fine. (but i
think we should only depends on jquery rather than including another
framework) .

- benoit

Re: Roadmap for the 1.3.0 release

[Bob Dionne]

That sounds really neat, a number of folks have asked for such a thing. 

Right, the ddocs, validation funs, etc.. currently aren't stored globally, 
which requires clustered calls to retrieve them

On Feb 15, 2012, at 8:21 AM, Benoit Chesneau wrote:

 On Wed, Feb 15, 2012 at 2:13 PM, Bob Dionne
 dio...@dionne-associates.com wrote:
 
 On Feb 14, 2012, at 9:22 PM, Brian Mitchell wrote:
 
 Has there been any discussion around BigCouch integration strategies? It 
 seems like it would fit the bill for the next undertaking on the general 
 couch side. Does anyone from Cloudant have a suggestion for the timeline 
 here?
 
 There's been a lot of discussion in #couchdb and #couchdb-dev but little on 
 the ml. I'm not sure about timeline. There seems to be a lot of issues, most 
 of them minor technical ones (the type that readily bog down once more than 
 3 people get involved). BigCouch embeds couchdb and was architected to be 
 the clustering layer that couchdb lacks, so in that sense I think we're in 
 pretty good shape.
 
 Ideally we'd have one common code base but it may be that some configuration 
 of components is done at build time, perhaps driven by 3 types, mobile, 
 single instance, and clustered
 
 Does it make sense to anyone to think of this in the opposite direction, 
 .i.e. upgrade/enhance BigCouch to the latest couchdb and then call that 
 couchdb 2.0?
 
 I was thinking that having a single instance that you could upgrade as
 a cluster member with just a configuration swicth would be a better
 plan. With smart rebalancing I could create cluster really
 dynamically. I understand  that currently it will be hard technically
 to do that since couch embedded in bigcouch has been modified to get
 some infos from the cluster (like the design doc, validate func ...)
 but it's still possible. Not sure if it should happen first, but I
 really wish we follow this way rather than creating different
 instances types.
 
 
 - benoît

Roadmap for the 1.3.0 release

[Noah Slater]

Devs,

Please use this thread to discuss roadmap items for the 1.3.0 release.

The current hot topic seems to be number handling. We'd like to formalise,
improve, and document how we handle numbers.

What else?

Thanks,

N

Re: Roadmap for the 1.3.0 release

[Noah Slater]

So far we have:

https://issues.apache.org/jira/browse/COUCHDB-1410

On Tue, Feb 14, 2012 at 12:16 PM, Noah Slater nsla...@tumbolia.org wrote:

 Devs,

 Please use this thread to discuss roadmap items for the 1.3.0 release.

 The current hot topic seems to be number handling. We'd like to formalise,
 improve, and document how we handle numbers.

 What else?

 Thanks,

 N

Re: Roadmap for the 1.3.0 release

[Brian Mitchell]

Has there been any discussion around BigCouch integration strategies? It seems 
like it would fit the bill for the next undertaking on the general couch side. 
Does anyone from Cloudant have a suggestion for the timeline here?

Any other work from mobile builds and the like might be interesting to support. 
Were there any interesting changes to pull in from the mobile and embedded 
device ports? Also, I'm not sure where the view engine refactoring work ended 
up… I'll look at JIRA but maybe there are specific follow-ups to be made here.

Finally, for our included javascript libraries and Futon runtime, are we going 
to replace or update anything here? I'd imagine a newer jQuery could be an 
advantage for those doing CouchApps.

Brian.

On Tuesday, February 14, 2012 at 7:28 , Noah Slater wrote:  
 So far we have:
  
 https://issues.apache.org/jira/browse/COUCHDB-1410
  

Re: roadmap

[Gabriel Farrell]

On Thu, Feb 10, 2011 at 11:44 AM, Jan Lehnardt j...@apache.org wrote:

 On 10 Feb 2011, at 17:29, Gabriel Farrell wrote:

 On Tue, Feb 8, 2011 at 11:37 AM, Robert Newson robert.new...@gmail.com 
 wrote:
 You're absolutely right. 1.0.2 was ready to go quite some time ago but
 several bugs were found as we were releasing. We decided, as a team,
 that we couldn't ship with the bugs that were found, so we elected to
 fix them and delay the release. I think that was the right decision.
 We should only release when we feel the software is ready, not when
 some ultimately arbitrary day looms.

 I completely agree here: there should be no arbitrary release dates.
 However, I'm also in favor of increasing the frequency of minor point
 releases. Node.js is really inspiring in this area, with new minor
 point releases coming out every week or so (ooh, and 0.4.0 just got
 released 6 hours ago!). I know the process for an Apache project has
 more overhead, we don't have a BDFL, and the older community may not
 appreciate a release cycle that's quite so frantic (interpret that as
 you like), but there's still something to be learned from the rapid
 development and enthusiastic community around Node.


 Yup, I totally agree with node showing amazing momentum. But they do
 have the luxury of being able to break backwards compatibility with
 any release, really, and we don't have that :) — I think the 1.0.2 time
 frame was an outlier and that we are in pretty good shape to push maintenance
 releases quickly, if needed. — Now we only need to demonstrate that :)

Ryan's been a bit more careful about backwards compatibility lately
with the move to stable even and unstable odd branch releases.
Backwards compatibility is a concern for any project as it matures.
So, yeah, more agreement -- let's keep that concern in mind as we
quickly push releases. :)

 Cheers
 Jan
 --


 B.

 On Tue, Feb 8, 2011 at 4:32 PM, Noah Slater nsla...@apache.org wrote:

 On 8 Feb 2011, at 16:14, Dirkjan Ochtman wrote:

 Still, the problem I have is that it seems like there is a tendency to
 make releases large; it seems like there's little control against devs
 wanting to add their one more thing. Particularly for bugfix
 releases; from 1.0.1 it took almost 6 months for 1.0.2 to get
 released. In between, there were a little under 100 revisions on the
 1.0.x branch, presumably most of those fixing bugs users could
 actually run into. It seems valuable to me if the community could have
 gotten some of these fixes sooner.

 I need someone else to weigh in on this, but I believe the reason was 
 because of a few critical bugs that were being worked on. And not, as you 
 suggest, because we were suffering from a Just One More Thing problem. I'd 
 really need Jan or Chris to comment though as I use them as a conduit for 
 judging this stuff.

Re: roadmap

[Gabriel Farrell]

On Tue, Feb 8, 2011 at 11:37 AM, Robert Newson robert.new...@gmail.com wrote:
 You're absolutely right. 1.0.2 was ready to go quite some time ago but
 several bugs were found as we were releasing. We decided, as a team,
 that we couldn't ship with the bugs that were found, so we elected to
 fix them and delay the release. I think that was the right decision.
 We should only release when we feel the software is ready, not when
 some ultimately arbitrary day looms.

I completely agree here: there should be no arbitrary release dates.
However, I'm also in favor of increasing the frequency of minor point
releases. Node.js is really inspiring in this area, with new minor
point releases coming out every week or so (ooh, and 0.4.0 just got
released 6 hours ago!). I know the process for an Apache project has
more overhead, we don't have a BDFL, and the older community may not
appreciate a release cycle that's quite so frantic (interpret that as
you like), but there's still something to be learned from the rapid
development and enthusiastic community around Node.

 B.

 On Tue, Feb 8, 2011 at 4:32 PM, Noah Slater nsla...@apache.org wrote:

 On 8 Feb 2011, at 16:14, Dirkjan Ochtman wrote:

 Still, the problem I have is that it seems like there is a tendency to
 make releases large; it seems like there's little control against devs
 wanting to add their one more thing. Particularly for bugfix
 releases; from 1.0.1 it took almost 6 months for 1.0.2 to get
 released. In between, there were a little under 100 revisions on the
 1.0.x branch, presumably most of those fixing bugs users could
 actually run into. It seems valuable to me if the community could have
 gotten some of these fixes sooner.

 I need someone else to weigh in on this, but I believe the reason was 
 because of a few critical bugs that were being worked on. And not, as you 
 suggest, because we were suffering from a Just One More Thing problem. I'd 
 really need Jan or Chris to comment though as I use them as a conduit for 
 judging this stuff.

Re: roadmap

[Jan Lehnardt]

On 10 Feb 2011, at 17:29, Gabriel Farrell wrote:

 On Tue, Feb 8, 2011 at 11:37 AM, Robert Newson robert.new...@gmail.com 
 wrote:
 You're absolutely right. 1.0.2 was ready to go quite some time ago but
 several bugs were found as we were releasing. We decided, as a team,
 that we couldn't ship with the bugs that were found, so we elected to
 fix them and delay the release. I think that was the right decision.
 We should only release when we feel the software is ready, not when
 some ultimately arbitrary day looms.
 
 I completely agree here: there should be no arbitrary release dates.
 However, I'm also in favor of increasing the frequency of minor point
 releases. Node.js is really inspiring in this area, with new minor
 point releases coming out every week or so (ooh, and 0.4.0 just got
 released 6 hours ago!). I know the process for an Apache project has
 more overhead, we don't have a BDFL, and the older community may not
 appreciate a release cycle that's quite so frantic (interpret that as
 you like), but there's still something to be learned from the rapid
 development and enthusiastic community around Node.


Yup, I totally agree with node showing amazing momentum. But they do
have the luxury of being able to break backwards compatibility with
any release, really, and we don't have that :) — I think the 1.0.2 time
frame was an outlier and that we are in pretty good shape to push maintenance
releases quickly, if needed. — Now we only need to demonstrate that :)

Cheers
Jan
-- 

 
 B.
 
 On Tue, Feb 8, 2011 at 4:32 PM, Noah Slater nsla...@apache.org wrote:
 
 On 8 Feb 2011, at 16:14, Dirkjan Ochtman wrote:
 
 Still, the problem I have is that it seems like there is a tendency to
 make releases large; it seems like there's little control against devs
 wanting to add their one more thing. Particularly for bugfix
 releases; from 1.0.1 it took almost 6 months for 1.0.2 to get
 released. In between, there were a little under 100 revisions on the
 1.0.x branch, presumably most of those fixing bugs users could
 actually run into. It seems valuable to me if the community could have
 gotten some of these fixes sooner.
 
 I need someone else to weigh in on this, but I believe the reason was 
 because of a few critical bugs that were being worked on. And not, as you 
 suggest, because we were suffering from a Just One More Thing problem. I'd 
 really need Jan or Chris to comment though as I use them as a conduit for 
 judging this stuff.
 

Re: roadmap

[Simon Metson]

Hi,

On 9 Feb 2011, at 08:29, Benoit Chesneau wrote:

 It's not clear to me that couchdb would benefit by bundling
 clustering and search. Lucene has an approach that might work for
 us, namely where there's an explicit core project, with a number of
 supplementary parts. Releases are aligned for all of them, but the
 separation is pretty clear.
 
 I agree, wasn't speaking to make a bundle, but we could have official
 plugins supported by the apache project like hadoop or solr have.

Something like Apache Extras might be of interest (though afaict it's really 
just a branded google code). 
https://blogs.apache.org/foundation/entry/the_apache_software_foundation_launches

At the very least I think specific couchapps (implementations of wikis, blogs 
etc) should be encouraged to go there for hosting. I'm not sure couchdb-lucene, 
couchapp itself or geocouch (and other future plugins) are entirely suitable, 
but I guess it depends on how apache extras evolve over time.
Cheers
Simon

Re: roadmap

[Simon Metson]

Hi,

 - a really good plugin story (geocouch, lucene search, easy to compile
 against couchdb sources)

I don't know how possible this is (I can think of a number of issues without 
trying) but having these plugins uploadable into a database in a similar manner 
to views would be super sweet. It would mean you could find a vanilla couch 
hosting company and tailor it to your databases needs directly over HTTP.
Cheers
Simon

Re: roadmap

[Matt Adams]

Randall Leeds wrote:


My priorities are:
- a really good embedding story (android, desktop apps, couchbase, etc ...)


...


- a really good build story (particularly android, windows)


Having recently worked with the Android port (see build instructions on 
wiki -- soon to be updated  improved) we might be able to help with 
these things.


We're releasing a collaborative mobile data collection product in the 
next few weeks that uses Open Data Kit technology and CouchDB for 
persistence and synchronization.


More to come on this, obviously.


Cheers,

Matt
--
Matt Adams
Radical Dynamic
www.radicaldynamic.com

Re: roadmap

[Benoit Chesneau]

On Tue, Feb 8, 2011 at 10:02 PM, Noah Slater nsla...@apache.org wrote:

 On 8 Feb 2011, at 20:53, Benoit Chesneau wrote:

 What is it supposed to mean ? A roadmap is a a detailed plan to guide
 progress toward a goal  . Why couldn't we define goals ?

 I think Jan's point is that we use the JIRA roadmap as an advisory only, and 
 never state that we are committing to it. That means that if I create a 
 ticket for CouchDB to be able to read and send email, it doesn't hold-up the 
 project.

 Every program attempts to expand until it can read mail. Those programs 
 which cannot so expand are replaced by ones which can.

        — Jamie Zawinski


hum, ok yes a ticket is only a ticket. Although I was under the
impression that we can give some priority to the tickets or close them
if they doesn't enter in project goals.

Re: roadmap

[Benoit Chesneau]

On Tue, Feb 8, 2011 at 5:19 PM, Robert Newson robert.new...@gmail.com wrote:
 We do need to improve at releases, though I think we can all agree
 that 1.0.2 was just a particularly difficult one, pretty atypical.

 As for roadmap, I can see a need to revisit it. I'm not sure what
 should be on it, I suspect everyone has their pet feature or ten to
 add. So, I'm wary about fragmenting/diluting what we have by trying to
 put too much inside couchdb itself.

 Instead, I think Benoit is on to something by talking about
 pluggability. At the very least, couchdb can make it easy to embed and
 extend it in ways that are minimally likely to be broken by us in
 future releases.

 It's not clear to me that couchdb would benefit by bundling
 clustering and search. Lucene has an approach that might work for
 us, namely where there's an explicit core project, with a number of
 supplementary parts. Releases are aligned for all of them, but the
 separation is pretty clear.

I agree, wasn't speaking to make a bundle, but we could have official
plugins supported by the apache project like hadoop or solr have.

Re: roadmap

[Benoit Chesneau]

On Tue, Feb 8, 2011 at 6:42 PM, Peter Nolan peterwno...@gmail.com wrote:

 I would like a more formal direction on the future of couchapps.  I am
 unsure how couchapps will proceed, but one thing i would like to see is the
 ability of your basic internet users to be able to form their own couchapps
 by easily integrating other couchapps into their databases, modifying to
 their content (most 'simple' users are content with just basic color and
 image changes).


 For example, a user may 'see' something cool on someones elses site/couch
 and would like to copy it to theirs.  This 'copying' should be as easy as
 humanly possible.  Any complications will only hinder the adoption of
 couchapps to the public in my opinion.  Ideally this 'copying' should be
 done with a simple push of a button.

 This leads me to another need couchapps (and they're couchAPERS) must
 discuss - Standardization.  I imagine it will be 'hard' if we don't begin to
 standardize some of the aspects of couchapps.  For example - blog posts.
  There should be a standard way of 'inputting' blog posts into ones of
 couch, such that others may easily 'replicate' or 'pull from' peoples blogs
 and have them appear on their couches.  Of course this 'standardization'
 should also not limit ones ability to change the code.

There is a discussion started on the CouchApp mailing-list about that.
I also think a couchapp spec is welcome, so we can develop compatible
engine and clients. At the end we should only have to choose the
hosting or the tool we like, period.

Re: roadmap

[Jan Lehnardt]

On 9 Feb 2011, at 09:26, Benoit Chesneau wrote:

 On Tue, Feb 8, 2011 at 10:02 PM, Noah Slater nsla...@apache.org wrote:
 
 On 8 Feb 2011, at 20:53, Benoit Chesneau wrote:
 
 What is it supposed to mean ? A roadmap is a a detailed plan to guide
 progress toward a goal  . Why couldn't we define goals ?
 
 I think Jan's point is that we use the JIRA roadmap as an advisory only, and 
 never state that we are committing to it. That means that if I create a 
 ticket for CouchDB to be able to read and send email, it doesn't hold-up the 
 project.
 
 Every program attempts to expand until it can read mail. Those programs 
 which cannot so expand are replaced by ones which can.
 
— Jamie Zawinski
 
 
 hum, ok yes a ticket is only a ticket. Although I was under the
 impression that we can give some priority to the tickets or close them
 if they doesn't enter in project goals.

All I'm saying is that just creating a ticket and assigning it a version
number won't make us commit to delivering that.

Of course we should organise all tickets into versions and come up with
a sensible batch of stuff to work towards for every release.

Cheers
Jan
-- 

Re: roadmap

[Robert Newson]

We should be clear that just because Jira has that helpful 'Roadmap'
panel, doesn't mean it's our official roadmap. It really isn't, though
that is how Jira would like us to do things. I can't speak for
everyone, but Jira, to me, is just a tool, it's not the boss of me.

B.

On 9 February 2011 12:30, Jan Lehnardt j...@apache.org wrote:

 On 9 Feb 2011, at 09:26, Benoit Chesneau wrote:

 On Tue, Feb 8, 2011 at 10:02 PM, Noah Slater nsla...@apache.org wrote:

 On 8 Feb 2011, at 20:53, Benoit Chesneau wrote:

 What is it supposed to mean ? A roadmap is a a detailed plan to guide
 progress toward a goal  . Why couldn't we define goals ?

 I think Jan's point is that we use the JIRA roadmap as an advisory only, 
 and never state that we are committing to it. That means that if I create a 
 ticket for CouchDB to be able to read and send email, it doesn't hold-up 
 the project.

 Every program attempts to expand until it can read mail. Those programs 
 which cannot so expand are replaced by ones which can.

        — Jamie Zawinski


 hum, ok yes a ticket is only a ticket. Although I was under the
 impression that we can give some priority to the tickets or close them
 if they doesn't enter in project goals.

 All I'm saying is that just creating a ticket and assigning it a version
 number won't make us commit to delivering that.

 Of course we should organise all tickets into versions and come up with
 a sensible batch of stuff to work towards for every release.

 Cheers
 Jan
 --

Re: roadmap

[Noah Slater]

What do you mean by official here?

On 9 Feb 2011, at 12:39, Robert Newson wrote:

 We should be clear that just because Jira has that helpful 'Roadmap'
 panel, doesn't mean it's our official roadmap. It really isn't, though
 that is how Jira would like us to do things. I can't speak for
 everyone, but Jira, to me, is just a tool, it's not the boss of me.
 
 B.
 
 On 9 February 2011 12:30, Jan Lehnardt j...@apache.org wrote:
 
 On 9 Feb 2011, at 09:26, Benoit Chesneau wrote:
 
 On Tue, Feb 8, 2011 at 10:02 PM, Noah Slater nsla...@apache.org wrote:
 
 On 8 Feb 2011, at 20:53, Benoit Chesneau wrote:
 
 What is it supposed to mean ? A roadmap is a a detailed plan to guide
 progress toward a goal  . Why couldn't we define goals ?
 
 I think Jan's point is that we use the JIRA roadmap as an advisory only, 
 and never state that we are committing to it. That means that if I create 
 a ticket for CouchDB to be able to read and send email, it doesn't hold-up 
 the project.
 
 Every program attempts to expand until it can read mail. Those programs 
 which cannot so expand are replaced by ones which can.
 
— Jamie Zawinski
 
 
 hum, ok yes a ticket is only a ticket. Although I was under the
 impression that we can give some priority to the tickets or close them
 if they doesn't enter in project goals.
 
 All I'm saying is that just creating a ticket and assigning it a version
 number won't make us commit to delivering that.
 
 Of course we should organise all tickets into versions and come up with
 a sensible batch of stuff to work towards for every release.
 
 Cheers
 Jan
 --
 
 
 
 

roadmap

[Benoit Chesneau]

Hi all,

With the announce of another /couchdb fork/project embedding couchdb/ I
think it's the perfect time to define ourself for next releases.

What will be CouchDB 1.2 or 2.0, what do we target, what is our
timeline. What is the CouchDB core that will be used by other projects
basically. Having a defined timeline, ie defined deadlines for freeze,
release, ... is important in this context. It means that releases don't
depend on external factors: we, the Apache CouchDB project follows its
own agenda.  There are other reasons for that of course.

So can we try to define this time a good old roadmap? What will be the
next couchdb?

For me:

- Plugin support
- improved CouchApp engine
- Improve possibilities to integrate CouchDB in other projects
- Clean API, so couchdb features can be called more easily in other
  erlang programs or plugins

Other feature I wish:

- Official Apache CouchDB clustering layer plug
- Official Search Plugin based on Apache Lucene that can be used like
  riak search or cloudant search

... and you ?


- benoît

Re: roadmap

[Dirkjan Ochtman]

On Tue, Feb 8, 2011 at 10:24, Benoit Chesneau bchesn...@gmail.com wrote:
 ... and you ?

Most of all, I want a better schedule/insight into the release
process. Even when reading the dev list, it's completely unclear when
I might expect the next release or what the blockers are. Releases
seem to just keep slipping, or maybe releasing isn't a very big
priority, at least that's what it looks like from the outside.

For the rest, I would like more documentation. The CouchOne API
documentation is a step in the right direction, having good docs about
the query server protocol would also be very helpful. Yes, I realize I
can read the couchjs JS code, but that's not a substitute for good
docs.

The new Futon that Mikeal was working on would be nice to have.

As for more pie-in-the-sky things, it would be nice to have more
efficient view indexing (using multiple processes, for example --
map/reduce should enable that, right?) and an official way to do
full-text search.

Cheers,

Dirkjan

Re: roadmap

[Juhani Ränkimies]

On Tue, Feb 8, 2011 at 11:24 AM, Benoit Chesneau bchesn...@gmail.comwrote:

 ... and you ?


- erlang api / plugin support
- easier build process on Windows

-juhani

Re: roadmap

[Zachary Zolton]

+1 full-text search
+1 documentation

On Tue, Feb 8, 2011 at 4:07 AM, Juhani Ränkimies juh...@juranki.com wrote:
 On Tue, Feb 8, 2011 at 11:24 AM, Benoit Chesneau bchesn...@gmail.comwrote:

 ... and you ?


 - erlang api / plugin support
 - easier build process on Windows

 -juhani

Re: roadmap

[till]

On Tue, Feb 8, 2011 at 10:33 AM, Dirkjan Ochtman dirk...@ochtman.nl wrote:
 On Tue, Feb 8, 2011 at 10:24, Benoit Chesneau bchesn...@gmail.com wrote:
 ... and you ?

 Most of all, I want a better schedule/insight into the release
 process. Even when reading the dev list, it's completely unclear when
 I might expect the next release or what the blockers are. Releases
 seem to just keep slipping, or maybe releasing isn't a very big
 priority, at least that's what it looks like from the outside.

 For the rest, I would like more documentation. The CouchOne API
 documentation is a step in the right direction, having good docs about
 the query server protocol would also be very helpful. Yes, I realize I
 can read the couchjs JS code, but that's not a substitute for good
 docs.

 The new Futon that Mikeal was working on would be nice to have.

 As for more pie-in-the-sky things, it would be nice to have more
 efficient view indexing (using multiple processes, for example --
 map/reduce should enable that, right?) and an official way to do
 full-text search.

 Cheers,

 Dirkjan


+1 to the above

Re: roadmap

[Dirkjan Ochtman]

On Tue, Feb 8, 2011 at 16:57, Noah Slater nsla...@apache.org wrote:
 For step 2, we follow the roadmap that is generated in JIRA. That is, the 
 roadmap is constantly maintained through ticket work and ticket maintenance. 
 A link to this is even included prominently on the project homepage. We have 
 found that maintaining a HTML roadmap separately to JIRA is too much trouble. 
 Or, at least, we could infer that from the fact it was never updated by 
 anyone. The roadmap (as a JIRA view) is effectively discussed on this list, 
 and then codified through the tickets. I think that's Good Enough. And if 
 it's not, then maybe we need to be smarter about how we use JIRA.

Okay, the roadmap is somewhat useful, and I wasn't aware of it.

Still, the problem I have is that it seems like there is a tendency to
make releases large; it seems like there's little control against devs
wanting to add their one more thing. Particularly for bugfix
releases; from 1.0.1 it took almost 6 months for 1.0.2 to get
released. In between, there were a little under 100 revisions on the
1.0.x branch, presumably most of those fixing bugs users could
actually run into. It seems valuable to me if the community could have
gotten some of these fixes sooner.

I do realize the Apache release process is rather heavyweight, and
that CouchDB only recently got a second RM, but it would still be nice
if it was possible to have more timely releases.

(And I also agree with till's sentiment about roadmaps.)

Cheers,

Dirkjan

Re: roadmap

[Robert Newson]

We do need to improve at releases, though I think we can all agree
that 1.0.2 was just a particularly difficult one, pretty atypical.

As for roadmap, I can see a need to revisit it. I'm not sure what
should be on it, I suspect everyone has their pet feature or ten to
add. So, I'm wary about fragmenting/diluting what we have by trying to
put too much inside couchdb itself.

Instead, I think Benoit is on to something by talking about
pluggability. At the very least, couchdb can make it easy to embed and
extend it in ways that are minimally likely to be broken by us in
future releases.

It's not clear to me that couchdb would benefit by bundling
clustering and search. Lucene has an approach that might work for
us, namely where there's an explicit core project, with a number of
supplementary parts. Releases are aligned for all of them, but the
separation is pretty clear.

B.

On Tue, Feb 8, 2011 at 4:14 PM, Dirkjan Ochtman dirk...@ochtman.nl wrote:
 On Tue, Feb 8, 2011 at 16:57, Noah Slater nsla...@apache.org wrote:
 For step 2, we follow the roadmap that is generated in JIRA. That is, the 
 roadmap is constantly maintained through ticket work and ticket maintenance. 
 A link to this is even included prominently on the project homepage. We have 
 found that maintaining a HTML roadmap separately to JIRA is too much 
 trouble. Or, at least, we could infer that from the fact it was never 
 updated by anyone. The roadmap (as a JIRA view) is effectively discussed on 
 this list, and then codified through the tickets. I think that's Good 
 Enough. And if it's not, then maybe we need to be smarter about how we use 
 JIRA.

 Okay, the roadmap is somewhat useful, and I wasn't aware of it.

 Still, the problem I have is that it seems like there is a tendency to
 make releases large; it seems like there's little control against devs
 wanting to add their one more thing. Particularly for bugfix
 releases; from 1.0.1 it took almost 6 months for 1.0.2 to get
 released. In between, there were a little under 100 revisions on the
 1.0.x branch, presumably most of those fixing bugs users could
 actually run into. It seems valuable to me if the community could have
 gotten some of these fixes sooner.

 I do realize the Apache release process is rather heavyweight, and
 that CouchDB only recently got a second RM, but it would still be nice
 if it was possible to have more timely releases.

 (And I also agree with till's sentiment about roadmaps.)

 Cheers,

 Dirkjan

Re: roadmap

[Noah Slater]

On 8 Feb 2011, at 16:08, till wrote:

 IMHO a roadmap is defined by more than there's a new jira issue, we
 need to fix it with the next release.

I think you're misunderstanding me. In JIRA, you can pin tickets to release 
versions. This is a perfectly good way of constructing a roadmap. And if the 
thing you want isn't in JIRA as a ticket to pin to a release version, that's a 
good opportunity for you to create one.

Re: roadmap

[Noah Slater]

On 8 Feb 2011, at 16:14, Dirkjan Ochtman wrote:

 Still, the problem I have is that it seems like there is a tendency to
 make releases large; it seems like there's little control against devs
 wanting to add their one more thing. Particularly for bugfix
 releases; from 1.0.1 it took almost 6 months for 1.0.2 to get
 released. In between, there were a little under 100 revisions on the
 1.0.x branch, presumably most of those fixing bugs users could
 actually run into. It seems valuable to me if the community could have
 gotten some of these fixes sooner.

I need someone else to weigh in on this, but I believe the reason was because 
of a few critical bugs that were being worked on. And not, as you suggest, 
because we were suffering from a Just One More Thing problem. I'd really need 
Jan or Chris to comment though as I use them as a conduit for judging this 
stuff.

Re: roadmap

[Robert Newson]

You're absolutely right. 1.0.2 was ready to go quite some time ago but
several bugs were found as we were releasing. We decided, as a team,
that we couldn't ship with the bugs that were found, so we elected to
fix them and delay the release. I think that was the right decision.
We should only release when we feel the software is ready, not when
some ultimately arbitrary day looms.

B.

On Tue, Feb 8, 2011 at 4:32 PM, Noah Slater nsla...@apache.org wrote:

 On 8 Feb 2011, at 16:14, Dirkjan Ochtman wrote:

 Still, the problem I have is that it seems like there is a tendency to
 make releases large; it seems like there's little control against devs
 wanting to add their one more thing. Particularly for bugfix
 releases; from 1.0.1 it took almost 6 months for 1.0.2 to get
 released. In between, there were a little under 100 revisions on the
 1.0.x branch, presumably most of those fixing bugs users could
 actually run into. It seems valuable to me if the community could have
 gotten some of these fixes sooner.

 I need someone else to weigh in on this, but I believe the reason was because 
 of a few critical bugs that were being worked on. And not, as you suggest, 
 because we were suffering from a Just One More Thing problem. I'd really need 
 Jan or Chris to comment though as I use them as a conduit for judging this 
 stuff.

Re: roadmap

[Peter Nolan]

 I would like a more formal direction on the future of couchapps.  I am
 unsure how couchapps will proceed, but one thing i would like to see is the
 ability of your basic internet users to be able to form their own couchapps
 by easily integrating other couchapps into their databases, modifying to
 their content (most 'simple' users are content with just basic color and
 image changes).


For example, a user may 'see' something cool on someones elses site/couch
and would like to copy it to theirs.  This 'copying' should be as easy as
humanly possible.  Any complications will only hinder the adoption of
couchapps to the public in my opinion.  Ideally this 'copying' should be
done with a simple push of a button.

This leads me to another need couchapps (and they're couchAPERS) must
discuss - Standardization.  I imagine it will be 'hard' if we don't begin to
standardize some of the aspects of couchapps.  For example - blog posts.
 There should be a standard way of 'inputting' blog posts into ones of
couch, such that others may easily 'replicate' or 'pull from' peoples blogs
and have them appear on their couches.  Of course this 'standardization'
should also not limit ones ability to change the code.

Same thing goes for messaging.  It would be nice if we would start forming a
standardized 'messaging' system such that messages posted on users A blog
will (to steal from Jan) automagically appear on users B blog.  (think
I.R.C. amongst couches).


In short, i believe the future of couchapps depends upon getting your mother
to be able to make her own (and feel like she made her own).


-Pete

Re: roadmap

[Matt Adams]

I'm going to chime in here and say that improved build support for 
Android would be terrific.  There are already some patches available for 
this and it would be nice to see them included in the official releases.


Hopefully this doesn't equate with asking for ponies.

I'd be happy to assist with testing or anything else that would help in 
this regard.



Cheers,

Matt
--
Matt Adams
Radical Dynamic
www.radicaldynamic.com

Re: roadmap

[Randall Leeds]

On Tue, Feb 8, 2011 at 01:24, Benoit Chesneau bchesn...@gmail.com wrote:

 Hi all,

 With the announce of another /couchdb fork/project embedding couchdb/ I
 think it's the perfect time to define ourself for next releases.

 What will be CouchDB 1.2 or 2.0, what do we target, what is our
 timeline. What is the CouchDB core that will be used by other projects
 basically. Having a defined timeline, ie defined deadlines for freeze,
 release, ... is important in this context. It means that releases don't
 depend on external factors: we, the Apache CouchDB project follows its
 own agenda.  There are other reasons for that of course.

 So can we try to define this time a good old roadmap? What will be the
 next couchdb?

 For me:

 - Plugin support
 - improved CouchApp engine
 - Improve possibilities to integrate CouchDB in other projects
 - Clean API, so couchdb features can be called more easily in other
  erlang programs or plugins

 Other feature I wish:

 - Official Apache CouchDB clustering layer plug
 - Official Search Plugin based on Apache Lucene that can be used like
  riak search or cloudant search

 ... and you ?


 - benoît


I agree mostly with Benoit. My priorities are:
- a really good embedding story (android, desktop apps, couchbase, etc ...)
- a really good plugin story (geocouch, lucene search, easy to compile
against couchdb sources)
- a really good build story (particularly android, windows)
- clean internal APIs

-Randall

Re: roadmap

[Jan Lehnardt]

On 8 Feb 2011, at 17:32, Noah Slater wrote:

 
 On 8 Feb 2011, at 16:14, Dirkjan Ochtman wrote:
 
 Still, the problem I have is that it seems like there is a tendency to
 make releases large; it seems like there's little control against devs
 wanting to add their one more thing. Particularly for bugfix
 releases; from 1.0.1 it took almost 6 months for 1.0.2 to get
 released. In between, there were a little under 100 revisions on the
 1.0.x branch, presumably most of those fixing bugs users could
 actually run into. It seems valuable to me if the community could have
 gotten some of these fixes sooner.
 
 I need someone else to weigh in on this, but I believe the reason was because 
 of a few critical bugs that were being worked on. And not, as you suggest, 
 because we were suffering from a Just One More Thing problem. I'd really need 
 Jan or Chris to comment though as I use them as a conduit for judging this 
 stuff.

Robert already confirmed this, but I'd like to point out that Noah's analysis
is apt.

--

As for the suggestions for more transparency regarding what new features
are being worked on and when do they land in which version I agree that we
can do better and I'll take on doing some of the legwork here.

I also like the proposed features, but I don't think committing to ship 
pony-features without seeing any code is a good idea — just to paint an
extreme, so far nobody suggested that.

Cheers
Jan
-- 

Re: roadmap

[Benoit Chesneau]

On Tue, Feb 8, 2011 at 9:40 PM, Jan Lehnardt j...@apache.org wrote:

 On 8 Feb 2011, at 17:32, Noah Slater wrote:


 On 8 Feb 2011, at 16:14, Dirkjan Ochtman wrote:

 Still, the problem I have is that it seems like there is a tendency to
 make releases large; it seems like there's little control against devs
 wanting to add their one more thing. Particularly for bugfix
 releases; from 1.0.1 it took almost 6 months for 1.0.2 to get
 released. In between, there were a little under 100 revisions on the
 1.0.x branch, presumably most of those fixing bugs users could
 actually run into. It seems valuable to me if the community could have
 gotten some of these fixes sooner.

 I need someone else to weigh in on this, but I believe the reason was 
 because of a few critical bugs that were being worked on. And not, as you 
 suggest, because we were suffering from a Just One More Thing problem. I'd 
 really need Jan or Chris to comment though as I use them as a conduit for 
 judging this stuff.

 Robert already confirmed this, but I'd like to point out that Noah's analysis
 is apt.

 --

 As for the suggestions for more transparency regarding what new features
 are being worked on and when do they land in which version I agree that we
 can do better and I'll take on doing some of the legwork here.

 I also like the proposed features, but I don't think committing to ship
 pony-features without seeing any code is a good idea — just to paint an
 extreme, so far nobody suggested that.

 Cheers
 Jan
 --


What is it supposed to mean ? A roadmap is a a detailed plan to guide
progress toward a goal  . Why couldn't we define goals ?

- benoît

Re: roadmap

[Noah Slater]

On 8 Feb 2011, at 20:53, Benoit Chesneau wrote:

 What is it supposed to mean ? A roadmap is a a detailed plan to guide
 progress toward a goal  . Why couldn't we define goals ?

I think Jan's point is that we use the JIRA roadmap as an advisory only, and 
never state that we are committing to it. That means that if I create a ticket 
for CouchDB to be able to read and send email, it doesn't hold-up the project.

Every program attempts to expand until it can read mail. Those programs which 
cannot so expand are replaced by ones which can.

— Jamie Zawinski

Re: Auth Roadmap

[Benoit Chesneau]

On Fri, Feb 12, 2010 at 7:10 AM, Chris Anderson jch...@apache.org wrote:
 On Tue, Feb 9, 2010 at 2:52 PM, Chris Anderson jch...@apache.org wrote:
 Devs,

 I've been getting a lot of feedback about the authentication 
 authorization work that I did over the holidays and over the last few
 weeks. There are also some enhancements I've been thinking about for a
 while. Here's a quick list of what I see as the important things to
 do. I'm not concerned here with releases / feature freeze etc as in my
 opinion CouchDB development is expected to continue even after we
 reach 1.0.

 1) Extensible password storage.

 Thanks Brian Candler for the links to the OpenLDAP style of storage. I
 think we should do this asap so we don't have to worry about backwards
 compatibility with the current storage mechanism until the end of
 time. The relevant message:
 http://permalink.gmane.org/gmane.comp.db.couchdb.devel/7588

 I'm helping Filipe Manana with an implementation of this, which will
 be backwards compatible with existing admin passwords (stored in
 config). We won't try to be backwards compatible with old _users dbs
 (it should be simple to write an upgrade script if you have crucial
 data). This doesn't need to block 0.11 but it could go in 0.11.1 as
 it'd be nice to get it out there for people who want better crypto.



Actually I think I wouild prefer a json object so we don't have to
parse string a such

{

  auth-type: brypt-sha1,
  key: ...
}

and in ini

[crypt-handlers]
{ bcrypt-sha1, couch_httpd_auth, brcrypt_sha1_encode,
brcrypt_sha1_decode }

Somethink like it. Which is basically the same as having all in one
string but is more jsonful. Also it remove the need to parse the
string or do more pattern matching than needed.

Is there any ticket open about it ?

- benoît

Re: Auth Roadmap

[Filipe David Manana]

Benoit,

The .ini file suggestion seems a good idea, as it allows for pluggable
password hash validation. Now we're thinking about OpenLDAP schemes, but
tomorrow someone might suggest or require something else.

About the JSON object, containing the hash scheme in an attribute and the
hash in a different one, I'm not 100% sure about it. Usually the scheme,
password hash and used salt (if any) travel in the same string. Take a look
at OpenLDAP:

$ slappasswd -s 'teste_password123' -h '{SSHA}'
{SSHA}Pmnj6Jg3CBccqLuYttGMfTYh1GKMLLyC

$ slappasswd -s 'teste_password123' -h '{SMD5}'
{SMD5}ojYaLtvJga+4hJqMSgi34JGUV/8=

and at RFC 3112 ( http://www.ietf.org/rfc/rfc3112.txt ) -

SHA1$RzqH67DY3uQ=$atAcDs1eS+IJwPy7V4UDXEoBrDI=

I would prefer having a .ini file with:

[couch_httpd_auth]
password_hash_scheme = -hashed-  ; the default scheme to use for hashing
passwords
password_validators= {couch_httpd_auth, hashed_password_validator},
{couch_httpd_auth, openldap_password_validator}

[password_hash_creators]
; scheme = {module, function}
-hashed- = {couch_httpd_auth, hash_password}
{SSHA} = {couch_httpd_auth, openldap_hash_password}
{SMD5} = {couch_httpd_auth, openldap_hash_password}

To validate a user supplied password, we would execute each
password_validator in turn until one of them succeeds. Exactly like we do
now with the authentication_handlers.

Each password validator would receive 2 paremeters: the user supplied
password and the password hash previously computed.

Each password hash creator would receive 2 parameters: the scheme and the
password to hash.

In a _users doc, we would drop the attribute salt and rename
password_sha to password_hash.

What do you think?



On Fri, Feb 12, 2010 at 1:20 PM, Benoit Chesneau bchesn...@gmail.comwrote:

 On Fri, Feb 12, 2010 at 7:10 AM, Chris Anderson jch...@apache.org wrote:
  On Tue, Feb 9, 2010 at 2:52 PM, Chris Anderson jch...@apache.org
 wrote:
  Devs,
 
  I've been getting a lot of feedback about the authentication 
  authorization work that I did over the holidays and over the last few
  weeks. There are also some enhancements I've been thinking about for a
  while. Here's a quick list of what I see as the important things to
  do. I'm not concerned here with releases / feature freeze etc as in my
  opinion CouchDB development is expected to continue even after we
  reach 1.0.
 
  1) Extensible password storage.
 
  Thanks Brian Candler for the links to the OpenLDAP style of storage. I
  think we should do this asap so we don't have to worry about backwards
  compatibility with the current storage mechanism until the end of
  time. The relevant message:
  http://permalink.gmane.org/gmane.comp.db.couchdb.devel/7588
 
  I'm helping Filipe Manana with an implementation of this, which will
  be backwards compatible with existing admin passwords (stored in
  config). We won't try to be backwards compatible with old _users dbs
  (it should be simple to write an upgrade script if you have crucial
  data). This doesn't need to block 0.11 but it could go in 0.11.1 as
  it'd be nice to get it out there for people who want better crypto.
 
 

 Actually I think I wouild prefer a json object so we don't have to
 parse string a such

 {

  auth-type: brypt-sha1,
  key: ...
 }

 and in ini

 [crypt-handlers]
 { bcrypt-sha1, couch_httpd_auth, brcrypt_sha1_encode,
 brcrypt_sha1_decode }

 Somethink like it. Which is basically the same as having all in one
 string but is more jsonful. Also it remove the need to parse the
 string or do more pattern matching than needed.

 Is there any ticket open about it ?

 - benoît




-- 
Filipe David Manana,
fdman...@gmail.com
PGP key - http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xC569452B

Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men.

Re: Auth Roadmap

[Brian Candler]

 I think what you say has merit, but it doesn't jibe with my
 understanding of our implementation in a logical way. I'm ready to
 ship the basic programming model we have - it maps cleanly onto the
 underlying infrastructure (less abstraction can be a good thing).

With respect, I think we're actually talking about the same thing :-)

You've already said that it may make sense to lump _readers and _admins into
_security. If so, then I think this is what you would get:

  {
_readers:{
  names:[foo,bar],
  roles:[baz]
},
_admins:{
  names:[jan,brian],
  roles:[support]
},
other_security_stuff:{...}
  }

You've also suggested adding some more granular capabilities, such as
_replicators and create-only (dropbox).  Then _security would become:

  {
_readers:{
  names:[foo,bar],
  roles:[baz]
},
_admins:{
  names:[jan,brian],
  roles:[support]
},
_replicators:{
  names:[cron],
  roles:[baz],
},
_creators:{
  names:[foo],
  roles:[_anon],
},
other_security_stuff:{...}
  }

Now, the top-level keys there are exactly what I called rights, and would
be enforced in erlang by check_is_admin, check_is_reader,
check_is_replicator etc.

You can keep the data structure exactly like that. All I'm offering is that
you *could* turn the structure on its head like this:

  {
names:{
  foo:[_reader,_creator],
  bar:[_reader],
  jan:[_admin],
  brian:[_admin],
  cron:[_replicator]
},
roles:{
  baz:[_reader,_replicator],
  support:[_admin],
  _anon:[_creator]
},
other_security_stuff:{...}
  }

I prefer this format because all the rights for one user/role are in a
single place; consequently it's easier to remove all access for a user.

The other benefit is that Futon can also display application-specific rights
without being confused by other stuff at the top level of the _security
object; and, if extra couchdb rights are added, Futon doesn't need to
change.

For example, it would be hard for Futon to display the extra 'doctors'
rights from this:

  {
_readers:{
  names:[foo,bar],
  roles:[baz]
},
_admins:{
  names:[jan,brian],
  roles:[support]
},
doctors:{
  names:[foo,jim],
},
hmac_key:abc12345,
  }

(doctors having significance in validate_doc_update and _show/_list)
because it wouldn't know which of doctors and hmac_key should be
displayed as an ACL.  But if you restructure it as

  {
names:{
  foo:[_reader,doctor],
  bar:[_reader],
  jan:[_admin],
  jim:[doctor],
  brian:[_admin],
},
roles:{
  baz:[_reader],
  support:[_admin]
},
hmac_key:abc12345
  }

then futon can show that jim is a doctor, and allow the doctor right to be
added to other users.

The functionality is ultimately the same though, so I'm not wedded to the
second format.

Regards,

Brian.

Re: Auth Roadmap

[Chris Anderson]

On Thu, Feb 11, 2010 at 4:20 AM, Brian Candler b.cand...@pobox.com wrote:
 I think what you say has merit, but it doesn't jibe with my
 understanding of our implementation in a logical way. I'm ready to
 ship the basic programming model we have - it maps cleanly onto the
 underlying infrastructure (less abstraction can be a good thing).

 With respect, I think we're actually talking about the same thing :-)

 You've already said that it may make sense to lump _readers and _admins into
 _security. If so, then I think this is what you would get:

  {
    _readers:{
      names:[foo,bar],
      roles:[baz]
    },
    _admins:{
      names:[jan,brian],
      roles:[support]
    },
    other_security_stuff:{...}
  }


 You've also suggested adding some more granular capabilities, such as
 _replicators and create-only (dropbox).  Then _security would become:

  {
    _readers:{
      names:[foo,bar],
      roles:[baz]
    },
    _admins:{
      names:[jan,brian],
      roles:[support]
    },
    _replicators:{
      names:[cron],
      roles:[baz],
    },
    _creators:{
      names:[foo],
      roles:[_anon],
    },
    other_security_stuff:{...}
  }



To be clear, I'm not suggesting this at all.

It'd be more like (pardon my earlier accidental _underscores):

{
readers:{
  names:[foo,bar],
  roles:[baz, _replicator, doctor]
},
admins:{
  names:[jan,brian],
  roles:[support, (_admin is an implied member)]
},
other_security_stuff:{...}
}


_replicator, etc wouldn't go in as top level keys with special
meaning. They'd be just another role like doctor or foo. I actually
can't think of a time you'd  put _replicator into the ACLs. It'd be
more likely something checked by the validation function. (Eg only
allow docs to be written if doc.author == userCtx.name, unless we are
replicating and the replication was triggered with both _admin and
_replicator roles.) The more I think about it the more of a bad idea I
think a _replicator role could potentially be.

As far as drop box, I think it'd just be a boolean flag (allow anyone
to write to this database even if they aren't in the readers list)
which should be easy to add in the future without changing any of the
existing stuff.

There's no need for a _creators list as that can be implemented in the
validation function.

 Now, the top-level keys there are exactly what I called rights, and would
 be enforced in erlang by check_is_admin, check_is_reader,
 check_is_replicator etc.

Something like check_is_replicator can be written in javascript and be
part of the validation function.


 You can keep the data structure exactly like that. All I'm offering is that
 you *could* turn the structure on its head like this:

  {
    names:{
      foo:[_reader,_creator],
      bar:[_reader],
      jan:[_admin],
      brian:[_admin],
      cron:[_replicator]
    },
    roles:{
      baz:[_reader,_replicator],
      support:[_admin],
      _anon:[_creator]
    },
    other_security_stuff:{...}
  }

 I prefer this format because all the rights for one user/role are in a
 single place; consequently it's easier to remove all access for a user.

 The other benefit is that Futon can also display application-specific rights
 without being confused by other stuff at the top level of the _security
 object; and, if extra couchdb rights are added, Futon doesn't need to
 change.

 For example, it would be hard for Futon to display the extra 'doctors'
 rights from this:

  {
    _readers:{
      names:[foo,bar],
      roles:[baz]
    },
    _admins:{
      names:[jan,brian],
      roles:[support]
    },
    doctors:{
      names:[foo,jim],
    },
    hmac_key:abc12345,
  }

 (doctors having significance in validate_doc_update and _show/_list)
 because it wouldn't know which of doctors and hmac_key should be
 displayed as an ACL.  But if you restructure it as

  {
    names:{
      foo:[_reader,doctor],
      bar:[_reader],
      jan:[_admin],
      jim:[doctor],
      brian:[_admin],
    },
    roles:{
      baz:[_reader],
      support:[_admin]
    },
    hmac_key:abc12345
  }

 then futon can show that jim is a doctor, and allow the doctor right to be
 added to other users.

 The functionality is ultimately the same though, so I'm not wedded to the
 second format.

 Regards,

 Brian.




-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Re: Auth Roadmap

[Brian Candler]

On Thu, Feb 11, 2010 at 08:32:49AM -0800, Chris Anderson wrote:
 To be clear, I'm not suggesting this at all.
 
 It'd be more like (pardon my earlier accidental _underscores):
 
 {
 readers:{
   names:[foo,bar],
   roles:[baz, _replicator, doctor]
 },
 admins:{
   names:[jan,brian],
   roles:[support, (_admin is an implied member)]
 },
 other_security_stuff:{...}
 }

Oh I see. When you replicate, you give the credentials for the remote host,
but perhaps the local side should pick up a _replicator role.  (Or perhaps
not, if it runs with the credentials of the user who started the
replication)

I can imagine readers splitting in future though: an indirect reader
capability which can access _show/_list/_update but nothing else would be
able to enforce controls at the document and view row level, since those
points all have access to userCtx.

Regards,

Brian.

Re: Auth Roadmap

[Chris Anderson]

On Tue, Feb 9, 2010 at 2:52 PM, Chris Anderson jch...@apache.org wrote:
 Devs,

 I've been getting a lot of feedback about the authentication 
 authorization work that I did over the holidays and over the last few
 weeks. There are also some enhancements I've been thinking about for a
 while. Here's a quick list of what I see as the important things to
 do. I'm not concerned here with releases / feature freeze etc as in my
 opinion CouchDB development is expected to continue even after we
 reach 1.0.

 1) Extensible password storage.

 Thanks Brian Candler for the links to the OpenLDAP style of storage. I
 think we should do this asap so we don't have to worry about backwards
 compatibility with the current storage mechanism until the end of
 time. The relevant message:
 http://permalink.gmane.org/gmane.comp.db.couchdb.devel/7588

I'm helping Filipe Manana with an implementation of this, which will
be backwards compatible with existing admin passwords (stored in
config). We won't try to be backwards compatible with old _users dbs
(it should be simple to write an upgrade script if you have crucial
data). This doesn't need to block 0.11 but it could go in 0.11.1 as
it'd be nice to get it out there for people who want better crypto.


 2) ACLs / Security Object

 I couldn't originally think of a reason the validation funs would need
 to see the per-db admins / readers lists. Brian has a use case for
 this. Also, I think I can accomplish this feature while also
 simplifying the implementation. I'd like to make it so each db has an
 /_security object that contains admins and readers (in their current
 form, but as fields on the object, not as separate object at different
 URLs). This entire object would be made available to the validation
 functions.

This is done and in trunk. I plan to backport this to the 0.11.x
branch unless there are objections.


 3) More system roles.

 Brian also mentioned something about _user and _anon roles which could
 be applied to the userCtx automatically. This would be handy in both
 per-db access control and in validation functions. This will be a bit
 harder to implement as it touches more of the codebase. I'm also
 uneasy about these roles as they raise the burden for implementors of
 pluggable authentication modules.

 Going forward we maybe want to add a _replicator role (or maybe that's
 a horrible idea). We should also think about making it possible for
 _admins to interact with the database without the _admin role. They
 could trigger admin actions with something like sudo. I want to put
 this off for now, because it's complicated and worse-case scenario is
 people don't realize they need to sudo to get things done, and come
 away thinking CouchDB is fighting with them.

we can punt on this for 1.0, it won't break backward compat to add it later


 4) _temp_views and _all_dbs (maybe more)

 Regardless of whether the above is done, for 1.0 we should clean up
 any bugs like these.


Please file tickets (especially with JavaScript test cases) if you
find anything that needs fixing for 1.0. These are important bugs to
fix but they don't need to block 0.11

 5) drop box


this can happen after 1.0

Thanks,

Chris

-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Re: Auth Roadmap

[Jan Lehnardt]

On 11 Feb 2010, at 22:10, Chris Anderson wrote:
 
 2) ACLs / Security Object
 
 I couldn't originally think of a reason the validation funs would need
 to see the per-db admins / readers lists. Brian has a use case for
 this. Also, I think I can accomplish this feature while also
 simplifying the implementation. I'd like to make it so each db has an
 /_security object that contains admins and readers (in their current
 form, but as fields on the object, not as separate object at different
 URLs). This entire object would be made available to the validation
 functions.
 
 This is done and in trunk. I plan to backport this to the 0.11.x
 branch unless there are objections.

Go for it. It looks like you got rid of some extra code there too :)

Cheers
Jan
--
http://couch.io/

Re: Auth Roadmap

[Brian Candler]

On Wed, Feb 10, 2010 at 12:24:26AM +0100, Benoit Chesneau wrote:
 I've read all the thread, and I'm not conviced all readers and admins
 should be in one doc. List could be long also it would require to
 check if one already exists some stuff like it. Why not putting all in
 their docs and making a view on it ?

I'd prefer a real doc or docs too.

But if we have to have an opaque non-scalable non-document object, it would
be better to have one of these things rather than three of them.

  3) More system roles.
 
  Brian also mentioned something about _user and _anon roles which could
  be applied to the userCtx automatically. This would be handy in both
  per-db access control and in validation functions. This will be a bit
  harder to implement as it touches more of the codebase. I'm also
  uneasy about these roles as they raise the burden for implementors of
  pluggable authentication modules.
 
 Everyone could be _anon instead of _admin though.

Maybe _anon isn't even needed, because they will have name:null.

But you'd want a way of saying any logged-in user, which could be as
simple as adding a fixed role like _user to everyone who has a non-null
name. And if you're going to do that, you might as well add _anon role to
everyone who has a null name. Sounds easy to me, but I've not tried coding
it yet.

  We should also think about making it possible for
  _admins to interact with the database without the _admin role. They
  could trigger admin actions with something like sudo.

That's where having a real document would win, because you could use
validate_doc_update for this logic.  Such logic already exists in the _users
database, to prevent non-admins from adding roles to themselves.

Personally I think that if a system admin is a role, then a database admin
should be a role, and a database reader should be a role (and by extension,
you could have a system reader role)

Suppose the per-database _security doc or object looked like this:

  {
names:{
  brian:[_admin],  # brian is database-level admin
  jan:[_reader],
  bob:[_reader,foo]  # foo role has local significance
}
roles:{
  support:[_reader]# anyone with support role can read this DB
}
  }

You'd merge the roles from the userCtx with these roles to get the effective
set of roles for this user when accessing this particular database.

 Just to be sure everything is one user db ? Just want to make sure
 it's possible to replicate all authentification for a node or even a
 cluster.

That's another reason for doing this in docs. Either make _security a real
doc within each database, which can replicate along with it; or put the
authorizations into the _users database (which is easy with
database-specific roles).

The only objection I've heard to the latter idea is that if a database is
filesystem-copied or renamed, the authorizations won't move with it. 
Authorizing against a database uuid rather than database name would solve
that.

Also: given that bob on db1 could be a different person than bob on db2,
it seems dangerous to replicate or copy the authorizations from one database
to another independently of the users database.

  5) drop box
 
  If we had the concept of DBs that you could write to but not read
  from, it would make read-controlled _users dbs compatible with new
  user signups. This would be not that fun to implement, but potentially
  useful for lots of other things too.
 
 
 +1 on this.

Interesting idea - also for logging databases, audit databases etc. Need to
think about the semantics.

Possibly POST to the database itself could have its own role, and it would
ignore _rev so it could only be used to create new docs.  But still some
other mechanism would be required for users to change their own passwords.

One idea is to split _reader into two; one role would permit access to
_show, _list and _update whilst blocking all direct access.

Regards,

Brian.

Re: Auth Roadmap

[Chris Anderson]

On Wed, Feb 10, 2010 at 2:12 AM, Brian Candler b.cand...@pobox.com wrote:
 On Wed, Feb 10, 2010 at 12:24:26AM +0100, Benoit Chesneau wrote:
 I've read all the thread, and I'm not conviced all readers and admins
 should be in one doc. List could be long also it would require to
 check if one already exists some stuff like it. Why not putting all in
 their docs and making a view on it ?

 I'd prefer a real doc or docs too.

 But if we have to have an opaque non-scalable non-document object, it would
 be better to have one of these things rather than three of them.

  3) More system roles.
 
  Brian also mentioned something about _user and _anon roles which could
  be applied to the userCtx automatically. This would be handy in both
  per-db access control and in validation functions. This will be a bit
  harder to implement as it touches more of the codebase. I'm also
  uneasy about these roles as they raise the burden for implementors of
  pluggable authentication modules.

 Everyone could be _anon instead of _admin though.

 Maybe _anon isn't even needed, because they will have name:null.

 But you'd want a way of saying any logged-in user, which could be as
 simple as adding a fixed role like _user to everyone who has a non-null
 name. And if you're going to do that, you might as well add _anon role to
 everyone who has a null name. Sounds easy to me, but I've not tried coding
 it yet.

  We should also think about making it possible for
  _admins to interact with the database without the _admin role. They
  could trigger admin actions with something like sudo.

 That's where having a real document would win, because you could use
 validate_doc_update for this logic.  Such logic already exists in the _users
 database, to prevent non-admins from adding roles to themselves.

 Personally I think that if a system admin is a role, then a database admin
 should be a role, and a database reader should be a role (and by extension,
 you could have a system reader role)

 Suppose the per-database _security doc or object looked like this:

  {
    names:{
      brian:[_admin],      # brian is database-level admin
      jan:[_reader],
      bob:[_reader,foo]  # foo role has local significance
    }
    roles:{
      support:[_reader]    # anyone with support role can read this DB
    }
  }

The problem with this approach, imho, is that currently users have the
same set of roles in every db. That is, your userCtx doesn't change
depending on the db you are accessing. I can see how adding that
capability increases flexibility. But it's the sort of flexibility
that I see having a nasty complexity tax. Eg: currently the /_session
resource is server-wide. As soon as some roles are available on
certain dbs and not others, we need to make it /db/_session which is
misleading as you don't actually log into dbs, you log into the
server. The consequences of per-db roles being applied to users would
ripple through the system as complexity in all sorts of places.

The thing you've got above I'd do as:

jan and bob both have foobar in their list of roles in their
documents in the _users db.

the security object looks like:

{
readers: {
names : [],
roles : [foobar, support],
}
admins : {
names : [brian]
roles : []
}
whatever else...
}

I think we don't want this object to replicate, as moving data around
shouldn't have a chance of affecting ACLs on the target db.

This is why it is not a document. (It could be a _local document and
have similar semantics, but O(log n) read cost instead of O(1) read
cost. This is why we use a special member.)



 You'd merge the roles from the userCtx with these roles to get the effective
 set of roles for this user when accessing this particular database.

 Just to be sure everything is one user db ? Just want to make sure
 it's possible to replicate all authentification for a node or even a
 cluster.

 That's another reason for doing this in docs. Either make _security a real
 doc within each database, which can replicate along with it; or put the
 authorizations into the _users database (which is easy with
 database-specific roles).

 The only objection I've heard to the latter idea is that if a database is
 filesystem-copied or renamed, the authorizations won't move with it.
 Authorizing against a database uuid rather than database name would solve
 that.

 Also: given that bob on db1 could be a different person than bob on db2,
 it seems dangerous to replicate or copy the authorizations from one database
 to another independently of the users database.

  5) drop box
 
  If we had the concept of DBs that you could write to but not read
  from, it would make read-controlled _users dbs compatible with new
  user signups. This would be not that fun to implement, but potentially
  useful for lots of other things too.
 

 +1 on this.

 Interesting idea - also for logging databases, audit databases etc. Need to
 think about the semantics.

 Possibly POST to the database itself could have its own 

Re: Auth Roadmap

[Chris Anderson]

On Wed, Feb 10, 2010 at 2:27 PM, Brian Candler b.cand...@pobox.com wrote:
 On Wed, Feb 10, 2010 at 07:26:00AM -0800, Chris Anderson wrote:
 The problem with this approach, imho, is that currently users have the
 same set of roles in every db. That is, your userCtx doesn't change
 depending on the db you are accessing. I can see how adding that
 capability increases flexibility. But it's the sort of flexibility
 that I see having a nasty complexity tax. Eg: currently the /_session
 resource is server-wide. As soon as some roles are available on
 certain dbs and not others, we need to make it /db/_session which is
 misleading as you don't actually log into dbs, you log into the
 server.

 Yes this could be confusing, especially adding db-roles to system-roles.

 Maybe it's clearer to think of it as groups and rights:

 * groups are what your user is a member of (in your global _session).
 * rights are what you can do in a particular database.
 * rights within a db can be assigned to individual users or to groups.
 * The _admin group picks up _admin rights automatically on all dbs.


I think what you say has merit, but it doesn't jibe with my
understanding of our implementation in a logical way. I'm ready to
ship the basic programming model we have - it maps cleanly onto the
underlying infrastructure (less abstraction can be a good thing).

I think the kinds of apps you want to write can work in the current
model, and I hope the trade off for less abstraction (eg no _rev on
the _security object) in favor of simplicity will turn out to be the
right choice.

I'd like to help you find an architecture that works for your app.
Maybe db(s) per user + filtered _admin controlled replication, which
also maps really cleanly to usage patterns.

Thanks for all the feedback.


 Regards,

 Brian.




-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Re: Roadmap 0.11?

[Benoit Chesneau]

On Tue, Feb 9, 2010 at 3:51 AM, Mark Hammond skippy.hamm...@gmail.com wrote:
 On 9/02/2010 7:05 AM, Per Ejeklint wrote:

 And will official windows builds be available with it?

 I've been putting together the official binaries for 0.10 and will be
 making some just as official for 0.11.  I expect that for 0.11 and later
 though, the builds I put together will be listed on the main download page
 making them really official instead of just official :)

 http://people.apache.org/~mhammond/dist/

 Mark.


There Is an old threads about what will/could be in 0.11. I think it
would be good to have a roadmap published. It's definitely needed for
non developers people that want to have an idea of futures
developments and orientation.

- benoît

Auth Roadmap

[Chris Anderson]

Devs,

I've been getting a lot of feedback about the authentication 
authorization work that I did over the holidays and over the last few
weeks. There are also some enhancements I've been thinking about for a
while. Here's a quick list of what I see as the important things to
do. I'm not concerned here with releases / feature freeze etc as in my
opinion CouchDB development is expected to continue even after we
reach 1.0.

1) Extensible password storage.

Thanks Brian Candler for the links to the OpenLDAP style of storage. I
think we should do this asap so we don't have to worry about backwards
compatibility with the current storage mechanism until the end of
time. The relevant message:
http://permalink.gmane.org/gmane.comp.db.couchdb.devel/7588

2) ACLs / Security Object

I couldn't originally think of a reason the validation funs would need
to see the per-db admins / readers lists. Brian has a use case for
this. Also, I think I can accomplish this feature while also
simplifying the implementation. I'd like to make it so each db has an
/_security object that contains admins and readers (in their current
form, but as fields on the object, not as separate object at different
URLs). This entire object would be made available to the validation
functions.

3) More system roles.

Brian also mentioned something about _user and _anon roles which could
be applied to the userCtx automatically. This would be handy in both
per-db access control and in validation functions. This will be a bit
harder to implement as it touches more of the codebase. I'm also
uneasy about these roles as they raise the burden for implementors of
pluggable authentication modules.

Going forward we maybe want to add a _replicator role (or maybe that's
a horrible idea). We should also think about making it possible for
_admins to interact with the database without the _admin role. They
could trigger admin actions with something like sudo. I want to put
this off for now, because it's complicated and worse-case scenario is
people don't realize they need to sudo to get things done, and come
away thinking CouchDB is fighting with them.

4) _temp_views and _all_dbs (maybe more)

Regardless of whether the above is done, for 1.0 we should clean up
any bugs like these.

5) drop box

If we had the concept of DBs that you could write to but not read
from, it would make read-controlled _users dbs compatible with new
user signups. This would be not that fun to implement, but potentially
useful for lots of other things too.


I hope this clears things up for people. The only other thing I feel
compelled to add is that CouchDB has never had (and probably never
will have) any concept of per-document read control. It is too fraught
with pitfalls, and even the best ideas about how to enforce
per-document read-control put an unacceptable burden on implementors
of alternate view engines.

Again, this is open-source, so scratch your own itch, feel free to
write patches, argue about all this, etc.

That said, if nothing but #4 ever gets done I'd still be proud of the
system as it stands today, as I think it's more or less the simplest
thing that could possibly work.

Cheers,
Chris

-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Roadmap 0.11?

[Per Ejeklint]

Jan asked me to take this in dev@ so here I am. :-)

The official roadmap page (http://couchdb.apache.org/roadmap.html) is a tiiiny 
little outdated. As I'm trying to convince my new customer to make an epic 
technology leap I need as much info as possible about the near future for 
CouchDB. 0.11 seems to feature rich enough for our production environment. So 
can you with insights comment on about when we will have a 0.11 release? And 
will official windows builds be available with it?

Cheers,

/Per

Re: Updating the CouchDB roadmap

[Brian Candler]

On Sun, Jan 24, 2010 at 09:33:02PM -0800, Chris Anderson wrote:
 To round out this list, I think
 
 * Reader ACLs
...
 
 look like they will make it into 0.11.

That's the jchris/readeracl branch presumably?

I was hoping to turn my counter-proposal(*) into code, but I've not had any
time to do so unfortunately.

Regards,

Brian.

(*) which was, in summary:

1. user record has roles like foo:_reader or [foo,_reader]

2. _anon user has roles of db:_reader for all public databases

3. you can read database foo only if you have one of
 foo:_reader, foo:_admin, _reader or _admin roles

4. /_all_dbs lists only those databases to which you or _anon have read access
   (but shows every database if you have _reader or _admin roles)

5. userdb validate_doc_update allows someone with foo:_admin to add and
   remove roles foo:*. Also foo:_manager to add and remove roles foo:*
   apart from foo:_admin

Re: Updating the CouchDB roadmap

[Chris Anderson]

On Mon, Jan 25, 2010 at 1:14 AM, Brian Candler b.cand...@pobox.com wrote:
 On Sun, Jan 24, 2010 at 09:33:02PM -0800, Chris Anderson wrote:
 To round out this list, I think

 * Reader ACLs
 ...

 look like they will make it into 0.11.

 That's the jchris/readeracl branch presumably?

 I was hoping to turn my counter-proposal(*) into code, but I've not had any
 time to do so unfortunately.

 Regards,

 Brian.

 (*) which was, in summary:

 1. user record has roles like foo:_reader or [foo,_reader]

 2. _anon user has roles of db:_reader for all public databases

 3. you can read database foo only if you have one of
     foo:_reader, foo:_admin, _reader or _admin roles

 4. /_all_dbs lists only those databases to which you or _anon have read access
   (but shows every database if you have _reader or _admin roles)

Thanks for reminding me that I should set _all_dbs to hide dbs the
curertn user can't read if that doesn't incur much additional
overhead.

Also, I plan to put a Futon interface on the reader and admin lists.

And, the security object still needs work, to round out the capability
set to be something like what you describe here.


 5. userdb validate_doc_update allows someone with foo:_admin to add and
   remove roles foo:*. Also foo:_manager to add and remove roles foo:*
   apart from foo:_admin




-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Re: Updating the CouchDB roadmap

[Brian Candler]

 Thanks for reminding me that I should set _all_dbs to hide dbs the
 curertn user can't read if that doesn't incur much additional
 overhead.

I think that it will incur a huge overhead, if there are a large number of
databases and the reader rights are stored within the databases themselves. 
It'll have to open and read every single database file on disk, even if the
user only has access to one.

Storing the rights within the user record avoids this problem completely.

Re: Updating the CouchDB roadmap

[Dirkjan Ochtman]

On Fri, Dec 18, 2009 at 20:30, Damien Katz dam...@apache.org wrote:
 I think we should set a hard date of Feb 1. for feature freeze, 0.11.0 is the 
 last new feature release.

Did this ever turn into some kind of consensus, or is there no idea
yet about a 0.11 release time frame?

Cheers,

Dirkjan

Re: Updating the CouchDB roadmap

[Benoit Chesneau]

On Thu, Dec 3, 2009 at 8:29 PM, Noah Slater nsla...@tumbolia.org wrote:
 Hey,

 As part of the release procedure, this is a request to update the roadmap 
 document:

        http://couchdb.apache.org/roadmap.html

 Please comment on what should be added, and what should be removed.

 After a we reach consensus, I will update the roadmap.

 Thanks,

 Noah

This a quick summary of what have been proposed on this thread. If I
forgot one, sorry, and don't hesitate to update :) I'm not sure all of
this is in jira. But maybe once they are, we could extract a roadmap
from it ?

*) New tasks:
- Support for compressed attachments
- authorization per database
- a global notification handler which would allow an indexer (or
anything else) to listen changes without opening a connection per db.
Some work started here
- Better RFC 2616 compliance: some work started, couchdb webmachine
branch on github and other commits
- Windows support for compaction
- design doc refactoring: work started by jchris
- Account's tab for Futon

*) Old tasks that could be removed from the roadmap :
- Continuous replication
- Pure tail-append storage
- Update API to parse client requests in JavaScript
- Cookie authentication support
- Windows build supports

Date for 0.11: 1/02/20010


- benoit

Re: Updating the CouchDB roadmap

[Filipe David Manana]

An updated (and no longer breaking 4 test cases) patch for storing
compressed attachments was added to tickets 583 and 437.

cheers

On Sat, Dec 19, 2009 at 8:46 AM, Filipe David Manana fdman...@gmail.comwrote:

 After rev 891077 (ticket 558, md5 integrity check) I need to update that
 patch for storing gzip compressed attachments.
 The md5 calculated by couch_stream corresponds to the md5 of the gzipped
 content, therefore not matching the attachment content sent by the client
 for those attachments having a mime type listed in the additional config
 file.

 I'll be fixing this, adding a config option for the compression level and
 do the necessary modifications on the existing test cases. A new patch will
 be coming soon :)

 cheers


 On Fri, Dec 18, 2009 at 8:31 PM, Damien Katz dam...@apache.org wrote:

 FYI, I haven't looked at this patch's code, but I like it's concepts and I
 hope of the other committers will have a chance to look at it soon and work
 out any issues to get it checked into trunk. If not I eventually will get
 around to it, but I can't promise when.

 -Damien


 On Dec 9, 2009, at 8:07 AM, Filipe David Manana wrote:

  that's ticket 583 - https://issues.apache.org/jira/browse/COUCHDB-583
 
  The ticket's title is no longer fair. One of the last comments
  mentions the possibility of storing attachments in gzip compressed
  form (suggestion from Damien).
  I submitted a patch for that feature yesterday.
 
  cheers
 
  On Fri, Dec 4, 2009 at 7:17 PM, Robert Newson robert.new...@gmail.com
 wrote:
  Support for compressed attachments? I think there's a ticket for it,
  but the basic idea is to support Accept-Encoding/Content-Encoding form
  the client and store the attachment with compression (and
  decompressing on demand).
 
  A bulk insertion endpoint that included attachments (without base64
  inflation) would also be nice.
 
  On Fri, Dec 4, 2009 at 5:55 PM, Dirkjan Ochtman djc.ocht...@gmail.com
 wrote:
  On Fri, Dec 4, 2009 at 18:32, Noah Slater nsla...@tumbolia.org
 wrote:
  Hey, does anyone want to start the discussion off?
 
  Question from one of those trailing around on actual releases, that
  may help getting started: what big-ticket features are in trunk, but
  not 0.10.1?
 
  Cheers,
 
  Dirkjan
 
 
 
 
 
  --
  Filipe David Manana,
  fdman...@gmail.com
  PGP key - http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xC569452B
 
  Reasonable men adapt themselves to the world.
  Unreasonable men adapt the world to themselves.
  That's why all progress depends on unreasonable men.




 --
 Filipe David Manana,
 fdman...@gmail.com
 PGP key - http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xC569452B

 Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men.




-- 
Filipe David Manana,
fdman...@gmail.com
PGP key - http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xC569452B

Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men.

Re: Updating the CouchDB roadmap

[Benoit Chesneau]

On Sat, Dec 19, 2009 at 9:08 AM, Chris Anderson jch...@apache.org wrote:
 On Fri, Dec 18, 2009 at 9:02 AM, Noah Slater nsla...@tumbolia.org wrote:

 On 18 Dec 2009, at 16:45, Dirkjan Ochtman wrote:

 That said, it's perfectly alright if there's no roadmap at all, I'm
 just surprised by the lack of thoughts from the core developers (who
 are usually the ones to have these ideas) in this thread.

 Three of the core developers have been busy attending to real life for the 
 last few weeks. :)

 Heh,

 Mostly I've been dealing with real life by buckling down and writing code.

 I wrote up my ideas for 0.11 / 1.0 in this thread Nov 1st:

 http://mail-archives.apache.org/mod_mbox/couchdb-dev/200911.mbox/%3ce282921e0911011048sec6a907xbeac06651ae2c...@mail.gmail.com%3e

 Re-reading it they are still features I'm interested in, although I'm
 much less ambitious now. What I'd really like to see for 0.11:

 Account's tab for Futon:
  There's no reason app developers should have to write user account
 code or login/logout code. Users should just be able to log in and out
 of CouchDB (and sign up for accounts) via something like a Futon page
 that ships with the db. The userCtx is already available to Ajax apps
 via /_session so once we have a reliable way to log in and out this
 will be useful.

Maybe we could also make the auth really useful when you don't access
to CouchDB behind a proxy. My plan about this is :

- having authentification a little more modular than it is : cascading
across authentification modules and also add authentification via a
script
- authorization: having a way to protect read/write(so delete too) on
a db for a list of users or roles
- Protect access to the user database. Having it readable by public
isn't good for security purpose. Regardless of the length of the key
or the algorithm used, there will be a time where it isn't enough. Imo
we should hide access to this db behind an api.


 Design Doc Refactor:
  I've been heads down working on this patch in my spare time for the
 last 3 weeks. It is 90% a pure refactor (cleanup code not add new
 features.) It is nearing completion. This will change the query server
 protocol but not the HTTP API. You can look at what I've got here:
 http://github.com/jchris/couchdb/tree/ddoc-qs

What will be the changes in protocol ?


 Query Server Security:
  I haven't started on this yet but my plan is just to work from
 http://github.com/rcoder/couchdb/commits/master to make it more clear
 when the curl bindings are available to the JS sandbox. If anyone will
 pick this up and run with it I will jump for joy.

I failed to find the relation with security :) Isn't the point of this
fork to add an http client ?


- benoit

Re: Updating the CouchDB roadmap

[Benoit Chesneau]

On Fri, Dec 18, 2009 at 5:02 PM, Paul Davis paul.joseph.da...@gmail.com wrote:


 I'm not really a fan of roadmaps. We get new features when people
 write them. Everyone has the ability to write code, submit it and get
 a new feature. I can't tell you what feature you're going to write, so
 why would I tell everyone else that I expect you to write a given
 feature?

 Granted, I would be perfectly happy referring to versions by sha1 so
 maybe I'm a bit crazy.

 HTH,
 Paul Davis

Well roadmap in a sense would allow everyone to know who working on
what. Or if not who, what is the work in progress/planned.  Maybe more
than a roadmap we can put a todo list and at a time extract some items
done to make a release ?

- benoit

Re: Updating the CouchDB roadmap

[Robert Dionne]

I sort of agree with you, but in all fairness some of the core programmers have 
weighed in.

To repeat my earlier post, I think it's a good time to put some stakes in the 
ground with respect to direction. I'm sure there are a number of parties 
interested in extending and making use of the internals and a lot has been 
discussed in terms of what could be done. A roadmap would be nice, especially 
since there is growing corporate support, a key selling point for CouchDB.




On Dec 18, 2009, at 2:39 AM, Dirkjan Ochtman wrote:

 It seems weird that none of the core developers have bothered to write
 up some of their ideas here, for ten days running now. I understand
 you're all busy with Relaxed, but it doesn't have to take so long,
 does it? IMO it would be nice to have some kind of idea of where
 CouchDB is going and/or what will be in the next release.
 
 Cheers,
 
 Dirkjan

Re: Updating the CouchDB roadmap

[Paul Davis]

On Fri, Dec 18, 2009 at 2:39 AM, Dirkjan Ochtman djc.ocht...@gmail.com wrote:
 It seems weird that none of the core developers have bothered to write
 up some of their ideas here, for ten days running now. I understand
 you're all busy with Relaxed, but it doesn't have to take so long,
 does it? IMO it would be nice to have some kind of idea of where
 CouchDB is going and/or what will be in the next release.

 Cheers,

 Dirkjan


I'm not really a fan of roadmaps. We get new features when people
write them. Everyone has the ability to write code, submit it and get
a new feature. I can't tell you what feature you're going to write, so
why would I tell everyone else that I expect you to write a given
feature?

Granted, I would be perfectly happy referring to versions by sha1 so
maybe I'm a bit crazy.

HTH,
Paul Davis

Re: Updating the CouchDB roadmap

[Noah Slater]

Hey,

Does JIRA have a page what we could use as the roadmap?

That would have the added advantage of updating itself automatically as we 
edited tickets.

I found the roadmap tab from the main CouchDB page, but it doesn't have any 
tickets for post 0.10 on it.

Thoughts?

Thanks,

Noah

Re: Updating the CouchDB roadmap

[Paul Davis]

Link might be handy:

http://forrest.apache.org/forrest-issues.html

On Fri, Dec 18, 2009 at 12:09 PM, Paul Davis
paul.joseph.da...@gmail.com wrote:
 On Fri, Dec 18, 2009 at 12:05 PM, Noah Slater nsla...@tumbolia.org wrote:
 Hey,

 Does JIRA have a page what we could use as the roadmap?

 That would have the added advantage of updating itself automatically as we 
 edited tickets.

 I found the roadmap tab from the main CouchDB page, but it doesn't have any 
 tickets for post 0.10 on it.

 Thoughts?

 Thanks,

 Noah

 There was that one page that listed tickets in a format that you said
 you were going to include in the release procedure. That's about all I
 can think of other than saving a query which isn't nearly as awesome.

 There's also an example from the Apache Forrest site [1] that lists
 open issues for the next release on their site which might also be an
 interesting approach.

 HTH,
 Paul Davis

 Paul Davis

Re: Updating the CouchDB roadmap

[Robert Newson]

Jira does build a roadmap for you, but it depends on you filing
bugs/tasks and assigning them as fix for version. I'm not sure we
all can add new versions (that should be locked down if it's not
already), but that's how roadmaps are handled in jira.

B.

On Fri, Dec 18, 2009 at 5:10 PM, Paul Davis paul.joseph.da...@gmail.com wrote:
 Link might be handy:

 http://forrest.apache.org/forrest-issues.html

 On Fri, Dec 18, 2009 at 12:09 PM, Paul Davis
 paul.joseph.da...@gmail.com wrote:
 On Fri, Dec 18, 2009 at 12:05 PM, Noah Slater nsla...@tumbolia.org wrote:
 Hey,

 Does JIRA have a page what we could use as the roadmap?

 That would have the added advantage of updating itself automatically as we 
 edited tickets.

 I found the roadmap tab from the main CouchDB page, but it doesn't have any 
 tickets for post 0.10 on it.

 Thoughts?

 Thanks,

 Noah

 There was that one page that listed tickets in a format that you said
 you were going to include in the release procedure. That's about all I
 can think of other than saving a query which isn't nearly as awesome.

 There's also an example from the Apache Forrest site [1] that lists
 open issues for the next release on their site which might also be an
 interesting approach.

 HTH,
 Paul Davis

 Paul Davis

Re: Updating the CouchDB roadmap

[Noah Slater]

On 18 Dec 2009, at 17:37, Robert Newson wrote:

 Jira does build a roadmap for you, but it depends on you filing
 bugs/tasks and assigning them as fix for version. I'm not sure we
 all can add new versions (that should be locked down if it's not
 already), but that's how roadmaps are handled in jira.

Only me and the JIRA admins can add new versions.

Re: Updating the CouchDB roadmap

[Damien Katz]

Apologies for not replying, I have a lot of stuff going on right.

For 1.0 features, I only want a few more enhancements to continuous replication 
(they might already be there) and full windows support.

I think we should set a hard date of Feb 1. for feature freeze, 0.11.0 is the 
last new feature release. Then after that, all new code is for stabilization 
until we hit 1.0. There will be exceptions some features, and we'll vote on 
them as they come up. But generally the rules should be post-0.11.0, no new 
features.

1.0 isn't the end of the line for your desired features though, While 1.0.x is 
for bug fixes , 1.1, 1.2 etc can have new stuff after a beta period, etc.

That's my take on the roadmap.

-Damien


On Dec 18, 2009, at 7:58 AM, Robert Dionne wrote:

 I sort of agree with you, but in all fairness some of the core programmers 
 have weighed in.
 
 To repeat my earlier post, I think it's a good time to put some stakes in the 
 ground with respect to direction. I'm sure there are a number of parties 
 interested in extending and making use of the internals and a lot has been 
 discussed in terms of what could be done. A roadmap would be nice, especially 
 since there is growing corporate support, a key selling point for CouchDB.
 
 
 
 
 On Dec 18, 2009, at 2:39 AM, Dirkjan Ochtman wrote:
 
 It seems weird that none of the core developers have bothered to write
 up some of their ideas here, for ten days running now. I understand
 you're all busy with Relaxed, but it doesn't have to take so long,
 does it? IMO it would be nice to have some kind of idea of where
 CouchDB is going and/or what will be in the next release.
 
 Cheers,
 
 Dirkjan
 

Re: Updating the CouchDB roadmap

[Damien Katz]

FYI, I haven't looked at this patch's code, but I like it's concepts and I hope 
of the other committers will have a chance to look at it soon and work out any 
issues to get it checked into trunk. If not I eventually will get around to it, 
but I can't promise when.

-Damien


On Dec 9, 2009, at 8:07 AM, Filipe David Manana wrote:

 that's ticket 583 - https://issues.apache.org/jira/browse/COUCHDB-583
 
 The ticket's title is no longer fair. One of the last comments
 mentions the possibility of storing attachments in gzip compressed
 form (suggestion from Damien).
 I submitted a patch for that feature yesterday.
 
 cheers
 
 On Fri, Dec 4, 2009 at 7:17 PM, Robert Newson robert.new...@gmail.com wrote:
 Support for compressed attachments? I think there's a ticket for it,
 but the basic idea is to support Accept-Encoding/Content-Encoding form
 the client and store the attachment with compression (and
 decompressing on demand).
 
 A bulk insertion endpoint that included attachments (without base64
 inflation) would also be nice.
 
 On Fri, Dec 4, 2009 at 5:55 PM, Dirkjan Ochtman djc.ocht...@gmail.com 
 wrote:
 On Fri, Dec 4, 2009 at 18:32, Noah Slater nsla...@tumbolia.org wrote:
 Hey, does anyone want to start the discussion off?
 
 Question from one of those trailing around on actual releases, that
 may help getting started: what big-ticket features are in trunk, but
 not 0.10.1?
 
 Cheers,
 
 Dirkjan
 
 
 
 
 
 -- 
 Filipe David Manana,
 fdman...@gmail.com
 PGP key - http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xC569452B
 
 Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men.

Re: Updating the CouchDB roadmap

[Filipe David Manana]

After rev 891077 (ticket 558, md5 integrity check) I need to update that
patch for storing gzip compressed attachments.
The md5 calculated by couch_stream corresponds to the md5 of the gzipped
content, therefore not matching the attachment content sent by the client
for those attachments having a mime type listed in the additional config
file.

I'll be fixing this, adding a config option for the compression level and do
the necessary modifications on the existing test cases. A new patch will be
coming soon :)

cheers

On Fri, Dec 18, 2009 at 8:31 PM, Damien Katz dam...@apache.org wrote:

 FYI, I haven't looked at this patch's code, but I like it's concepts and I
 hope of the other committers will have a chance to look at it soon and work
 out any issues to get it checked into trunk. If not I eventually will get
 around to it, but I can't promise when.

 -Damien


 On Dec 9, 2009, at 8:07 AM, Filipe David Manana wrote:

  that's ticket 583 - https://issues.apache.org/jira/browse/COUCHDB-583
 
  The ticket's title is no longer fair. One of the last comments
  mentions the possibility of storing attachments in gzip compressed
  form (suggestion from Damien).
  I submitted a patch for that feature yesterday.
 
  cheers
 
  On Fri, Dec 4, 2009 at 7:17 PM, Robert Newson robert.new...@gmail.com
 wrote:
  Support for compressed attachments? I think there's a ticket for it,
  but the basic idea is to support Accept-Encoding/Content-Encoding form
  the client and store the attachment with compression (and
  decompressing on demand).
 
  A bulk insertion endpoint that included attachments (without base64
  inflation) would also be nice.
 
  On Fri, Dec 4, 2009 at 5:55 PM, Dirkjan Ochtman djc.ocht...@gmail.com
 wrote:
  On Fri, Dec 4, 2009 at 18:32, Noah Slater nsla...@tumbolia.org
 wrote:
  Hey, does anyone want to start the discussion off?
 
  Question from one of those trailing around on actual releases, that
  may help getting started: what big-ticket features are in trunk, but
  not 0.10.1?
 
  Cheers,
 
  Dirkjan
 
 
 
 
 
  --
  Filipe David Manana,
  fdman...@gmail.com
  PGP key - http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xC569452B
 
  Reasonable men adapt themselves to the world.
  Unreasonable men adapt the world to themselves.
  That's why all progress depends on unreasonable men.




-- 
Filipe David Manana,
fdman...@gmail.com
PGP key - http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xC569452B

Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men.

Re: Updating the CouchDB roadmap

[Dirkjan Ochtman]

It seems weird that none of the core developers have bothered to write
up some of their ideas here, for ten days running now. I understand
you're all busy with Relaxed, but it doesn't have to take so long,
does it? IMO it would be nice to have some kind of idea of where
CouchDB is going and/or what will be in the next release.

Cheers,

Dirkjan

Re: Updating the CouchDB roadmap

[Benoit Chesneau]

On Thu, Dec 3, 2009 at 8:29 PM, Noah Slater nsla...@tumbolia.org wrote:
 Hey,

 As part of the release procedure, this is a request to update the roadmap 
 document:

        http://couchdb.apache.org/roadmap.html

 Please comment on what should be added, and what should be removed.


for next release I would like to have :
- authorization per database
- a global notification handler which would allow an indexer (or
anything else) to listen changes without opening a connection per db.
Also maybe we should define a global way to index document content and
send it on an indexer like we do for view server ?
- Better RFC 2616 compliance: davisp and me started to write on
webmachine couchdb

Other stuff but I don't think it's possible for next release :

native couchdb clustering : possibility to dispatch database and docs
across multiple nodes that could be added on the fly.


I think that's thse features could be removed from the roadmap :

Continuous replication
Pure tail-append storage
Update API to parse client requests in JavaScript
Cookie authentication support
Windows build supports


- benoit