[jira] [Commented] (COUCHDB-2237) Add a 'live' sugar for 'continuous'
[ https://issues.apache.org/jira/browse/COUCHDB-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14395820#comment-14395820 ] Jan Lehnardt commented on COUCHDB-2237: --- As I said on the GH ticket, you (Robert K) have enough momentum to land this and no -1. This is not “doesn’t have a lot of fans” :) I do like `live` over `stream`. tl;dr: do land the code :) Add a 'live' sugar for 'continuous' --- Key: COUCHDB-2237 URL: https://issues.apache.org/jira/browse/COUCHDB-2237 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Components: HTTP Interface Reporter: Dale Harvey With PouchDB we generally try to stick to the same param names as Couch, we are even changing some we implemented first to be compatible (https://github.com/pouchdb/pouchdb/issues/2193) However 'continuous' sucks to type, its confusing to type and spell and everyone gets it wrong, we still support it but switched to documenting it as 'live' and life is awesome again -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2638) CouchDB should not be writing /etc/couchdb/local.ini
[ https://issues.apache.org/jira/browse/COUCHDB-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363136#comment-14363136 ] Jan Lehnardt commented on COUCHDB-2638: --- In addition, CouchDB has behaved like this since it had a config system (0.7.0 or so, back in 2007 or 08), and has happily been on FreeBSD ever since and this is the first time this comes up :) CouchDB should not be writing /etc/couchdb/local.ini Key: COUCHDB-2638 URL: https://issues.apache.org/jira/browse/COUCHDB-2638 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Reporter: Yuri Fix For: 2.0.0 I am getting such messages in log on FreeBSD: Could not write config file /usr/local/etc/couchdb/local.ini: permission denied The problem is that CoachDB supplies the original copy of local.ini, and it is treated as a template for this configuration file. It is placed into /usr/local/etc/couchdb/local.ini.sample, and its copy is placed into /usr/local/etc/couchdb/local.ini. Everything under /etc is what admin configures. Ideally admin can compare local.ini and local.ini.sample and see if anything in default configuration was modified compared to the suggested sample. When the executable itself modifies local.ini too, this makes it very confusing. Admin will be confused if he should or shouldn't touch this file. My suggestion is that CouchDB should copy local.ini under /var/db/, or somewhere else, and write it there. /etc isn't supposed to be writable by the process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2638) CouchDB should not be writing /etc/couchdb/local.ini
[ https://issues.apache.org/jira/browse/COUCHDB-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363535#comment-14363535 ] Jan Lehnardt commented on COUCHDB-2638: --- Rest assured, CouchDB is getting plenty of use. I don’t consider this a bug :) CouchDB should not be writing /etc/couchdb/local.ini Key: COUCHDB-2638 URL: https://issues.apache.org/jira/browse/COUCHDB-2638 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Reporter: Yuri Fix For: 2.0.0 I am getting such messages in log on FreeBSD: Could not write config file /usr/local/etc/couchdb/local.ini: permission denied The problem is that CoachDB supplies the original copy of local.ini, and it is treated as a template for this configuration file. It is placed into /usr/local/etc/couchdb/local.ini.sample, and its copy is placed into /usr/local/etc/couchdb/local.ini. Everything under /etc is what admin configures. Ideally admin can compare local.ini and local.ini.sample and see if anything in default configuration was modified compared to the suggested sample. When the executable itself modifies local.ini too, this makes it very confusing. Admin will be confused if he should or shouldn't touch this file. My suggestion is that CouchDB should copy local.ini under /var/db/, or somewhere else, and write it there. /etc isn't supposed to be writable by the process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2638) CouchDB should not be writing /etc/couchdb/local.ini
[ https://issues.apache.org/jira/browse/COUCHDB-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363534#comment-14363534 ] Jan Lehnardt commented on COUCHDB-2638: --- Rest assured, CouchDB is getting plenty of use. I don’t consider this a bug :) CouchDB should not be writing /etc/couchdb/local.ini Key: COUCHDB-2638 URL: https://issues.apache.org/jira/browse/COUCHDB-2638 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Reporter: Yuri Fix For: 2.0.0 I am getting such messages in log on FreeBSD: Could not write config file /usr/local/etc/couchdb/local.ini: permission denied The problem is that CoachDB supplies the original copy of local.ini, and it is treated as a template for this configuration file. It is placed into /usr/local/etc/couchdb/local.ini.sample, and its copy is placed into /usr/local/etc/couchdb/local.ini. Everything under /etc is what admin configures. Ideally admin can compare local.ini and local.ini.sample and see if anything in default configuration was modified compared to the suggested sample. When the executable itself modifies local.ini too, this makes it very confusing. Admin will be confused if he should or shouldn't touch this file. My suggestion is that CouchDB should copy local.ini under /var/db/, or somewhere else, and write it there. /etc isn't supposed to be writable by the process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (COUCHDB-2638) CouchDB should not be writing /etc/couchdb/local.ini
[ https://issues.apache.org/jira/browse/COUCHDB-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt updated COUCHDB-2638: -- Comment: was deleted (was: Rest assured, CouchDB is getting plenty of use. I don’t consider this a bug :)) CouchDB should not be writing /etc/couchdb/local.ini Key: COUCHDB-2638 URL: https://issues.apache.org/jira/browse/COUCHDB-2638 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Reporter: Yuri Fix For: 2.0.0 I am getting such messages in log on FreeBSD: Could not write config file /usr/local/etc/couchdb/local.ini: permission denied The problem is that CoachDB supplies the original copy of local.ini, and it is treated as a template for this configuration file. It is placed into /usr/local/etc/couchdb/local.ini.sample, and its copy is placed into /usr/local/etc/couchdb/local.ini. Everything under /etc is what admin configures. Ideally admin can compare local.ini and local.ini.sample and see if anything in default configuration was modified compared to the suggested sample. When the executable itself modifies local.ini too, this makes it very confusing. Admin will be confused if he should or shouldn't touch this file. My suggestion is that CouchDB should copy local.ini under /var/db/, or somewhere else, and write it there. /etc isn't supposed to be writable by the process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (COUCHDB-2638) CouchDB should not be writing /etc/couchdb/local.ini
[ https://issues.apache.org/jira/browse/COUCHDB-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt closed COUCHDB-2638. - CouchDB should not be writing /etc/couchdb/local.ini Key: COUCHDB-2638 URL: https://issues.apache.org/jira/browse/COUCHDB-2638 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Reporter: Yuri Fix For: 2.0.0 I am getting such messages in log on FreeBSD: Could not write config file /usr/local/etc/couchdb/local.ini: permission denied The problem is that CoachDB supplies the original copy of local.ini, and it is treated as a template for this configuration file. It is placed into /usr/local/etc/couchdb/local.ini.sample, and its copy is placed into /usr/local/etc/couchdb/local.ini. Everything under /etc is what admin configures. Ideally admin can compare local.ini and local.ini.sample and see if anything in default configuration was modified compared to the suggested sample. When the executable itself modifies local.ini too, this makes it very confusing. Admin will be confused if he should or shouldn't touch this file. My suggestion is that CouchDB should copy local.ini under /var/db/, or somewhere else, and write it there. /etc isn't supposed to be writable by the process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2635) Replace hardcoded list of systems dbs with registration approach
[ https://issues.apache.org/jira/browse/COUCHDB-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355614#comment-14355614 ] Jan Lehnardt commented on COUCHDB-2635: --- I don’t think making this end-user configurable is a good idea. Replace hardcoded list of systems dbs with registration approach Key: COUCHDB-2635 URL: https://issues.apache.org/jira/browse/COUCHDB-2635 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Reporter: ILYA Assignee: ILYA To replace this https://github.com/apache/couchdb-couch/blob/master/include/couch_db.hrl#L43:L49 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2310) Add a bulk API for revs open_revs
[
https://issues.apache.org/jira/browse/COUCHDB-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358514#comment-14358514
]
Jan Lehnardt commented on COUCHDB-2310:
---
Ok, trying to sum up this discussion. I’m aiming for least amount of work to
get something tangible going.
1. Let us only consider 2.0 and beyond.
2. Let us only focus on an intermediate way to make PouchDB, Couchbase Mobile
etc. replication faster. A fully streaming API is out of scope for this ticket
(although we should work on it, once this lands).
3. I propose to use the existing `_bulk_get` spec from Couchbase Sync Server /
Mobile / rcouch ([~benoitc] if you do have that CouchDB 2.0-compatible patch
for _bulk_get, it’d be nice to see now, even if not 100% ready).
4. As [~rnewson] notes, we add this as `_bulk_revs` with an immediately
deprecated alias `_bulk_get`. The two benefits here are easy feature detection
(_bulk_revs/_get = 404 on older CouchDB versions) and immediate compatibility
with existing versions of the other replicating stores.
5. PouchDB will have to be updated to use the `_bulk_get` endpoint. [~dholth]’s
work, as far as I can tell, would only need minor adjustments. Would PouchDB
accept such a patch into core ([~nolanlawson]).
Does this work for everyone?
Add a bulk API for revs open_revs
---
Key: COUCHDB-2310
URL: https://issues.apache.org/jira/browse/COUCHDB-2310
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: HTTP Interface
Reporter: Nolan Lawson
CouchDB replication is too slow.
And what makes it so slow is that it's just so unnecessarily chatty. During
replication, you have to do a separate GET for each individual document, in
order to get the full {{_revisions}} object for that document (using the
{{revs}} and {{open_revs}} parameters ndash; refer to [the TouchDB
writeup|https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithm]
or [Benoit's writeup|http://dataprotocols.org/couchdb-replication/] if you
need a refresher).
So for example, let's say you've got a database full of 10,000 documents, and
you replicate using a batch size of 500 (batch sizes are configurable in
PouchDB). The conversation for a single batch basically looks like this:
{code}
- REPLICATOR: gimme 500 changes since seq X (1 GET request)
- SOURCE: okay
- REPLICATOR: gimme the _revs_diff for these 500 docs/_revs (1 POST request)
- SOURCE: okay
- repeat 500 times:
- REPLICATOR: gimme the _revisions for doc n with _revs [...] (1 GET
request)
- SOURCE: okay
- REPLICATOR: here's a _bulk_docs with 500 documents (1 POST request)
- TARGET: okay
{code}
See the problem here? That 500-loop, where we have to do a GET for each one
of 500 documents, is a lot of unnecessary back-and-forth, considering that
the replicator already knows what it needs before the loop starts. You can
parallelize, but if you assume a browser (e.g. for PouchDB), most browsers
only let you do ~8 simultaneous requests at once. Plus, there's latency and
HTTP headers to consider. So overall, it's not cool.
So why do we even need to do the separate requests? Shouldn't {{_all_docs}}
be good enough? Turns out it's not, because we need this special
{{_revisions}} object.
For example, consider a document {{'foo'}} with 10 revisions. You may compact
the database, in which case revisions {{1-x}} through {{9-x}} are no longer
retrievable. However, if you query using {{revs}} and {{open_revs}}, those
rev IDs are still available:
{code}
$ curl 'http://nolan.iriscouch.com/test/foo?revs=trueopen_revs=all'
{
_id: foo,
_rev: 10-c78e199ad5e996b240c9d6482907088e,
_revisions: {
start: 10,
ids: [
c78e199ad5e996b240c9d6482907088e,
f560283f1968a05046f0c38e468006bb,
0091198554171c632c27c8342ddec5af,
e0a023e2ea59db73f812ad773ea08b17,
65d7f8b8206a244035edd9f252f206ad,
069d1432a003c58bdd23f01ff80b718f,
d21f26bb604b7fe9eba03ce4562cf37b,
31d380f99a6e54875855e1c24469622d,
3b4791360024426eadafe31542a2c34b,
967a00dff5e02add41819138abb3284d
]
}
}
{code}
And in the replication algorithm, _this full \_revisions object is required_
at the point when you copy the document from one database to another, which
is accomplished with a POST to {{_bulk_docs}} using {{new_edits=false}}. If
you don't have the full {{_revisions}} object, CouchDB accepts the new
revision, but considers it to be a conflict. (The exception is with
generation-1 documents, since they have no history, so as it says in the
TouchDB writeup, you can safely just use {{_all_docs}} as an optimization for
such documents.)
And unfortunately, this {{_revision}} object is only available from the {{GET
[jira] [Commented] (COUCHDB-2310) Add a bulk API for revs open_revs
[
https://issues.apache.org/jira/browse/COUCHDB-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358633#comment-14358633
]
Jan Lehnardt commented on COUCHDB-2310:
---
[~dholth] According to
https://github.com/apache/couchdb-couch/pull/18#issuecomment-67550634 yes :)
Add a bulk API for revs open_revs
---
Key: COUCHDB-2310
URL: https://issues.apache.org/jira/browse/COUCHDB-2310
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: HTTP Interface
Reporter: Nolan Lawson
CouchDB replication is too slow.
And what makes it so slow is that it's just so unnecessarily chatty. During
replication, you have to do a separate GET for each individual document, in
order to get the full {{_revisions}} object for that document (using the
{{revs}} and {{open_revs}} parameters ndash; refer to [the TouchDB
writeup|https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithm]
or [Benoit's writeup|http://dataprotocols.org/couchdb-replication/] if you
need a refresher).
So for example, let's say you've got a database full of 10,000 documents, and
you replicate using a batch size of 500 (batch sizes are configurable in
PouchDB). The conversation for a single batch basically looks like this:
{code}
- REPLICATOR: gimme 500 changes since seq X (1 GET request)
- SOURCE: okay
- REPLICATOR: gimme the _revs_diff for these 500 docs/_revs (1 POST request)
- SOURCE: okay
- repeat 500 times:
- REPLICATOR: gimme the _revisions for doc n with _revs [...] (1 GET
request)
- SOURCE: okay
- REPLICATOR: here's a _bulk_docs with 500 documents (1 POST request)
- TARGET: okay
{code}
See the problem here? That 500-loop, where we have to do a GET for each one
of 500 documents, is a lot of unnecessary back-and-forth, considering that
the replicator already knows what it needs before the loop starts. You can
parallelize, but if you assume a browser (e.g. for PouchDB), most browsers
only let you do ~8 simultaneous requests at once. Plus, there's latency and
HTTP headers to consider. So overall, it's not cool.
So why do we even need to do the separate requests? Shouldn't {{_all_docs}}
be good enough? Turns out it's not, because we need this special
{{_revisions}} object.
For example, consider a document {{'foo'}} with 10 revisions. You may compact
the database, in which case revisions {{1-x}} through {{9-x}} are no longer
retrievable. However, if you query using {{revs}} and {{open_revs}}, those
rev IDs are still available:
{code}
$ curl 'http://nolan.iriscouch.com/test/foo?revs=trueopen_revs=all'
{
_id: foo,
_rev: 10-c78e199ad5e996b240c9d6482907088e,
_revisions: {
start: 10,
ids: [
c78e199ad5e996b240c9d6482907088e,
f560283f1968a05046f0c38e468006bb,
0091198554171c632c27c8342ddec5af,
e0a023e2ea59db73f812ad773ea08b17,
65d7f8b8206a244035edd9f252f206ad,
069d1432a003c58bdd23f01ff80b718f,
d21f26bb604b7fe9eba03ce4562cf37b,
31d380f99a6e54875855e1c24469622d,
3b4791360024426eadafe31542a2c34b,
967a00dff5e02add41819138abb3284d
]
}
}
{code}
And in the replication algorithm, _this full \_revisions object is required_
at the point when you copy the document from one database to another, which
is accomplished with a POST to {{_bulk_docs}} using {{new_edits=false}}. If
you don't have the full {{_revisions}} object, CouchDB accepts the new
revision, but considers it to be a conflict. (The exception is with
generation-1 documents, since they have no history, so as it says in the
TouchDB writeup, you can safely just use {{_all_docs}} as an optimization for
such documents.)
And unfortunately, this {{_revision}} object is only available from the {{GET
/:dbid/:docid}} endpoint. Trust me; I've tried the other APIs. You can't get
it anywhere else.
This is a huge problem, especially in PouchDB where we often have to deal
with CORS, meaning the number of HTTP requests is doubled. So for those 500
GETs, it's an extra 500 OPTIONs, which is just unacceptable.
Replication does not have to be slow. While we were experimenting with ways
of fetching documents in bulk, we tried a technique that just relied on using
{{_changes}} with {{include_docs=true}}
([|\#2472|https://github.com/pouchdb/pouchdb/pull/2472]). This pushed
conflicts into the target database, but on the upside, you can sync ~95k
documents from npm's skimdb repository to the browser in less than 20
minutes! (See [npm-browser.com|http://npm-browser.com] for a demo.)
What an amazing story we could tell about the beauty of CouchDB replication,
if only this trick actually worked!
My proposal is a simple one: just add the {{revs}} and {{open_revs}} options
to
[jira] [Commented] (COUCHDB-2310) Add a bulk API for revs open_revs
[
https://issues.apache.org/jira/browse/COUCHDB-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358558#comment-14358558
]
Jan Lehnardt commented on COUCHDB-2310:
---
[~kxepal] Like I said, we will make lives better for people who are on existing
versions who support _bulk_get. And it doesn’t cost us anything, it’s just an
alias.
Add a bulk API for revs open_revs
---
Key: COUCHDB-2310
URL: https://issues.apache.org/jira/browse/COUCHDB-2310
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: HTTP Interface
Reporter: Nolan Lawson
CouchDB replication is too slow.
And what makes it so slow is that it's just so unnecessarily chatty. During
replication, you have to do a separate GET for each individual document, in
order to get the full {{_revisions}} object for that document (using the
{{revs}} and {{open_revs}} parameters ndash; refer to [the TouchDB
writeup|https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithm]
or [Benoit's writeup|http://dataprotocols.org/couchdb-replication/] if you
need a refresher).
So for example, let's say you've got a database full of 10,000 documents, and
you replicate using a batch size of 500 (batch sizes are configurable in
PouchDB). The conversation for a single batch basically looks like this:
{code}
- REPLICATOR: gimme 500 changes since seq X (1 GET request)
- SOURCE: okay
- REPLICATOR: gimme the _revs_diff for these 500 docs/_revs (1 POST request)
- SOURCE: okay
- repeat 500 times:
- REPLICATOR: gimme the _revisions for doc n with _revs [...] (1 GET
request)
- SOURCE: okay
- REPLICATOR: here's a _bulk_docs with 500 documents (1 POST request)
- TARGET: okay
{code}
See the problem here? That 500-loop, where we have to do a GET for each one
of 500 documents, is a lot of unnecessary back-and-forth, considering that
the replicator already knows what it needs before the loop starts. You can
parallelize, but if you assume a browser (e.g. for PouchDB), most browsers
only let you do ~8 simultaneous requests at once. Plus, there's latency and
HTTP headers to consider. So overall, it's not cool.
So why do we even need to do the separate requests? Shouldn't {{_all_docs}}
be good enough? Turns out it's not, because we need this special
{{_revisions}} object.
For example, consider a document {{'foo'}} with 10 revisions. You may compact
the database, in which case revisions {{1-x}} through {{9-x}} are no longer
retrievable. However, if you query using {{revs}} and {{open_revs}}, those
rev IDs are still available:
{code}
$ curl 'http://nolan.iriscouch.com/test/foo?revs=trueopen_revs=all'
{
_id: foo,
_rev: 10-c78e199ad5e996b240c9d6482907088e,
_revisions: {
start: 10,
ids: [
c78e199ad5e996b240c9d6482907088e,
f560283f1968a05046f0c38e468006bb,
0091198554171c632c27c8342ddec5af,
e0a023e2ea59db73f812ad773ea08b17,
65d7f8b8206a244035edd9f252f206ad,
069d1432a003c58bdd23f01ff80b718f,
d21f26bb604b7fe9eba03ce4562cf37b,
31d380f99a6e54875855e1c24469622d,
3b4791360024426eadafe31542a2c34b,
967a00dff5e02add41819138abb3284d
]
}
}
{code}
And in the replication algorithm, _this full \_revisions object is required_
at the point when you copy the document from one database to another, which
is accomplished with a POST to {{_bulk_docs}} using {{new_edits=false}}. If
you don't have the full {{_revisions}} object, CouchDB accepts the new
revision, but considers it to be a conflict. (The exception is with
generation-1 documents, since they have no history, so as it says in the
TouchDB writeup, you can safely just use {{_all_docs}} as an optimization for
such documents.)
And unfortunately, this {{_revision}} object is only available from the {{GET
/:dbid/:docid}} endpoint. Trust me; I've tried the other APIs. You can't get
it anywhere else.
This is a huge problem, especially in PouchDB where we often have to deal
with CORS, meaning the number of HTTP requests is doubled. So for those 500
GETs, it's an extra 500 OPTIONs, which is just unacceptable.
Replication does not have to be slow. While we were experimenting with ways
of fetching documents in bulk, we tried a technique that just relied on using
{{_changes}} with {{include_docs=true}}
([|\#2472|https://github.com/pouchdb/pouchdb/pull/2472]). This pushed
conflicts into the target database, but on the upside, you can sync ~95k
documents from npm's skimdb repository to the browser in less than 20
minutes! (See [npm-browser.com|http://npm-browser.com] for a demo.)
What an amazing story we could tell about the beauty of CouchDB replication,
if only this trick actually worked!
My proposal is a simple
[jira] [Commented] (COUCHDB-2635) Replace hardcoded list of systems dbs with registration approach
[ https://issues.apache.org/jira/browse/COUCHDB-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355624#comment-14355624 ] Jan Lehnardt commented on COUCHDB-2635: --- ah thanks, still don’t think this is a good idea. What problem is this solving? Replace hardcoded list of systems dbs with registration approach Key: COUCHDB-2635 URL: https://issues.apache.org/jira/browse/COUCHDB-2635 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Reporter: ILYA Assignee: ILYA To replace this https://github.com/apache/couchdb-couch/blob/master/include/couch_db.hrl#L43:L49 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2635) Replace hardcoded list of systems dbs with registration approach
[ https://issues.apache.org/jira/browse/COUCHDB-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355645#comment-14355645 ] Jan Lehnardt commented on COUCHDB-2635: --- I’m weary because we then have to make sure to properly abstract that in all places it becomes relevant. If we can ensure that, the plugin argument gets me over the fence, but I warn against the ensuing complexity. Would also love to hear from [~pjdavis1970] or [~chewbranca]. Replace hardcoded list of systems dbs with registration approach Key: COUCHDB-2635 URL: https://issues.apache.org/jira/browse/COUCHDB-2635 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Reporter: ILYA Assignee: ILYA To replace this https://github.com/apache/couchdb-couch/blob/master/include/couch_db.hrl#L43:L49 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (COUCHDB-2585) Make admin check extensible in global_change
[ https://issues.apache.org/jira/browse/COUCHDB-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt reopened COUCHDB-2585: --- Assignee: Jan Lehnardt (was: ILYA) Make admin check extensible in global_change Key: COUCHDB-2585 URL: https://issues.apache.org/jira/browse/COUCHDB-2585 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Components: Database Core Reporter: ILYA Assignee: Jan Lehnardt Currently there is no way to use different user here https://github.com/apache/couchdb-global-changes/blob/master/src/global_changes_httpd.erl#L46 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2585) Make admin check extensible in global_change
[ https://issues.apache.org/jira/browse/COUCHDB-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340238#comment-14340238 ] Jan Lehnardt commented on COUCHDB-2585: --- Can somebody explain what the `allowed_owner` setting does? And can we talk about naming, so far we don’t have the concept of “owners” in CouchDB. We have server admins, database admins and database members. How does an owner fit into this? Make admin check extensible in global_change Key: COUCHDB-2585 URL: https://issues.apache.org/jira/browse/COUCHDB-2585 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Components: Database Core Reporter: ILYA Assignee: ILYA Currently there is no way to use different user here https://github.com/apache/couchdb-global-changes/blob/master/src/global_changes_httpd.erl#L46 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2629) Hide internal config details from other applications
[ https://issues.apache.org/jira/browse/COUCHDB-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340700#comment-14340700 ] Jan Lehnardt commented on COUCHDB-2629: --- +1 Hide internal config details from other applications Key: COUCHDB-2629 URL: https://issues.apache.org/jira/browse/COUCHDB-2629 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Components: Database Core Reporter: Russell Branca Throughout the code base we do things like `config:get(couch_httpd_auth, authentication_db, _users)` which exposes internal details to other applications that don't need to be aware of the particulars. This duplicates code unnecessarily and causes problematic typos to go unnoticed like in COUCHDB-2628. One possible approach that I'm a fan of is to go into the application responsible for relevant config value, and add a function to hide the details. For instance, this is done in `cassim_metadata_cache:metadata_db`: https://github.com/apache/couchdb-cassim/blob/windsor-merge/src/cassim_metadata_cache.erl#L58-L59 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2620) Rename cassim db to _metadata
[ https://issues.apache.org/jira/browse/COUCHDB-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14333801#comment-14333801 ] Jan Lehnardt commented on COUCHDB-2620: --- To clarify: we discussed this in IRC and we are *proposing* a rename. We can’t obviously decide anything on IRC :) Rename cassim db to _metadata - Key: COUCHDB-2620 URL: https://issues.apache.org/jira/browse/COUCHDB-2620 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Components: Database Core Reporter: Alexander Shorin After IRC conversation with [~janl] and [~chewbranca] we decided to rename cassim database into something more specific and concrete. This database currently holds only _security of all cluster databases, but is designed for more. In future, we could use it to handle config-per-database feature and etc. The _metadata name reflects nicely such propose. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2620) Rename cassim db to _metadata
[ https://issues.apache.org/jira/browse/COUCHDB-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14333819#comment-14333819 ] Jan Lehnardt commented on COUCHDB-2620: --- thanks :) Rename cassim db to _metadata - Key: COUCHDB-2620 URL: https://issues.apache.org/jira/browse/COUCHDB-2620 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Components: Database Core Reporter: Alexander Shorin After IRC conversation with [~janl] and [~chewbranca] we -decided- are proposing to rename cassim database into something more specific and concrete. This database currently holds only _security of all cluster databases, but is designed for more. In future, we could use it to handle config-per-database feature and etc. The _metadata name reflects nicely such propose. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2621) Hide _metadata database from the world by default
[ https://issues.apache.org/jira/browse/COUCHDB-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14333812#comment-14333812 ] Jan Lehnardt commented on COUCHDB-2621: --- again clarifying that the “decision” was just a quorum among the IRC participants, not a project-level decision. this all is just for your all consideration :) A few clarifications: - the lack of validation logic means that unsuspecting end-users could write documents to the database that screw up database internals - replicating of metadata is definitely interesting and we could look at that in the future, but this isn’t a blocker for 2.0 - I’d suggest /_metadata returns 404, e.g. we just don’t hook it up to chttpd. - in the future, we might want to add a config option that hooks up the database to chttpd, so expert users can do with it what they need to do. This is also not blocking for 2.0. Hide _metadata database from the world by default - Key: COUCHDB-2621 URL: https://issues.apache.org/jira/browse/COUCHDB-2621 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Components: Database Core Reporter: Alexander Shorin After IRC conversation with [~janl] and [~chewbranca] we decided to hide _metadata database from the world by blocking all the access to it via HTTP API. The motivation is the following: - it lacks of validation logic, but adding such may hurt overall performance a more; - it creates a confusion about database security management: should users use /db/_security endpoint of /_metadata/db@security one? - technically, it allows to replicate databases metadata across the cluster, but this use case is a dark room of black cats; - there is something about quorum thing, but I didn't get that completely (: The proposal is to add chttpd handler which returns a HTTP error (403 Forbidden?) on /_metadata request and only allows to access to it from HTTP after setting couchdb/expose_metadata_database with value true in config file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2310) Add a bulk API for revs open_revs
[
https://issues.apache.org/jira/browse/COUCHDB-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14333758#comment-14333758
]
Jan Lehnardt commented on COUCHDB-2310:
---
*nudge* :)
Add a bulk API for revs open_revs
---
Key: COUCHDB-2310
URL: https://issues.apache.org/jira/browse/COUCHDB-2310
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: HTTP Interface
Reporter: Nolan Lawson
CouchDB replication is too slow.
And what makes it so slow is that it's just so unnecessarily chatty. During
replication, you have to do a separate GET for each individual document, in
order to get the full {{_revisions}} object for that document (using the
{{revs}} and {{open_revs}} parameters ndash; refer to [the TouchDB
writeup|https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithm]
or [Benoit's writeup|http://dataprotocols.org/couchdb-replication/] if you
need a refresher).
So for example, let's say you've got a database full of 10,000 documents, and
you replicate using a batch size of 500 (batch sizes are configurable in
PouchDB). The conversation for a single batch basically looks like this:
{code}
- REPLICATOR: gimme 500 changes since seq X (1 GET request)
- SOURCE: okay
- REPLICATOR: gimme the _revs_diff for these 500 docs/_revs (1 POST request)
- SOURCE: okay
- repeat 500 times:
- REPLICATOR: gimme the _revisions for doc n with _revs [...] (1 GET
request)
- SOURCE: okay
- REPLICATOR: here's a _bulk_docs with 500 documents (1 POST request)
- TARGET: okay
{code}
See the problem here? That 500-loop, where we have to do a GET for each one
of 500 documents, is a lot of unnecessary back-and-forth, considering that
the replicator already knows what it needs before the loop starts. You can
parallelize, but if you assume a browser (e.g. for PouchDB), most browsers
only let you do ~8 simultaneous requests at once. Plus, there's latency and
HTTP headers to consider. So overall, it's not cool.
So why do we even need to do the separate requests? Shouldn't {{_all_docs}}
be good enough? Turns out it's not, because we need this special
{{_revisions}} object.
For example, consider a document {{'foo'}} with 10 revisions. You may compact
the database, in which case revisions {{1-x}} through {{9-x}} are no longer
retrievable. However, if you query using {{revs}} and {{open_revs}}, those
rev IDs are still available:
{code}
$ curl 'http://nolan.iriscouch.com/test/foo?revs=trueopen_revs=all'
{
_id: foo,
_rev: 10-c78e199ad5e996b240c9d6482907088e,
_revisions: {
start: 10,
ids: [
c78e199ad5e996b240c9d6482907088e,
f560283f1968a05046f0c38e468006bb,
0091198554171c632c27c8342ddec5af,
e0a023e2ea59db73f812ad773ea08b17,
65d7f8b8206a244035edd9f252f206ad,
069d1432a003c58bdd23f01ff80b718f,
d21f26bb604b7fe9eba03ce4562cf37b,
31d380f99a6e54875855e1c24469622d,
3b4791360024426eadafe31542a2c34b,
967a00dff5e02add41819138abb3284d
]
}
}
{code}
And in the replication algorithm, _this full \_revisions object is required_
at the point when you copy the document from one database to another, which
is accomplished with a POST to {{_bulk_docs}} using {{new_edits=false}}. If
you don't have the full {{_revisions}} object, CouchDB accepts the new
revision, but considers it to be a conflict. (The exception is with
generation-1 documents, since they have no history, so as it says in the
TouchDB writeup, you can safely just use {{_all_docs}} as an optimization for
such documents.)
And unfortunately, this {{_revision}} object is only available from the {{GET
/:dbid/:docid}} endpoint. Trust me; I've tried the other APIs. You can't get
it anywhere else.
This is a huge problem, especially in PouchDB where we often have to deal
with CORS, meaning the number of HTTP requests is doubled. So for those 500
GETs, it's an extra 500 OPTIONs, which is just unacceptable.
Replication does not have to be slow. While we were experimenting with ways
of fetching documents in bulk, we tried a technique that just relied on using
{{_changes}} with {{include_docs=true}}
([|\#2472|https://github.com/pouchdb/pouchdb/pull/2472]). This pushed
conflicts into the target database, but on the upside, you can sync ~95k
documents from npm's skimdb repository to the browser in less than 20
minutes! (See [npm-browser.com|http://npm-browser.com] for a demo.)
What an amazing story we could tell about the beauty of CouchDB replication,
if only this trick actually worked!
My proposal is a simple one: just add the {{revs}} and {{open_revs}} options
to {{_all_docs}}. Presumably this would be aligned with {{keys}}, so similar
to how {{keys}} takes an
[jira] [Commented] (COUCHDB-2191) Please consider including couchperuser in core
[
https://issues.apache.org/jira/browse/COUCHDB-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328981#comment-14328981
]
Jan Lehnardt commented on COUCHDB-2191:
---
I’d be in favour of this.
Please consider including couchperuser in core
--
Key: COUCHDB-2191
URL: https://issues.apache.org/jira/browse/COUCHDB-2191
Project: CouchDB
Issue Type: Improvement
Security Level: public(Regular issues)
Reporter: Nolan Lawson
I would love to be able to use CouchDB as the exclusive backend for all my
webapps. The {{_users}} database with the automatic password salting/hashing
and session cookies is brilliant, and saves a lot of developer effort while
still ensuring I don't shoot myself in the foot trying to implement password
security.
However, without creating a database per user, it's impossible to silo user
data in any way other than through {{validate_doc_update}} - i.e. every user
can see everybody else's data, but they can only write to theirs. This use
case does exist (e.g. Twitter), but it's much less common than the case where
users can only read/write their own data.
The plugin ecosystem is great and all, and I totally understand not wanting
to include the kitchen sink in Couch core, but I strongly feel
[couchperuser|https://github.com/etrepum/couchperuser] (or something like it)
should be a checkbox I can tick in the Couch config, rather than a plugin I
have to install manually. It's just too common of a use case in typical
webapps.
Some background: this was prompted by a [discussion in
PouchDB|https://github.com/daleharvey/pouchdb/issues/1575]; Dale has written
a fine solution in
[couch-persona|https://github.com/daleharvey/couch-persona], but I really
think the why Pouch/Couch? story would be more compelling if you could do
it in pure Couch without an extra server process.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (COUCHDB-2588) dereferencing type-punned pointer will break strict-aliasing rule
[
https://issues.apache.org/jira/browse/COUCHDB-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Lehnardt updated COUCHDB-2588:
--
Description:
Happens on FreeBSD 9:
{code}
# gcc -v
Using built-in specs.
Target: i386-undermydesk-freebsd
Configured with: FreeBSD/i386 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]
== b64url (compile)
Compiled src/b64url.erl
Compiling /root/couchdb/src/b64url/c_src/b64url.c
cc1: warnings being treated as errors
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_encode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:424: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_decode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:579: warning: dereferencing type-punned
pointer will break strict-aliasing rules
ERROR: compile failed while processing /root/couchdb/src/b64url: rebar_abort
*** [couch] Error code 1
== khash (compile)
Compiled src/khash.erl
Compiling c_src/hash.c
Compiling /root/couchdb/src/khash/c_src/khash.c
cc1: warnings being treated as errors
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_to_list':
/root/couchdb/src/khash/c_src/khash.c:232: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_clear':
/root/couchdb/src/khash/c_src/khash.c:264: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_lookup':
/root/couchdb/src/khash/c_src/khash.c:303: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_get':
/root/couchdb/src/khash/c_src/khash.c:337: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_put':
/root/couchdb/src/khash/c_src/khash.c:369: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_del':
/root/couchdb/src/khash/c_src/khash.c:409: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_size':
/root/couchdb/src/khash/c_src/khash.c:441: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_iter':
/root/couchdb/src/khash/c_src/khash.c:465: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_iter_next':
/root/couchdb/src/khash/c_src/khash.c:514: warning: dereferencing type-punned
pointer will break strict-aliasing rules
ERROR: compile failed while processing /root/couchdb/src/khash: rebar_abort
*** [couch] Error code 1
{code}
was:
Happens on FreeBSD:
{code}
# gcc -v
Using built-in specs.
Target: i386-undermydesk-freebsd
Configured with: FreeBSD/i386 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]
== b64url (compile)
Compiled src/b64url.erl
Compiling /root/couchdb/src/b64url/c_src/b64url.c
cc1: warnings being treated as errors
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_encode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:424: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_decode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:579: warning: dereferencing type-punned
pointer will break strict-aliasing rules
ERROR: compile failed while processing /root/couchdb/src/b64url: rebar_abort
*** [couch] Error code 1
== khash (compile)
Compiled src/khash.erl
Compiling c_src/hash.c
Compiling /root/couchdb/src/khash/c_src/khash.c
cc1: warnings being treated as errors
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_to_list':
/root/couchdb/src/khash/c_src/khash.c:232: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_clear':
/root/couchdb/src/khash/c_src/khash.c:264: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_lookup':
/root/couchdb/src/khash/c_src/khash.c:303: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_get':
/root/couchdb/src/khash/c_src/khash.c:337: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_put':
/root/couchdb/src/khash/c_src/khash.c:369: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_del':
[jira] [Commented] (COUCHDB-2588) dereferencing type-punned pointer will break strict-aliasing rule
[
https://issues.apache.org/jira/browse/COUCHDB-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328136#comment-14328136
]
Jan Lehnardt commented on COUCHDB-2588:
---
For anyone looking at this, the sources are at
https://github.com/apache/couchdb-b64url and
https://github.com/apache/couchdb-khash
dereferencing type-punned pointer will break strict-aliasing rule
-
Key: COUCHDB-2588
URL: https://issues.apache.org/jira/browse/COUCHDB-2588
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: Database Core
Affects Versions: 2.0.0
Environment: FreeBSD 9
Reporter: Alexander Shorin
Priority: Blocker
Fix For: 2.0.0
Happens on FreeBSD 9:
{code}
# gcc -v
Using built-in specs.
Target: i386-undermydesk-freebsd
Configured with: FreeBSD/i386 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]
== b64url (compile)
Compiled src/b64url.erl
Compiling /root/couchdb/src/b64url/c_src/b64url.c
cc1: warnings being treated as errors
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_encode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:424: warning: dereferencing
type-punned pointer will break strict-aliasing rules
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_decode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:579: warning: dereferencing
type-punned pointer will break strict-aliasing rules
ERROR: compile failed while processing /root/couchdb/src/b64url: rebar_abort
*** [couch] Error code 1
== khash (compile)
Compiled src/khash.erl
Compiling c_src/hash.c
Compiling /root/couchdb/src/khash/c_src/khash.c
cc1: warnings being treated as errors
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_to_list':
/root/couchdb/src/khash/c_src/khash.c:232: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_clear':
/root/couchdb/src/khash/c_src/khash.c:264: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_lookup':
/root/couchdb/src/khash/c_src/khash.c:303: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_get':
/root/couchdb/src/khash/c_src/khash.c:337: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_put':
/root/couchdb/src/khash/c_src/khash.c:369: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_del':
/root/couchdb/src/khash/c_src/khash.c:409: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_size':
/root/couchdb/src/khash/c_src/khash.c:441: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_iter':
/root/couchdb/src/khash/c_src/khash.c:465: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_iter_next':
/root/couchdb/src/khash/c_src/khash.c:514: warning: dereferencing type-punned
pointer will break strict-aliasing rules
ERROR: compile failed while processing /root/couchdb/src/khash: rebar_abort
*** [couch] Error code 1
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (COUCHDB-2588) dereferencing type-punned pointer will break strict-aliasing rule
[
https://issues.apache.org/jira/browse/COUCHDB-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328133#comment-14328133
]
Jan Lehnardt commented on COUCHDB-2588:
---
clang based FreeBSD 10+ not affected.
dereferencing type-punned pointer will break strict-aliasing rule
-
Key: COUCHDB-2588
URL: https://issues.apache.org/jira/browse/COUCHDB-2588
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: Database Core
Affects Versions: 2.0.0
Environment: FreeBSD 9
Reporter: Alexander Shorin
Priority: Blocker
Fix For: 2.0.0
Happens on FreeBSD 9:
{code}
# gcc -v
Using built-in specs.
Target: i386-undermydesk-freebsd
Configured with: FreeBSD/i386 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]
== b64url (compile)
Compiled src/b64url.erl
Compiling /root/couchdb/src/b64url/c_src/b64url.c
cc1: warnings being treated as errors
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_encode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:424: warning: dereferencing
type-punned pointer will break strict-aliasing rules
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_decode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:579: warning: dereferencing
type-punned pointer will break strict-aliasing rules
ERROR: compile failed while processing /root/couchdb/src/b64url: rebar_abort
*** [couch] Error code 1
== khash (compile)
Compiled src/khash.erl
Compiling c_src/hash.c
Compiling /root/couchdb/src/khash/c_src/khash.c
cc1: warnings being treated as errors
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_to_list':
/root/couchdb/src/khash/c_src/khash.c:232: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_clear':
/root/couchdb/src/khash/c_src/khash.c:264: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_lookup':
/root/couchdb/src/khash/c_src/khash.c:303: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_get':
/root/couchdb/src/khash/c_src/khash.c:337: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_put':
/root/couchdb/src/khash/c_src/khash.c:369: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_del':
/root/couchdb/src/khash/c_src/khash.c:409: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_size':
/root/couchdb/src/khash/c_src/khash.c:441: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_iter':
/root/couchdb/src/khash/c_src/khash.c:465: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_iter_next':
/root/couchdb/src/khash/c_src/khash.c:514: warning: dereferencing type-punned
pointer will break strict-aliasing rules
ERROR: compile failed while processing /root/couchdb/src/khash: rebar_abort
*** [couch] Error code 1
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (COUCHDB-2588) dereferencing type-punned pointer will break strict-aliasing rule
[
https://issues.apache.org/jira/browse/COUCHDB-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Lehnardt updated COUCHDB-2588:
--
Environment: FreeBSD 9
dereferencing type-punned pointer will break strict-aliasing rule
-
Key: COUCHDB-2588
URL: https://issues.apache.org/jira/browse/COUCHDB-2588
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: Database Core
Affects Versions: 2.0.0
Environment: FreeBSD 9
Reporter: Alexander Shorin
Priority: Blocker
Fix For: 2.0.0
Happens on FreeBSD 9:
{code}
# gcc -v
Using built-in specs.
Target: i386-undermydesk-freebsd
Configured with: FreeBSD/i386 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]
== b64url (compile)
Compiled src/b64url.erl
Compiling /root/couchdb/src/b64url/c_src/b64url.c
cc1: warnings being treated as errors
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_encode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:424: warning: dereferencing
type-punned pointer will break strict-aliasing rules
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_decode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:579: warning: dereferencing
type-punned pointer will break strict-aliasing rules
ERROR: compile failed while processing /root/couchdb/src/b64url: rebar_abort
*** [couch] Error code 1
== khash (compile)
Compiled src/khash.erl
Compiling c_src/hash.c
Compiling /root/couchdb/src/khash/c_src/khash.c
cc1: warnings being treated as errors
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_to_list':
/root/couchdb/src/khash/c_src/khash.c:232: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_clear':
/root/couchdb/src/khash/c_src/khash.c:264: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_lookup':
/root/couchdb/src/khash/c_src/khash.c:303: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_get':
/root/couchdb/src/khash/c_src/khash.c:337: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_put':
/root/couchdb/src/khash/c_src/khash.c:369: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_del':
/root/couchdb/src/khash/c_src/khash.c:409: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_size':
/root/couchdb/src/khash/c_src/khash.c:441: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_iter':
/root/couchdb/src/khash/c_src/khash.c:465: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_iter_next':
/root/couchdb/src/khash/c_src/khash.c:514: warning: dereferencing type-punned
pointer will break strict-aliasing rules
ERROR: compile failed while processing /root/couchdb/src/khash: rebar_abort
*** [couch] Error code 1
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (COUCHDB-2597) Release: make targets
[ https://issues.apache.org/jira/browse/COUCHDB-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328172#comment-14328172 ] Jan Lehnardt commented on COUCHDB-2597: --- This will all be covered in https://github.com/apache/couchdb/pull/302 Docs building is already part of this. `make dist` is now `make release` and still work in progress Release: make targets - Key: COUCHDB-2597 URL: https://issues.apache.org/jira/browse/COUCHDB-2597 Project: CouchDB Issue Type: Task Security Level: public(Regular issues) Components: Build System Reporter: Robert Kowalski Assignee: Jan Lehnardt Priority: Blocker make dist and make docs are still missing for 2.0. Additionaly the Thanks.in file that was generated by autotools when preparing a release must be readded -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2602) delayed deletion as safety net
[ https://issues.apache.org/jira/browse/COUCHDB-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328171#comment-14328171 ] Jan Lehnardt commented on COUCHDB-2602: --- it’s less a timer and more a “soft delete” where we move databases out of the way, but don’t delete them right away. delayed deletion as safety net -- Key: COUCHDB-2602 URL: https://issues.apache.org/jira/browse/COUCHDB-2602 Project: CouchDB Issue Type: Task Security Level: public(Regular issues) Components: Database Core Affects Versions: 2.1 Reporter: Robert Kowalski Delay deletion as a safety net for users. Make the delay configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2595) Packaging/Release: tarball
[ https://issues.apache.org/jira/browse/COUCHDB-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328176#comment-14328176 ] Jan Lehnardt commented on COUCHDB-2595: --- This will be part of https://github.com/apache/couchdb/pull/302 Packaging/Release: tarball -- Key: COUCHDB-2595 URL: https://issues.apache.org/jira/browse/COUCHDB-2595 Project: CouchDB Issue Type: Task Security Level: public(Regular issues) Components: Build System Reporter: Robert Kowalski Assignee: Jan Lehnardt Priority: Blocker We have to create a tarball for the release of 2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2588) dereferencing type-punned pointer will break strict-aliasing rule
[
https://issues.apache.org/jira/browse/COUCHDB-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328224#comment-14328224
]
Jan Lehnardt commented on COUCHDB-2588:
---
Trusted pointer wrangler Justin McCormack suggests to compile with
-Wno-strict-aliasing
https://twitter.com/justincormack/status/568537462676402176
dereferencing type-punned pointer will break strict-aliasing rule
-
Key: COUCHDB-2588
URL: https://issues.apache.org/jira/browse/COUCHDB-2588
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: Database Core
Affects Versions: 2.0.0
Environment: FreeBSD 9
Reporter: Alexander Shorin
Priority: Blocker
Fix For: 2.0.0
Happens on FreeBSD 9:
{code}
# gcc -v
Using built-in specs.
Target: i386-undermydesk-freebsd
Configured with: FreeBSD/i386 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]
== b64url (compile)
Compiled src/b64url.erl
Compiling /root/couchdb/src/b64url/c_src/b64url.c
cc1: warnings being treated as errors
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_encode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:424: warning: dereferencing
type-punned pointer will break strict-aliasing rules
/root/couchdb/src/b64url/c_src/b64url.c: In function 'b64url_decode_cont':
/root/couchdb/src/b64url/c_src/b64url.c:579: warning: dereferencing
type-punned pointer will break strict-aliasing rules
ERROR: compile failed while processing /root/couchdb/src/b64url: rebar_abort
*** [couch] Error code 1
== khash (compile)
Compiled src/khash.erl
Compiling c_src/hash.c
Compiling /root/couchdb/src/khash/c_src/khash.c
cc1: warnings being treated as errors
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_to_list':
/root/couchdb/src/khash/c_src/khash.c:232: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_clear':
/root/couchdb/src/khash/c_src/khash.c:264: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_lookup':
/root/couchdb/src/khash/c_src/khash.c:303: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_get':
/root/couchdb/src/khash/c_src/khash.c:337: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_put':
/root/couchdb/src/khash/c_src/khash.c:369: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_del':
/root/couchdb/src/khash/c_src/khash.c:409: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_size':
/root/couchdb/src/khash/c_src/khash.c:441: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_iter':
/root/couchdb/src/khash/c_src/khash.c:465: warning: dereferencing type-punned
pointer will break strict-aliasing rules
/root/couchdb/src/khash/c_src/khash.c: In function 'khash_iter_next':
/root/couchdb/src/khash/c_src/khash.c:514: warning: dereferencing type-punned
pointer will break strict-aliasing rules
ERROR: compile failed while processing /root/couchdb/src/khash: rebar_abort
*** [couch] Error code 1
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (COUCHDB-2310) Add a bulk API for revs open_revs
[
https://issues.apache.org/jira/browse/COUCHDB-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293463#comment-14293463
]
Jan Lehnardt commented on COUCHDB-2310:
---
Cool, thanks, no worries :) — Also feel free to post incomplete stuff, happy to
pick it up.
Add a bulk API for revs open_revs
---
Key: COUCHDB-2310
URL: https://issues.apache.org/jira/browse/COUCHDB-2310
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: HTTP Interface
Reporter: Nolan Lawson
CouchDB replication is too slow.
And what makes it so slow is that it's just so unnecessarily chatty. During
replication, you have to do a separate GET for each individual document, in
order to get the full {{_revisions}} object for that document (using the
{{revs}} and {{open_revs}} parameters ndash; refer to [the TouchDB
writeup|https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithm]
or [Benoit's writeup|http://dataprotocols.org/couchdb-replication/] if you
need a refresher).
So for example, let's say you've got a database full of 10,000 documents, and
you replicate using a batch size of 500 (batch sizes are configurable in
PouchDB). The conversation for a single batch basically looks like this:
{code}
- REPLICATOR: gimme 500 changes since seq X (1 GET request)
- SOURCE: okay
- REPLICATOR: gimme the _revs_diff for these 500 docs/_revs (1 POST request)
- SOURCE: okay
- repeat 500 times:
- REPLICATOR: gimme the _revisions for doc n with _revs [...] (1 GET
request)
- SOURCE: okay
- REPLICATOR: here's a _bulk_docs with 500 documents (1 POST request)
- TARGET: okay
{code}
See the problem here? That 500-loop, where we have to do a GET for each one
of 500 documents, is a lot of unnecessary back-and-forth, considering that
the replicator already knows what it needs before the loop starts. You can
parallelize, but if you assume a browser (e.g. for PouchDB), most browsers
only let you do ~8 simultaneous requests at once. Plus, there's latency and
HTTP headers to consider. So overall, it's not cool.
So why do we even need to do the separate requests? Shouldn't {{_all_docs}}
be good enough? Turns out it's not, because we need this special
{{_revisions}} object.
For example, consider a document {{'foo'}} with 10 revisions. You may compact
the database, in which case revisions {{1-x}} through {{9-x}} are no longer
retrievable. However, if you query using {{revs}} and {{open_revs}}, those
rev IDs are still available:
{code}
$ curl 'http://nolan.iriscouch.com/test/foo?revs=trueopen_revs=all'
{
_id: foo,
_rev: 10-c78e199ad5e996b240c9d6482907088e,
_revisions: {
start: 10,
ids: [
c78e199ad5e996b240c9d6482907088e,
f560283f1968a05046f0c38e468006bb,
0091198554171c632c27c8342ddec5af,
e0a023e2ea59db73f812ad773ea08b17,
65d7f8b8206a244035edd9f252f206ad,
069d1432a003c58bdd23f01ff80b718f,
d21f26bb604b7fe9eba03ce4562cf37b,
31d380f99a6e54875855e1c24469622d,
3b4791360024426eadafe31542a2c34b,
967a00dff5e02add41819138abb3284d
]
}
}
{code}
And in the replication algorithm, _this full \_revisions object is required_
at the point when you copy the document from one database to another, which
is accomplished with a POST to {{_bulk_docs}} using {{new_edits=false}}. If
you don't have the full {{_revisions}} object, CouchDB accepts the new
revision, but considers it to be a conflict. (The exception is with
generation-1 documents, since they have no history, so as it says in the
TouchDB writeup, you can safely just use {{_all_docs}} as an optimization for
such documents.)
And unfortunately, this {{_revision}} object is only available from the {{GET
/:dbid/:docid}} endpoint. Trust me; I've tried the other APIs. You can't get
it anywhere else.
This is a huge problem, especially in PouchDB where we often have to deal
with CORS, meaning the number of HTTP requests is doubled. So for those 500
GETs, it's an extra 500 OPTIONs, which is just unacceptable.
Replication does not have to be slow. While we were experimenting with ways
of fetching documents in bulk, we tried a technique that just relied on using
{{_changes}} with {{include_docs=true}}
([|\#2472|https://github.com/pouchdb/pouchdb/pull/2472]). This pushed
conflicts into the target database, but on the upside, you can sync ~95k
documents from npm's skimdb repository to the browser in less than 20
minutes! (See [npm-browser.com|http://npm-browser.com] for a demo.)
What an amazing story we could tell about the beauty of CouchDB replication,
if only this trick actually worked!
My proposal is a simple one: just add the {{revs}} and {{open_revs}} options
to {{_all_docs}}.
[jira] [Commented] (COUCHDB-2310) Add a bulk API for revs open_revs
[
https://issues.apache.org/jira/browse/COUCHDB-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293450#comment-14293450
]
Jan Lehnardt commented on COUCHDB-2310:
---
[~benoitc] any news? :)
Add a bulk API for revs open_revs
---
Key: COUCHDB-2310
URL: https://issues.apache.org/jira/browse/COUCHDB-2310
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: HTTP Interface
Reporter: Nolan Lawson
CouchDB replication is too slow.
And what makes it so slow is that it's just so unnecessarily chatty. During
replication, you have to do a separate GET for each individual document, in
order to get the full {{_revisions}} object for that document (using the
{{revs}} and {{open_revs}} parameters ndash; refer to [the TouchDB
writeup|https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithm]
or [Benoit's writeup|http://dataprotocols.org/couchdb-replication/] if you
need a refresher).
So for example, let's say you've got a database full of 10,000 documents, and
you replicate using a batch size of 500 (batch sizes are configurable in
PouchDB). The conversation for a single batch basically looks like this:
{code}
- REPLICATOR: gimme 500 changes since seq X (1 GET request)
- SOURCE: okay
- REPLICATOR: gimme the _revs_diff for these 500 docs/_revs (1 POST request)
- SOURCE: okay
- repeat 500 times:
- REPLICATOR: gimme the _revisions for doc n with _revs [...] (1 GET
request)
- SOURCE: okay
- REPLICATOR: here's a _bulk_docs with 500 documents (1 POST request)
- TARGET: okay
{code}
See the problem here? That 500-loop, where we have to do a GET for each one
of 500 documents, is a lot of unnecessary back-and-forth, considering that
the replicator already knows what it needs before the loop starts. You can
parallelize, but if you assume a browser (e.g. for PouchDB), most browsers
only let you do ~8 simultaneous requests at once. Plus, there's latency and
HTTP headers to consider. So overall, it's not cool.
So why do we even need to do the separate requests? Shouldn't {{_all_docs}}
be good enough? Turns out it's not, because we need this special
{{_revisions}} object.
For example, consider a document {{'foo'}} with 10 revisions. You may compact
the database, in which case revisions {{1-x}} through {{9-x}} are no longer
retrievable. However, if you query using {{revs}} and {{open_revs}}, those
rev IDs are still available:
{code}
$ curl 'http://nolan.iriscouch.com/test/foo?revs=trueopen_revs=all'
{
_id: foo,
_rev: 10-c78e199ad5e996b240c9d6482907088e,
_revisions: {
start: 10,
ids: [
c78e199ad5e996b240c9d6482907088e,
f560283f1968a05046f0c38e468006bb,
0091198554171c632c27c8342ddec5af,
e0a023e2ea59db73f812ad773ea08b17,
65d7f8b8206a244035edd9f252f206ad,
069d1432a003c58bdd23f01ff80b718f,
d21f26bb604b7fe9eba03ce4562cf37b,
31d380f99a6e54875855e1c24469622d,
3b4791360024426eadafe31542a2c34b,
967a00dff5e02add41819138abb3284d
]
}
}
{code}
And in the replication algorithm, _this full \_revisions object is required_
at the point when you copy the document from one database to another, which
is accomplished with a POST to {{_bulk_docs}} using {{new_edits=false}}. If
you don't have the full {{_revisions}} object, CouchDB accepts the new
revision, but considers it to be a conflict. (The exception is with
generation-1 documents, since they have no history, so as it says in the
TouchDB writeup, you can safely just use {{_all_docs}} as an optimization for
such documents.)
And unfortunately, this {{_revision}} object is only available from the {{GET
/:dbid/:docid}} endpoint. Trust me; I've tried the other APIs. You can't get
it anywhere else.
This is a huge problem, especially in PouchDB where we often have to deal
with CORS, meaning the number of HTTP requests is doubled. So for those 500
GETs, it's an extra 500 OPTIONs, which is just unacceptable.
Replication does not have to be slow. While we were experimenting with ways
of fetching documents in bulk, we tried a technique that just relied on using
{{_changes}} with {{include_docs=true}}
([|\#2472|https://github.com/pouchdb/pouchdb/pull/2472]). This pushed
conflicts into the target database, but on the upside, you can sync ~95k
documents from npm's skimdb repository to the browser in less than 20
minutes! (See [npm-browser.com|http://npm-browser.com] for a demo.)
What an amazing story we could tell about the beauty of CouchDB replication,
if only this trick actually worked!
My proposal is a simple one: just add the {{revs}} and {{open_revs}} options
to {{_all_docs}}. Presumably this would be aligned with {{keys}}, so similar
to how
[jira] [Commented] (COUCHDB-2541) couchdbctl
[
https://issues.apache.org/jira/browse/COUCHDB-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282636#comment-14282636
]
Jan Lehnardt commented on COUCHDB-2541:
---
Are you proposing we should have such a tool? Or do you have one?
There is https://github.com/dscape/futoncli which aims to do something similar
and afaik @dscape is up for donating it to the ASF.
Maybe we can roll it into the `couchdb` script (by way of clever wrapping, I
don’t want to do all this in pure shell :), so it is all one thing. (maybe this
is a bad idea, dunno :)
Either way though, I really like this :)
couchdbctl
--
Key: COUCHDB-2541
URL: https://issues.apache.org/jira/browse/COUCHDB-2541
Project: CouchDB
Issue Type: New Feature
Security Level: public(Regular issues)
Reporter: Alexander Shorin
couchdbctl is CouchDB control service utility that aims to help users manage
and inspect their CouchDB instance, keeping it well and health.
Few ideas what it may do:
- register users with easy:
{code}
$ couchdbctl users add Jan
Password:
{code}
hides routines to create /_users/org.couchdb.users:Jan document with all
required fields;
- run service operations:
{code}
$ couchdbctl db compact
{code}
- connect to node and provide shell:
{code}
$ couchdbctl attach
{code}
- run security audit as like as
[audit-couchdb|https://github.com/iriscouch/audit_couchdb] or
[couchdb-auditor|https://github.com/kxepal/python-couchdb-auditor] does;
{code}
$ couchdbctl audit
{code}
- see erltop;
{code}
$ couchdbctl top
{code}
- monitor stats and active tasks in realtime;
- run replications with single command without worry about JSON and required
fields:
{code}
$ couchdbctl replicate foo http://example.com/bar --continuous --create-target
{code}
- check cluster health, add/remove nodes to it;
- explain errors in logs:
{code}
$ couchdbctl errors
[error] [0.125.0] {error_report,0.30.0,
{0.125.0,crash_report,
[[{initial_call,{couch_file,init,['Argument__1']}},
{pid,0.125.0},
{registered_name,[]},
{error_info,
{exit,
{{badmatch,{error,eacces}},
[{couch_file,init,1,
...
eacess error - insufficient or invalid permissions, please verify that
couchdb user has all r permissions to the following paths:
/var/lib/couchdb - read+write
/var/log/couchdb - read+write
/etc/couchdb - read+write
/usr/lib64/couchdb/erlang/lib/couch-2.0.0/priv/lib - read
{code}
And so on and so forth. It's easy to find how this utility may improve users
experience with CouchDB by simplifying common maintaining routes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (COUCHDB-2541) couchdbctl
[
https://issues.apache.org/jira/browse/COUCHDB-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282741#comment-14282741
]
Jan Lehnardt commented on COUCHDB-2541:
---
`couchdb` script is one the made in bash? I don't feel that this will be a
simple in implementation, but indeed it will be simple in realization (sorry
Windows).
What I meant is that the cli name could maybe `couchdb` instead of
`couchdbctl`. I wouldn’t want to implement all the functionality in shell (I
think/hope we use plain sh, not bash), but pipe all commands to a binary where
such things are easier implemented. E.g. `couchdb start` runs the shell routine
that starts CouchDB, `couchdb db compact` would not match any shell routine, so
we pass this onto a sub-program that does the job.
couchdbctl
--
Key: COUCHDB-2541
URL: https://issues.apache.org/jira/browse/COUCHDB-2541
Project: CouchDB
Issue Type: New Feature
Security Level: public(Regular issues)
Reporter: Alexander Shorin
couchdbctl is CouchDB control service utility that aims to help users manage
and inspect their CouchDB instance, keeping it well and health.
Few ideas what it may do:
- register users with easy:
{code}
$ couchdbctl users add Jan
Password:
{code}
hides routines to create /_users/org.couchdb.users:Jan document with all
required fields;
- run service operations:
{code}
$ couchdbctl db compact
{code}
- connect to node and provide shell:
{code}
$ couchdbctl attach
{code}
- run security audit as like as
[audit-couchdb|https://github.com/iriscouch/audit_couchdb] or
[couchdb-auditor|https://github.com/kxepal/python-couchdb-auditor] does;
{code}
$ couchdbctl audit
{code}
- see erltop;
{code}
$ couchdbctl top
{code}
- monitor stats and active tasks in realtime;
- run replications with single command without worry about JSON and required
fields:
{code}
$ couchdbctl replicate foo http://example.com/bar --continuous --create-target
{code}
- check cluster health, add/remove nodes to it;
- explain errors in logs:
{code}
$ couchdbctl errors
[error] [0.125.0] {error_report,0.30.0,
{0.125.0,crash_report,
[[{initial_call,{couch_file,init,['Argument__1']}},
{pid,0.125.0},
{registered_name,[]},
{error_info,
{exit,
{{badmatch,{error,eacces}},
[{couch_file,init,1,
...
eacess error - insufficient or invalid permissions, please verify that
couchdb user has all r permissions to the following paths:
/var/lib/couchdb - read+write
/var/log/couchdb - read+write
/etc/couchdb - read+write
/usr/lib64/couchdb/erlang/lib/couch-2.0.0/priv/lib - read
{code}
And so on and so forth. It's easy to find how this utility may improve users
experience with CouchDB by simplifying common maintaining routes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (COUCHDB-2310) Add a bulk API for revs open_revs
[
https://issues.apache.org/jira/browse/COUCHDB-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14264487#comment-14264487
]
Jan Lehnardt commented on COUCHDB-2310:
---
[~benoitc] Just to make sure I read this correctly: You have a 2.x compatible
_bulk_get implementation? That’d be awesome!
I’m sure we can sort out the rest.
Add a bulk API for revs open_revs
---
Key: COUCHDB-2310
URL: https://issues.apache.org/jira/browse/COUCHDB-2310
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: HTTP Interface
Reporter: Nolan Lawson
CouchDB replication is too slow.
And what makes it so slow is that it's just so unnecessarily chatty. During
replication, you have to do a separate GET for each individual document, in
order to get the full {{_revisions}} object for that document (using the
{{revs}} and {{open_revs}} parameters ndash; refer to [the TouchDB
writeup|https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithm]
or [Benoit's writeup|http://dataprotocols.org/couchdb-replication/] if you
need a refresher).
So for example, let's say you've got a database full of 10,000 documents, and
you replicate using a batch size of 500 (batch sizes are configurable in
PouchDB). The conversation for a single batch basically looks like this:
{code}
- REPLICATOR: gimme 500 changes since seq X (1 GET request)
- SOURCE: okay
- REPLICATOR: gimme the _revs_diff for these 500 docs/_revs (1 POST request)
- SOURCE: okay
- repeat 500 times:
- REPLICATOR: gimme the _revisions for doc n with _revs [...] (1 GET
request)
- SOURCE: okay
- REPLICATOR: here's a _bulk_docs with 500 documents (1 POST request)
- TARGET: okay
{code}
See the problem here? That 500-loop, where we have to do a GET for each one
of 500 documents, is a lot of unnecessary back-and-forth, considering that
the replicator already knows what it needs before the loop starts. You can
parallelize, but if you assume a browser (e.g. for PouchDB), most browsers
only let you do ~8 simultaneous requests at once. Plus, there's latency and
HTTP headers to consider. So overall, it's not cool.
So why do we even need to do the separate requests? Shouldn't {{_all_docs}}
be good enough? Turns out it's not, because we need this special
{{_revisions}} object.
For example, consider a document {{'foo'}} with 10 revisions. You may compact
the database, in which case revisions {{1-x}} through {{9-x}} are no longer
retrievable. However, if you query using {{revs}} and {{open_revs}}, those
rev IDs are still available:
{code}
$ curl 'http://nolan.iriscouch.com/test/foo?revs=trueopen_revs=all'
{
_id: foo,
_rev: 10-c78e199ad5e996b240c9d6482907088e,
_revisions: {
start: 10,
ids: [
c78e199ad5e996b240c9d6482907088e,
f560283f1968a05046f0c38e468006bb,
0091198554171c632c27c8342ddec5af,
e0a023e2ea59db73f812ad773ea08b17,
65d7f8b8206a244035edd9f252f206ad,
069d1432a003c58bdd23f01ff80b718f,
d21f26bb604b7fe9eba03ce4562cf37b,
31d380f99a6e54875855e1c24469622d,
3b4791360024426eadafe31542a2c34b,
967a00dff5e02add41819138abb3284d
]
}
}
{code}
And in the replication algorithm, _this full \_revisions object is required_
at the point when you copy the document from one database to another, which
is accomplished with a POST to {{_bulk_docs}} using {{new_edits=false}}. If
you don't have the full {{_revisions}} object, CouchDB accepts the new
revision, but considers it to be a conflict. (The exception is with
generation-1 documents, since they have no history, so as it says in the
TouchDB writeup, you can safely just use {{_all_docs}} as an optimization for
such documents.)
And unfortunately, this {{_revision}} object is only available from the {{GET
/:dbid/:docid}} endpoint. Trust me; I've tried the other APIs. You can't get
it anywhere else.
This is a huge problem, especially in PouchDB where we often have to deal
with CORS, meaning the number of HTTP requests is doubled. So for those 500
GETs, it's an extra 500 OPTIONs, which is just unacceptable.
Replication does not have to be slow. While we were experimenting with ways
of fetching documents in bulk, we tried a technique that just relied on using
{{_changes}} with {{include_docs=true}}
([|\#2472|https://github.com/pouchdb/pouchdb/pull/2472]). This pushed
conflicts into the target database, but on the upside, you can sync ~95k
documents from npm's skimdb repository to the browser in less than 20
minutes! (See [npm-browser.com|http://npm-browser.com] for a demo.)
What an amazing story we could tell about the beauty of CouchDB replication,
if only this trick actually worked!
My proposal is a simple one: just
[jira] [Commented] (COUCHDB-2310) Add a bulk API for revs open_revs
[
https://issues.apache.org/jira/browse/COUCHDB-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252457#comment-14252457
]
Jan Lehnardt commented on COUCHDB-2310:
---
How about this for an option:
1. We include the rcouch code in 1.7.0 as _bulk_get.
2. We *may* make a chttpd version in 2.0.
3. We mark it/them as deprecated from the start.
4. We design a desirable replication stream for 2.x and forward together with
the PouchDB and TouchDB folks. Roughly, this would be a multipart stream that
mirrors /_changes + include_docs + open_revs (plus handwaving, bear with me).
Something like /db/_replication_stream or so.
We can’t get around POST APIs for potentially large key-requests as per real
world HTTP constraints. Maybe HTTP2 with its multiplexing helps? I don’t know.
We should also definitely support the GET version of these requests if the
client knows that the amount of data it has to upload to get the right response
is within any practical limits for the given setup.
I’d say though, for the discussion of this ticket, that is out of scope.
Add a bulk API for revs open_revs
---
Key: COUCHDB-2310
URL: https://issues.apache.org/jira/browse/COUCHDB-2310
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: HTTP Interface
Reporter: Nolan Lawson
CouchDB replication is too slow.
And what makes it so slow is that it's just so unnecessarily chatty. During
replication, you have to do a separate GET for each individual document, in
order to get the full {{_revisions}} object for that document (using the
{{revs}} and {{open_revs}} parameters ndash; refer to [the TouchDB
writeup|https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithm]
or [Benoit's writeup|http://dataprotocols.org/couchdb-replication/] if you
need a refresher).
So for example, let's say you've got a database full of 10,000 documents, and
you replicate using a batch size of 500 (batch sizes are configurable in
PouchDB). The conversation for a single batch basically looks like this:
{code}
- REPLICATOR: gimme 500 changes since seq X (1 GET request)
- SOURCE: okay
- REPLICATOR: gimme the _revs_diff for these 500 docs/_revs (1 POST request)
- SOURCE: okay
- repeat 500 times:
- REPLICATOR: gimme the _revisions for doc n with _revs [...] (1 GET
request)
- SOURCE: okay
- REPLICATOR: here's a _bulk_docs with 500 documents (1 POST request)
- TARGET: okay
{code}
See the problem here? That 500-loop, where we have to do a GET for each one
of 500 documents, is a lot of unnecessary back-and-forth, considering that
the replicator already knows what it needs before the loop starts. You can
parallelize, but if you assume a browser (e.g. for PouchDB), most browsers
only let you do ~8 simultaneous requests at once. Plus, there's latency and
HTTP headers to consider. So overall, it's not cool.
So why do we even need to do the separate requests? Shouldn't {{_all_docs}}
be good enough? Turns out it's not, because we need this special
{{_revisions}} object.
For example, consider a document {{'foo'}} with 10 revisions. You may compact
the database, in which case revisions {{1-x}} through {{9-x}} are no longer
retrievable. However, if you query using {{revs}} and {{open_revs}}, those
rev IDs are still available:
{code}
$ curl 'http://nolan.iriscouch.com/test/foo?revs=trueopen_revs=all'
{
_id: foo,
_rev: 10-c78e199ad5e996b240c9d6482907088e,
_revisions: {
start: 10,
ids: [
c78e199ad5e996b240c9d6482907088e,
f560283f1968a05046f0c38e468006bb,
0091198554171c632c27c8342ddec5af,
e0a023e2ea59db73f812ad773ea08b17,
65d7f8b8206a244035edd9f252f206ad,
069d1432a003c58bdd23f01ff80b718f,
d21f26bb604b7fe9eba03ce4562cf37b,
31d380f99a6e54875855e1c24469622d,
3b4791360024426eadafe31542a2c34b,
967a00dff5e02add41819138abb3284d
]
}
}
{code}
And in the replication algorithm, _this full \_revisions object is required_
at the point when you copy the document from one database to another, which
is accomplished with a POST to {{_bulk_docs}} using {{new_edits=false}}. If
you don't have the full {{_revisions}} object, CouchDB accepts the new
revision, but considers it to be a conflict. (The exception is with
generation-1 documents, since they have no history, so as it says in the
TouchDB writeup, you can safely just use {{_all_docs}} as an optimization for
such documents.)
And unfortunately, this {{_revision}} object is only available from the {{GET
/:dbid/:docid}} endpoint. Trust me; I've tried the other APIs. You can't get
it anywhere else.
This is a huge problem, especially in PouchDB where we often have to deal
with CORS, meaning the number of
[jira] [Created] (COUCHDB-2495) Switch PBKDF2 to SHA256
Jan Lehnardt created COUCHDB-2495: - Summary: Switch PBKDF2 to SHA256 Key: COUCHDB-2495 URL: https://issues.apache.org/jira/browse/COUCHDB-2495 Project: CouchDB Issue Type: Improvement Security Level: public (Regular issues) Components: Database Core Reporter: Jan Lehnardt We currently use SHA1 for PBKDF2 hashing. While the way SHA1 is used, this doesn’t pose a security issue, it is generally advisable to use a newer hash function, e.g. SHA256. [~kxepal] noted on the user@ list, that this would leave older Erlang versions (R14 and R15) behind, so we can’t do it right now, but we should think about it for the future. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2488) Respawn no longer works
[
https://issues.apache.org/jira/browse/COUCHDB-2488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14234171#comment-14234171
]
Jan Lehnardt commented on COUCHDB-2488:
---
Curious! I am fairly sure we didn’t change anything there.
In addition, I think the 1.6.1 behaviour is correct, as you shouldn’t get
restarts when you normally shut down the CouchDB instance.
A better test would be to kill the beam[.smp] process and waiting for a respawn.
What are your respective Erlang versions?
(also, hi waves from Kotti :)
Respawn no longer works
---
Key: COUCHDB-2488
URL: https://issues.apache.org/jira/browse/COUCHDB-2488
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Reporter: Kristine Jetzke
I noticed that CouchDB does not get restarted anymore if it was killed. Were
there any changes between 1.5 and 1.6?
Using 1.5.1:
{code}
tine$ couchdb -r 5 -b
Apache CouchDB has started, time to relax.
tine$ couchdb -k
Apache CouchDB crashed, restarting in 5 seconds.
Apache CouchDB has been killed.
tine$ curl localhost:5984
{couchdb:Welcome,uuid:9ed01bc924270ea8c58fb659db662eb7,version:1.5.1,vendor:{version:1.6.1-1,name:Homebrew}}
{code}
Using 1.6.1:
{code}
tine$ couchdb -r 5 -b
Apache CouchDB has started, time to relax.
tine$ couchdb -k
Apache CouchDB has been killed.
tine$ curl localhost:5984
curl: (7) Failed to connect to localhost port 5984: Connection refused
{code}
I'm on Mac OS X Yosemite. I used homebrew to install CouchDB and {{brew
switch couchdb 1.5.1}} and {{brew switch couchdb 1.6.1_1}} to test the
different versions. However, I initially noticed the problem on a system with
only 1.6.1 installed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (COUCHDB-1577) Refuse start when underlying Erlang version has changed
[ https://issues.apache.org/jira/browse/COUCHDB-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14217677#comment-14217677 ] Jan Lehnardt commented on COUCHDB-1577: --- [~kxepal] yeah, I think so™ :) Refuse start when underlying Erlang version has changed --- Key: COUCHDB-1577 URL: https://issues.apache.org/jira/browse/COUCHDB-1577 Project: CouchDB Issue Type: Bug Affects Versions: 1.2 Environment: All Reporter: Jan Lehnardt When a user installs CouchDB from source and then updates the Erlang version on the system, spurious errors can occur (like COUCHDB-1571). On install, we should detect the Erlang version and store it with the CouchDB installation, and then bail if things don't match up during start time. I recon this could be coded entirely in bin/couchdb, no erlang required. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-732) ruby versions of the query server spec functions
[
https://issues.apache.org/jira/browse/COUCHDB-732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14205209#comment-14205209
]
Jan Lehnardt commented on COUCHDB-732:
--
Word!
ruby versions of the query server spec functions
Key: COUCHDB-732
URL: https://issues.apache.org/jira/browse/COUCHDB-732
Project: CouchDB
Issue Type: Improvement
Components: Test Suite
Reporter: Matt Lyon
Assignee: Robert Kowalski
Priority: Trivial
In the process of creating the a ruby version of the query server, I wrote
these to determine it was working correctly. Even though the repository for
the ruby query server contains its own test suite, I figure these might be
welcomed into the main query server specs. If not, no big deal. They assume
the ruby query server's repository is in the same parent directory as couchdb.
diff --git a/test/view_server/query_server_spec.rb
b/test/view_server/query_server_spec.rb
index 1de8e5b..c427e35 100644
--- a/test/view_server/query_server_spec.rb
+++ b/test/view_server/query_server_spec.rb
@@ -117,7 +117,8 @@ class QueryServerRunner OSProcessRunner
COMMANDS = {
js = #{COUCH_ROOT}/bin/couchjs_dev
#{COUCH_ROOT}/share/server/main.js,
-erlang = #{COUCH_ROOT}/test/view_server/run_native_process.es
+erlang = #{COUCH_ROOT}/test/view_server/run_native_process.es,
+ruby = /usr/bin/env ruby --
#{COUCH_ROOT}/../couchdb-ruby-query-server/bin/couchdb_view_server --safe
}
def self.run_command
@@ -137,13 +138,14 @@ end
functions = {
emit-twice = {
js = %{function(doc){emit(foo,doc.a); emit(bar,doc.a)}},
-erlang = -ERLANG
+erlang = -ERLANG,
fun({Doc}) -
A = proplists:get_value(a, Doc, null),
Emit(foo, A),
Emit(bar, A)
end.
ERLANG
+ruby = lambda{|doc| emit('foo',doc['a']); emit('bar',doc['a']) }
},
emit-once = {
js = -JS,
@@ -151,20 +153,39 @@ functions = {
emit(baz,doc.a)
}
JS
-erlang = -ERLANG
+erlang = -ERLANG,
fun({Doc}) -
A = proplists:get_value(a, Doc, null),
Emit(baz, A)
end.
ERLANG
+ruby = -RUBY
+ lambda {|doc| emit(baz, doc['a']) }
+RUBY
+ },
+ map-invalid-expression = {
+js = %{function(doc {emit(foo, doc.a);}},
+erlang = %|fun({Doc}|,
+ruby = lambda{
+ },
+ map-non-function-expression = {
+js = 3,
+erlang = 3,
+ruby = 3
+ },
+ map-logging = {
+js = %{function(doc){ log(doc); emit(logged, doc.a);}},
+ruby = %{lambda{|doc| log(doc); emit(logged, doc['a']) }}
},
reduce-values-length = {
js = %{function(keys, values, rereduce) { return values.length; }},
-erlang = %{fun(Keys, Values, ReReduce) - length(Values) end.}
+erlang = %{fun(Keys, Values, ReReduce) - length(Values) end.},
+ruby = %{lambda{|keys, values, rereduce| values.size }}
},
reduce-values-sum = {
js = %{function(keys, values, rereduce) { return sum(values); }},
-erlang = %{fun(Keys, Values, ReReduce) - lists:sum(Values) end.}
+erlang = %{fun(Keys, Values, ReReduce) - lists:sum(Values) end.},
+ruby = %{lambda{|keys, values, rereduce| values.inject(0){|sum, val|
sum += val} }}
},
validate-forbidden = {
js = -JS,
@@ -173,7 +194,7 @@ functions = {
throw({forbidden:bad doc}); foo bar;
}
JS
-erlang = -ERLANG
+erlang = -ERLANG,
fun({NewDoc}, _OldDoc, _UserCtx) -
case proplists:get_value(bad, NewDoc) of
undefined - 1;
@@ -181,6 +202,13 @@ functions = {
end
end.
ERLANG
+ruby = -RUBY
+ lambda{|new_doc, old_doc, user_ctx|
+if (new_doc['bad'])
+ throw(:forbidden, bad doc)
+end
+ }
+RUBY
},
show-simple = {
js = -JS,
@@ -189,7 +217,7 @@ functions = {
return [doc.title, doc.body].join(' - ');
}
JS
-erlang = -ERLANG
+erlang = -ERLANG,
fun({Doc}, Req) -
Title = proplists:get_value(title, Doc),
Body = proplists:get_value(body, Doc),
@@ -197,6 +225,9 @@ functions = {
{[{body, Resp}]}
end.
ERLANG
+ruby = -RUBY
+ lambda{|doc, req| [doc['title'], doc['body']].join(' - ') }
+RUBY
},
show-headers = {
js = -JS,
@@ -206,7 +237,7 @@ functions = {
return resp;
}
JS
-erlang = -ERLANG
+erlang = -ERLANG,
fun({Doc}, Req) -
Title = proplists:get_value(title, Doc),
Body = proplists:get_value(body, Doc),
@@ -218,6 +249,12 @@ functions = {
]}
end.
ERLANG
+ruby = -RUBY
+ lambda {|doc,
[jira] [Created] (COUCHDB-2433) Allow downloading of views
Jan Lehnardt created COUCHDB-2433: - Summary: Allow downloading of views Key: COUCHDB-2433 URL: https://issues.apache.org/jira/browse/COUCHDB-2433 Project: CouchDB Issue Type: Improvement Security Level: public (Regular issues) Components: Fauxton Reporter: Jan Lehnardt I have a non-technical user of CouchDB that would like a “JSON Export” of views in Fauxton. They can’t use the command line where they can just `curl $viewurl export.json`. One way to implement this is add a new query parameter to view queries could be `/db/_design/_view/name?download=true` which adds a Content-Disposition header so browsers offer the link as a download (and whatever else is necessary to make browsers behave :) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (COUCHDB-2433) Allow downloading of views
[ https://issues.apache.org/jira/browse/COUCHDB-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt updated COUCHDB-2433: -- Priority: Minor (was: Major) Affects Version/s: 2.0.0 Skill Level: New Contributors Level (Easy) Allow downloading of views -- Key: COUCHDB-2433 URL: https://issues.apache.org/jira/browse/COUCHDB-2433 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Components: Fauxton Affects Versions: 2.0.0 Reporter: Jan Lehnardt Priority: Minor I have a non-technical user of CouchDB that would like a “JSON Export” of views in Fauxton. They can’t use the command line where they can just `curl $viewurl export.json`. One way to implement this is add a new query parameter to view queries could be `/db/_design/_view/name?download=true` which adds a Content-Disposition header so browsers offer the link as a download (and whatever else is necessary to make browsers behave :) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (COUCHDB-1985) ./bootstrap feature deprecation warnings
[ https://issues.apache.org/jira/browse/COUCHDB-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt reopened COUCHDB-1985: --- ./bootstrap feature deprecation warnings Key: COUCHDB-1985 URL: https://issues.apache.org/jira/browse/COUCHDB-1985 Project: CouchDB Issue Type: Bug Components: Build System Affects Versions: 1.x.x Reporter: Jan Lehnardt Running ./bootstrap on master gives me: ./bootstrap configure.ac:29: installing 'build-aux/compile' configure.ac:34: installing 'build-aux/config.guess' configure.ac:34: installing 'build-aux/config.sub' configure.ac:27: installing 'build-aux/install-sh' configure.ac:27: installing 'build-aux/missing' src/couchdb/priv/Makefile.am:44: warning: source file 'couch_ejson_compare/couch_ejson_compare.c' is in a subdirectory, src/couchdb/priv/Makefile.am:44: but option 'subdir-objects' is disabled automake: warning: possible forward-incompatibility. automake: At least a source file is in a subdirectory, but the 'subdir-objects' automake: automake option hasn't been enabled. For now, the corresponding output automake: object file(s) will be placed in the top-level directory. However, automake: this behaviour will change in future Automake versions: they will automake: unconditionally cause object files to be placed in the same subdirectory automake: of the corresponding sources. automake: You are advised to start using 'subdir-objects' option throughout your automake: project, to avoid future incompatibilities. src/couchdb/priv/Makefile.am:52: warning: source file 'icu_driver/couch_icu_driver.c' is in a subdirectory, src/couchdb/priv/Makefile.am:52: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/http.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/main.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/utf8.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/util.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:103: warning: source file 'spawnkillable/couchspawnkillable_win.c' is in a subdirectory, src/couchdb/priv/Makefile.am:103: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am: installing 'build-aux/depcomp' src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_alloc.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_buf.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_encode.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_gen.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_lex.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_parser.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy-sinksource.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy-stubs-internal.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled You have bootstrapped Apache CouchDB, time to relax. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (COUCHDB-1985) ./bootstrap feature deprecation warnings
[ https://issues.apache.org/jira/browse/COUCHDB-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt updated COUCHDB-1985: -- Affects Version/s: 1.x.x ./bootstrap feature deprecation warnings Key: COUCHDB-1985 URL: https://issues.apache.org/jira/browse/COUCHDB-1985 Project: CouchDB Issue Type: Bug Components: Build System Affects Versions: 1.x.x Reporter: Jan Lehnardt Running ./bootstrap on master gives me: ./bootstrap configure.ac:29: installing 'build-aux/compile' configure.ac:34: installing 'build-aux/config.guess' configure.ac:34: installing 'build-aux/config.sub' configure.ac:27: installing 'build-aux/install-sh' configure.ac:27: installing 'build-aux/missing' src/couchdb/priv/Makefile.am:44: warning: source file 'couch_ejson_compare/couch_ejson_compare.c' is in a subdirectory, src/couchdb/priv/Makefile.am:44: but option 'subdir-objects' is disabled automake: warning: possible forward-incompatibility. automake: At least a source file is in a subdirectory, but the 'subdir-objects' automake: automake option hasn't been enabled. For now, the corresponding output automake: object file(s) will be placed in the top-level directory. However, automake: this behaviour will change in future Automake versions: they will automake: unconditionally cause object files to be placed in the same subdirectory automake: of the corresponding sources. automake: You are advised to start using 'subdir-objects' option throughout your automake: project, to avoid future incompatibilities. src/couchdb/priv/Makefile.am:52: warning: source file 'icu_driver/couch_icu_driver.c' is in a subdirectory, src/couchdb/priv/Makefile.am:52: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/http.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/main.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/utf8.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/util.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:103: warning: source file 'spawnkillable/couchspawnkillable_win.c' is in a subdirectory, src/couchdb/priv/Makefile.am:103: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am: installing 'build-aux/depcomp' src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_alloc.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_buf.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_encode.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_gen.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_lex.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_parser.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy-sinksource.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy-stubs-internal.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled You have bootstrapped Apache CouchDB, time to relax. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-1985) ./bootstrap feature deprecation warnings
[ https://issues.apache.org/jira/browse/COUCHDB-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14194387#comment-14194387 ] Jan Lehnardt commented on COUCHDB-1985: --- Let’s keep this open for 1.x.x. ./bootstrap feature deprecation warnings Key: COUCHDB-1985 URL: https://issues.apache.org/jira/browse/COUCHDB-1985 Project: CouchDB Issue Type: Bug Components: Build System Affects Versions: 1.x.x Reporter: Jan Lehnardt Running ./bootstrap on master gives me: ./bootstrap configure.ac:29: installing 'build-aux/compile' configure.ac:34: installing 'build-aux/config.guess' configure.ac:34: installing 'build-aux/config.sub' configure.ac:27: installing 'build-aux/install-sh' configure.ac:27: installing 'build-aux/missing' src/couchdb/priv/Makefile.am:44: warning: source file 'couch_ejson_compare/couch_ejson_compare.c' is in a subdirectory, src/couchdb/priv/Makefile.am:44: but option 'subdir-objects' is disabled automake: warning: possible forward-incompatibility. automake: At least a source file is in a subdirectory, but the 'subdir-objects' automake: automake option hasn't been enabled. For now, the corresponding output automake: object file(s) will be placed in the top-level directory. However, automake: this behaviour will change in future Automake versions: they will automake: unconditionally cause object files to be placed in the same subdirectory automake: of the corresponding sources. automake: You are advised to start using 'subdir-objects' option throughout your automake: project, to avoid future incompatibilities. src/couchdb/priv/Makefile.am:52: warning: source file 'icu_driver/couch_icu_driver.c' is in a subdirectory, src/couchdb/priv/Makefile.am:52: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/http.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/main.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/utf8.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/util.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:103: warning: source file 'spawnkillable/couchspawnkillable_win.c' is in a subdirectory, src/couchdb/priv/Makefile.am:103: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am: installing 'build-aux/depcomp' src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_alloc.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_buf.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_encode.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_gen.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_lex.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_parser.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy-sinksource.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy-stubs-internal.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled You have bootstrapped Apache CouchDB, time to relax. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-1985) ./bootstrap feature deprecation warnings
[ https://issues.apache.org/jira/browse/COUCHDB-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14194390#comment-14194390 ] Jan Lehnardt commented on COUCHDB-1985: --- it is because of an older version of automake, IIRC. All systems with it have this issue. ./bootstrap feature deprecation warnings Key: COUCHDB-1985 URL: https://issues.apache.org/jira/browse/COUCHDB-1985 Project: CouchDB Issue Type: Bug Components: Build System Affects Versions: 1.x.x Reporter: Jan Lehnardt Running ./bootstrap on master gives me: ./bootstrap configure.ac:29: installing 'build-aux/compile' configure.ac:34: installing 'build-aux/config.guess' configure.ac:34: installing 'build-aux/config.sub' configure.ac:27: installing 'build-aux/install-sh' configure.ac:27: installing 'build-aux/missing' src/couchdb/priv/Makefile.am:44: warning: source file 'couch_ejson_compare/couch_ejson_compare.c' is in a subdirectory, src/couchdb/priv/Makefile.am:44: but option 'subdir-objects' is disabled automake: warning: possible forward-incompatibility. automake: At least a source file is in a subdirectory, but the 'subdir-objects' automake: automake option hasn't been enabled. For now, the corresponding output automake: object file(s) will be placed in the top-level directory. However, automake: this behaviour will change in future Automake versions: they will automake: unconditionally cause object files to be placed in the same subdirectory automake: of the corresponding sources. automake: You are advised to start using 'subdir-objects' option throughout your automake: project, to avoid future incompatibilities. src/couchdb/priv/Makefile.am:52: warning: source file 'icu_driver/couch_icu_driver.c' is in a subdirectory, src/couchdb/priv/Makefile.am:52: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/http.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/main.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/utf8.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:61: warning: source file 'couch_js/util.c' is in a subdirectory, src/couchdb/priv/Makefile.am:61: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am:103: warning: source file 'spawnkillable/couchspawnkillable_win.c' is in a subdirectory, src/couchdb/priv/Makefile.am:103: but option 'subdir-objects' is disabled src/couchdb/priv/Makefile.am: installing 'build-aux/depcomp' src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_alloc.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_buf.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_encode.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_gen.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_lex.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/ejson/Makefile.am:21: warning: source file 'yajl/yajl_parser.c' is in a subdirectory, src/ejson/Makefile.am:21: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy-sinksource.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled src/snappy/Makefile.am:16: warning: source file 'google-snappy/snappy-stubs-internal.cc' is in a subdirectory, src/snappy/Makefile.am:16: but option 'subdir-objects' is disabled You have bootstrapped Apache CouchDB, time to relax. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2310) Add a bulk API for revs open_revs
[
https://issues.apache.org/jira/browse/COUCHDB-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182625#comment-14182625
]
Jan Lehnardt commented on COUCHDB-2310:
---
Heya Daniel, cool work thanks!
While we can’t use the lua implementation in CouchDB obviously, it is a great
prototype for any work we can do on the Erlang side.
FWIW, this would make a great first Erlang/CouchDB project, if you feel so
inclined :) — Or anyone, for that matter!
Add a bulk API for revs open_revs
---
Key: COUCHDB-2310
URL: https://issues.apache.org/jira/browse/COUCHDB-2310
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: HTTP Interface
Reporter: Nolan Lawson
CouchDB replication is too slow.
And what makes it so slow is that it's just so unnecessarily chatty. During
replication, you have to do a separate GET for each individual document, in
order to get the full {{_revisions}} object for that document (using the
{{revs}} and {{open_revs}} parameters ndash; refer to [the TouchDB
writeup|https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithm]
or [Benoit's writeup|http://dataprotocols.org/couchdb-replication/] if you
need a refresher).
So for example, let's say you've got a database full of 10,000 documents, and
you replicate using a batch size of 500 (batch sizes are configurable in
PouchDB). The conversation for a single batch basically looks like this:
{code}
- REPLICATOR: gimme 500 changes since seq X (1 GET request)
- SOURCE: okay
- REPLICATOR: gimme the _revs_diff for these 500 docs/_revs (1 POST request)
- SOURCE: okay
- repeat 500 times:
- REPLICATOR: gimme the _revisions for doc n with _revs [...] (1 GET
request)
- SOURCE: okay
- REPLICATOR: here's a _bulk_docs with 500 documents (1 POST request)
- TARGET: okay
{code}
See the problem here? That 500-loop, where we have to do a GET for each one
of 500 documents, is a lot of unnecessary back-and-forth, considering that
the replicator already knows what it needs before the loop starts. You can
parallelize, but if you assume a browser (e.g. for PouchDB), most browsers
only let you do ~8 simultaneous requests at once. Plus, there's latency and
HTTP headers to consider. So overall, it's not cool.
So why do we even need to do the separate requests? Shouldn't {{_all_docs}}
be good enough? Turns out it's not, because we need this special
{{_revisions}} object.
For example, consider a document {{'foo'}} with 10 revisions. You may compact
the database, in which case revisions {{1-x}} through {{9-x}} are no longer
retrievable. However, if you query using {{revs}} and {{open_revs}}, those
rev IDs are still available:
{code}
$ curl 'http://nolan.iriscouch.com/test/foo?revs=trueopen_revs=all'
{
_id: foo,
_rev: 10-c78e199ad5e996b240c9d6482907088e,
_revisions: {
start: 10,
ids: [
c78e199ad5e996b240c9d6482907088e,
f560283f1968a05046f0c38e468006bb,
0091198554171c632c27c8342ddec5af,
e0a023e2ea59db73f812ad773ea08b17,
65d7f8b8206a244035edd9f252f206ad,
069d1432a003c58bdd23f01ff80b718f,
d21f26bb604b7fe9eba03ce4562cf37b,
31d380f99a6e54875855e1c24469622d,
3b4791360024426eadafe31542a2c34b,
967a00dff5e02add41819138abb3284d
]
}
}
{code}
And in the replication algorithm, _this full \_revisions object is required_
at the point when you copy the document from one database to another, which
is accomplished with a POST to {{_bulk_docs}} using {{new_edits=false}}. If
you don't have the full {{_revisions}} object, CouchDB accepts the new
revision, but considers it to be a conflict. (The exception is with
generation-1 documents, since they have no history, so as it says in the
TouchDB writeup, you can safely just use {{_all_docs}} as an optimization for
such documents.)
And unfortunately, this {{_revision}} object is only available from the {{GET
/:dbid/:docid}} endpoint. Trust me; I've tried the other APIs. You can't get
it anywhere else.
This is a huge problem, especially in PouchDB where we often have to deal
with CORS, meaning the number of HTTP requests is doubled. So for those 500
GETs, it's an extra 500 OPTIONs, which is just unacceptable.
Replication does not have to be slow. While we were experimenting with ways
of fetching documents in bulk, we tried a technique that just relied on using
{{_changes}} with {{include_docs=true}}
([|\#2472|https://github.com/pouchdb/pouchdb/pull/2472]). This pushed
conflicts into the target database, but on the upside, you can sync ~95k
documents from npm's skimdb repository to the browser in less than 20
minutes! (See [npm-browser.com|http://npm-browser.com] for a demo.)
What an amazing story we
[jira] [Commented] (COUCHDB-2403) Build Fauxton for a different url paths?
[ https://issues.apache.org/jira/browse/COUCHDB-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182812#comment-14182812 ] Jan Lehnardt commented on COUCHDB-2403: --- [~kxepal] JIRA is fine for asking dev related questions. [~michaelcole] at this point I’m not aware of any other solution, maybe some of the Fauxton devs have more insight (I’m only looking at Fauxton from a packaging perspective and I have had to change the URL too :) CC [~garren] [~deathbear] [~robertkowalski] Build Fauxton for a different url paths? Key: COUCHDB-2403 URL: https://issues.apache.org/jira/browse/COUCHDB-2403 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Components: Fauxton Reporter: Michael Cole Hey, I'm trying to use Fauxton in another project: express-pouchdb We'd like Fauxton to not know it's URL, until it's being served. So someone could configure a PouchDB server in Express at their [own URL](https://github.com/MichaelJCole/express-pouchdb#example-usage) like this: /mydb/ -- PouchDB server /mydb/_utils -- Fauxton The best I could figure out to do was hard- code the URL in settings.json and use `grunt release` to build Fauxton for that specific URL. Is there a better way? Thanks! Mike -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (COUCHDB-2403) Build Fauxton for a different url paths?
[ https://issues.apache.org/jira/browse/COUCHDB-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt reopened COUCHDB-2403: --- Skill Level: Regular Contributors Level (Easy to Medium) Build Fauxton for a different url paths? Key: COUCHDB-2403 URL: https://issues.apache.org/jira/browse/COUCHDB-2403 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Components: Fauxton Reporter: Michael Cole Hey, I'm trying to use Fauxton in another project: express-pouchdb We'd like Fauxton to not know it's URL, until it's being served. So someone could configure a PouchDB server in Express at their [own URL](https://github.com/MichaelJCole/express-pouchdb#example-usage) like this: /mydb/ -- PouchDB server /mydb/_utils -- Fauxton The best I could figure out to do was hard- code the URL in settings.json and use `grunt release` to build Fauxton for that specific URL. Is there a better way? Thanks! Mike -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-1521) multipart parser gets multiple attachments mixed up
[ https://issues.apache.org/jira/browse/COUCHDB-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166604#comment-14166604 ] Jan Lehnardt commented on COUCHDB-1521: --- I think we still do the “process according to json order” bit, though. multipart parser gets multiple attachments mixed up --- Key: COUCHDB-1521 URL: https://issues.apache.org/jira/browse/COUCHDB-1521 Project: CouchDB Issue Type: Bug Components: HTTP Interface Affects Versions: 1.2 Reporter: Jens Alfke Assignee: Randall Leeds When receiving a document PUT in multipart format, CouchDB gets the attachments and MIME parts mixed up. Instead of looking at the headers of a MIME part to identify which attachment it is (most likely by using the 'filename' property of the 'Content-Disposition:' header), it processes the attachments according to the order in which their metadata objects appear in the JSON body's '_attachments:' object. The problem with this is that JSON objects (dictionaries) are _not_ ordered collections. I know that Erlang's implementation of them (as linked lists of key/value pairs) happens to be ordered, and I think some JavaScript implementations have the side effect of preserving order; but in many languages these are implemented as hash tables and genuinely unordered. This means that when a program written in such a language converts a native object to JSON, it has no control over (and probably no knowledge of) the order in which the keys of the JSON object are written out. This makes it impossible to then write the attachments in the same order. The only workaround seems to be for the program to implement its own custom JSON encoder just so that it can write object keys in a known order (probably sorted), which then enables it to write the attachment bodies in the same order. NOTE: This is the flip side of COUCHDB-1368 which I filed last year; that bug has to do with the same ordering issue when CouchDB _generates_ multipart responses (and presents similar problems for clients not written in Erlang.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-1863) Documentation link should be on top links of front page
[ https://issues.apache.org/jira/browse/COUCHDB-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163915#comment-14163915 ] Jan Lehnardt commented on COUCHDB-1863: --- I think it doesn’t matter that the navigation metaphor is “broken”. I think we need “Doc” or “Documentation” (literally) in that top row. I’m happy for other things to move around, but a rename to “Learn Discuss” just to keep a navigation metaphor is as good as not having it. I’d prefer to revert to the previous model until we have a new website. Documentation link should be on top links of front page --- Key: COUCHDB-1863 URL: https://issues.apache.org/jira/browse/COUCHDB-1863 Project: CouchDB Issue Type: Bug Components: Website Reporter: Dale Harvey Assignee: Noah Slater Its about the most important link there and nobody (including me) can find it. Can send in a patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-1863) Documentation link should be on top links of front page
[ https://issues.apache.org/jira/browse/COUCHDB-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163923#comment-14163923 ] Jan Lehnardt commented on COUCHDB-1863: --- I just noticed that Docs was gone earlier, I’m vehemently against that change and I suggest we re-add it ASAP until any new changes are all finished. I’m sorry I didn’t catch this earlier, but I feel this is a really really bad change. Documentation link should be on top links of front page --- Key: COUCHDB-1863 URL: https://issues.apache.org/jira/browse/COUCHDB-1863 Project: CouchDB Issue Type: Bug Components: Website Reporter: Dale Harvey Assignee: Noah Slater Its about the most important link there and nobody (including me) can find it. Can send in a patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-1863) Documentation link should be on top links of front page
[ https://issues.apache.org/jira/browse/COUCHDB-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163949#comment-14163949 ] Jan Lehnardt commented on COUCHDB-1863: --- heya, sorry for jumping in, please keep discussing the proposed change as usual. My vote is for having “Docs” or “Documentation” front and center. I consider not having the link there that our site is broken and I decided to re-add it right now as a hot-fix. Please don’t let hat influence this particular discussion (aside from my vote above). Sorry for the noise. Documentation link should be on top links of front page --- Key: COUCHDB-1863 URL: https://issues.apache.org/jira/browse/COUCHDB-1863 Project: CouchDB Issue Type: Bug Components: Website Reporter: Dale Harvey Assignee: Noah Slater Its about the most important link there and nobody (including me) can find it. Can send in a patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-1863) Documentation link should be on top links of front page
[ https://issues.apache.org/jira/browse/COUCHDB-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163982#comment-14163982 ] Jan Lehnardt commented on COUCHDB-1863: --- I really like the Learn Discuss section, maybe we can just keep it as is *and* have the direct link Docs in the nav bar, with a symbol that signifies that the link is external (like you suggest). I kind of have Wikipedia in mind, that has a little arrow type thing. I think having a direct link to the RTD docs up top is really important in a “remove frustration” kind of way, e.g. if I’m in JUST NEED DOCS” mode, I don’t want to learn what other stuff there is, I want “The Docs” (I understand it is a little myopic, but I can just see myself in that situation, and I’m sure you can, too :) Documentation link should be on top links of front page --- Key: COUCHDB-1863 URL: https://issues.apache.org/jira/browse/COUCHDB-1863 Project: CouchDB Issue Type: Bug Components: Website Reporter: Dale Harvey Assignee: Noah Slater Its about the most important link there and nobody (including me) can find it. Can send in a patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2350) Finish move of the wiki documentation; clean up references in docs.couchdb.org
[ https://issues.apache.org/jira/browse/COUCHDB-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160207#comment-14160207 ] Jan Lehnardt commented on COUCHDB-2350: --- [~candeira] What is your username on the old wiki? Finish move of the wiki documentation; clean up references in docs.couchdb.org -- Key: COUCHDB-2350 URL: https://issues.apache.org/jira/browse/COUCHDB-2350 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Components: Documentation, Website Reporter: Javier Candeira It occurs to me that as a new inductee into the project I'm in a privileged position to update and restructure the documentation as I take the project in, and it would probably be a better first task than to go after individual bugs. This is how I'd go about working on restructuring the documentation: - move the old wiki content to confluence and 301 all wiki.apache.org pages to the new wiki. No new content added. - track all links and references to old wiki in docs.couchdb.org, and rewrite them to point at new wiki. Still no new content added. - then I would start triaging documentation bugs. There are many tasks that are better done by a newcomer, since we need to follow the documentation or be confused by it. This is what I'd need: - To be added to the confluence wiki contributors list (username: candeira) - To be added to the old wiki contributors list (username: JavierCandeira) - Optionally, to have a test confluence wiki so I can test migrating the old one to the new one via scripts without making public changes until bugs have been ironed out. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2350) Finish move of the wiki documentation; clean up references in docs.couchdb.org
[ https://issues.apache.org/jira/browse/COUCHDB-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160304#comment-14160304 ] Jan Lehnardt commented on COUCHDB-2350: --- Heya, you are already in the contributors list, you should be able to edit everything! :) Finish move of the wiki documentation; clean up references in docs.couchdb.org -- Key: COUCHDB-2350 URL: https://issues.apache.org/jira/browse/COUCHDB-2350 Project: CouchDB Issue Type: Improvement Security Level: public(Regular issues) Components: Documentation, Website Reporter: Javier Candeira It occurs to me that as a new inductee into the project I'm in a privileged position to update and restructure the documentation as I take the project in, and it would probably be a better first task than to go after individual bugs. This is how I'd go about working on restructuring the documentation: - move the old wiki content to confluence and 301 all wiki.apache.org pages to the new wiki. No new content added. - track all links and references to old wiki in docs.couchdb.org, and rewrite them to point at new wiki. Still no new content added. - then I would start triaging documentation bugs. There are many tasks that are better done by a newcomer, since we need to follow the documentation or be confused by it. This is what I'd need: - To be added to the confluence wiki contributors list (username: candeira) - To be added to the old wiki contributors list (username: JavierCandeira) - Optionally, to have a test confluence wiki so I can test migrating the old one to the new one via scripts without making public changes until bugs have been ironed out. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2257) BulkRequest, when conflict include _rev as well to e.g. ease retry bulk delete
[ https://issues.apache.org/jira/browse/COUCHDB-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116675#comment-14116675 ] Jan Lehnardt commented on COUCHDB-2257: --- +1 BulkRequest, when conflict include _rev as well to e.g. ease retry bulk delete -- Key: COUCHDB-2257 URL: https://issues.apache.org/jira/browse/COUCHDB-2257 Project: CouchDB Issue Type: New Feature Security Level: public(Regular issues) Components: HTTP Interface Reporter: Daniel Wertheim When doing a BulkRequest and e.g. including document headers (_id, _rev) for deletion and there are conflicts. Can the current _rev be returned in the response as well. That would ease e.g. doing a new bulk delete. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (COUCHDB-2310) Add a bulk API for revs open_revs
[
https://issues.apache.org/jira/browse/COUCHDB-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116848#comment-14116848
]
Jan Lehnardt commented on COUCHDB-2310:
---
(bikeshed) or we make it a new endpoint /_bulk_revs (or something) that we can
build up as we like.
Add a bulk API for revs open_revs
---
Key: COUCHDB-2310
URL: https://issues.apache.org/jira/browse/COUCHDB-2310
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: HTTP Interface
Reporter: Nolan Lawson
CouchDB replication is too slow.
And what makes it so slow is that it's just so unnecessarily chatty. During
replication, you have to do a separate GET for each individual document, in
order to get the full {{_revisions}} object for that document (using the
{{revs}} and {{open_revs}} parameters ndash; refer to [the TouchDB
writeup|https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithm]
or [Benoit's writeup|http://dataprotocols.org/couchdb-replication/] if you
need a refresher).
So for example, let's say you've got a database full of 10,000 documents, and
you replicate using a batch size of 500 (batch sizes are configurable in
PouchDB). The conversation for a single batch basically looks like this:
{code}
- REPLICATOR: gimme 500 changes since seq X (1 GET request)
- SOURCE: okay
- REPLICATOR: gimme the _revs_diff for these 500 docs/_revs (1 POST request)
- SOURCE: okay
- repeat 500 times:
- REPLICATOR: gimme the _revisions for doc n with _revs [...] (1 GET
request)
- SOURCE: okay
- REPLICATOR: here's a _bulk_docs with 500 documents (1 POST request)
- TARGET: okay
{code}
See the problem here? That 500-loop, where we have to do a GET for each one
of 500 documents, is a lot of unnecessary back-and-forth, considering that
the replicator already knows what it needs before the loop starts. You can
parallelize, but if you assume a browser (e.g. for PouchDB), most browsers
only let you do ~8 simultaneous requests at once. Plus, there's latency and
HTTP headers to consider. So overall, it's not cool.
So why do we even need to do the separate requests? Shouldn't {{_all_docs}}
be good enough? Turns out it's not, because we need this special
{{_revisions}} object.
For example, consider a document {{'foo'}} with 10 revisions. You may compact
the database, in which case revisions {{1-x}} through {{9-x}} are no longer
retrievable. However, if you query using {{revs}} and {{open_revs}}, those
rev IDs are still available:
{code}
$ curl 'http://nolan.iriscouch.com/test/foo?revs=trueopen_revs=all'
{
_id: foo,
_rev: 10-c78e199ad5e996b240c9d6482907088e,
_revisions: {
start: 10,
ids: [
c78e199ad5e996b240c9d6482907088e,
f560283f1968a05046f0c38e468006bb,
0091198554171c632c27c8342ddec5af,
e0a023e2ea59db73f812ad773ea08b17,
65d7f8b8206a244035edd9f252f206ad,
069d1432a003c58bdd23f01ff80b718f,
d21f26bb604b7fe9eba03ce4562cf37b,
31d380f99a6e54875855e1c24469622d,
3b4791360024426eadafe31542a2c34b,
967a00dff5e02add41819138abb3284d
]
}
}
{code}
And in the replication algorithm, _this full \_revisions object is required_
at the point when you copy the document from one database to another, which
is accomplished with a POST to {{_bulk_docs}} using {{new_edits=false}}. If
you don't have the full {{_revisions}} object, CouchDB accepts the new
revision, but considers it to be a conflict. (The exception is with
generation-1 documents, since they have no history, so as it says in the
TouchDB writeup, you can safely just use {{_all_docs}} as an optimization for
such documents.)
And unfortunately, this {{_revision}} object is only available from the {{GET
/:dbid/:docid}} endpoint. Trust me; I've tried the other APIs. You can't get
it anywhere else.
This is a huge problem, especially in PouchDB where we often have to deal
with CORS, meaning the number of HTTP requests is doubled. So for those 500
GETs, it's an extra 500 OPTIONs, which is just unacceptable.
Replication does not have to be slow. While we were experimenting with ways
of fetching documents in bulk, we tried a technique that just relied on using
{{_changes}} with {{include_docs=true}}
([|\#2472|https://github.com/pouchdb/pouchdb/pull/2472]). This pushed
conflicts into the target database, but on the upside, you can sync ~95k
documents from npm's skimdb repository to the browser in less than 20
minutes! (See [npm-browser.com|http://npm-browser.com] for a demo.)
What an amazing story we could tell about the beauty of CouchDB replication,
if only this trick actually worked!
My proposal is a simple one: just add the {{revs}} and {{open_revs}} options
to
[jira] [Closed] (COUCHDB-2197) Invalid couchjs raises EPIPE, should raise (useful) ENOENT
[ https://issues.apache.org/jira/browse/COUCHDB-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt closed COUCHDB-2197. - Resolution: Duplicate Invalid couchjs raises EPIPE, should raise (useful) ENOENT -- Key: COUCHDB-2197 URL: https://issues.apache.org/jira/browse/COUCHDB-2197 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Reporter: Isaac Z. Schlueter If you accidentally point your couchjs config at a file that doesn't exist, then it'll raise an EPIPE when it tries to write to the process's stdin. However, it would be much more useful if it instead raised the ENOENT that it got when it tried to run the process. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (COUCHDB-1994) merge rcouch with couchdb 1.6 in a branch
[ https://issues.apache.org/jira/browse/COUCHDB-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13971863#comment-13971863 ] Jan Lehnardt commented on COUCHDB-1994: --- Adding this post from Benoit from dev@ for reference: I did many posts during the merge here asking for a review: https://www.mail-archive.com/search?l=dev%40couchdb.apache.orgq=rcouch+status That should help any dev interested in the review. COUCHDB-1994 can be used to track any review. wohali started some here: https://issues.apache.org/jira/browse/COUCHDB-1994 Any information on the rcouch wiki are still valid also: https://wiki.refuge.io/display/RCOUCH/rcouch Hope it helps. merge rcouch with couchdb 1.6 in a branch - Key: COUCHDB-1994 URL: https://issues.apache.org/jira/browse/COUCHDB-1994 Project: CouchDB Issue Type: Task Components: Build System, Database Core, HTTP Interface, JavaScript View Server Reporter: Benoit Chesneau -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (COUCHDB-2008) CI: Create repeatable images for different operating system configurations
[ https://issues.apache.org/jira/browse/COUCHDB-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13939102#comment-13939102 ] Jan Lehnardt commented on COUCHDB-2008: --- [~crigor] From APT. CI: Create repeatable images for different operating system configurations -- Key: COUCHDB-2008 URL: https://issues.apache.org/jira/browse/COUCHDB-2008 Project: CouchDB Issue Type: New Feature Components: Infrastructure Reporter: Jan Lehnardt Dave suggests to look into Ansible Packer to create images for CI. The idea would be to run these configurations on the CI server and controlled by Jenkins. It should also be possible to set up a configuration locally that matches the CI box exactly, so one can do debugging under the same circumstances the CI server runs tests. The requirement for now is that it all needs to work with a set of VMs that we have SSH access to, but we don’t manage the host. I’m not too familiar with all the automation and VM image packaging tools, so I don’t know if one or another is useful in that context or not. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (COUCHDB-1986) 04-replication-large-atts.t times out
[ https://issues.apache.org/jira/browse/COUCHDB-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13913692#comment-13913692 ] Jan Lehnardt commented on COUCHDB-1986: --- [~andywenk] can you re-run this? The 200 test is also one of the ones that fails spuriously, albeit a lot less often. 04-replication-large-atts.t times out - Key: COUCHDB-1986 URL: https://issues.apache.org/jira/browse/COUCHDB-1986 Project: CouchDB Issue Type: Bug Components: Replication Affects Versions: 1.5.0 Reporter: Jan Lehnardt 04-replication-large-atts.t gets stuck around 558, sometimes a little earlier or later, but it times out eventually, regardless of the timeout. I tried doubling and such. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (COUCHDB-1780) Upgrade users password hash on login, not password change
[ https://issues.apache.org/jira/browse/COUCHDB-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt updated COUCHDB-1780: -- Fix Version/s: 1.7.0 Upgrade users password hash on login, not password change - Key: COUCHDB-1780 URL: https://issues.apache.org/jira/browse/COUCHDB-1780 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Fix For: 1.7.0 We can upgrade users hashes to PBKDF2 on login, we don't need to wait for a password change. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-2066) Don't allow stupid storage of passwords
[ https://issues.apache.org/jira/browse/COUCHDB-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902584#comment-13902584 ] Jan Lehnardt commented on COUCHDB-2066: --- Since 1.3.0, CouchDB uses PBKDF2 for password hashing for user accounts. CouchDB will accept the old sha+salt format to not break BC, since it was introduced in a minor version update. Users are discouraged to use the old sha/salt ones though, we just kept the API intact. What we should do, though, is adding a transparent layer to turn sha+salt-style PWs into pbkdf2 PWs transparently. I’m not sure how to swing that without a API BC break, and we should run this plan by a proper cryptographer. Don't allow stupid storage of passwords --- Key: COUCHDB-2066 URL: https://issues.apache.org/jira/browse/COUCHDB-2066 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Reporter: Isaac Z. Schlueter If a password_sha/salt combination is PUT into the _users db, wrap that up in PBKDF2. Discussion: https://twitter.com/janl/status/434818855626502144 https://twitter.com/izs/status/434835388213899264 https://twitter.com/janl/status/434835614790586368 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-2066) Don't allow stupid storage of passwords
[ https://issues.apache.org/jira/browse/COUCHDB-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902590#comment-13902590 ] Jan Lehnardt commented on COUCHDB-2066: --- [~isaacs] I totally follow that train of though. But I also know that crypto is tricky and I don’t want to end up in a situation where we double XOR and then double ROT13 our PWs :) Don't allow stupid storage of passwords --- Key: COUCHDB-2066 URL: https://issues.apache.org/jira/browse/COUCHDB-2066 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Reporter: Isaac Z. Schlueter If a password_sha/salt combination is PUT into the _users db, wrap that up in PBKDF2. Discussion: https://twitter.com/janl/status/434818855626502144 https://twitter.com/izs/status/434835388213899264 https://twitter.com/janl/status/434835614790586368 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-2066) Don't allow stupid storage of passwords
[ https://issues.apache.org/jira/browse/COUCHDB-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902593#comment-13902593 ] Jan Lehnardt commented on COUCHDB-2066: --- This ticket is now for a way to upgrade old style sha+salt PWs to PBKDF2 in a non-disruptive as possible way. For all other security related discussions, please use the secur...@couchdb.apache.org mailing list. Thanks! Don't allow stupid storage of passwords --- Key: COUCHDB-2066 URL: https://issues.apache.org/jira/browse/COUCHDB-2066 Project: CouchDB Issue Type: Bug Security Level: public(Regular issues) Reporter: Isaac Z. Schlueter If a password_sha/salt combination is PUT into the _users db, wrap that up in PBKDF2. Discussion: https://twitter.com/janl/status/434818855626502144 https://twitter.com/izs/status/434835388213899264 https://twitter.com/janl/status/434835614790586368 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-2052) Add API for discovering feature availability
[
https://issues.apache.org/jira/browse/COUCHDB-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894894#comment-13894894
]
Jan Lehnardt commented on COUCHDB-2052:
---
[~snej] Thanks for your patience, this is discussion is extremely fruitful,
please continue :) — I am sure that we will find a good solution for this once
we’ve talked it all over. Team CouchDB is definitely interested in getting this
right.
Add API for discovering feature availability
Key: COUCHDB-2052
URL: https://issues.apache.org/jira/browse/COUCHDB-2052
Project: CouchDB
Issue Type: Improvement
Security Level: public(Regular issues)
Components: HTTP Interface
Reporter: Jens Alfke
I propose adding to the response of GET / a property called features or
extensions whose value is an array of strings, each string being an
agreed-upon identifier of a specific optional feature. For example:
{couchdb: welcome, features: [_bulk_get, persona]}, vendor:
…
Rationale:
Features are being added to CouchDB over time, plug-ins may add features, and
there are compatible servers that may have nonstandard features (like
_bulk_get). But there isn't a clear way for a client (which might be another
server's replicator) to determine what features a server has. Currently a
client looking at the response of a GET / has to figure out what server and
version thereof it's talking to, and then has to consult hardcoded knowledge
that version X of server Y supports feature Z.
(True, you can often get away without needing to check, by assuming a feature
exists but falling back to standard behavior if you get an error. But not all
features may be so easy to detect — the behavior of an unaware server might
be to ignore the feature and do the wrong thing, rather than returning an
error — and anyway this adds extra round-trips that slow down the operation.)
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (COUCHDB-2043) [replicator] handle non-200 response of _changes-feed items gracefully
Jan Lehnardt created COUCHDB-2043: - Summary: [replicator] handle non-200 response of _changes-feed items gracefully Key: COUCHDB-2043 URL: https://issues.apache.org/jira/browse/COUCHDB-2043 Project: CouchDB Issue Type: Bug Components: Replication Reporter: Jan Lehnardt See http://markmail.org/message/jmrvq6spfcmrvzcy for details. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-1960) Provide pagination for /_all_dbs
[
https://issues.apache.org/jira/browse/COUCHDB-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863188#comment-13863188
]
Jan Lehnardt commented on COUCHDB-1960:
---
Excellent point. Forward compatibly is definitely desired now that we have the
opportunity *and* the foreknowledge.
Provide pagination for /_all_dbs
Key: COUCHDB-1960
URL: https://issues.apache.org/jira/browse/COUCHDB-1960
Project: CouchDB
Issue Type: Improvement
Components: Fauxton, HTTP Interface
Reporter: Benjamin Young
The database-per-user design pattern has increased in popularity and
consequently causes long lists of databases when one hits {{/_all_dbs}}
To keep this sane, we need some pagination.
{{?skip=10limit=10}} style seems to be the simplest to implement (and there
should be existing code from Cloudant for this one).
Thanks!
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (COUCHDB-1960) Provide pagination for /_all_dbs
[
https://issues.apache.org/jira/browse/COUCHDB-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863286#comment-13863286
]
Jan Lehnardt commented on COUCHDB-1960:
---
heh, the point is that skip is not recommended for pagination on larger
clustered dbs, I think Russell suggests we also add means to do the regular
linked-list-pagination with start key and limit+1 for the next startkey.
Provide pagination for /_all_dbs
Key: COUCHDB-1960
URL: https://issues.apache.org/jira/browse/COUCHDB-1960
Project: CouchDB
Issue Type: Improvement
Components: Fauxton, HTTP Interface
Reporter: Benjamin Young
The database-per-user design pattern has increased in popularity and
consequently causes long lists of databases when one hits {{/_all_dbs}}
To keep this sane, we need some pagination.
{{?skip=10limit=10}} style seems to be the simplest to implement (and there
should be existing code from Cloudant for this one).
Thanks!
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (COUCHDB-1989) move src/couchjs-node in an `extra` repository
[ https://issues.apache.org/jira/browse/COUCHDB-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13862538#comment-13862538 ] Jan Lehnardt commented on COUCHDB-1989: --- is there a rebar / erlang convention for optional sources or are we inventing something on the fly here? Why aren’t we using src/extra or src/optional? move src/couchjs-node in an `extra` repository -- Key: COUCHDB-1989 URL: https://issues.apache.org/jira/browse/COUCHDB-1989 Project: CouchDB Issue Type: Improvement Components: Build System, JavaScript View Server Reporter: Benoit Chesneau Since the build of couchjs-node is optional we should put it outside of the src/ folder and move it to an `extra` folder (and the same for all extras). The reasoning behind is that we should separate more clearly what is an addon from what is the core product. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-1960) Provide pagination for /_all_dbs
[
https://issues.apache.org/jira/browse/COUCHDB-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13862541#comment-13862541
]
Jan Lehnardt commented on COUCHDB-1960:
---
Just one nitpick: Skip0 and Limit0 can be called Skip and Limit in
handle_all_dbs_req/1. I assume all_dbs_fun/2 was factored out of there at a
later time and Skip0 and Limit0 are remains of that.
Provide pagination for /_all_dbs
Key: COUCHDB-1960
URL: https://issues.apache.org/jira/browse/COUCHDB-1960
Project: CouchDB
Issue Type: Improvement
Components: Fauxton, HTTP Interface
Reporter: Benjamin Young
The database-per-user design pattern has increased in popularity and
consequently causes long lists of databases when one hits {{/_all_dbs}}
To keep this sane, we need some pagination.
{{?skip=10limit=10}} style seems to be the simplest to implement (and there
should be existing code from Cloudant for this one).
Thanks!
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (COUCHDB-1960) Provide pagination for /_all_dbs
[
https://issues.apache.org/jira/browse/COUCHDB-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13862542#comment-13862542
]
Jan Lehnardt commented on COUCHDB-1960:
---
Oh, and regular docs are still missing :)
Provide pagination for /_all_dbs
Key: COUCHDB-1960
URL: https://issues.apache.org/jira/browse/COUCHDB-1960
Project: CouchDB
Issue Type: Improvement
Components: Fauxton, HTTP Interface
Reporter: Benjamin Young
The database-per-user design pattern has increased in popularity and
consequently causes long lists of databases when one hits {{/_all_dbs}}
To keep this sane, we need some pagination.
{{?skip=10limit=10}} style seems to be the simplest to implement (and there
should be existing code from Cloudant for this one).
Thanks!
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (COUCHDB-2023) Plugins: use better mechanism to ensure compatibility than CouchDB Erlang Version
Jan Lehnardt created COUCHDB-2023: - Summary: Plugins: use better mechanism to ensure compatibility than CouchDB Erlang Version Key: COUCHDB-2023 URL: https://issues.apache.org/jira/browse/COUCHDB-2023 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt Currently a plugin can only be auto-installed if there is a binary available that has been built with the same CouchDB and Erlang version as the target CouchDB. The reason for this is ensuring that only plugins that are compatible can be installed. A problem with this is, that plugins for 1.2 also work in 1.2, 1.3 and 1.4 and 1.5. So the mechanism above is a but coarse and it puts a burden on the Plugin author to recompile their plugins with each of CouchDB’s now relatively frequent releases. Since we started using SemVer in earnest now, it should be enough to match on the major version number for CouchDB and Erlang version. Are there cases where this isn’t holding compatibility? -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-1989) move src/couchjs-node in an `extra` repository
[ https://issues.apache.org/jira/browse/COUCHDB-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13862553#comment-13862553 ] Jan Lehnardt commented on COUCHDB-1989: --- I don’t mind the change, I just want to do the right thing. Would love to hear from other folks here as well. This is an excellent bikeshed, don’t pass on the opportunity to participate. move src/couchjs-node in an `extra` repository -- Key: COUCHDB-1989 URL: https://issues.apache.org/jira/browse/COUCHDB-1989 Project: CouchDB Issue Type: Improvement Components: Build System, JavaScript View Server Reporter: Benoit Chesneau Since the build of couchjs-node is optional we should put it outside of the src/ folder and move it to an `extra` folder (and the same for all extras). The reasoning behind is that we should separate more clearly what is an addon from what is the core product. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-2016) Plugins meet BigCouch
[ https://issues.apache.org/jira/browse/COUCHDB-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13862558#comment-13862558 ] Jan Lehnardt commented on COUCHDB-2016: --- I talked this over a bit with [~rnewson]. Money quote: “rnewson so couchdb plugins are erlang applications that we have scripting to ease their integration into the couchdb release (and the inverse).” There are multiple issues intermingled here: 1. How to (un)install and start a CouchDB Plugin on an installation that uses Erlang Releases (BigCouch rcouch up)? 2. How to ensure a plugin gets installed into every node of a cluster? As for 1., Erlang Releases can be upgraded to newer versions while they are running. A plugin installation is simply an upgrade to a newer release. The difference from regular release updates is that these releases would be triggered by CouchDB itself (e.g. the couch_plugins application). We would have to figure out how to do version numbers in this case and everything else that retains Erlang Release semantics. 2. Installation of plugins should be done on each node separately. Once a plugin is installed on all nodes, the nodes can be upgraded as per 1. to activate the plugin. The API for this could disallow installation and upgrade if the cluster is in a unhealthy state. That way, we don’t have to worry about nodes that don’t receive an update instruction. Allowing this for unhealthy clusters could be tackled later, but is punted on for now. Plugins meet BigCouch - Key: COUCHDB-2016 URL: https://issues.apache.org/jira/browse/COUCHDB-2016 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt Currently Plugins don’t work with BigCouch. Two options that I can see so far: 1. Find a way to dynamically install plugins into a cluster, surviving offline nodes that get a delayed installation on recovery (or only allow plugins into a fully online cluster). 2. Make plugin building a static affair where plugins are installed with the rest of CouchDB as an Erlang release and do not allow dynamic updates of apps. I’d prefer 1, but any other options are welcome too. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (COUCHDB-2015) Plugin Registry
Jan Lehnardt created COUCHDB-2015: - Summary: Plugin Registry Key: COUCHDB-2015 URL: https://issues.apache.org/jira/browse/COUCHDB-2015 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt - Instead of hardcoding a list of plugins into Futon/Fauxton, we load a list of applicable plugins from a central (and configurable) plugins repository. - This allows plugin authors to publish new plugins and new versions of existing plugins independently. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (COUCHDB-2016) Plugins meet BigCouch
Jan Lehnardt created COUCHDB-2016: - Summary: Plugins meet BigCouch Key: COUCHDB-2016 URL: https://issues.apache.org/jira/browse/COUCHDB-2016 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt Currently Plugins don’t work with BigCouch. Two options that I can see so far: 1. Find a way to dynamically install plugins into a cluster, surviving offline nodes that get a delayed installation on recovery (or only allow plugins into a fully online cluster). 2. Make plugin building a static affair where plugins are installed with the rest of CouchDB as an Erlang release and do not allow dynamic updates of apps. I’d prefer 1, but any other options are welcome too. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (COUCHDB-2017) Plugins meet Erlang Releases
Jan Lehnardt created COUCHDB-2017: - Summary: Plugins meet Erlang Releases Key: COUCHDB-2017 URL: https://issues.apache.org/jira/browse/COUCHDB-2017 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt In the near future, CouchDB’s distribution will be based around Erlang releases and zero downtime upgrades. Plugins will have to play well with that world. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (COUCHDB-2018) Plugins with external dependencies
Jan Lehnardt created COUCHDB-2018: - Summary: Plugins with external dependencies Key: COUCHDB-2018 URL: https://issues.apache.org/jira/browse/COUCHDB-2018 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt Figure out how to handle plugins that have native ports, C library, or Java dependencies (or whatever else people would like to wrap into plugins) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (COUCHDB-2015) Plugin Registry
[ https://issues.apache.org/jira/browse/COUCHDB-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt updated COUCHDB-2015: -- Description: - Instead of hardcoding a list of plugins into Futon/Fauxton, we load a list of applicable plugins from a central (and configurable) plugins repository. - This allows plugin authors to publish new plugins and new versions of existing plugins independently. The focus should be on ease of publishing. was: - Instead of hardcoding a list of plugins into Futon/Fauxton, we load a list of applicable plugins from a central (and configurable) plugins repository. - This allows plugin authors to publish new plugins and new versions of existing plugins independently. Plugin Registry --- Key: COUCHDB-2015 URL: https://issues.apache.org/jira/browse/COUCHDB-2015 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt - Instead of hardcoding a list of plugins into Futon/Fauxton, we load a list of applicable plugins from a central (and configurable) plugins repository. - This allows plugin authors to publish new plugins and new versions of existing plugins independently. The focus should be on ease of publishing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (COUCHDB-2020) Tie plugins to identities / sign plugins
Jan Lehnardt created COUCHDB-2020: - Summary: Tie plugins to identities / sign plugins Key: COUCHDB-2020 URL: https://issues.apache.org/jira/browse/COUCHDB-2020 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt It should be possible to cryptographically ensure that a plugin has been provided by its supposed author. A web of trust for plugin authors could be a useful thing to have. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (COUCHDB-2019) Handle CouchDB crashes during plugin installation gracefully
Jan Lehnardt created COUCHDB-2019: - Summary: Handle CouchDB crashes during plugin installation gracefully Key: COUCHDB-2019 URL: https://issues.apache.org/jira/browse/COUCHDB-2019 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt Currently installing a plugin and crashing CouchDB somewhere in between results in an unknown state. Installation should be more resilient. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (COUCHDB-2021) Plugins should install tests docs
Jan Lehnardt created COUCHDB-2021: - Summary: Plugins should install tests docs Key: COUCHDB-2021 URL: https://issues.apache.org/jira/browse/COUCHDB-2021 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt currently plugins can’t install tests or docs. That should be fixed. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (COUCHDB-2022) ./couchdb should honour --additional_plugin_dir=DIR
Jan Lehnardt created COUCHDB-2022: - Summary: ./couchdb should honour --additional_plugin_dir=DIR Key: COUCHDB-2022 URL: https://issues.apache.org/jira/browse/COUCHDB-2022 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt It should be possible to point to another directory with plugins. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-2015) Plugin Registry
[ https://issues.apache.org/jira/browse/COUCHDB-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13862420#comment-13862420 ] Jan Lehnardt commented on COUCHDB-2015: --- I’d like to steal as many ideas from npm as possible as they seem to have gotten things pretty right. Also, they use CouchDB underneath, so yeah :) Plugin Registry --- Key: COUCHDB-2015 URL: https://issues.apache.org/jira/browse/COUCHDB-2015 Project: CouchDB Issue Type: New Feature Components: Plugins Reporter: Jan Lehnardt - Instead of hardcoding a list of plugins into Futon/Fauxton, we load a list of applicable plugins from a central (and configurable) plugins repository. - This allows plugin authors to publish new plugins and new versions of existing plugins independently. The focus should be on ease of publishing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-2008) CI: Create repeatable images for different operating system configurations
[ https://issues.apache.org/jira/browse/COUCHDB-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13857664#comment-13857664 ] Jan Lehnardt commented on COUCHDB-2008: --- [~crigor] sounds good, looking forward to your Ansible work! We can get new VMs with any OS configuration on demand, I opted for the older ones since they are still supported and the current CI system and many devs cover the current ubuntus. CI: Create repeatable images for different operating system configurations -- Key: COUCHDB-2008 URL: https://issues.apache.org/jira/browse/COUCHDB-2008 Project: CouchDB Issue Type: New Feature Components: Infrastructure Reporter: Jan Lehnardt Dave suggests to look into Ansible Packer to create images for CI. The idea would be to run these configurations on the CI server and controlled by Jenkins. It should also be possible to set up a configuration locally that matches the CI box exactly, so one can do debugging under the same circumstances the CI server runs tests. The requirement for now is that it all needs to work with a set of VMs that we have SSH access to, but we don’t manage the host. I’m not too familiar with all the automation and VM image packaging tools, so I don’t know if one or another is useful in that context or not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-1946) Trying to replicate NPM grinds to a halt after 40GB
[ https://issues.apache.org/jira/browse/COUCHDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13857118#comment-13857118 ] Jan Lehnardt commented on COUCHDB-1946: --- [~indexzero] Are you replicating form a single CouchDB or through a load balancer with multiple couches behind it? Trying to replicate NPM grinds to a halt after 40GB --- Key: COUCHDB-1946 URL: https://issues.apache.org/jira/browse/COUCHDB-1946 Project: CouchDB Issue Type: Bug Components: Database Core Reporter: Marc Trudel Attachments: couch.log I have been able to replicate the Node.js NPM database until 40G or so, then I get this: https://gist.github.com/stelcheck/7723362 I one case I have gotten a flat-out OOM error, but I didn't take a dump of the log output at the time. CentOS6.4 with CouchDB 1.5 (also tried 1.3.1, but to no avail). Also tried to restart replication from scratch - twice - bot cases stalling at 40GB. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Closed] (COUCHDB-1987) make distcheck broken
[ https://issues.apache.org/jira/browse/COUCHDB-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt closed COUCHDB-1987. - Resolution: Fixed make distcheck broken - Key: COUCHDB-1987 URL: https://issues.apache.org/jira/browse/COUCHDB-1987 Project: CouchDB Issue Type: Bug Components: Build System Reporter: Jan Lehnardt make[1]: Entering directory `/vagrant/src' make[1]: *** No rule to make target `fauxton/app/initialize.js', needed by `distdir'. Stop. make[1]: Leaving directory `/vagrant/src' make: *** [distdir] Error 1 -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Created] (COUCHDB-2007) Building docs under CI fails
Jan Lehnardt created COUCHDB-2007:
-
Summary: Building docs under CI fails
Key: COUCHDB-2007
URL: https://issues.apache.org/jira/browse/COUCHDB-2007
Project: CouchDB
Issue Type: Bug
Components: Build System, Documentation
Reporter: Jan Lehnardt
Building the PDF version of the docs fails under CI due to release name:
The CouchDB.tex file that gets generated includes a line that looks like this:
\release{1.6.0+build.jenkins-ERLANG_VERSION=R14B04,label=Mac-OS-10-8-2-832-76-g2996574}
It isn’t clear whether this is plain to long or whether there are invalid
characters in there.
I don’t quite understand how docs inherit the release name, so I need to ask
for help here.
The release name otherwise is valid, as it encodes the different build matrix
settings.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (COUCHDB-2007) Building docs under CI fails
[
https://issues.apache.org/jira/browse/COUCHDB-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13852992#comment-13852992
]
Jan Lehnardt commented on COUCHDB-2007:
---
Builds started failing after this commit:
https://github.com/apache/couchdb/commit/dd20a0fa69e85e985f70e65d6291416d529a2c34
— Maybe we can add a clause that strips the long CI name down somehow?
Building docs under CI fails
Key: COUCHDB-2007
URL: https://issues.apache.org/jira/browse/COUCHDB-2007
Project: CouchDB
Issue Type: Bug
Components: Build System, Documentation
Reporter: Jan Lehnardt
Building the PDF version of the docs fails under CI due to release name:
The CouchDB.tex file that gets generated includes a line that looks like this:
\release{1.6.0+build.jenkins-ERLANG_VERSION=R14B04,label=Mac-OS-10-8-2-832-76-g2996574}
It isn’t clear whether this is plain to long or whether there are invalid
characters in there.
I don’t quite understand how docs inherit the release name, so I need to ask
for help here.
The release name otherwise is valid, as it encodes the different build matrix
settings.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (COUCHDB-2007) Building docs under CI fails
[
https://issues.apache.org/jira/browse/COUCHDB-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13853022#comment-13853022
]
Jan Lehnardt commented on COUCHDB-2007:
---
The linked branch includes a commit that does the job, however the python,
while it works, is not a prettiest. I wouldn’t mind an idiomatic adjustment :)
Building docs under CI fails
Key: COUCHDB-2007
URL: https://issues.apache.org/jira/browse/COUCHDB-2007
Project: CouchDB
Issue Type: Bug
Components: Build System, Documentation
Reporter: Jan Lehnardt
Building the PDF version of the docs fails under CI due to release name:
The CouchDB.tex file that gets generated includes a line that looks like this:
\release{1.6.0+build.jenkins-ERLANG_VERSION=R14B04,label=Mac-OS-10-8-2-832-76-g2996574}
It isn’t clear whether this is plain to long or whether there are invalid
characters in there.
I don’t quite understand how docs inherit the release name, so I need to ask
for help here.
The release name otherwise is valid, as it encodes the different build matrix
settings.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Created] (COUCHDB-2008) CI: Create repeatable images for different operating system configurations
Jan Lehnardt created COUCHDB-2008: - Summary: CI: Create repeatable images for different operating system configurations Key: COUCHDB-2008 URL: https://issues.apache.org/jira/browse/COUCHDB-2008 Project: CouchDB Issue Type: New Feature Components: Infrastructure Reporter: Jan Lehnardt Dave suggests to look into Ansible Packer to create images for CI. The idea would be to run these configurations on the CI server and controlled by Jenkins. It should also be possible to set up a configuration locally that matches the CI box exactly, so one can do debugging under the same circumstances the CI server runs tests. The requirement for now is that it all needs to work with a set of VMs that we have SSH access to, but we don’t manage the host. I’m not too familiar with all the automation and VM image packaging tools, so I don’t know if one or another is useful in that context or not. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Closed] (COUCHDB-2007) Building docs under CI fails
[
https://issues.apache.org/jira/browse/COUCHDB-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Lehnardt closed COUCHDB-2007.
-
Resolution: Fixed
Fix Version/s: 1.6.0
1.5.1
Building docs under CI fails
Key: COUCHDB-2007
URL: https://issues.apache.org/jira/browse/COUCHDB-2007
Project: CouchDB
Issue Type: Bug
Components: Build System, Documentation
Reporter: Jan Lehnardt
Fix For: 1.5.1, 1.6.0
Building the PDF version of the docs fails under CI due to release name:
The CouchDB.tex file that gets generated includes a line that looks like this:
\release{1.6.0+build.jenkins-ERLANG_VERSION=R14B04,label=Mac-OS-10-8-2-832-76-g2996574}
It isn’t clear whether this is plain to long or whether there are invalid
characters in there.
I don’t quite understand how docs inherit the release name, so I need to ask
for help here.
The release name otherwise is valid, as it encodes the different build matrix
settings.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (COUCHDB-1986) 04-replication-large-atts.t times out
[ https://issues.apache.org/jira/browse/COUCHDB-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849272#comment-13849272 ] Jan Lehnardt commented on COUCHDB-1986: --- Some more testing today, bumping the test timeout waiting for replication to finish from 30 to 60 makes this pass again. Looking at the request log can see that PUTing docs to the remote db takes disproportionally long (1 minute + a few seconds each): [Mon, 16 Dec 2013 12:23:27 GMT] [info] [0.321.0] 127.0.0.1 - - PUT /couch_test_rep_db_b/_local/8831f8738bd5d17e00da7 1581b23 201 [Mon, 16 Dec 2013 12:24:30 GMT] [info] [0.322.0] 127.0.0.1 - - PUT /couch_test_rep_db_b/doc5?new_edits=false 201 [Mon, 16 Dec 2013 12:25:38 GMT] [info] [0.322.0] 127.0.0.1 - - PUT /couch_test_rep_db_b/doc4?new_edits=false 201 [Mon, 16 Dec 2013 12:26:51 GMT] [info] [0.322.0] 127.0.0.1 - - PUT /couch_test_rep_db_b/doc3?new_edits=false 201 [Mon, 16 Dec 2013 12:27:58 GMT] [info] [0.322.0] 127.0.0.1 - - PUT /couch_test_rep_db_b/doc2?new_edits=false 201 [Mon, 16 Dec 2013 12:29:06 GMT] [info] [0.322.0] 127.0.0.1 - - PUT /couch_test_rep_db_b/doc11?new_edits=false 201 [Mon, 16 Dec 2013 12:30:13 GMT] [info] [0.322.0] 127.0.0.1 - - PUT /couch_test_rep_db_b/doc10?new_edits=false 201 [Mon, 16 Dec 2013 12:31:20 GMT] [info] [0.322.0] 127.0.0.1 - - PUT /couch_test_rep_db_b/doc1?new_edits=false 201 [Mon, 16 Dec 2013 12:31:20 GMT] [info] [0.322.0] 127.0.0.1 - - POST /couch_test_rep_db_b/_ensure_full_commit 201 After the last one all other requests come in at expected speeds. I’m not too familiar with this part of the replicator code, so I couldn’t quickly find why either the requests take that long, or why the replicator waits a minute until it continues. Another thing I tried was seeing whether the new R16B scheduling behaviour introduced this, but running the test with ERL_FLAGS=+sws legacy ./test/etap/run -v src/couch_replicator/test/04-replication-large-atts.t and the original timeout still times out. 04-replication-large-atts.t times out - Key: COUCHDB-1986 URL: https://issues.apache.org/jira/browse/COUCHDB-1986 Project: CouchDB Issue Type: Bug Components: Replication Affects Versions: 1.5.0 Reporter: Jan Lehnardt 04-replication-large-atts.t gets stuck around 558, sometimes a little earlier or later, but it times out eventually, regardless of the timeout. I tried doubling and such. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (COUCHDB-1987) make distcheck broken
[ https://issues.apache.org/jira/browse/COUCHDB-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848582#comment-13848582 ] Jan Lehnardt commented on COUCHDB-1987: --- Noah said he’s going to fix this in his build-fauxton branch, but I’d like to explain this for posterity. make indeed throws this error, I didn’t suggest it is a grunt.js error, but it’s the way we use Autotools. In particular, in how we use Autotools for packaging. Each file that is supposed to end up in our final apache-couchdb-x.y.z.tar.gz tarball needs to be marked as such in a Makefile.am. For Fauxton that means we maintain a list of Fauxton files in src/Makefila.am specified as `FAUXTON_FILES`: FAUXTON_FILES = \ fauxton/app/addons/activetasks/assets/less/activetasks.less \ fauxton/app/addons/activetasks/base.js \ fauxton/app/addons/activetasks/resources.js \ …and so on In that list, there is a line: `fauxton/app/initialize.js`, e.g. where we say that file is supposed to go into the release tarball. That file however does not exist at the time `make dist` is run, so make complains. I assume the solution is that make needs to call grunt at some point to turn fauxton/app/initialize.underscore into fauxton/app/initialize.js and I think that is what Noah is doing in his branch. I just wanted to explain, why this isn’t a make error. make distcheck broken - Key: COUCHDB-1987 URL: https://issues.apache.org/jira/browse/COUCHDB-1987 Project: CouchDB Issue Type: Bug Components: Build System Reporter: Jan Lehnardt make[1]: Entering directory `/vagrant/src' make[1]: *** No rule to make target `fauxton/app/initialize.js', needed by `distdir'. Stop. make[1]: Leaving directory `/vagrant/src' make: *** [distdir] Error 1 -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Created] (COUCHDB-1999) Merge BigCouch and rcouch
Jan Lehnardt created COUCHDB-1999:
-
Summary: Merge BigCouch and rcouch
Key: COUCHDB-1999
URL: https://issues.apache.org/jira/browse/COUCHDB-1999
Project: CouchDB
Issue Type: New Feature
Reporter: Jan Lehnardt
This issue is the master issue to track the BigCouch and rcouch.
{noformat}
rebase
nebraska master done
+---x+--++X---
|| ||+
|| |||
integration |v vvv
+---x+
| ^^ ^ ^ ^ |
| || | | | |
rcouch | || | | + |
+---x-++---+---+--X--|
||
||
master ||
+---xv
merge to master
{noformat}’
via http://www.asciiflow.com/#Draw3086755538105541081/1497478309
1. nebraska and rcouch branches rebase from master from tag merge-rebase-target
/ d5f0976.
2. nebraska continues with its work and periodically pushes to integration.
3. rcouch continues with its work and periodically pushes to integration.
4. at point `done`, both rcouch and nebraska can verify all their stuff works.
5. at point `merge to master` the end result goes into CouchDB master and we
can make a release.
Please link all related issues with this one.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (COUCHDB-1993) Make fabric work with the refactored view engine
[ https://issues.apache.org/jira/browse/COUCHDB-1993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt updated COUCHDB-1993: -- Component/s: (was: Database Core) BigCouch Make fabric work with the refactored view engine Key: COUCHDB-1993 URL: https://issues.apache.org/jira/browse/COUCHDB-1993 Project: CouchDB Issue Type: Improvement Components: BigCouch Reporter: Volker Mische -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Assigned] (COUCHDB-1992) move build to an Erlang release
[ https://issues.apache.org/jira/browse/COUCHDB-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt reassigned COUCHDB-1992: - Assignee: Jan Lehnardt move build to an Erlang release --- Key: COUCHDB-1992 URL: https://issues.apache.org/jira/browse/COUCHDB-1992 Project: CouchDB Issue Type: Improvement Reporter: Benoit Chesneau Assignee: Jan Lehnardt Backports changes from rcouch -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (COUCHDB-1992) move build to an Erlang release
[ https://issues.apache.org/jira/browse/COUCHDB-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt updated COUCHDB-1992: -- Component/s: BigCouch move build to an Erlang release --- Key: COUCHDB-1992 URL: https://issues.apache.org/jira/browse/COUCHDB-1992 Project: CouchDB Issue Type: Improvement Components: BigCouch Reporter: Benoit Chesneau Assignee: Jan Lehnardt Backports changes from rcouch -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (COUCHDB-1991) improve supervision of o=couchdb
[ https://issues.apache.org/jira/browse/COUCHDB-1991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lehnardt updated COUCHDB-1991: -- Component/s: BigCouch improve supervision of o=couchdb Key: COUCHDB-1991 URL: https://issues.apache.org/jira/browse/COUCHDB-1991 Project: CouchDB Issue Type: Improvement Components: BigCouch Reporter: Benoit Chesneau Improve the supervision by backporting changes from rcouch mixed with the nebraska branch -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (COUCHDB-1999) Merge BigCouch and rcouch
[
https://issues.apache.org/jira/browse/COUCHDB-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Lehnardt updated COUCHDB-1999:
--
Description:
This issue is the master issue to track the BigCouch and rcouch.
{noformat}
rebase
nebraska master done
+---x+--++X---
|| ||+
|| |||
integration |v vvv
+---x+
| ^^ ^ ^ ^ |
| || | | | |
rcouch | || | | + |
+---x-++---+---+--X--|
||
||
master ||
+---xv
merge to master
{noformat}
via http://www.asciiflow.com/#Draw3086755538105541081/1497478309
1. nebraska and rcouch branches rebase from master from tag merge-rebase-target
/ d5f0976.
2. nebraska continues with its work and periodically pushes to integration.
3. rcouch continues with its work and periodically pushes to integration.
4. at point `done`, both rcouch and nebraska can verify all their stuff works.
5. at point `merge to master` the end result goes into CouchDB master and we
can make a release.
Please link all related issues with this one.
was:
This issue is the master issue to track the BigCouch and rcouch.
{noformat}
rebase
nebraska master done
+---x+--++X---
|| ||+
|| |||
integration |v vvv
+---x+
| ^^ ^ ^ ^ |
| || | | | |
rcouch | || | | + |
+---x-++---+---+--X--|
||
||
master ||
+---xv
merge to master
{noformat}’
via http://www.asciiflow.com/#Draw3086755538105541081/1497478309
1. nebraska and rcouch branches rebase from master from tag merge-rebase-target
/ d5f0976.
2. nebraska continues with its work and periodically pushes to integration.
3. rcouch continues with its work and periodically pushes to integration.
4. at point `done`, both rcouch and nebraska can verify all their stuff works.
5. at point `merge to master` the end result goes into CouchDB master and we
can make a release.
Please link all related issues with this one.
Merge BigCouch and rcouch
-
Key: COUCHDB-1999
URL: https://issues.apache.org/jira/browse/COUCHDB-1999
Project: CouchDB
Issue Type: New Feature
Reporter: Jan Lehnardt
This issue is the master issue to track the BigCouch and rcouch.
{noformat}
rebase
nebraska master done
+---x+--++X---
|| ||+
|| |||
integration |v vvv
+---x+
| ^^ ^ ^ ^ |
| || | | | |
rcouch | || | | + |
+---x-++---+---+--X--|
||
||
master ||
+---xv
merge to master
{noformat}
via http://www.asciiflow.com/#Draw3086755538105541081/1497478309
1. nebraska and rcouch branches rebase from master from tag
merge-rebase-target / d5f0976.
2. nebraska continues with its work and periodically pushes to integration.
3. rcouch continues with its work and periodically pushes to integration.
4. at point `done`, both rcouch and nebraska can verify all their stuff works.
5. at point `merge
