Re: [VOTE] Apache CouchDB 1.0.2 Release, Round 2
For goodness' sake, release it already! ;) +1 On Mon, Jan 10, 2011 at 9:01 PM, Paul Davis wrote: > This is the second release vote for Apache CouchDB 1.0.2 > > Changes since the last round: > > * Fix share/www/image/spinner.gif > * Fix OOM error when compacting documents with many conflicts > * Upgraded ibrowse to 2.1.2 to fix more replicator bugs > * Fix attachment compression for MIME types with parameters > * Fix replicator to respect HTTP settings in the configuration > * Fixed spurious replicator bug due to ibrowse dropping connections > * Fix for frequently edited documents in multi-master deployments > being duplicated in _changes and _all_docs [See COUCHDB-968]. > * Detect corrupt views due to document duplication bug and warn > that the view should be rebuilt [See COUCHDB-999]. > > We apologize for the delay due to fixing COUCHDB-968 but we felt it was > sufficiently serious that it warranted an immediate fix. > > We encourage the whole community to download and test these release > artifacts so that any critical issues can be resolved before the release > is made. Everyone is free to vote on this release. Please report your > results and vote to this thread. > > We are voting on the following release artifacts: > > http://people.apache.org/~davisp/dist/1.0.2/ > > These artifacts have been built from the 1.0.2 tag in Subversion: > > http://svn.apache.org/repos/asf/couchdb/tags/1.0.2/ > > Release the voters! >
[VOTE] Apache CouchDB 1.0.2 Release, Round 2
This is the second release vote for Apache CouchDB 1.0.2 Changes since the last round: * Fix share/www/image/spinner.gif * Fix OOM error when compacting documents with many conflicts * Upgraded ibrowse to 2.1.2 to fix more replicator bugs * Fix attachment compression for MIME types with parameters * Fix replicator to respect HTTP settings in the configuration * Fixed spurious replicator bug due to ibrowse dropping connections * Fix for frequently edited documents in multi-master deployments being duplicated in _changes and _all_docs [See COUCHDB-968]. * Detect corrupt views due to document duplication bug and warn that the view should be rebuilt [See COUCHDB-999]. We apologize for the delay due to fixing COUCHDB-968 but we felt it was sufficiently serious that it warranted an immediate fix. We encourage the whole community to download and test these release artifacts so that any critical issues can be resolved before the release is made. Everyone is free to vote on this release. Please report your results and vote to this thread. We are voting on the following release artifacts: http://people.apache.org/~davisp/dist/1.0.2/ These artifacts have been built from the 1.0.2 tag in Subversion: http://svn.apache.org/repos/asf/couchdb/tags/1.0.2/ Release the voters!
[jira] Resolved: (COUCHDB-998) aggressive use of encodeURIComponent on view names inside Futon
[ https://issues.apache.org/jira/browse/COUCHDB-998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Newson resolved COUCHDB-998. --- Resolution: Fixed Jan committed a fix for this bug to 1.0.x, 1.1.x and trunk. > aggressive use of encodeURIComponent on view names inside Futon > --- > > Key: COUCHDB-998 > URL: https://issues.apache.org/jira/browse/COUCHDB-998 > Project: CouchDB > Issue Type: Bug > Components: Futon >Affects Versions: 1.0, 1.0.1, 1.0.2, 1.1, 1.2, 2.0 >Reporter: Gordon Stratton >Assignee: Robert Newson >Priority: Minor > Fix For: 1.0.2, 1.1, 1.2, 2.0 > > Attachments: futon.browse.js.patch > > Original Estimate: 0.08h > Remaining Estimate: 0.08h > > Steps to reproduce: > 1. Enter a database containing a design document view > 2. Select one of the design document views from the view list dropdown > 3. Navigate back to the Overview section > 4. Navigate back to the database you were just in > As you can tell by looking at the Location bar, the view name has had > encodeURIComponent applied to it, and so the view won't be found by CouchDB. > I'm going to attach a patch that fixes the issue for me, but it needs review > in case the fix needs to happen in some other place inside Futon. I'm judging > this patch based on the way the view switching works in the view list > dropdown. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (COUCHDB-998) aggressive use of encodeURIComponent on view names inside Futon
[ https://issues.apache.org/jira/browse/COUCHDB-998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Newson reassigned COUCHDB-998: - Assignee: Robert Newson > aggressive use of encodeURIComponent on view names inside Futon > --- > > Key: COUCHDB-998 > URL: https://issues.apache.org/jira/browse/COUCHDB-998 > Project: CouchDB > Issue Type: Bug > Components: Futon >Affects Versions: 1.0, 1.0.1, 1.0.2, 1.1, 1.2, 2.0 >Reporter: Gordon Stratton >Assignee: Robert Newson >Priority: Minor > Fix For: 1.0.2, 1.1, 1.2, 2.0 > > Attachments: futon.browse.js.patch > > Original Estimate: 0.08h > Remaining Estimate: 0.08h > > Steps to reproduce: > 1. Enter a database containing a design document view > 2. Select one of the design document views from the view list dropdown > 3. Navigate back to the Overview section > 4. Navigate back to the database you were just in > As you can tell by looking at the Location bar, the view name has had > encodeURIComponent applied to it, and so the view won't be found by CouchDB. > I'm going to attach a patch that fixes the issue for me, but it needs review > in case the fix needs to happen in some other place inside Futon. I'm judging > this patch based on the way the view switching works in the view list > dropdown. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
new replicator
Hi all,
It's been some months now that I resumed Damien's initial work on a
new replicator. It's now in a state that's perfectly usable
(hopefully), functional and with all the features of the current
replicator.
Its code is at:
https://github.com/fdmanana/couchdb/compare/trunk_new_replicator
(the ASF's svn branch "new_replicator" is too aged now and has several
issues with pull replications)
What it brings to the table:
- When new revisions of a document need to be replicated to the
target, it only replicates the attachments introduced in the missing
revisions, that is, unlike the current replicator it doesn't replicate
all the attachments (introduced since the first revision) - this makes
a huge difference for databases with many and/or large attachments;
- It exploits parallelism more aggressively - the current replicator
uses a single process for finding which revisions are missing in the
target, a single process to write documents to the target and for push
replications a single process that reads documents from the local
source (for pull replications it spawns several processes to read
documents from the remote source). The new replicator uses several (#
is configurable) processes to find the missing revisions and copy
documents (read from source and write to target);
- Progress is done faster when attachments are present. For a batch of
documents, the current replicator fetches first the body of the
documents and only when it decides to flush them to disk, for each
attachment it opens a separate connection to download each. This means
that only after all the attachments for the documents in the batch are
fetched, a checkpoint is done. The new replicator fetches documents
together with their attachments (in compressed form if they're
compressed at the source) in a single request and flushes them as soon
as possible;
- Better error isolation. Currently, HTTP connections are shared by
different replications. This behaviour is more like a "side effect" of
using ibrowse's load balancer. The new replicator has its own load
balancer implementation which ensures that all the requests in the
pipeline of a connection belong to the same replication;
- It was completely rewritten from scratch. Hopefully the code is
better organised now and slightly shorter;
- Better logging (more meaningful information about connectivity
errors and document write failures) and integration with the
replicator database.
There are now some new .ini replicator configuration options (
https://github.com/fdmanana/couchdb/blob/trunk_new_replicator/etc/couchdb/default.ini.tpl.in#L138
) under the [replicator] section:
"worker_processes" - the number of processes that copy documents from
the source database to the target database. For each one of them, a
missing revisions process is also spawned. For example, if set to 4
(default) we get 8 processes: 4 for copying documents and 4 to find
which revisions are missing in the target;
"worker_batch_size" - the maximum number of consecutive source
sequence numbers each worker processes at once. Default value is 1000.
Lower values mean that checkpoints are done more frequently and are
recommended when the source has very large documents and/or most
documents with many and/or large attachments. Higher values can make
the replicator process faster only when most documents are very small
(below a few kilobytes) and there are none or very few with
attachments;
"http_connections" - the maximum number of HTTP connections per replication;
"http_pipeline_size" - the maximum pipeline size for each HTTP connection;
"connection_timeout" - the period of time (in milliseconds) after
which a connection is considered dead because no data was received
(and up to 10 retries are done). The default value is 30 000
milliseconds;
"socket_options" - options to pass to the TCP sockets. All the
available options are listed in the man page of the Erlang module
'inet' (as well as in the man page for the system call setsockopt).
Some of these options, such as priority, sndbuf and recbuf, can make a
significant difference if set to the correct values, which depend on
the OS, hardware and network characteristics - a good tutorial for
this can be found at:
http://www.ibm.com/developerworks/linux/library/l-hisock.html
Any of these options can also be specified for individual replications
in the replication document/object. Example:
$ curl -H 'Content-Type: application/json' -X POST
http://myserver.com/_replicate -d '{ "source": "http://abc.com/foo";,
"target": "bar", "worker_processes": 6, "http_connections": 20,
"http_pipeline_size": 100}
Now, for a comparison with the current replicator, here are some
results for replications done from scratch (empty target) in a local
Wifi network and from my laptop (Europe, Portugal) to my CouchOne
account ( http://fdmanana.couchone.com/_utils/ ). Time was measured
using the 'time' command like this:
$ time curl -X POST http://foobar.com/_replicate .
--- P
couchdb rebar version based on bigcouch
Hi all, On another work that don't use it any more I've made a rebared version of couchdb based on bigcouch. It's a trunk version of couchdb. You can find it here : https://github.com/benoitc/couchdb/tree/rebar or : https://github.com/benoitc/couchdb/tree/rebar-split Second version is a spitted version of couchdb using davisp' script. For now couch_view isn't splitted because couch_db rely on it to make doc validation (js function using couch_query_servers) . This couchdb version could be use in environnement that need a more erlangish way to montior their application, or simply embed couchdb in your own app. I'm not sure how much time I will put on this during next days, so I'm releasing it hoping it could be useful for someone. Enjoy, - benoît
[jira] Updated: (COUCHDB-902) Attachments that have recovered from conflict do not accept attachments.
[ https://issues.apache.org/jira/browse/COUCHDB-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Klaus Trainer updated COUCHDB-902: -- Attachment: 0001-Fix-COUCHDB-902.patch > Attachments that have recovered from conflict do not accept attachments. > > > Key: COUCHDB-902 > URL: https://issues.apache.org/jira/browse/COUCHDB-902 > Project: CouchDB > Issue Type: Bug > Components: Database Core > Environment: trunk >Reporter: Paul Joseph Davis >Priority: Critical > Attachments: 0001-Fix-COUCHDB-902.patch, couchdb-902-test-case.py > > > Apparently if a document has been in a conflict, they will reject requests to > add an attachment with a conflict error. > I've tracked this down to couch_db_updater.erl line 501, but I'm not too > familiar with this part of the code so I figured I'd fill out a ticket in > case anyone else can go through this more quickly than me. > Sure would be nice if I could attach a file when I create an issue... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-902) Attachments that have recovered from conflict do not accept attachments.
[ https://issues.apache.org/jira/browse/COUCHDB-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Klaus Trainer updated COUCHDB-902: -- Attachment: (was: 0001-Fix-COUCHDB-902.patch) > Attachments that have recovered from conflict do not accept attachments. > > > Key: COUCHDB-902 > URL: https://issues.apache.org/jira/browse/COUCHDB-902 > Project: CouchDB > Issue Type: Bug > Components: Database Core > Environment: trunk >Reporter: Paul Joseph Davis >Priority: Critical > Attachments: 0001-Fix-COUCHDB-902.patch, couchdb-902-test-case.py > > > Apparently if a document has been in a conflict, they will reject requests to > add an attachment with a conflict error. > I've tracked this down to couch_db_updater.erl line 501, but I'm not too > familiar with this part of the code so I figured I'd fill out a ticket in > case anyone else can go through this more quickly than me. > Sure would be nice if I could attach a file when I create an issue... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: rewriter needed changes
On Mon, Jan 10, 2011 at 12:03 PM, Volker Mische wrote: > > I'm +1 for a more powerful rewriter. Though I haven't quite understood how > those mongrel2 style rewrites will actually look like. I understand how to > match a pattern, but how is it rewritten after that? > Each patterns become a var you can reuse in your rewritten url. Following discussion on irc, I started an implementation as a couchdb module, so it will be more easier to understand. Hopefully I will have something to show on wed. - benoît
[jira] Updated: (COUCHDB-902) Attachments that have recovered from conflict do not accept attachments.
[ https://issues.apache.org/jira/browse/COUCHDB-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Klaus Trainer updated COUCHDB-902: -- Attachment: (was: 0001-Fix-COUCHDB-902.patch) > Attachments that have recovered from conflict do not accept attachments. > > > Key: COUCHDB-902 > URL: https://issues.apache.org/jira/browse/COUCHDB-902 > Project: CouchDB > Issue Type: Bug > Components: Database Core > Environment: trunk >Reporter: Paul Joseph Davis >Priority: Critical > Attachments: 0001-Fix-COUCHDB-902.patch, couchdb-902-test-case.py > > > Apparently if a document has been in a conflict, they will reject requests to > add an attachment with a conflict error. > I've tracked this down to couch_db_updater.erl line 501, but I'm not too > familiar with this part of the code so I figured I'd fill out a ticket in > case anyone else can go through this more quickly than me. > Sure would be nice if I could attach a file when I create an issue... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-902) Attachments that have recovered from conflict do not accept attachments.
[ https://issues.apache.org/jira/browse/COUCHDB-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Klaus Trainer updated COUCHDB-902: -- Attachment: 0001-Fix-COUCHDB-902.patch Here is a new version of my patch I'd propose. In contrast to the previous version, this one seems to be fine. > Attachments that have recovered from conflict do not accept attachments. > > > Key: COUCHDB-902 > URL: https://issues.apache.org/jira/browse/COUCHDB-902 > Project: CouchDB > Issue Type: Bug > Components: Database Core > Environment: trunk >Reporter: Paul Joseph Davis >Priority: Critical > Attachments: 0001-Fix-COUCHDB-902.patch, couchdb-902-test-case.py > > > Apparently if a document has been in a conflict, they will reject requests to > add an attachment with a conflict error. > I've tracked this down to couch_db_updater.erl line 501, but I'm not too > familiar with this part of the code so I figured I'd fill out a ticket in > case anyone else can go through this more quickly than me. > Sure would be nice if I could attach a file when I create an issue... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: rewriter needed changes
Hi, On 10.01.2011 01:32, Benoit Chesneau wrote: 2. Use mongrel2 pattern matching: URL patterns always match from the start, routes are broken into prefix and pattern part. We uses the routes to find the longest matching prefix and then tests the pattern. If the pattern matches, then the route works. If the route doesn't have a pattern, then it's assumed to match, and you're done. The only caveat is you have to wrap your pattern parts in parenthesis, but these don't mean anything other than to delimit where a pattern starts. So instead of /images/.⋆.jpg, write /images/(.⋆.jpg) for it to work. [...] This solution is really simple, remove the useless things you have in regexp and give complete power to the users. Also this kind of parsing is relatively easy to do in erlang. [...] > Any thoughts ? - benoît I'm +1 for a more powerful rewriter. Though I haven't quite understood how those mongrel2 style rewrites will actually look like. I understand how to match a pattern, but how is it rewritten after that? Cheers, Volker
