[jira] [Commented] (COUCHDB-1946) Trying to replicate NPM grinds to a halt after 40GB

[Emilien Kenler (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13844045#comment-13844045
 ] 

Emilien Kenler commented on COUCHDB-1946:
-

Hi,

As I said, I was able to replicate the npm registry with only 1GB of memory on 
CouchDB 1.2.0.
Even if the replication works on CouchDB 1.5.0, it should work with a similar 
amount of memory.
I don't think that it's an Erlang related issue, because I use the same version 
of Erlang for my tests.
I use the default configuration of CouchDB, so I think it's strange to have to 
change something to make the replication work on 1.5.0 if it works without any 
change on 1.2.0.

I can provide access to my two test VM, and I can provide new VM if needed.

> Trying to replicate NPM grinds to a halt after 40GB
> ---
>
> Key: COUCHDB-1946
> URL: https://issues.apache.org/jira/browse/COUCHDB-1946
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Marc Trudel
> Attachments: couch.log
>
>
> I have been able to replicate the Node.js NPM database until 40G or so, then 
> I get this:
> https://gist.github.com/stelcheck/7723362
> I one case I have gotten a flat-out OOM error, but I didn't take a dump of 
> the log output at the time.
> CentOS6.4 with CouchDB 1.5 (also tried 1.3.1, but to no avail). Also tried to 
> restart replication from scratch - twice - bot cases stalling at 40GB.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Dale Harvey (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843730#comment-13843730
 ] 

Dale Harvey commented on COUCHDB-1950:
--

So the problem being that in a system where we manage to get conflicts to 
happen simultaneously on seperate nodes, instead of having a 3 way merge with 
the original documents, this will result in a 2 way merge of the result of the 
'resolved' documents (potentially lost information). That is entirely true of 
how we currently document how conflict resolution should be done now.

There isnt a problem with this suggestion per se, more that its just exposing 
what is already a 'flaw' in the current documented implementation.

A read resolver would be a much easier / safer thing to implement, however it 
comes with its own problems, to make a read resolver any way API consistent 
between all_docs / views / changes etc will mean a huge increase in the number 
of disk fetches each api call has to make, as it now needs to fetch each 
conflicting document and run them through a view engine 

Currently we hide all these complexities behind the winning revision mechanism, 
this is hiding what most people to consider CouchDB's killer feature so I do 
think its worth figuring out how to get it right

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Nathan Vander Wilt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843678#comment-13843678
 ] 

Nathan Vander Wilt commented on COUCHDB-1950:
-

So the book (http://guide.couchdb.org/draft/conflicts.html) and the wiki 
(http://wiki.apache.org/couchdb/Replication_and_conflicts) both tell me as a 
user to do ± the following:

1. `GET somedoc?conflicts=true`
2. if there are conflicts, figure out what document state one the *app* wants 
to win, making it so by (optionally) writing a new revision of one of the leafs 
and then `DELETE somedoc?rev=unwanted` all the others

This glosses over the resolution strategy and any error handling for the 
writes, but I think is a fair summary of what I was told to do as an app 
developer?

Basically my job as an app developer is to (correctly…NOT glossing over error 
handling!) write the boilerplate that turns:

{code}
  ,--> r2a -> r3a
r1 --> r2b -> r3b
  `--> r2c -> r3c
{code}

into:

{code}
  ,--> r2a -> r3a
r1 --> r2b -> r3b -> (r4b deleted)
  `--> r2c -> r3c -> (r4c deleted)
{code}

…or:
{code}
  ,--> r2a -> r3a -> r4a
r1 --> r2b -> r3b -> (r4b deleted)
  `--> r2c -> r3c -> (r4c deleted)
{code}

Again, this is correct, right?


If this is how I should be handling conflicts, then — and AFAICT this is the 
same point others have been trying to make — what is so problematic about 
having People Who Understand CouchDB enapsulate the proper "detect+resolve 
conflicts" meta-logic in such a way that all People Like Me have to write is 
the app-specific logic as a JavaScript callback?

Isn't this what we already have for map/reduce, for validate_doc_update, for 
update handlers…? Putting the logic closer to the data, abstracting it in such 
a way as to corral implementations towards correctness, paving the cowpaths so 
as to reduce boilerplate — how would doing the same for conflict resolution be 
problematic?

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1946) Trying to replicate NPM grinds to a halt after 40GB

[Dave Cottlehuber (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843588#comment-13843588
 ] 

Dave Cottlehuber commented on COUCHDB-1946:
---

I'm pretty much finished a full replication of the registry with 1.5.0, 
multiple pipelines etc, filling my adsl link as much as possible, and I've not 
had any issues. Maximum RAM went up to ~ 12GiB (I have plenty to spare) but no 
hitches, need to copy individual docs across, nor restart replication, despite 
many transient network issues & apparent npm disconnects.

What's people's thoughts on this?

- is this a bug or simply we propose some tuning enhancements?
- we could clearly reduce memory consumption though as [~kocolosk] pointed out 
this will likely come with a cpu hit


> Trying to replicate NPM grinds to a halt after 40GB
> ---
>
> Key: COUCHDB-1946
> URL: https://issues.apache.org/jira/browse/COUCHDB-1946
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Marc Trudel
> Attachments: couch.log
>
>
> I have been able to replicate the Node.js NPM database until 40G or so, then 
> I get this:
> https://gist.github.com/stelcheck/7723362
> I one case I have gotten a flat-out OOM error, but I didn't take a dump of 
> the log output at the time.
> CentOS6.4 with CouchDB 1.5 (also tried 1.3.1, but to no avail). Also tried to 
> restart replication from scratch - twice - bot cases stalling at 40GB.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-348) / should redirect to a human readable page when accessed from a browser

[Joan Touzet (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843559#comment-13843559
 ] 

Joan Touzet commented on COUCHDB-348:
-

Maybe we "had this backwards" from the start, but it's too late to change it 
now.

We know for a fact that your suggestion breaks some programmatic clients 
hitting /. Earlier this year, [~rnewson] and [~drsm79] I believe had a list of 
specific clients that broke on this, and it's a non-insignificant percentage of 
the world. It would also break naive curl samples and how-tos that have been 
around for years - it's way, way more than just Futon and Fauxton that hit / !

> / should redirect to a human readable page when accessed from a browser
> ---
>
> Key: COUCHDB-348
> URL: https://issues.apache.org/jira/browse/COUCHDB-348
> Project: CouchDB
>  Issue Type: Improvement
>  Components: HTTP Interface
>Reporter: Chris Anderson
>Assignee: Robert Newson
> Attachments: COUCHDB_348_01.patch, COUCHDB_348_02.patch, 
> COUCHDB_348_03.patch, encs.erl
>
>
> As more people are adopting CouchDB, the number of questions along the lines 
> of "All I see is {"couchdb":"Welcome","version":"0.10.0a773399"}, what now?"
> If this page redirected to /_utils/ (or maybe something more user-focussed) 
> when the accept headers included html, we could make the first-time user 
> experience more palatable.
> There's some code in COUCHDB-234 that might be helpful here.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1955) Logs should use UTC timestamps

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843498#comment-13843498
 ] 

Jan Lehnardt commented on COUCHDB-1955:
---

Also: RFC 822 is from August 13, 1982 :)

This isn’t to say that any one format is incorrect. But I believe UTC is more 
commonly used for computer systems these days.


> Logs should use UTC timestamps
> --
>
> Key: COUCHDB-1955
> URL: https://issues.apache.org/jira/browse/COUCHDB-1955
> Project: CouchDB
>  Issue Type: Bug
>Reporter: Jan Lehnardt
>
> We currently log GMT while the rest of the world uses UTC (IIRC).
> This ticket is to figure out if that’s actually true and if it is, what we 
> can do to transition.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1955) Logs should use UTC timestamps

[Alexander Shorin (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843494#comment-13843494
 ] 

Alexander Shorin commented on COUCHDB-1955:
---

I think this issue also depends from COUCHDB-1401. Having better logging 
subsystem will allow everyone use those log format that they like/need and this 
issue wouldn't break too much things.

But anyway, currently CouchDB 
[logs|https://github.com/apache/couchdb/blob/master/src/couchdb/couch_log.erl#L210]
 are using 
[timestamp|https://github.com/apache/couchdb/blob/master/src/couchdb/couch_util.erl#L450]
 in [RFC-1123|http://tools.ietf.org/html/rfc1123.html#section-5.2.14] format 
which based on [RFC-822|http://tools.ietf.org/html/rfc822.html#section-5.1] 
that doesn't have any UTC, but GMT and 
[UT|https://en.wikipedia.org/wiki/Universal_Time]. UT seems to be also 
different from UTC in details.

> Logs should use UTC timestamps
> --
>
> Key: COUCHDB-1955
> URL: https://issues.apache.org/jira/browse/COUCHDB-1955
> Project: CouchDB
>  Issue Type: Bug
>Reporter: Jan Lehnardt
>
> We currently log GMT while the rest of the world uses UTC (IIRC).
> This ticket is to figure out if that’s actually true and if it is, what we 
> can do to transition.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1955) Logs should use UTC timestamps

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843467#comment-13843467
 ] 

Jan Lehnardt commented on COUCHDB-1955:
---

For reference: https://en.wikipedia.org/wiki/UTC and 
https://en.wikipedia.org/wiki/GMT

> Logs should use UTC timestamps
> --
>
> Key: COUCHDB-1955
> URL: https://issues.apache.org/jira/browse/COUCHDB-1955
> Project: CouchDB
>  Issue Type: Bug
>Reporter: Jan Lehnardt
>
> We currently log GMT while the rest of the world uses UTC (IIRC).
> This ticket is to figure out if that’s actually true and if it is, what we 
> can do to transition.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Created] (COUCHDB-1955) Logs should use UTC timestamps

[Jan Lehnardt (JIRA)]

Jan Lehnardt created COUCHDB-1955:
-

 Summary: Logs should use UTC timestamps
 Key: COUCHDB-1955
 URL: https://issues.apache.org/jira/browse/COUCHDB-1955
 Project: CouchDB
  Issue Type: Bug
Reporter: Jan Lehnardt


We currently log GMT while the rest of the world uses UTC (IIRC).

This ticket is to figure out if that’s actually true and if it is, what we can 
do to transition.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1955) Logs should use UTC timestamps

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843463#comment-13843463
 ] 

Jan Lehnardt commented on COUCHDB-1955:
---

Changing the log timestamp time zone could be considered a backwards 
compatibility break.

> Logs should use UTC timestamps
> --
>
> Key: COUCHDB-1955
> URL: https://issues.apache.org/jira/browse/COUCHDB-1955
> Project: CouchDB
>  Issue Type: Bug
>Reporter: Jan Lehnardt
>
> We currently log GMT while the rest of the world uses UTC (IIRC).
> This ticket is to figure out if that’s actually true and if it is, what we 
> can do to transition.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1303) Add a _bulk_update handler similar to _update but for bulk document changes

[Benjamin Young (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843302#comment-13843302
 ] 

Benjamin Young commented on COUCHDB-1303:
-

No, this would (if built) handle multiple documents sent in likely in a format 
similar to what {{_bulk_docs}} receives and process them in the same manner, 
but with a JS function sitting in front.

The "wrapper" comment was merely about the JS function wrapping the Erlang that 
does the {{_bulk_docs}} processing. Nothing more, and certainly with no 
requirements outside of CouchDB.

There are ways (like what you describe) to get close to that now, but that 
conversation's best for the user's email list. Thanks!

> Add a _bulk_update handler similar to _update but for bulk document changes
> ---
>
> Key: COUCHDB-1303
> URL: https://issues.apache.org/jira/browse/COUCHDB-1303
> Project: CouchDB
>  Issue Type: New Feature
>Reporter: Benjamin Young
>  Labels: api, update_request_handler
>
> _update handlers are great (and getting better!) for building RESTful API's 
> inside CouchDB. One limitation I found tonight is that _update can only do a 
> single document at a time. If the API I'm building needs to update multiple 
> docs (in a similar fashion to _bulk_docs), then an outside "proxy" script is 
> required. It would be ideal to have a _bulk_update handler to allow for the 
> same functionality as _update, but with the ability to insert multiple 
> documents at once.
> Perhaps the current _update handler API could be extended to support multiple 
> IDs/documents, but a separate API endpoint would be seem reasonable if needed.
> Thanks for considering this idea.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1303) Add a _bulk_update handler similar to _update but for bulk document changes

[Ari Najarian (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843295#comment-13843295
 ] 

Ari Najarian commented on COUCHDB-1303:
---

Thanks, all, for revisiting this discussion.

Just so I'm clear, what Benjamin is proposing (the wrapper for _bulk_docs) 
would still require some kind of middleware in order to 'prepare' the list of 
documents to send, correct? I'd likely have to get client-side JS / Node / 
Lasso / PHP / whatever to query ddoc/_view/viewname?include_docs=true, grab 
those results, and pipe them into this update wrapper.

Is there a design reason why piping a view directly into the wrapper wouldn't 
work? If it's technically feasible, this would cut down on the round-trips to 
the server (and latency, and bandwidth usage, and overall throughput between 
CouchDB and the client). It also means less code in less places for developers 
(like me!).

> Add a _bulk_update handler similar to _update but for bulk document changes
> ---
>
> Key: COUCHDB-1303
> URL: https://issues.apache.org/jira/browse/COUCHDB-1303
> Project: CouchDB
>  Issue Type: New Feature
>Reporter: Benjamin Young
>  Labels: api, update_request_handler
>
> _update handlers are great (and getting better!) for building RESTful API's 
> inside CouchDB. One limitation I found tonight is that _update can only do a 
> single document at a time. If the API I'm building needs to update multiple 
> docs (in a similar fashion to _bulk_docs), then an outside "proxy" script is 
> required. It would be ideal to have a _bulk_update handler to allow for the 
> same functionality as _update, but with the ability to insert multiple 
> documents at once.
> Perhaps the current _update handler API could be extended to support multiple 
> IDs/documents, but a separate API endpoint would be seem reasonable if needed.
> Thanks for considering this idea.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1954) Clarify Futon/Fauxton transition

[Benjamin Young (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843249#comment-13843249
 ] 

Benjamin Young commented on COUCHDB-1954:
-

1. we keep them around until the next *major* point release.
2. added some thoughts about {{/}} routing to 
[COUCHDB-348|https://issues.apache.org/jira/browse/COUCHDB-348?focusedCommentId=13843248]
3. we should be sure that Fauxton routes those to their new equivalent 
(ideally) or en lieu of that, not leave the user stranded (at a broken loading 
page or the like), but route them to the default screen (at least).

Just ideas. :)

> Clarify Futon/Fauxton transition
> 
>
> Key: COUCHDB-1954
> URL: https://issues.apache.org/jira/browse/COUCHDB-1954
> Project: CouchDB
>  Issue Type: Task
>  Components: Futon
>Reporter: Jan Lehnardt
>
> At some point Fauxton will replace Futon. There are a few questions that we 
> should clarify before we get there:
> 1. what happens to resources like jquery.js and jquery.couch.js that are 
> currently in well-known places where CouchApps depend on them?
> 2. there are several other tickets regarding serving a human readable HTML 
> page on `/` if the client is clearly a browser. (so people don’t have to type 
> the `/_utils` manually, how does that play into rolling out Fauxton?
> 3. cool URLs don’t change: what happens to links into Futon that no longer 
> work in Fauxton that might be coming in from the docs and other tutorials, 
> including people’s in-house documentation?



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-348) / should redirect to a human readable page when accessed from a browser

[Benjamin Young (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843248#comment-13843248
 ] 

Benjamin Young commented on COUCHDB-348:


What if we had this backwards? What if {{/}} defaulted to {{text/html}} unless 
the client (aka, CouchDB libraries, HTTP libs, etc) explicitly requested 
otherwise?

Browsers, then get Futon/Fauxton handed to them. Clients that hit the {{/}} 
resource (mostly Futon & Fauxton, I'd recon), then be sure they're asking for 
{{application/json}} when hitting that URL. They should be anyway if that's 
what they want. :)

That seems like a lighter, faster fix that won't break (m)any things. :) One 
hopes...

> / should redirect to a human readable page when accessed from a browser
> ---
>
> Key: COUCHDB-348
> URL: https://issues.apache.org/jira/browse/COUCHDB-348
> Project: CouchDB
>  Issue Type: Improvement
>  Components: HTTP Interface
>Reporter: Chris Anderson
>Assignee: Robert Newson
> Attachments: COUCHDB_348_01.patch, COUCHDB_348_02.patch, 
> COUCHDB_348_03.patch, encs.erl
>
>
> As more people are adopting CouchDB, the number of questions along the lines 
> of "All I see is {"couchdb":"Welcome","version":"0.10.0a773399"}, what now?"
> If this page redirected to /_utils/ (or maybe something more user-focussed) 
> when the accept headers included html, we could make the first-time user 
> experience more palatable.
> There's some code in COUCHDB-234 that might be helpful here.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1303) Add a _bulk_update handler similar to _update but for bulk document changes

[Benjamin Young (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843222#comment-13843222
 ] 

Benjamin Young commented on COUCHDB-1303:
-

Good points. :)

Given that, we'd need a straight "wrapper" for the _bulk_docs endpoint, but 
with some user-created handling code:
http://docs.couchdb.org/en/latest/api/database/bulk-api.html#post--db-_bulk_docs

Likely the perfect thing for someone to explore as a plugin. :)

> Add a _bulk_update handler similar to _update but for bulk document changes
> ---
>
> Key: COUCHDB-1303
> URL: https://issues.apache.org/jira/browse/COUCHDB-1303
> Project: CouchDB
>  Issue Type: New Feature
>Reporter: Benjamin Young
>  Labels: api, update_request_handler
>
> _update handlers are great (and getting better!) for building RESTful API's 
> inside CouchDB. One limitation I found tonight is that _update can only do a 
> single document at a time. If the API I'm building needs to update multiple 
> docs (in a similar fashion to _bulk_docs), then an outside "proxy" script is 
> required. It would be ideal to have a _bulk_update handler to allow for the 
> same functionality as _update, but with the ability to insert multiple 
> documents at once.
> Perhaps the current _update handler API could be extended to support multiple 
> IDs/documents, but a separate API endpoint would be seem reasonable if needed.
> Thanks for considering this idea.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1303) Add a _bulk_update handler similar to _update but for bulk document changes

[Robert Newson (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843214#comment-13843214
 ] 

Robert Newson commented on COUCHDB-1303:


validate_doc_update runs on all updates. An update handler is just a function 
that runs *before* a database update, the return value of the update handler 
includes the update to be attempted. This is also why an update handler (for a 
single document) can fail with a 409.

"all or nothing" cannot happen, we are not building a distributed transaction 
engine. :)

> Add a _bulk_update handler similar to _update but for bulk document changes
> ---
>
> Key: COUCHDB-1303
> URL: https://issues.apache.org/jira/browse/COUCHDB-1303
> Project: CouchDB
>  Issue Type: New Feature
>Reporter: Benjamin Young
>  Labels: api, update_request_handler
>
> _update handlers are great (and getting better!) for building RESTful API's 
> inside CouchDB. One limitation I found tonight is that _update can only do a 
> single document at a time. If the API I'm building needs to update multiple 
> docs (in a similar fashion to _bulk_docs), then an outside "proxy" script is 
> required. It would be ideal to have a _bulk_update handler to allow for the 
> same functionality as _update, but with the ability to insert multiple 
> documents at once.
> Perhaps the current _update handler API could be extended to support multiple 
> IDs/documents, but a separate API endpoint would be seem reasonable if needed.
> Thanks for considering this idea.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1303) Add a _bulk_update handler similar to _update but for bulk document changes

[Benjamin Young (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843210#comment-13843210
 ] 

Benjamin Young commented on COUCHDB-1303:
-

Yeah, that needs exploration, discussion, etc. There'd certainly be value in 
{{_update}} (or similar) handling multiple document saving. Likely, there'd 
need to be ways to do some or all of the following:
a. report per-doc failure info
b. handle document-saving in the function (this seems like a Bad Thing due to 
blocking the engine)
c. provide an "all or nothing" style feature (or make it mandatory, maybe) 
where nothing gets stored if they all can't be stored.

Does {{validate_doc_update}} run on {{_update}}?
Are conflicts the key issue (as we'd not have read any previous docs prior to 
write-attempts in the mentioned setup at least)?

The error condition handling is where the complexity/bloat/etc will happen...as 
ever...

> Add a _bulk_update handler similar to _update but for bulk document changes
> ---
>
> Key: COUCHDB-1303
> URL: https://issues.apache.org/jira/browse/COUCHDB-1303
> Project: CouchDB
>  Issue Type: New Feature
>Reporter: Benjamin Young
>  Labels: api, update_request_handler
>
> _update handlers are great (and getting better!) for building RESTful API's 
> inside CouchDB. One limitation I found tonight is that _update can only do a 
> single document at a time. If the API I'm building needs to update multiple 
> docs (in a similar fashion to _bulk_docs), then an outside "proxy" script is 
> required. It would be ideal to have a _bulk_update handler to allow for the 
> same functionality as _update, but with the ability to insert multiple 
> documents at once.
> Perhaps the current _update handler API could be extended to support multiple 
> IDs/documents, but a separate API endpoint would be seem reasonable if needed.
> Thanks for considering this idea.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Robert Newson (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843208#comment-13843208
 ] 

Robert Newson commented on COUCHDB-1950:


What I think we *can* safely do is;

"read_resolve":
function(leafs) {
  leafs.sort(function(a,b) { return b.last_updated - a.last_updated });
  return leafs[0];
}

so that instead of our winner algorithm, the user would see some semantically 
'latest' version (a bit like Riak, I think?). That function isn't perfectly 
deterministic, of course, two leafs could have identical last_updated values, 
but the user (well, app developer) would understand that. More importantly, 
only resolving on read will not impact eventual consistency or cause 
replicator-induced edit cycles.

As to how we could automate the removal of losing edit branches (or even the 
removal of deleted branches), I don't know if there is a perfect answer. An 
imperfect one, that I hesitate to even mention, would be wall clock-based. A 
user could tell couchdb to only keep losing branches for a certain period of 
time, after which it may assume that the update has been replicated everywhere. 
The duration chosen for this will be app-specific, of course. At Cloudant, for 
the conflicts that are generated wholly within the system, as opposed to the 
ones from users, we might set that at a few hours or even less. In general, no 
value is guaranteed to be safe. I would not like to see such a thing in core, 
though.

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843195#comment-13843195
 ] 

Jan Lehnardt commented on COUCHDB-1950:
---

What then, is our story for app-level resolution functions? A _changes-feed 
powered app-level-resolver will create the same issues as far as I can tell and 
I am happy to use a different thing altogether.

Whatever we suggest that people do in their apps, I’d like to take a piece out 
of the boilerplate that they need to set up for running CouchDB apps. (I hope 
that clarifies my motivation).

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843196#comment-13843196
 ] 

Jan Lehnardt commented on COUCHDB-1950:
---

(And THANKS for elaborating!)

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1946) Trying to replicate NPM grinds to a halt after 40GB

[Thor Anker Lange (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843191#comment-13843191
 ] 

Thor Anker Lange commented on COUCHDB-1946:
---

I am doing an replication with CouchDB 1.2.2-R15B03 and this seems to have 
gotten past the document causing problems - currently at seq 772595.

> Trying to replicate NPM grinds to a halt after 40GB
> ---
>
> Key: COUCHDB-1946
> URL: https://issues.apache.org/jira/browse/COUCHDB-1946
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Marc Trudel
> Attachments: couch.log
>
>
> I have been able to replicate the Node.js NPM database until 40G or so, then 
> I get this:
> https://gist.github.com/stelcheck/7723362
> I one case I have gotten a flat-out OOM error, but I didn't take a dump of 
> the log output at the time.
> CentOS6.4 with CouchDB 1.5 (also tried 1.3.1, but to no avail). Also tried to 
> restart replication from scratch - twice - bot cases stalling at 40GB.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Robert Newson (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843189#comment-13843189
 ] 

Robert Newson commented on COUCHDB-1950:


So, an update is made, a resolver runs, removing some leafs. A running 
replication is triggered, copying that resolution to other nodes. Oops, it 
introduces a conflict, a resolver runs, removing some leafs. A running 
replication is triggered, copying that resolution to others nodes. Oops...

The problem with automatic removal of leafs contemporaneously with document 
updates is that it eliminates the existing window where all leafs propagate to 
all replicas where, because winner selection is deterministic, all replicas 
present the same leaf as the winner. This problem is eliminated if we allow 
customisation of the read time winning algorithm.

We all agree that some pruning needs to happen, since performance drops as 
revision trees widen (and that includes branches that end in deleted leafs, 
since we must keep those forever too). It's almost impossible to imagine doing 
so in CouchDB, given its intended design (and core value). To be done safely, 
we could only prune once every leaf at every replica has reached every other 
replica, which obviously degrades our offline story. Users can resolve 
conflicts at their app tier (or, much more typically, never do so) because they 
can determine when it's safe to do. The difficulty in knowing when (or if) it 
is safe to prune a revision tree already affects variants of CouchDB, like 
Cloudant, where multiple copies of a database are kept in sync using 
replication. In theory, at least, we could truly purge losing branches once 
that replication has completed for every pair of replicas. Doing so 
automatically for the general case seems very problematic.

Even if we accept all this, do you think we have a chance of conveying the 
implications of the ddoc-based conflict resolution feature to our users? Do you 
worry that the potential for data loss that users can introduce this way will 
be attributed to their own actions and not the database that let it happen?


> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1934) jquery.couch status

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843175#comment-13843175
 ] 

Jan Lehnardt commented on COUCHDB-1934:
---

> So when we replacing Futon with Fauxton (this will happens anyway) we also 
> stops shipping with jquery.couch.js and his deps, correct? I want to clarify 
> this moment since it is the main of this issue and to be sure that we all 
> thinks about the same.

Great question. This is out of scope for this ticket, I’d say. There are a few 
other factors that play into that (like what to server at / for no-json clients 
(e.g. a human-readable start page). Let’s file a new issue for all this: 
https://issues.apache.org/jira/browse/COUCHDB-1954

> jquery.couch status
> ---
>
> Key: COUCHDB-1934
> URL: https://issues.apache.org/jira/browse/COUCHDB-1934
> Project: CouchDB
>  Issue Type: Question
>Reporter: Alexander Shorin
>
> When [~nslater] had proposed to include [jquery.couch.js 
> docs|http://daleharvey.github.io/jquery.couch.js-docs/symbols/] into official 
> CouchDB one I recall old good time when we had discussed Futon.Next project.
> Suddenly, it was out of official mailing lists, so no references are 
> available, but [~garren], [~dch], [~bigbluehat] and [~ryanramage] were there 
> and our discussion was around jquery vs backbone vs pouchdb. Backbone won.
> Nowdays, we have Fauxton which doesn't used jquery.couch and based on 
> Backbone. So it seems if we'll replace Futon with him this client library 
> would be left orphan.
> So that's the question: what is the status of jquery.couch.js? Is it still 
> actual and supported or going to be deprecated?



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843177#comment-13843177
 ] 

Jan Lehnardt commented on COUCHDB-1950:
---

[~rnewson] as we don’t encourage users to leave conflicts to linger, I do mean 
potentially destructive resolution.

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Created] (COUCHDB-1954) Clarify Futon/Fauxton transition

[Jan Lehnardt (JIRA)]

Jan Lehnardt created COUCHDB-1954:
-

 Summary: Clarify Futon/Fauxton transition
 Key: COUCHDB-1954
 URL: https://issues.apache.org/jira/browse/COUCHDB-1954
 Project: CouchDB
  Issue Type: Task
  Components: Futon
Reporter: Jan Lehnardt


At some point Fauxton will replace Futon. There are a few questions that we 
should clarify before we get there:

1. what happens to resources like jquery.js and jquery.couch.js that are 
currently in well-known places where CouchApps depend on them?

2. there are several other tickets regarding serving a human readable HTML page 
on `/` if the client is clearly a browser. (so people don’t have to type the 
`/_utils` manually, how does that play into rolling out Fauxton?

3. cool URLs don’t change: what happens to links into Futon that no longer work 
in Fauxton that might be coming in from the docs and other tutorials, including 
people’s in-house documentation?



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Robert Newson (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843173#comment-13843173
 ] 

Robert Newson commented on COUCHDB-1950:


ok, so Dale does mean "resolve" in the "delete a leaf off disk" sense. Jan, 
which is your position?


> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Dale Harvey (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843170#comment-13843170
 ] 

Dale Harvey commented on COUCHDB-1950:
--

Sorry I meant 'prune'

A design document based conflict resolution function wouldnt discard users 
writes, its a mechanism to allow users to more easily do what we already 
suggest they do to resolve conflicts, write a new document and/or delete 
conflicting leafs, a read resolver is similiar however the conflicts would 
still exist, which isnt a particularly a bad thing however it makes the 
semantics of all_docs / views and changes more complicated

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Robert Newson (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843169#comment-13843169
 ] 

Robert Newson commented on COUCHDB-1950:


Jan, that answer did not clarify anything, I hope I clearly described two very 
different interpretations of "resolve". Can you clarify which you mean?

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843167#comment-13843167
 ] 

Jan Lehnardt commented on COUCHDB-1950:
---

I am suggesting moving a bit of user code from the app to the ddoc. It is still 
user code and codifies user-intent, so I’d be happy for it to properly resolve 
a conflict by picking a winning revision over others, like an app does today. 
There need to be a few limits on resolution functions, like we have for map 
functions, but I think that is entirely reasonable.

I’d be happy to make this an opt-in plugin, but my crystal ball says that it 
will quickly be the most installed plugin :)

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Robert Newson (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843166#comment-13843166
 ] 

Robert Newson commented on COUCHDB-1950:


and please don't say "purge". CouchDB has a special meaning for that term which 
doesn't apply to any interpretation of "resolve".

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1934) jquery.couch status

[Alexander Shorin (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843163#comment-13843163
 ] 

Alexander Shorin commented on COUCHDB-1934:
---

> Alexander Shorin Not quite, I propose that the URLs remain the same. Merely 
> the source code gets moved around. On release-packaging time, jquery.couch.js 
> is pulled in from the other repo. 

I mean in long term vision (e.g. for CouchDB 3.0+). It's oblivious that for 2.x 
it will also available by old URL and new one (if any). But ok.

> jquery.js should stay the way it is as long as we ship Futon.
So when we replacing Futon with Fauxton (this will happens anyway) we also 
stops shipping with jquery.couch.js and his deps, correct? I want to clarify 
this moment since it is the main of this issue and to be sure that we all 
thinks about the same.

Sorry for such persistent questions! Thanks(:

> jquery.couch status
> ---
>
> Key: COUCHDB-1934
> URL: https://issues.apache.org/jira/browse/COUCHDB-1934
> Project: CouchDB
>  Issue Type: Question
>Reporter: Alexander Shorin
>
> When [~nslater] had proposed to include [jquery.couch.js 
> docs|http://daleharvey.github.io/jquery.couch.js-docs/symbols/] into official 
> CouchDB one I recall old good time when we had discussed Futon.Next project.
> Suddenly, it was out of official mailing lists, so no references are 
> available, but [~garren], [~dch], [~bigbluehat] and [~ryanramage] were there 
> and our discussion was around jquery vs backbone vs pouchdb. Backbone won.
> Nowdays, we have Fauxton which doesn't used jquery.couch and based on 
> Backbone. So it seems if we'll replace Futon with him this client library 
> would be left orphan.
> So that's the question: what is the status of jquery.couch.js? Is it still 
> actual and supported or going to be deprecated?



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Robert Newson (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843164#comment-13843164
 ] 

Robert Newson commented on COUCHDB-1950:


I am saying that couchdb has a firm principle of not discarding a user's write, 
and it is that excellent principle I am defending here.

Allowing customisation of which of the leafs is presented as the winner sounds 
fine (assuming the code is deterministic).

Which of these two different meanings of "resolve" is this ticket about?


> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Dale Harvey (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843162#comment-13843162
 ] 

Dale Harvey commented on COUCHDB-1950:
--

The notion that conflicts should not be fixed at all is confusing to me, the 
distinction between 'we' isnt clear, this is talking about a simpler way for 
users to resolve their conflicts how they choose. Are you saying conflicts 
should just grow infinitely until they are purged?



> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843159#comment-13843159
 ] 

Jan Lehnardt commented on COUCHDB-1950:
---

(sorry if that was confusing)

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843158#comment-13843158
 ] 

Jan Lehnardt commented on COUCHDB-1950:
---

I am asking why we should not be doing conflict resolution at all. I think it 
would be beneficial for the discussion of this ticket to have that spelled out 
or have a pointer to where it is spelled out.

Does that other ticket exist, can you post a link if it does?

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Robert Newson (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843150#comment-13843150
 ] 

Robert Newson commented on COUCHDB-1950:


This is confusing. You're asking me to elaborate on the thing I said we should 
not do at all (actually resolve conflicts).

On another ticket, one about the read-time ddoc-mediate choice of 'winning' 
leaf, I would elaborate a bit more (though it seems pretty straightforward).


> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843149#comment-13843149
 ] 

Jan Lehnardt commented on COUCHDB-1950:
---

Can you elaborate on this? In my head it works like this:

- today all apps need to have a conflict resolution function that they run on 
conflicts.
- in the future, that function could live in a ddoc and be fed by a 
_changes?conflicts=true feed and run inside CouchDB instead of the app. As far 
as the core DB is concerned, the same thing happens.

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1934) jquery.couch status

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843147#comment-13843147
 ] 

Jan Lehnardt commented on COUCHDB-1934:
---

[~kxepal] Not quite, I propose that the URLs remain the same. Merely the source 
code gets moved around. On release-packaging time, jquery.couch.js is pulled in 
from the other repo. jquery.js should stay the way it is as long as we ship 
Futon.

For developers nothing changes unless they want to work on Futon. Only 
packagers will need to have the jquery.couch.js repo checked out as well (the 
details of which are to be determined and I’d love to hear from [~nslater] & 
[~djc] about options.

> jquery.couch status
> ---
>
> Key: COUCHDB-1934
> URL: https://issues.apache.org/jira/browse/COUCHDB-1934
> Project: CouchDB
>  Issue Type: Question
>Reporter: Alexander Shorin
>
> When [~nslater] had proposed to include [jquery.couch.js 
> docs|http://daleharvey.github.io/jquery.couch.js-docs/symbols/] into official 
> CouchDB one I recall old good time when we had discussed Futon.Next project.
> Suddenly, it was out of official mailing lists, so no references are 
> available, but [~garren], [~dch], [~bigbluehat] and [~ryanramage] were there 
> and our discussion was around jquery vs backbone vs pouchdb. Backbone won.
> Nowdays, we have Fauxton which doesn't used jquery.couch and based on 
> Backbone. So it seems if we'll replace Futon with him this client library 
> would be left orphan.
> So that's the question: what is the status of jquery.couch.js? Is it still 
> actual and supported or going to be deprecated?



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Robert Newson (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843143#comment-13843143
 ] 

Robert Newson commented on COUCHDB-1950:


Allowing programmatic control over which leaf is returning by GET /dbname/docid 
is much less controversial than "ddoc-based conflict resolution", since that 
brings the real danger of destroying eventual consistency and creating positive 
edit cycle feedback. I'd support the former, but not the latter.



> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1934) jquery.couch status

[Alexander Shorin (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843142#comment-13843142
 ] 

Alexander Shorin commented on COUCHDB-1934:
---

[~janl] , sounds very good! I'm happy to +1, but one question: 
> bundle it on build-time with CouchDB, so CouchApps don’t break right away.
So as for CouchDB users we just moving jquery.couch _and_ his dependencies 
(like jquery.js), but not futon one(like jquery.ui.js etc.) to some other 
location (like /_static/scripts). Correct? Btw, having separate from 
Futon/Fauxton resource for such static content that supposed to be used by 
couchapp might be good idea.

As for developers jquery.couch will live in his own new repo and when they 
builds the CouchDB they may run ./configure with option `--with-jquery-couch` 
(or without?) to let autotools grab jquery.couch with all his actual deps to 
the release artifact? 

> jquery.couch status
> ---
>
> Key: COUCHDB-1934
> URL: https://issues.apache.org/jira/browse/COUCHDB-1934
> Project: CouchDB
>  Issue Type: Question
>Reporter: Alexander Shorin
>
> When [~nslater] had proposed to include [jquery.couch.js 
> docs|http://daleharvey.github.io/jquery.couch.js-docs/symbols/] into official 
> CouchDB one I recall old good time when we had discussed Futon.Next project.
> Suddenly, it was out of official mailing lists, so no references are 
> available, but [~garren], [~dch], [~bigbluehat] and [~ryanramage] were there 
> and our discussion was around jquery vs backbone vs pouchdb. Backbone won.
> Nowdays, we have Fauxton which doesn't used jquery.couch and based on 
> Backbone. So it seems if we'll replace Futon with him this client library 
> would be left orphan.
> So that's the question: what is the status of jquery.couch.js? Is it still 
> actual and supported or going to be deprecated?



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1952) How exactly are underscore-prefixed fields reserved?

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843141#comment-13843141
 ] 

Jan Lehnardt commented on COUCHDB-1952:
---

We should still clarify this in the new docs.

> How exactly are underscore-prefixed fields reserved?
> 
>
> Key: COUCHDB-1952
> URL: https://issues.apache.org/jira/browse/COUCHDB-1952
> Project: CouchDB
>  Issue Type: Bug
>  Components: Documentation
>Reporter: Nathan Vander Wilt
>
> The wiki stated "Note that any top-level fields within a JSON document 
> containing a name that starts with a _ prefix are reserved for use by CouchDB 
> itself." http://wiki.apache.org/couchdb/HTTP_Document_API#Special_Fields
> In the new documentation the closest equivalent I can find is the hand-wavy 
> "As a general rule, URL components and JSON fields starting with the _ 
> (underscore) character represent a special component or entity within the 
> server or returned object." 
> http://docs.couchdb.org/en/latest/api/index.html?highlight=underscore
> I'd like to see the official documentation clarified:
> - are underscore fields still reserved? (I'm assuming so.)
> - are they reserved only at the top level? (I'm hoping so.)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1950) ddoc-based conflict resolution

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843140#comment-13843140
 ] 

Jan Lehnardt commented on COUCHDB-1950:
---

Yeah, I’ve wanted ddoc-based default resolution functions for the longest time!

> ddoc-based conflict resolution
> --
>
> Key: COUCHDB-1950
> URL: https://issues.apache.org/jira/browse/COUCHDB-1950
> Project: CouchDB
>  Issue Type: Improvement
>Reporter: Nathan Vander Wilt
>
> This was discussed at CouchConf in Vancouver last month, but didn't see a 
> hook here I could refer to in another conversation, so…
> It'd be great if a design document could include a conflict resolver 
> function, in the vein of other "app logic" handler hooks like 
> validate_doc_write. I imagine it would look something like either "function 
> (currentWinner, nextWinningestLoser, parent)" (simply called multiple times 
> if more than 2 leafs) or simply "function (arrayOfDocs, revisionHistor)" — if 
> it returns a document, that's the winner, if not the next design document in 
> line gets a pass at it. (Bonus: if it throws, the conflict stays no matter 
> what other resolvers say?)



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1303) Add a _bulk_update handler similar to _update but for bulk document changes

[Robert Newson (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843135#comment-13843135
 ] 

Robert Newson commented on COUCHDB-1303:


Is a bulk version of update handlers desirable given that each item in the bulk 
update can fail independently? Wouldn't that be very confusing? How would users 
handle that?


> Add a _bulk_update handler similar to _update but for bulk document changes
> ---
>
> Key: COUCHDB-1303
> URL: https://issues.apache.org/jira/browse/COUCHDB-1303
> Project: CouchDB
>  Issue Type: New Feature
>Reporter: Benjamin Young
>  Labels: api, update_request_handler
>
> _update handlers are great (and getting better!) for building RESTful API's 
> inside CouchDB. One limitation I found tonight is that _update can only do a 
> single document at a time. If the API I'm building needs to update multiple 
> docs (in a similar fashion to _bulk_docs), then an outside "proxy" script is 
> required. It would be ideal to have a _bulk_update handler to allow for the 
> same functionality as _update, but with the ability to insert multiple 
> documents at once.
> Perhaps the current _update handler API could be extended to support multiple 
> IDs/documents, but a separate API endpoint would be seem reasonable if needed.
> Thanks for considering this idea.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1303) Add a _bulk_update handler similar to _update but for bulk document changes

[Benjamin Young (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843130#comment-13843130
 ] 

Benjamin Young commented on COUCHDB-1303:
-

It would seem that allowing `_update` to store multiple documents would be the 
easier approach.

This is the current code for saving documents and responding to an `_update` 
request:
https://github.com/apache/couchdb/blob/master/src/couch_mrview/src/couch_mrview_show.erl#L131

Perhaps making the first element of the return array support multiple JSON 
objects (i.e. return [[doc1, doc2], {..headers..}]; ) would be a good first 
step.

The next step would be to allow `_update` (with no doc_id specified in the URL 
support the bulk_docs JSON format for POSTs).

Likely this could be broken into two separate commits (and Jira tickets) as 
they have separate value:
1. sending in a single `_update` request and having multiple documents changed
2. sending in a bulk_docs formatted `_update` POST and having those documents 
handled and/or other documents generated/updated.

I don't think (now) that a `_bulk_update` endpoint would be needed.

Additionally, the use of redirect responses from an `_update` handler might 
work now has a work around, it's not something I'd care to depend on (nor see 
shipped in CouchDB) due to there being no guarantee that the Location header 
would actually be followed by the client, and therefore no guarantee that the 
second (or more) updates would actually be done. It could certainly work if you 
control the whole stack and are OK with the potential of the redirects failing 
at times.

Thanks for drawing attention to this issue in any case!

> Add a _bulk_update handler similar to _update but for bulk document changes
> ---
>
> Key: COUCHDB-1303
> URL: https://issues.apache.org/jira/browse/COUCHDB-1303
> Project: CouchDB
>  Issue Type: New Feature
>Reporter: Benjamin Young
>  Labels: api, update_request_handler
>
> _update handlers are great (and getting better!) for building RESTful API's 
> inside CouchDB. One limitation I found tonight is that _update can only do a 
> single document at a time. If the API I'm building needs to update multiple 
> docs (in a similar fashion to _bulk_docs), then an outside "proxy" script is 
> required. It would be ideal to have a _bulk_update handler to allow for the 
> same functionality as _update, but with the ability to insert multiple 
> documents at once.
> Perhaps the current _update handler API could be extended to support multiple 
> IDs/documents, but a separate API endpoint would be seem reasonable if needed.
> Thanks for considering this idea.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1934) jquery.couch status

[Garren Smith (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843124#comment-13843124
 ] 

Garren Smith commented on COUCHDB-1934:
---

+1 that will work nicely.

> jquery.couch status
> ---
>
> Key: COUCHDB-1934
> URL: https://issues.apache.org/jira/browse/COUCHDB-1934
> Project: CouchDB
>  Issue Type: Question
>Reporter: Alexander Shorin
>
> When [~nslater] had proposed to include [jquery.couch.js 
> docs|http://daleharvey.github.io/jquery.couch.js-docs/symbols/] into official 
> CouchDB one I recall old good time when we had discussed Futon.Next project.
> Suddenly, it was out of official mailing lists, so no references are 
> available, but [~garren], [~dch], [~bigbluehat] and [~ryanramage] were there 
> and our discussion was around jquery vs backbone vs pouchdb. Backbone won.
> Nowdays, we have Fauxton which doesn't used jquery.couch and based on 
> Backbone. So it seems if we'll replace Futon with him this client library 
> would be left orphan.
> So that's the question: what is the status of jquery.couch.js? Is it still 
> actual and supported or going to be deprecated?



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1934) jquery.couch status

[Dale Harvey (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843121#comment-13843121
 ] 

Dale Harvey commented on COUCHDB-1934:
--

+1

> jquery.couch status
> ---
>
> Key: COUCHDB-1934
> URL: https://issues.apache.org/jira/browse/COUCHDB-1934
> Project: CouchDB
>  Issue Type: Question
>Reporter: Alexander Shorin
>
> When [~nslater] had proposed to include [jquery.couch.js 
> docs|http://daleharvey.github.io/jquery.couch.js-docs/symbols/] into official 
> CouchDB one I recall old good time when we had discussed Futon.Next project.
> Suddenly, it was out of official mailing lists, so no references are 
> available, but [~garren], [~dch], [~bigbluehat] and [~ryanramage] were there 
> and our discussion was around jquery vs backbone vs pouchdb. Backbone won.
> Nowdays, we have Fauxton which doesn't used jquery.couch and based on 
> Backbone. So it seems if we'll replace Futon with him this client library 
> would be left orphan.
> So that's the question: what is the status of jquery.couch.js? Is it still 
> actual and supported or going to be deprecated?



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1934) jquery.couch status

[Jan Lehnardt (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843120#comment-13843120
 ] 

Jan Lehnardt commented on COUCHDB-1934:
---

Excellent discussion everyone!

How about this?

- give jquery.couch.js its on repo on git-asf + mirror to GitHUb, so future 
maintainers have it easier to maintain.
- put it on npm, so there can be releases of it independent of CouchDB’s 
releases.
- bundle it on build-time with CouchDB, so CouchApps don’t break right away.
- deprecate the canonical URL for it for the n+2 major release (i.e. deprecated 
in 2.0.0 and remove in 3.0.0, e.g.)


> jquery.couch status
> ---
>
> Key: COUCHDB-1934
> URL: https://issues.apache.org/jira/browse/COUCHDB-1934
> Project: CouchDB
>  Issue Type: Question
>Reporter: Alexander Shorin
>
> When [~nslater] had proposed to include [jquery.couch.js 
> docs|http://daleharvey.github.io/jquery.couch.js-docs/symbols/] into official 
> CouchDB one I recall old good time when we had discussed Futon.Next project.
> Suddenly, it was out of official mailing lists, so no references are 
> available, but [~garren], [~dch], [~bigbluehat] and [~ryanramage] were there 
> and our discussion was around jquery vs backbone vs pouchdb. Backbone won.
> Nowdays, we have Fauxton which doesn't used jquery.couch and based on 
> Backbone. So it seems if we'll replace Futon with him this client library 
> would be left orphan.
> So that's the question: what is the status of jquery.couch.js? Is it still 
> actual and supported or going to be deprecated?



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Re: Next release planning

[Simon Metson]

> Yes, but the URL is `/_utils/fauxton/`, isn’t it?
>  


It is indeed

Re: Next release planning

[Jan Lehnardt]

On 08 Dec 2013, at 16:17 , Andy Wenk  wrote:

> On 6 December 2013 16:17, Jan Lehnardt  wrote:
> 
> On 27 Nov 2013, at 09:37 , Andy Wenk  wrote:
> 
> > On 26 November 2013 17:10, Benjamin Young  wrote:
> >
> >> This is a nit...maybe...but can we be sure that `_utils/fauxton` and
> >> `_utils/fauxton/` both work? :)
> >>
> >> Would prevent "rock in shoe" sort of frustrations. :)
> >>
> >
> > true :) but that would mean you are not allowed to create a database named
> > fauxton which would be requested via _utils/fauxton/
> >
> > This has already been discussed on IRC and the "solution" was imo more or
> > less to wait till fauxton will replace futon. Then you will call it via
> > well known _utils/.
> 
> There are no discussions on IRC that are binding. If any of this happens,
> please make sure that dev@ is informed. This is important to ensure
> transparency of the development process/
> 
> Jan, thanks for pointing me to this. 
>  
> I don’t see why `/_utils/fauxton/` would collide with a database called
> `fauxton` as that would live at `/_fauxton`, but I might be missing something
> subtle.
> 
> I don't get the point here. When calling a database called fauxton, the API 
> call would be sth. like /fauxton/all_docs . Wouldn't that collide with 
> /fauxton as the Webinterface? 

Yes, but the URL is `/_utils/fauxton/`, isn’t it?

Best
Jan
--



> 
> Cheers
> 
> Andy
> 
> -- 
> Andy Wenk
> Hamburg - Germany
> RockIt!
> 
> http://www.couchdb-buch.de
> http://www.pg-praxisbuch.de
> 
> GPG fingerprint: C044 8322 9E12 1483 4FEC  9452 B65D 6BE3 9ED3 9588



signature.asc
Description: Message signed with OpenPGP using GPGMail

[jira] [Commented] (COUCHDB-1946) Trying to replicate NPM grinds to a halt after 40GB

[Thor Anker Lange (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843029#comment-13843029
 ] 

Thor Anker Lange commented on COUCHDB-1946:
---

I have been using the default replicator settings before I started trying to 
adjust the batch size - which did not help. Here is the last setup I tried:

{code}
{
 "_id": "npmjs_repl",
 "source":"http://isaacs.iriscouch.com/registry/";,
 "target":"registry",
 "continuous":false,
 "worker_batch_size":50,
 "user_ctx": {"name":"xpaadmin", "roles":["_admin"]}
}
{code}

> Trying to replicate NPM grinds to a halt after 40GB
> ---
>
> Key: COUCHDB-1946
> URL: https://issues.apache.org/jira/browse/COUCHDB-1946
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Marc Trudel
> Attachments: couch.log
>
>
> I have been able to replicate the Node.js NPM database until 40G or so, then 
> I get this:
> https://gist.github.com/stelcheck/7723362
> I one case I have gotten a flat-out OOM error, but I didn't take a dump of 
> the log output at the time.
> CentOS6.4 with CouchDB 1.5 (also tried 1.3.1, but to no avail). Also tried to 
> restart replication from scratch - twice - bot cases stalling at 40GB.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1946) Trying to replicate NPM grinds to a halt after 40GB

[Dave Cottlehuber (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843018#comment-13843018
 ] 

Dave Cottlehuber commented on COUCHDB-1946:
---

[~stelcheck] agreed
[~thor.lange]

There's something with replicating this specific doc that seems to trigger 
issues. Here's what I used to identify it (call source db and use since= 
http://isaacs.iriscouch.com/registry/_changes\?limit\=2\&since\=701251

here's some things you can try:

# option 1

-  delete all existing replications
- compact your DB if there's a big difference between data size and on-disk 
size. jq is awesome for this.

curl -s http://localhost:5984/registry | jq ' (.disk_size| tonumber) - 
(.data_size |tonumber)'

http://stedolan.github.io/jq/

This is a good spot to copy the registry.couch file if you have space, in case 
you need to revert back to it.

-  replicate the single failing document by POSTing this to _replicator. This 
could take a *while*.

{{code}}
{
   "source": "http://isaacs.iriscouch.com/registry";,
   "target": "registry",
   "doc_ids": [
   "as-stream"
   ],
   "owner": "admin",
   }
}
{{code}}

- this is simply replicating the single stuck document. If you do this, I would 
love an ngrep or tcpdump of the traffic to see what happens on the wire during 
these stuck transfers

- once this is completed, you can then run the normal replication again.

# option 2

Install an older release of CouchDB and see if it doesn't get stuck here:

https://archive.apache.org/dist/couchdb/binary/win/1.2.2/

If you *can* please try the R15B03-1 release first, report back, and then the 
R14B04 one. It's not yet clear to me if the issue we are seeing is also related 
to garbage collection differences in Erlang/OTP between releases, or solely 
within CouchDB.

# option 3

Sometime later (hopefully today), I should have a bitttorrent accessible 
version of npm. I need to update & compact first, this is pretty much IO 
limited :-).


> Trying to replicate NPM grinds to a halt after 40GB
> ---
>
> Key: COUCHDB-1946
> URL: https://issues.apache.org/jira/browse/COUCHDB-1946
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Marc Trudel
> Attachments: couch.log
>
>
> I have been able to replicate the Node.js NPM database until 40G or so, then 
> I get this:
> https://gist.github.com/stelcheck/7723362
> I one case I have gotten a flat-out OOM error, but I didn't take a dump of 
> the log output at the time.
> CentOS6.4 with CouchDB 1.5 (also tried 1.3.1, but to no avail). Also tried to 
> restart replication from scratch - twice - bot cases stalling at 40GB.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1946) Trying to replicate NPM grinds to a halt after 40GB

[Marc Trudel (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843016#comment-13843016
 ] 

Marc Trudel commented on COUCHDB-1946:
--

I tried all sorts of settings, but always ended up with the same outcome.
Just took more or less time.






-- 
Marc Trudel-Belisle
Chief Technology Officer | Wizcorp Inc. 
--
TECH . GAMING . OPEN-SOURCE WIZARDS+ 81
3-4550-1448|Website


> Trying to replicate NPM grinds to a halt after 40GB
> ---
>
> Key: COUCHDB-1946
> URL: https://issues.apache.org/jira/browse/COUCHDB-1946
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Marc Trudel
> Attachments: couch.log
>
>
> I have been able to replicate the Node.js NPM database until 40G or so, then 
> I get this:
> https://gist.github.com/stelcheck/7723362
> I one case I have gotten a flat-out OOM error, but I didn't take a dump of 
> the log output at the time.
> CentOS6.4 with CouchDB 1.5 (also tried 1.3.1, but to no avail). Also tried to 
> restart replication from scratch - twice - bot cases stalling at 40GB.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1946) Trying to replicate NPM grinds to a halt after 40GB

[Alexander Shorin (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843011#comment-13843011
 ] 

Alexander Shorin commented on COUCHDB-1946:
---

Hey, [~stelcheck], [~thor.lange], [~MiLk]

Are you using default replicator settings?  Just to be sure that we are all on 
the same configuration since following [~dch] suggestion from first post helps 
a lot with reducing system resources usage during replication.

> Trying to replicate NPM grinds to a halt after 40GB
> ---
>
> Key: COUCHDB-1946
> URL: https://issues.apache.org/jira/browse/COUCHDB-1946
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Marc Trudel
> Attachments: couch.log
>
>
> I have been able to replicate the Node.js NPM database until 40G or so, then 
> I get this:
> https://gist.github.com/stelcheck/7723362
> I one case I have gotten a flat-out OOM error, but I didn't take a dump of 
> the log output at the time.
> CentOS6.4 with CouchDB 1.5 (also tried 1.3.1, but to no avail). Also tried to 
> restart replication from scratch - twice - bot cases stalling at 40GB.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Re: [jira] [Commented] (COUCHDB-1946) Trying to replicate NPM grinds to a halt after 40GB

[Nick North]

When I tried replicating the NPM registry on Windows it bluescreened the
machine when the Erlang process reached about 1GB. On the plus side, once
the machine was rebooted, replication continued past 40GB, though it did
begin to slow down at that point. I don't really want the registry, so
stopped replicating at that point, but I got the feeling that you could get
the full registry onto a Windows CouchDb installation if you stop and
restart the CouchDb service when memory consumption gets too high.

Nick


On 9 December 2013 09:10, Thor Anker Lange (JIRA)  wrote:

>
> [
> https://issues.apache.org/jira/browse/COUCHDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842996#comment-13842996]
>
> Thor Anker Lange commented on COUCHDB-1946:
> ---
>
> I am seeing the exact same behaviour when reaching 30-40 GB running
> CouchDB on Windows (yes, unfortunally I do not have a choice of server OS).
> The source start sequence is around 701252.
>
> Marc, how did you manage to get a complete replica of the npm registry?
>
> > Trying to replicate NPM grinds to a halt after 40GB
> > ---
> >
> > Key: COUCHDB-1946
> > URL: https://issues.apache.org/jira/browse/COUCHDB-1946
> > Project: CouchDB
> >  Issue Type: Bug
> >  Components: Database Core
> >Reporter: Marc Trudel
> > Attachments: couch.log
> >
> >
> > I have been able to replicate the Node.js NPM database until 40G or so,
> then I get this:
> > https://gist.github.com/stelcheck/7723362
> > I one case I have gotten a flat-out OOM error, but I didn't take a dump
> of the log output at the time.
> > CentOS6.4 with CouchDB 1.5 (also tried 1.3.1, but to no avail). Also
> tried to restart replication from scratch - twice - bot cases stalling at
> 40GB.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.1.4#6159)
>

[jira] [Commented] (COUCHDB-1946) Trying to replicate NPM grinds to a halt after 40GB

[Marc Trudel (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842999#comment-13842999
 ] 

Marc Trudel commented on COUCHDB-1946:
--

I downloaded the .couch  file from someone :)

I'd really suggest to people running NPM to make nightlies of this file,
and make it available for download. Regardless of this bug, it would be
much faster than starting with replication (and pretty similar to standard
procedures for setting replicates in other DBMS, like MySQL)






-- 
Marc Trudel-Belisle
Chief Technology Officer | Wizcorp Inc. 
--
TECH . GAMING . OPEN-SOURCE WIZARDS+ 81
3-4550-1448|Website


> Trying to replicate NPM grinds to a halt after 40GB
> ---
>
> Key: COUCHDB-1946
> URL: https://issues.apache.org/jira/browse/COUCHDB-1946
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Marc Trudel
> Attachments: couch.log
>
>
> I have been able to replicate the Node.js NPM database until 40G or so, then 
> I get this:
> https://gist.github.com/stelcheck/7723362
> I one case I have gotten a flat-out OOM error, but I didn't take a dump of 
> the log output at the time.
> CentOS6.4 with CouchDB 1.5 (also tried 1.3.1, but to no avail). Also tried to 
> restart replication from scratch - twice - bot cases stalling at 40GB.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (COUCHDB-1946) Trying to replicate NPM grinds to a halt after 40GB

[Thor Anker Lange (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842996#comment-13842996
 ] 

Thor Anker Lange commented on COUCHDB-1946:
---

I am seeing the exact same behaviour when reaching 30-40 GB running CouchDB on 
Windows (yes, unfortunally I do not have a choice of server OS). The source 
start sequence is around 701252.

Marc, how did you manage to get a complete replica of the npm registry?

> Trying to replicate NPM grinds to a halt after 40GB
> ---
>
> Key: COUCHDB-1946
> URL: https://issues.apache.org/jira/browse/COUCHDB-1946
> Project: CouchDB
>  Issue Type: Bug
>  Components: Database Core
>Reporter: Marc Trudel
> Attachments: couch.log
>
>
> I have been able to replicate the Node.js NPM database until 40G or so, then 
> I get this:
> https://gist.github.com/stelcheck/7723362
> I one case I have gotten a flat-out OOM error, but I didn't take a dump of 
> the log output at the time.
> CentOS6.4 with CouchDB 1.5 (also tried 1.3.1, but to no avail). Also tried to 
> restart replication from scratch - twice - bot cases stalling at 40GB.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)