[jira] [Updated] (COUCHDB-1791) Initial index of views is highly fragmented
[ https://issues.apache.org/jira/browse/COUCHDB-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Howe updated COUCHDB-1791: Description: {code:javascript} function(doc) { if (doc.a === 'b') { emit(doc._id, null); } } {code} This view, running on CouchDB 1.2.1, takes 16 minutes to index from scratch on an inactive database, creating a 2.5GB file. After compaction, the file is then only 157MB. Can we do something about this waste of disk I/O, such that the initial index is smaller to start with? was: function(doc) { if (doc.a = 'b') { emit(doc._id, null); } } This view, running on CouchDB 1.2.1, takes 16 minutes to index from scratch on an inactive database, creating a 2.5GB file. After compaction, the file is then only 157MB. Can we do something about this waste of disk I/O, such that the initial index is smaller to start with? > Initial index of views is highly fragmented > --- > > Key: COUCHDB-1791 > URL: https://issues.apache.org/jira/browse/COUCHDB-1791 > Project: CouchDB > Issue Type: Improvement > Components: JavaScript View Server >Reporter: James Howe > > {code:javascript} > function(doc) { > if (doc.a === 'b') { emit(doc._id, null); } > } > {code} > This view, running on CouchDB 1.2.1, takes 16 minutes to index from scratch > on an inactive database, creating a 2.5GB file. > After compaction, the file is then only 157MB. > Can we do something about this waste of disk I/O, such that the initial index > is smaller to start with? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1791) Initial index of views is highly fragmented
[ https://issues.apache.org/jira/browse/COUCHDB-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Howe updated COUCHDB-1791: Labels: performance (was: ) > Initial index of views is highly fragmented > --- > > Key: COUCHDB-1791 > URL: https://issues.apache.org/jira/browse/COUCHDB-1791 > Project: CouchDB > Issue Type: Improvement > Components: JavaScript View Server >Affects Versions: 1.2.1 >Reporter: James Howe > Labels: performance > > {code:javascript} > function(doc) { > if (doc.a === 'b') { emit(doc._id, null); } > } > {code} > This view, running on CouchDB 1.2.1, takes 16 minutes to index from scratch > on an inactive database, creating a 2.5GB file. > After compaction, the file is then only 157MB. > Can we do something about this waste of disk I/O, such that the initial index > is smaller to start with? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1791) Initial index of views is highly fragmented
[ https://issues.apache.org/jira/browse/COUCHDB-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13659437#comment-13659437 ] James Howe commented on COUCHDB-1791: - Fixed, was just a typo here, not in my actual code. > Initial index of views is highly fragmented > --- > > Key: COUCHDB-1791 > URL: https://issues.apache.org/jira/browse/COUCHDB-1791 > Project: CouchDB > Issue Type: Improvement > Components: JavaScript View Server >Reporter: James Howe > > {code:javascript} > function(doc) { > if (doc.a === 'b') { emit(doc._id, null); } > } > {code} > This view, running on CouchDB 1.2.1, takes 16 minutes to index from scratch > on an inactive database, creating a 2.5GB file. > After compaction, the file is then only 157MB. > Can we do something about this waste of disk I/O, such that the initial index > is smaller to start with? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1791) Initial index of views is highly fragmented
[ https://issues.apache.org/jira/browse/COUCHDB-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Howe updated COUCHDB-1791: Affects Version/s: 1.2.1 > Initial index of views is highly fragmented > --- > > Key: COUCHDB-1791 > URL: https://issues.apache.org/jira/browse/COUCHDB-1791 > Project: CouchDB > Issue Type: Improvement > Components: JavaScript View Server >Affects Versions: 1.2.1 >Reporter: James Howe > > {code:javascript} > function(doc) { > if (doc.a === 'b') { emit(doc._id, null); } > } > {code} > This view, running on CouchDB 1.2.1, takes 16 minutes to index from scratch > on an inactive database, creating a 2.5GB file. > After compaction, the file is then only 157MB. > Can we do something about this waste of disk I/O, such that the initial index > is smaller to start with? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1791) Initial index of views is highly fragmented
[ https://issues.apache.org/jira/browse/COUCHDB-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13654524#comment-13654524 ] James Howe commented on COUCHDB-1791: - On [~rnewson]'s suggestion I tried increasing view_checkpoint_interval to 1000 (to match the compactor), but this had no effect on the index size or time taken. > Initial index of views is highly fragmented > --- > > Key: COUCHDB-1791 > URL: https://issues.apache.org/jira/browse/COUCHDB-1791 > Project: CouchDB > Issue Type: Improvement > Components: JavaScript View Server >Reporter: James Howe > > function(doc) { > if (doc.a = 'b') { emit(doc._id, null); } > } > This view, running on CouchDB 1.2.1, takes 16 minutes to index from scratch > on an inactive database, creating a 2.5GB file. > After compaction, the file is then only 157MB. > Can we do something about this waste of disk I/O, such that the initial index > is smaller to start with? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1791) Initial index of views is highly fragmented
James Howe created COUCHDB-1791: --- Summary: Initial index of views is highly fragmented Key: COUCHDB-1791 URL: https://issues.apache.org/jira/browse/COUCHDB-1791 Project: CouchDB Issue Type: Improvement Components: JavaScript View Server Reporter: James Howe function(doc) { if (doc.a = 'b') { emit(doc._id, null); } } This view, running on CouchDB 1.2.1, takes 16 minutes to index from scratch on an inactive database, creating a 2.5GB file. After compaction, the file is then only 157MB. Can we do something about this waste of disk I/O, such that the initial index is smaller to start with? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-536) CouchDB HTTP server stops accepting connections
[ https://issues.apache.org/jira/browse/COUCHDB-536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13548682#comment-13548682 ] James Howe commented on COUCHDB-536: Think I've hit the same thing in 1.2.0. After making lots of view queries (triggering a reindex on every one on the server with a script) the server got into a state where it would abort a random selection of requests (made at a more leisurely rate), including a number of those used by Futon. Didn't have a chance to investigate socket states on the server unfortunately. The first error was this: [Wed, 09 Jan 2013 16:00:03 GMT] [error] [<0.1528.3109>] {error_report,<0.32.0>, {<0.1528.3109>,std_error, [{application,mochiweb}, "Accept failed error", "{error,enfile}"]}} Lots of these: [Wed, 09 Jan 2013 16:43:46 GMT] [error] [<0.617.3110>] {error_report,<0.32.0>, {<0.617.3110>,crash_report, [[{initial_call, {mochiweb_acceptor,init, ['Argument__1','Argument__2', 'Argument__3']}}, {pid,<0.617.3110>}, {registered_name,[]}, {error_info, {exit, {error,accept_failed}, [{mochiweb_acceptor,init,3}, {proc_lib,init_p_do_apply,3}]}}, {ancestors, [couch_httpd,couch_secondary_services, couch_server_sup,<0.33.0>]}, {messages,[]}, {links,[<0.124.0>]}, {dictionary,[]}, {trap_exit,false}, {status,running}, {heap_size,233}, {stack_size,24}, {reductions,209}], []]}} And then eventually lots of these as well: [Wed, 09 Jan 2013 16:43:46 GMT] [error] [<0.124.0>] {error_report,<0.32.0>, {<0.124.0>,std_error, {mochiweb_socket_server,254, {acceptor_error,{error,accept_failed} > CouchDB HTTP server stops accepting connections > --- > > Key: COUCHDB-536 > URL: https://issues.apache.org/jira/browse/COUCHDB-536 > Project: CouchDB > Issue Type: Bug > Components: HTTP Interface >Affects Versions: 0.10, 1.1 > Environment: Ubuntu Linux 8.04 32bit and 64bit with Erlang R13B01 > or Ubuntu Linux 8.04 64bit with Erlang R14B02 >Reporter: Simon Eisenmann > Attachments: couchdb_database_open-week.png, > couchdb_httpd_request_methods-week.png, couchdb_httpd_response_codes-week.png > > > Having 3 Couches all replicating a couple of databases to each other (pull > replication with a update notification process) the HTTP service on any of > the Couches stops working at some point (when running for a couple of ours > with constant changes on all databases and servers). > This is the error when a new HTTP request comes in: > =ERROR REPORT 19-Oct-2009::10:18:55 === > application: mochiweb > "Accept failed error" > "{error,enfile}" > [error] [<0.21619.12>] {error_report,<0.24.0>, > {<0.21619.12>,crash_report, > [[{initial_call,{mochiweb_socket_server,acceptor_loop,['Argument__1']}}, >{pid,<0.21619.12>}, >{registered_name,[]}, >{error_info, >{exit, >{error,accept_failed}, >[{mochiweb_socket_server,acceptor_loop,1}, > {proc_lib,init_p_do_apply,3}]}}, >{ancestors, >[couch_httpd,couch_secondary_services,couch_server_sup,<0.1.0>]}, >{messages,[]}, >{links,[<0.66.0>]}, >{dictionary,[]}, >{trap_exit,false}, >{status,running}, >{heap_size,233}, >{stack_size,24}, >{reductions,202}], > []]}} > [error] [<0.66.0>] {error_report,<0.24.0>, > {<0.66.0>,std_error, > {mochiweb_socket_server,225,{acceptor_error,{error,accept_failed} > To me this seems like it runs out of threads or sockets to handle the new > connection or somewhat like this. > Also i see in this setup that if i put lots of changes in a short time at > some point the replication process hangs (never finishes) and when trying to > restart the same replication once again is not possible and resultin
[jira] [Commented] (COUCHDB-1506) View returns null value after adding a document
[ https://issues.apache.org/jira/browse/COUCHDB-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402939#comment-13402939 ] James Howe commented on COUCHDB-1506: - There's no guarantee over what values are going to be passed to your reduce function: they get split up according to how the btree is arranged. Therefore, there are going to be occasions where none of the values are of type 'customer', and your reduce is going to return null. I suggest you use one view to do the reduce (emitting only purchase and refill documents), and then you can just get the customer documents by their id in a separate call. > View returns null value after adding a document > --- > > Key: COUCHDB-1506 > URL: https://issues.apache.org/jira/browse/COUCHDB-1506 > Project: CouchDB > Issue Type: Bug > Components: JavaScript View Server >Affects Versions: 1.2 > Environment: Windows XP SP3 >Reporter: Marcin Stefaniuk > > I have three types of documents: customer, refill and purchase. I defined a > view that aggregates customer data with current balance (from refill / > purchase values). Values customer._id and purchase/refill.customer are > joining keys. Map function looks as follows: > function(doc) { > if (doc.type=='purchase' || doc.type=='refill') { > emit(doc.customer, {value: doc.value, type: doc.type}); > } else if (doc.type=='customer') { > emit(doc._id, doc); > } > } > Reduce function is: > function(keys, values, rereduce) { > var customer = null, sum = 0; > values.forEach(function(element) { > if (element.type=='purchase') { > sum -= element.value; > } else if (element.type=='refill') { > sum += element.value; > } else if (customer==null && element.type=='customer') { > customer = element; > } > }); > customer.credit = sum; > return customer; > } > This view is working with success but after several adding of purchase > documents it returns null as value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1496) Incomplete Replication on Several Circumstances
[ https://issues.apache.org/jira/browse/COUCHDB-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294275#comment-13294275 ] James Howe commented on COUCHDB-1496: - When the document is deleted, all other properties are removed, so the _deleted revisions are not passing "if (doc.type)". You need to either not delete documents with DELETE, or add "if (doc._deleted) { return true; }" to the filter. > Incomplete Replication on Several Circumstances > --- > > Key: COUCHDB-1496 > URL: https://issues.apache.org/jira/browse/COUCHDB-1496 > Project: CouchDB > Issue Type: Bug >Affects Versions: 1.2 > Environment: iriscouch >Reporter: Maya Alipin >Priority: Minor > > I have found that on several circumstances, CouchDB's continous filtered > replication doesn't do what I expect. > 1. deleted documents in source database ends up not deleted in destination > database > 2. I notice on continuous replication, the replication status will remain > incomplete until check-pointed sequence equals source sequence. On continuous > filtered replication, an update may not pass the filter and cause the > replication to become incomplete. > I found the replication remains incomplete even if there is an update that > pass the filter. I need to restart the replication to get the update > replicated to the destination database. > Test Scenario: > 1. Trigger a filtered continuous replication from database X to database Y on > _replicator database. Replication progress is 100% > 2. Add new document(A1) on database X. > 3. Document A1 pass the replication filter, thus A1 is replicated to database > Y. Replication progress is 100% > 4. Delete document A1 on database X. Replication progress become 99%, > document A1 still exists on database Y. > 6. Restart the filtered continuous replication from database X to database Y > by deleting document on _replicator database and create it again > 7. Replication progress stops at 99%. > 8. Add new document(A2) on database X. > 9. Document A2 pass the replication filter, thus A2 is replicated to database > Y. Replication progress become 100% > 10. Delete document A2 on database X. Replication progress become 99%, > document A2 still exists on database Y. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1481) Futon fails to page design documents
James Howe created COUCHDB-1481: --- Summary: Futon fails to page design documents Key: COUCHDB-1481 URL: https://issues.apache.org/jira/browse/COUCHDB-1481 Project: CouchDB Issue Type: Bug Components: Futon Affects Versions: 1.2 Reporter: James Howe The "Next Page" and "Previous Page" just reload the current page when the view is set to "Design documents". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1476) In view map function doc object does not have .constructor defined on any of the contained objects.
[ https://issues.apache.org/jira/browse/COUCHDB-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266468#comment-13266468 ] James Howe commented on COUCHDB-1476: - What's wrong with Array.isArray and typeof v === 'object', or even just JSON.parse to do JSON parsing? Take it up with the library developers if they're using broken context-dependent code. > In view map function doc object does not have .constructor defined on any of > the contained objects. > --- > > Key: COUCHDB-1476 > URL: https://issues.apache.org/jira/browse/COUCHDB-1476 > Project: CouchDB > Issue Type: Bug > Components: JavaScript View Server >Affects Versions: 1.2 > Environment: CouchDB 1.2 Release binary on Windows 2003 >Reporter: Andrew Reslan >Priority: Minor > > I am doing some complex view map processing using external libraries. > These libraries use the following checks when parsing a JSON document: > function _isObject(v) { > return (v !== null && !_isUndefined(v) && v.constructor === Object); > } > function _isArray(v) { > return (v !== null && !_isUndefined(v) && v.constructor === Array); > } > The 'doc' object passed to the mapping function does not appear to have the > .constructor property defined, nor on any of the enclosed objects, this > causes the JSON processing to fail. > A simple workaround is to stringily the doc object and parse the results > back, the fixed document then has the .constructor properties correctly set > on all objects > var fixed = JSON.parse(JSON.stringify(doc)); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1252) A way to have views return _deleted documents
[ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13107698#comment-13107698 ] James Howe commented on COUCHDB-1252: - Time's up :p > A way to have views return _deleted documents > - > > Key: COUCHDB-1252 > URL: https://issues.apache.org/jira/browse/COUCHDB-1252 > Project: CouchDB > Issue Type: New Feature > Components: JavaScript View Server >Affects Versions: 1.1, 1.0.3 >Reporter: James Howe > > Given that documents can be 'soft' deleted / deleted with auditing data by > updating the document to include the _deleted property, it would be > incredibly useful if there were a way to access these documents in a map > function. Otherwise it is very difficult to find the auditing data - even > more so if the original ids are unknown. > I was thinking along the lines of a view query parameter 'include_deleted', > but don't really mind how this is implemented, as long as it is there. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1175) Improve content type negotiation for couchdb JSON responses
[ https://issues.apache.org/jira/browse/COUCHDB-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102742#comment-13102742 ] James Howe commented on COUCHDB-1175: - Just FYI, the JSONView Firefox extension gives you (optionally) an Accept of: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,application/json I would guess many people would have it, as it lets you use Couch APIs directly without struggling to match braces and quotes. > Improve content type negotiation for couchdb JSON responses > --- > > Key: COUCHDB-1175 > URL: https://issues.apache.org/jira/browse/COUCHDB-1175 > Project: CouchDB > Issue Type: Improvement >Affects Versions: 1.0.2 >Reporter: Robert Newson >Assignee: Robert Newson >Priority: Blocker > Fix For: 1.1.1, 1.2 > > > Currently we ignore qvalues when negotiation between 'application/json' and > 'text/plain' when returning JSON responses. > Specifically, we test directly for 'application/json' or 'text/plain' in the > Accept header. Different branches have different bugs, though. Trunk returns > 'application/json' if 'application/json' is present at all, even if it's less > preferred than 'text/plain' when qvalues are accounted for. > We should follow the standard. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1265) Replication can introduce duplicates into the seq_btree.
[ https://issues.apache.org/jira/browse/COUCHDB-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098788#comment-13098788 ] James Howe commented on COUCHDB-1265: - Presumably this bug could cause duplicate entries in views. Could it also cause views to contain deleted documents (that still had bodies which got them through the map)? > Replication can introduce duplicates into the seq_btree. > > > Key: COUCHDB-1265 > URL: https://issues.apache.org/jira/browse/COUCHDB-1265 > Project: CouchDB > Issue Type: Bug > Components: Database Core >Reporter: Paul Joseph Davis >Assignee: Paul Joseph Davis > Attachments: COUCHDB-1265.patch, replication-frenzy.py > > > Full description, test, and patch to follow shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1252) A way to have views return _deleted documents
[ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097067#comment-13097067 ] James Howe commented on COUCHDB-1252: - ETA on this patch Jason? > A way to have views return _deleted documents > - > > Key: COUCHDB-1252 > URL: https://issues.apache.org/jira/browse/COUCHDB-1252 > Project: CouchDB > Issue Type: New Feature > Components: JavaScript View Server >Affects Versions: 1.1, 1.0.3 >Reporter: James Howe > > Given that documents can be 'soft' deleted / deleted with auditing data by > updating the document to include the _deleted property, it would be > incredibly useful if there were a way to access these documents in a map > function. Otherwise it is very difficult to find the auditing data - even > more so if the original ids are unknown. > I was thinking along the lines of a view query parameter 'include_deleted', > but don't really mind how this is implemented, as long as it is there. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1179) erl.exe crash during view indexing
[ https://issues.apache.org/jira/browse/COUCHDB-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090866#comment-13090866 ] James Howe commented on COUCHDB-1179: - doc_count: 5477 doc_del_count: 33856 > erl.exe crash during view indexing > -- > > Key: COUCHDB-1179 > URL: https://issues.apache.org/jira/browse/COUCHDB-1179 > Project: CouchDB > Issue Type: Bug > Components: JavaScript View Server >Affects Versions: 1.0.2, 1.1 > Environment: Windows XP, 32-bit >Reporter: James Howe > > While querying new views consecutively, some of which are very large, erl.exe > crashes. First few sections of the dump: > =erl_crash_dump:0.1 > Thu May 26 11:34:32 2011 > Slogan: eheap_alloc: Cannot allocate 32097420 bytes of memory (of type > "old_heap"). > System version: Erlang R14B01 (erts-5.8.2) [source] [smp:4:4] [rq:4] > [async-threads:0] > Compiled: Fri Jan 21 22:17:42 2011 > Taints: crypto > Atoms: 9079 > =memory > total: 1566295328 > processes: 1556974804 > processes_used: 1556965252 > system: 9320524 > atom: 415297 > atom_used: 397915 > binary: 3483760 > code: 3026978 > ets: 1004308 > =hash_table:atom_tab > size: 6421 > used: 4864 > objs: 9079 > depth: 7 > =index_table:atom_tab > size: 9216 > limit: 1048576 > entries: 9079 > =hash_table:module_code > size: 197 > used: 126 > objs: 165 > depth: 3 > =index_table:module_code > size: 1024 > limit: 65536 > entries: 165 > =hash_table:export_list > size: 3203 > used: 2268 > objs: 3969 > depth: 6 > =index_table:export_list > size: 4096 > limit: 524288 > entries: 3969 > =hash_table:secondary_export_table > size: 97 > used: 0 > objs: 0 > depth: 0 > =hash_table:process_reg > size: 47 > used: 34 > objs: 56 > depth: 3 > =hash_table:fun_table > size: 797 > used: 543 > objs: 921 > depth: 6 > =hash_table:node_table > size: 11 > used: 1 > objs: 1 > depth: 1 > =hash_table:dist_table > size: 11 > used: 1 > objs: 1 > depth: 1 > =allocated_areas > sys_misc: 302763 > static: 499712 > atom_space: 131088 113730 > atom_table: 62629 > module_table: 4968 > export_table: 29280 > register_table: 252 > fun_table: 3250 > module_refs: 2048 > loaded_code: 2735300 > dist_table: 335 > node_table: 131 > bits_bufs_size: 0 > bif_timer: 40172 > link_lh: 0 > proc: 288316 285116 > atom_entry: 221580 221556 > export_entry: 192660 192132 > module_entry: 8740 8020 > reg_proc: 1492 1396 > monitor_sh: 41236 35556 > nlink_sh: 40532 39956 > fun_entry: 53044 51980 > db_tab: 12532 10452 > driver_event_data_state: 28 28 > driver_select_data_state: 356 260 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1254) Document design and view options
Document design and view options Key: COUCHDB-1254 URL: https://issues.apache.org/jira/browse/COUCHDB-1254 Project: CouchDB Issue Type: Improvement Components: Documentation Reporter: James Howe Priority: Minor Off the back of COUCH-1252, I've just discovered there are such things as design options and view options, including local_seq and include_design. Could somebody who knows what they do document them somewhere? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1252) A way to have views return _deleted documents
A way to have views return _deleted documents - Key: COUCHDB-1252 URL: https://issues.apache.org/jira/browse/COUCHDB-1252 Project: CouchDB Issue Type: New Feature Components: JavaScript View Server Affects Versions: 1.0.3, 1.1 Reporter: James Howe Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown. I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1179) erl.exe crash during view indexing
[ https://issues.apache.org/jira/browse/COUCHDB-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Howe updated COUCHDB-1179: Affects Version/s: 1.1 Happens with 1.1 too: =erl_crash_dump:0.1 Wed Aug 03 10:20:10 2011 Slogan: eheap_alloc: Cannot allocate 50152200 bytes of memory (of type "heap"). System version: Erlang R14B01 (erts-5.8.2) [source] [smp:4:4] [rq:4] [async-threads:4] Compiled: Mon May 9 10:06:59 2011 Taints: crypto Atoms: 9641 =memory total: 1453854888 processes: 1424142692 processes_used: 1424133548 system: 29712196 atom: 433545 atom_used: 425927 binary: 24181896 code: 3307903 ets: 403636 =hash_table:atom_tab size: 6421 used: 4979 objs: 9641 depth: 7 =index_table:atom_tab size: 10240 limit: 1048576 entries: 9641 =hash_table:module_code size: 197 used: 128 objs: 176 depth: 3 =index_table:module_code size: 1024 limit: 65536 entries: 176 =hash_table:export_list size: 3203 used: 2332 objs: 4176 depth: 7 =index_table:export_list size: 5120 limit: 524288 entries: 4176 =hash_table:secondary_export_table size: 97 used: 0 objs: 0 depth: 0 =hash_table:process_reg size: 47 used: 35 objs: 59 depth: 3 =hash_table:fun_table size: 797 used: 595 objs: 1116 depth: 6 =hash_table:node_table size: 11 used: 1 objs: 1 depth: 1 =hash_table:dist_table size: 11 used: 1 objs: 1 depth: 1 =allocated_areas sys_misc: 302356 static: 499712 atom_space: 131088 123926 atom_table: 66725 module_table: 4968 export_table: 33376 register_table: 252 fun_table: 3250 module_refs: 2048 loaded_code: 2990593 dist_table: 335 node_table: 131 bits_bufs_size: 0 bif_timer: 40172 link_lh: 0 proc: 168196 160196 atom_entry: 235732 235276 export_entry: 202340 202148 module_entry: 8740 8548 reg_proc: 1492 1468 monitor_sh: 21844 20964 nlink_sh: 20524 20284 fun_entry: 63196 62972 db_tab: 12532 11700 driver_event_data_state: 28 28 driver_select_data_state: 356 180 > erl.exe crash during view indexing > -- > > Key: COUCHDB-1179 > URL: https://issues.apache.org/jira/browse/COUCHDB-1179 > Project: CouchDB > Issue Type: Bug > Components: JavaScript View Server >Affects Versions: 1.0.2, 1.1 > Environment: Windows XP, 32-bit >Reporter: James Howe > > While querying new views consecutively, some of which are very large, erl.exe > crashes. First few sections of the dump: > =erl_crash_dump:0.1 > Thu May 26 11:34:32 2011 > Slogan: eheap_alloc: Cannot allocate 32097420 bytes of memory (of type > "old_heap"). > System version: Erlang R14B01 (erts-5.8.2) [source] [smp:4:4] [rq:4] > [async-threads:0] > Compiled: Fri Jan 21 22:17:42 2011 > Taints: crypto > Atoms: 9079 > =memory > total: 1566295328 > processes: 1556974804 > processes_used: 1556965252 > system: 9320524 > atom: 415297 > atom_used: 397915 > binary: 3483760 > code: 3026978 > ets: 1004308 > =hash_table:atom_tab > size: 6421 > used: 4864 > objs: 9079 > depth: 7 > =index_table:atom_tab > size: 9216 > limit: 1048576 > entries: 9079 > =hash_table:module_code > size: 197 > used: 126 > objs: 165 > depth: 3 > =index_table:module_code > size: 1024 > limit: 65536 > entries: 165 > =hash_table:export_list > size: 3203 > used: 2268 > objs: 3969 > depth: 6 > =index_table:export_list > size: 4096 > limit: 524288 > entries: 3969 > =hash_table:secondary_export_table > size: 97 > used: 0 > objs: 0 > depth: 0 > =hash_table:process_reg > size: 47 > used: 34 > objs: 56 > depth: 3 > =hash_table:fun_table > size: 797 > used: 543 > objs: 921 > depth: 6 > =hash_table:node_table > size: 11 > used: 1 > objs: 1 > depth: 1 > =hash_table:dist_table > size: 11 > used: 1 > objs: 1 > depth: 1 > =allocated_areas > sys_misc: 302763 > static: 499712 > atom_space: 131088 113730 > atom_table: 62629 > module_table: 4968 > export_table: 29280 > register_table: 252 > fun_table: 3250 > module_refs: 2048 > loaded_code: 2735300 > dist_table: 335 > node_table: 131 > bits_bufs_size: 0 > bif_timer: 40172 > link_lh: 0 > proc: 288316 285116 > atom_entry: 221580 221556 > export_entry: 192660 192132 > module_entry: 8740 8020 > reg_proc: 1492 1396 > monitor_sh: 41236 35556 > nlink_sh: 40532 39956 > fun_entry: 53044 51980 > db_tab: 12532 10452 > driver_event_data_state: 28 28 > driver_select_data_state: 356 260 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1227) Deleted validators are still run
Deleted validators are still run Key: COUCHDB-1227 URL: https://issues.apache.org/jira/browse/COUCHDB-1227 Project: CouchDB Issue Type: Bug Components: Database Core Affects Versions: 1.1 Environment: Windows XP Reporter: James Howe If design documents are deleted using an update to add _deleted while still retaining the validate_doc_update function, it is still run on updates. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1179) erl.exe crash during view indexing
erl.exe crash during view indexing -- Key: COUCHDB-1179 URL: https://issues.apache.org/jira/browse/COUCHDB-1179 Project: CouchDB Issue Type: Bug Components: JavaScript View Server Affects Versions: 1.0.2 Environment: Windows XP, 32-bit Reporter: James Howe While querying new views consecutively, some of which are very large, erl.exe crashes. First few sections of the dump: =erl_crash_dump:0.1 Thu May 26 11:34:32 2011 Slogan: eheap_alloc: Cannot allocate 32097420 bytes of memory (of type "old_heap"). System version: Erlang R14B01 (erts-5.8.2) [source] [smp:4:4] [rq:4] [async-threads:0] Compiled: Fri Jan 21 22:17:42 2011 Taints: crypto Atoms: 9079 =memory total: 1566295328 processes: 1556974804 processes_used: 1556965252 system: 9320524 atom: 415297 atom_used: 397915 binary: 3483760 code: 3026978 ets: 1004308 =hash_table:atom_tab size: 6421 used: 4864 objs: 9079 depth: 7 =index_table:atom_tab size: 9216 limit: 1048576 entries: 9079 =hash_table:module_code size: 197 used: 126 objs: 165 depth: 3 =index_table:module_code size: 1024 limit: 65536 entries: 165 =hash_table:export_list size: 3203 used: 2268 objs: 3969 depth: 6 =index_table:export_list size: 4096 limit: 524288 entries: 3969 =hash_table:secondary_export_table size: 97 used: 0 objs: 0 depth: 0 =hash_table:process_reg size: 47 used: 34 objs: 56 depth: 3 =hash_table:fun_table size: 797 used: 543 objs: 921 depth: 6 =hash_table:node_table size: 11 used: 1 objs: 1 depth: 1 =hash_table:dist_table size: 11 used: 1 objs: 1 depth: 1 =allocated_areas sys_misc: 302763 static: 499712 atom_space: 131088 113730 atom_table: 62629 module_table: 4968 export_table: 29280 register_table: 252 fun_table: 3250 module_refs: 2048 loaded_code: 2735300 dist_table: 335 node_table: 131 bits_bufs_size: 0 bif_timer: 40172 link_lh: 0 proc: 288316 285116 atom_entry: 221580 221556 export_entry: 192660 192132 module_entry: 8740 8020 reg_proc: 1492 1396 monitor_sh: 41236 35556 nlink_sh: 40532 39956 fun_entry: 53044 51980 db_tab: 12532 10452 driver_event_data_state: 28 28 driver_select_data_state: 356 260 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1163) Document returned by id, but cannot be found by rev
[ https://issues.apache.org/jira/browse/COUCHDB-1163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035502#comment-13035502 ] James Howe commented on COUCHDB-1163: - Just FYI we've already rescued all these documents into a clean database, taking whatever the GET without rev returned as "correct". Thus we don't really mind what rescuing you implement, but it seems most users would probably want to err on the side of caution if data loss is a possibility. > Document returned by id, but cannot be found by rev > --- > > Key: COUCHDB-1163 > URL: https://issues.apache.org/jira/browse/COUCHDB-1163 > Project: CouchDB > Issue Type: Bug > Components: Database Core >Affects Versions: 1.0.1, 1.0.2 >Reporter: James Howe >Priority: Blocker > Fix For: 1.0.3, 1.1 > > Attachments: COUCHDB-1163-1.1.x.patch, COUCHDB-1163.patch, Couch > logging for jira issue, cleaned-up.txt, formatted_rev_tree.txt > > > Somehow, our cluster has developed the following problem on a handful of > documents. Will post reproduction steps if we find them. All properties have > been redacted. All the documents this affects also have attachments, if that > is significant. Once a document is in this situation, it causes conflict > detection, replication and include_docs to behave incorrectly or outright > fail. > GET /database/4cdee83a118ea1cf3050b1d006144d46 returns > { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "10-df4bf65a6104ea240f100c30d3cb245d", > "foo": "bar" > } > GET /database/4cdee83a118ea1cf3050b1d006144d46?open_revs=all returns > [ > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "10-df4bf65a6104ea240f100c30d3cb245d", > "foo": "bar" > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "8-eea5e36daee12acd79a127abf36f7720", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "9-2cead1e4c813a4f0d10a9bc4aa28bfda", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "7-c3b44f004660caa496804409089b53d9", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "6-52e978041bb324d19e01a2ac5a243702", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "5-761bf28c6989f0fde41bdd5732c33159", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "4-abb005cf4b2d2dd12880a33af1e7066e", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "3-233e4624e620ec1c8b66f21a051832f8", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "10-55f0cdf9dd95ed230b733a2c826c842c", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "11-264c9d6c249ba2fc9b13df35cb447fd7", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "9-2cead1e4c813a4f0d10a9bc4aa28bfda", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "2-9f2df19059d9a460a12740a63a4d95e9", > _deleted: true > } > } > ] > GET > /database/4cdee83a118ea1cf3050b1d006144d46?rev=10-df4bf65a6104ea240f100c30d3cb245d > returns > { > "error": "not_found", > "reason": "missing" > } -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1163) Document returned by id, but cannot be found by rev
[ https://issues.apache.org/jira/browse/COUCHDB-1163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035473#comment-13035473 ] James Howe commented on COUCHDB-1163: - If there's doubt about whether the revision that's still there should be deleted or not, the tree-fixing code should probably preserve it above deletions, to avoid data loss. > Document returned by id, but cannot be found by rev > --- > > Key: COUCHDB-1163 > URL: https://issues.apache.org/jira/browse/COUCHDB-1163 > Project: CouchDB > Issue Type: Bug > Components: Database Core >Affects Versions: 1.0.1, 1.0.2 >Reporter: James Howe >Priority: Blocker > Fix For: 1.0.3, 1.1 > > Attachments: COUCHDB-1163-1.1.x.patch, COUCHDB-1163.patch, Couch > logging for jira issue, cleaned-up.txt, formatted_rev_tree.txt > > > Somehow, our cluster has developed the following problem on a handful of > documents. Will post reproduction steps if we find them. All properties have > been redacted. All the documents this affects also have attachments, if that > is significant. Once a document is in this situation, it causes conflict > detection, replication and include_docs to behave incorrectly or outright > fail. > GET /database/4cdee83a118ea1cf3050b1d006144d46 returns > { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "10-df4bf65a6104ea240f100c30d3cb245d", > "foo": "bar" > } > GET /database/4cdee83a118ea1cf3050b1d006144d46?open_revs=all returns > [ > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "10-df4bf65a6104ea240f100c30d3cb245d", > "foo": "bar" > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "8-eea5e36daee12acd79a127abf36f7720", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "9-2cead1e4c813a4f0d10a9bc4aa28bfda", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "7-c3b44f004660caa496804409089b53d9", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "6-52e978041bb324d19e01a2ac5a243702", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "5-761bf28c6989f0fde41bdd5732c33159", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "4-abb005cf4b2d2dd12880a33af1e7066e", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "3-233e4624e620ec1c8b66f21a051832f8", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "10-55f0cdf9dd95ed230b733a2c826c842c", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "11-264c9d6c249ba2fc9b13df35cb447fd7", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "9-2cead1e4c813a4f0d10a9bc4aa28bfda", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "2-9f2df19059d9a460a12740a63a4d95e9", > _deleted: true > } > } > ] > GET > /database/4cdee83a118ea1cf3050b1d006144d46?rev=10-df4bf65a6104ea240f100c30d3cb245d > returns > { > "error": "not_found", > "reason": "missing" > } -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1163) Document returned by id, but cannot be found by rev
[ https://issues.apache.org/jira/browse/COUCHDB-1163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034718#comment-13034718 ] James Howe commented on COUCHDB-1163: - I'm also trying to get a simple reproducible case, here are further details of our setup at the time these broken documents turned up. 8 couches, with each couch replicating to 3 others (continuous remote-remote replication with trivial filters). Validators present for all classes of document. Attachments being added to existing documents, creating conflicts (see COUCHDB-885). Due to a bug on our end, a lot of documents were updated on every couch at the same revision, repeatedly, causing lots more conflicts. At the same time, every 30 seconds, we queried for all conflicts (using a view with doc._conflicts) and did a bulk_docs POST for each doc, performing a no-change update to the deterministic couch winner, and deleting all others (i.e. {_id: foo, _rev: bar, _deleted: true}). This lasted for no more than a day after which we started noticing all kinds of things going wrong (replication getting stuck, documents that are impossible to update or delete, etc.) We're not in a position to run this exact setup again until we are certain corruption will not occur. > Document returned by id, but cannot be found by rev > --- > > Key: COUCHDB-1163 > URL: https://issues.apache.org/jira/browse/COUCHDB-1163 > Project: CouchDB > Issue Type: Bug > Components: Database Core >Affects Versions: 1.0.1, 1.0.2 >Reporter: James Howe >Priority: Critical > Attachments: Couch logging for jira issue > > > Somehow, our cluster has developed the following problem on a handful of > documents. Will post reproduction steps if we find them. All properties have > been redacted. All the documents this affects also have attachments, if that > is significant. Once a document is in this situation, it causes conflict > detection, replication and include_docs to behave incorrectly or outright > fail. > GET /database/4cdee83a118ea1cf3050b1d006144d46 returns > { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "10-df4bf65a6104ea240f100c30d3cb245d", > "foo": "bar" > } > GET /database/4cdee83a118ea1cf3050b1d006144d46?open_revs=all returns > [ > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "10-df4bf65a6104ea240f100c30d3cb245d", > "foo": "bar" > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "8-eea5e36daee12acd79a127abf36f7720", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "9-2cead1e4c813a4f0d10a9bc4aa28bfda", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "7-c3b44f004660caa496804409089b53d9", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "6-52e978041bb324d19e01a2ac5a243702", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "5-761bf28c6989f0fde41bdd5732c33159", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "4-abb005cf4b2d2dd12880a33af1e7066e", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "3-233e4624e620ec1c8b66f21a051832f8", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "10-55f0cdf9dd95ed230b733a2c826c842c", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "11-264c9d6c249ba2fc9b13df35cb447fd7", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "9-2cead1e4c813a4f0d10a9bc4aa28bfda", > _deleted: true > } > }, > { > "ok": { > "_id": "4cdee83a118ea1cf3050b1d006144d46", > "_rev": "2-9f2df19059d9a460a12740a63a4d95e9", > _deleted: true > } > } > ] > GET > /database/4cdee83a118ea1cf3050b1d006144d46?rev=10-df4bf65a6104ea240f100c30d3cb245d > returns > { > "error": "not_found", > "reason": "missing" > } -- This message is automatically genera
[jira] [Commented] (COUCHDB-885) Attachments introduce conflicts after replication.
[ https://issues.apache.org/jira/browse/COUCHDB-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034633#comment-13034633 ] James Howe commented on COUCHDB-885: Sorry, there is a _conflicts, it's not deleted at all. Having some JSON-blindness obviously. GET http://couch2/test/6028bdb0ea5f748321b3ef1e980ef0a5?open_revs=all [ { "ok": { "_id": "6028bdb0ea5f748321b3ef1e980ef0a5" "_rev": "2-49be403b7a6ddb504b657a86932df567" "_attachments": { "text_attachement": { "content_type": "application/octet-stream" "revpos": 2 "length": 7 "stub": true } } } } { "ok": { "_id": "6028bdb0ea5f748321b3ef1e980ef0a5" "_rev": "1-967a00dff5e02add41819138abb3284d" } } ] GET http://couch2/test/6028bdb0ea5f748321b3ef1e980ef0a5?conflicts=true { "_id": "6028bdb0ea5f748321b3ef1e980ef0a5" "_rev": "2-49be403b7a6ddb504b657a86932df567" "_conflicts": [ "1-967a00dff5e02add41819138abb3284d" ] "_attachments": { "text_attachement": { "content_type": "application/octet-stream" "revpos": 2 "length": 7 "stub": true } } } If this were caused by COUCHDB-902 then either both or neither should be fixed in 1.0.2 - this is my concern. > Attachments introduce conflicts after replication. > -- > > Key: COUCHDB-885 > URL: https://issues.apache.org/jira/browse/COUCHDB-885 > Project: CouchDB > Issue Type: Bug > Components: Database Core >Affects Versions: 1.0.1 > Environment: Mac OSX, Windows XP, Windows 7 >Reporter: Nikolai Teofilov > Fix For: 1.2 > > > Step to reproduce the bug: > 1. Make database "test" on a remote couchdb server that reside on a > different machine! > 2. Create new document: "http://remote-server:5984/test/doc"; > 3. Create database "replica" on the local couchdb server. > 4. Trigger pull replication http://remote-server:5984/test -> > http://localhost:5984/replica > 5. Attach a file to the replicated document on the local couchdb server. > 6. Trigger push replication http://localhost:5984/replica -> > http://remote-server:5984/test > The document in the test database shows one conflict. If instead attachment > the document is updated with a new field after the replication no conflict. > The same sequence shows no conflicts on CouchDB 0.11. > test.sh > #!/usr/bin/env bash > LOCAL="http://localhost:5984"; > REMOTE="http://koleto.couchone.com"; > echo "=" > echo "This sequence create a conflict after the replication" > echo "=" > echo "Cleanup" > curl --silent -X DELETE $LOCAL/replica > curl --silent -X DELETE $REMOTE/test > echo "Create database:" $REMOTE/test > curl --silent -X PUT $REMOTE/test > echo "Create empty document:" > curl --silent -d '{}' -X PUT $REMOTE/test/doc > echo "Create local databse:" $LOCAL/replica > curl --silent -X PUT $LOCAL/replica > echo "Pull replication:" REMOTE/test "->" $LOCAL/replica > curl --silent -X POST -d > "{\"source\":\"$REMOTE/test\",\"target\":\"$LOCAL/replica\"}" -H > 'Content-Type: application/json' $LOCAL/_replicate > echo "Put an attachment to the document:" > echo "foobar!" | curl --silent --upload-file - -X PUT > "$LOCAL/replica/doc/foo.txt?rev=1-967a00dff5e02add41819138abb3284d" > echo "Push replication:" $LOCAL/replica "->" $REMOTE/test > curl --silent -X POST -d > "{\"source\":\"$LOCAL/replica\",\"target\":\"$REMOTE/test\"}" -H > 'Content-Type: application/json' $LOCAL/_replicate > echo '## Get one conflic #' > curl --silent -X GET $REMOTE/test/doc?conflicts=true > echo "=" > echo "The same sequence with adding a field to the document" > echo "=" > echo "Cleanup" > curl --silent -X DELETE $LOCAL/replica > curl --silent -X DELETE $REMOTE/test > echo "Create database:" $REMOTE/test > curl --silent -X PUT $REMOTE/test > echo "Create empty document:" > curl --silent -d '{}' -X PUT $REMOTE/test/doc > echo "Create local databse:" $LOCAL/replica > curl --silent -X PUT $LOCAL/replica > echo "Pull replication:" REMOTE/test "->" $LOCAL/replica > curl --silent -X POST -d > "{\"source\":\"$REMOTE/test\",\"target\":\"$LOCAL/replica\"}" -H > 'Content-Type: application/json' $LOCAL/_replicate > echo "Adding a new field:" > curl --silent -d '{"foo":"bar" , "_rev" : > "1-967a00dff5e02add41819138abb3284d" }' -X PUT > $LOCAL/replica/doc?rev=1-967a00dff5e02ad
[jira] [Commented] (COUCHDB-885) Attachments introduce conflicts after replication.
[ https://issues.apache.org/jira/browse/COUCHDB-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034097#comment-13034097 ] James Howe commented on COUCHDB-885: And I think you're confusing deleted conflicts with multiple revisions. As expected, the _conflicts array is not there, as the conflict is created in a deleted state. If pull replication is performed - {source: http://couch1/test_db, target: test_db} - then the open_revs list shows the expected one open branch, not two. > Attachments introduce conflicts after replication. > -- > > Key: COUCHDB-885 > URL: https://issues.apache.org/jira/browse/COUCHDB-885 > Project: CouchDB > Issue Type: Bug > Components: Database Core >Affects Versions: 1.0.1 > Environment: Mac OSX, Windows XP, Windows 7 >Reporter: Nikolai Teofilov > Fix For: 1.2 > > > Step to reproduce the bug: > 1. Make database "test" on a remote couchdb server that reside on a > different machine! > 2. Create new document: "http://remote-server:5984/test/doc"; > 3. Create database "replica" on the local couchdb server. > 4. Trigger pull replication http://remote-server:5984/test -> > http://localhost:5984/replica > 5. Attach a file to the replicated document on the local couchdb server. > 6. Trigger push replication http://localhost:5984/replica -> > http://remote-server:5984/test > The document in the test database shows one conflict. If instead attachment > the document is updated with a new field after the replication no conflict. > The same sequence shows no conflicts on CouchDB 0.11. > test.sh > #!/usr/bin/env bash > LOCAL="http://localhost:5984"; > REMOTE="http://koleto.couchone.com"; > echo "=" > echo "This sequence create a conflict after the replication" > echo "=" > echo "Cleanup" > curl --silent -X DELETE $LOCAL/replica > curl --silent -X DELETE $REMOTE/test > echo "Create database:" $REMOTE/test > curl --silent -X PUT $REMOTE/test > echo "Create empty document:" > curl --silent -d '{}' -X PUT $REMOTE/test/doc > echo "Create local databse:" $LOCAL/replica > curl --silent -X PUT $LOCAL/replica > echo "Pull replication:" REMOTE/test "->" $LOCAL/replica > curl --silent -X POST -d > "{\"source\":\"$REMOTE/test\",\"target\":\"$LOCAL/replica\"}" -H > 'Content-Type: application/json' $LOCAL/_replicate > echo "Put an attachment to the document:" > echo "foobar!" | curl --silent --upload-file - -X PUT > "$LOCAL/replica/doc/foo.txt?rev=1-967a00dff5e02add41819138abb3284d" > echo "Push replication:" $LOCAL/replica "->" $REMOTE/test > curl --silent -X POST -d > "{\"source\":\"$LOCAL/replica\",\"target\":\"$REMOTE/test\"}" -H > 'Content-Type: application/json' $LOCAL/_replicate > echo '## Get one conflic #' > curl --silent -X GET $REMOTE/test/doc?conflicts=true > echo "=" > echo "The same sequence with adding a field to the document" > echo "=" > echo "Cleanup" > curl --silent -X DELETE $LOCAL/replica > curl --silent -X DELETE $REMOTE/test > echo "Create database:" $REMOTE/test > curl --silent -X PUT $REMOTE/test > echo "Create empty document:" > curl --silent -d '{}' -X PUT $REMOTE/test/doc > echo "Create local databse:" $LOCAL/replica > curl --silent -X PUT $LOCAL/replica > echo "Pull replication:" REMOTE/test "->" $LOCAL/replica > curl --silent -X POST -d > "{\"source\":\"$REMOTE/test\",\"target\":\"$LOCAL/replica\"}" -H > 'Content-Type: application/json' $LOCAL/_replicate > echo "Adding a new field:" > curl --silent -d '{"foo":"bar" , "_rev" : > "1-967a00dff5e02add41819138abb3284d" }' -X PUT > $LOCAL/replica/doc?rev=1-967a00dff5e02add41819138abb3284d > echo "Push replication:" $LOCAL/replica "->" $REMOTE/test > curl --silent -X POST -d > "{\"source\":\"$LOCAL/replica\",\"target\":\"$REMOTE/test\"}" -H > 'Content-Type: application/json' $LOCAL/_replicate > echo '## No conflicts after the replication ' > curl --silent -X GET $REMOTE/test/doc?conflicts=true > -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-885) Attachments introduce conflicts after replication.
[ https://issues.apache.org/jira/browse/COUCHDB-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034037#comment-13034037 ] James Howe commented on COUCHDB-885: This issue is still present in 1.0.2, even though COUCHDB-902 is fixed. Specifically the following process creates a conflict: couch1 - add new document A in test_db replicate {source: test_db, target: http://couch2/testdb} couch1 - add attachment to A replicate {source: test_db, target: http://couch2/testdb} couch2/testdb/A?open_revs=all now shows a conflict > Attachments introduce conflicts after replication. > -- > > Key: COUCHDB-885 > URL: https://issues.apache.org/jira/browse/COUCHDB-885 > Project: CouchDB > Issue Type: Bug > Components: Database Core >Affects Versions: 1.0.1 > Environment: Mac OSX, Windows XP, Windows 7 >Reporter: Nikolai Teofilov > Fix For: 1.2 > > > Step to reproduce the bug: > 1. Make database "test" on a remote couchdb server that reside on a > different machine! > 2. Create new document: "http://remote-server:5984/test/doc"; > 3. Create database "replica" on the local couchdb server. > 4. Trigger pull replication http://remote-server:5984/test -> > http://localhost:5984/replica > 5. Attach a file to the replicated document on the local couchdb server. > 6. Trigger push replication http://localhost:5984/replica -> > http://remote-server:5984/test > The document in the test database shows one conflict. If instead attachment > the document is updated with a new field after the replication no conflict. > The same sequence shows no conflicts on CouchDB 0.11. > test.sh > #!/usr/bin/env bash > LOCAL="http://localhost:5984"; > REMOTE="http://koleto.couchone.com"; > echo "=" > echo "This sequence create a conflict after the replication" > echo "=" > echo "Cleanup" > curl --silent -X DELETE $LOCAL/replica > curl --silent -X DELETE $REMOTE/test > echo "Create database:" $REMOTE/test > curl --silent -X PUT $REMOTE/test > echo "Create empty document:" > curl --silent -d '{}' -X PUT $REMOTE/test/doc > echo "Create local databse:" $LOCAL/replica > curl --silent -X PUT $LOCAL/replica > echo "Pull replication:" REMOTE/test "->" $LOCAL/replica > curl --silent -X POST -d > "{\"source\":\"$REMOTE/test\",\"target\":\"$LOCAL/replica\"}" -H > 'Content-Type: application/json' $LOCAL/_replicate > echo "Put an attachment to the document:" > echo "foobar!" | curl --silent --upload-file - -X PUT > "$LOCAL/replica/doc/foo.txt?rev=1-967a00dff5e02add41819138abb3284d" > echo "Push replication:" $LOCAL/replica "->" $REMOTE/test > curl --silent -X POST -d > "{\"source\":\"$LOCAL/replica\",\"target\":\"$REMOTE/test\"}" -H > 'Content-Type: application/json' $LOCAL/_replicate > echo '## Get one conflic #' > curl --silent -X GET $REMOTE/test/doc?conflicts=true > echo "=" > echo "The same sequence with adding a field to the document" > echo "=" > echo "Cleanup" > curl --silent -X DELETE $LOCAL/replica > curl --silent -X DELETE $REMOTE/test > echo "Create database:" $REMOTE/test > curl --silent -X PUT $REMOTE/test > echo "Create empty document:" > curl --silent -d '{}' -X PUT $REMOTE/test/doc > echo "Create local databse:" $LOCAL/replica > curl --silent -X PUT $LOCAL/replica > echo "Pull replication:" REMOTE/test "->" $LOCAL/replica > curl --silent -X POST -d > "{\"source\":\"$REMOTE/test\",\"target\":\"$LOCAL/replica\"}" -H > 'Content-Type: application/json' $LOCAL/_replicate > echo "Adding a new field:" > curl --silent -d '{"foo":"bar" , "_rev" : > "1-967a00dff5e02add41819138abb3284d" }' -X PUT > $LOCAL/replica/doc?rev=1-967a00dff5e02add41819138abb3284d > echo "Push replication:" $LOCAL/replica "->" $REMOTE/test > curl --silent -X POST -d > "{\"source\":\"$LOCAL/replica\",\"target\":\"$REMOTE/test\"}" -H > 'Content-Type: application/json' $LOCAL/_replicate > echo '## No conflicts after the replication ' > curl --silent -X GET $REMOTE/test/doc?conflicts=true > -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1163) Document returned by id, but cannot be found by rev
Document returned by id, but cannot be found by rev --- Key: COUCHDB-1163 URL: https://issues.apache.org/jira/browse/COUCHDB-1163 Project: CouchDB Issue Type: Bug Components: Database Core Affects Versions: 1.0.2, 1.0.1 Reporter: James Howe Priority: Critical Somehow, our cluster has developed the following problem on a handful of documents. Will post reproduction steps if we find them. All properties have been redacted. All the documents this affects also have attachments, if that is significant. Once a document is in this situation, it causes conflict detection, replication and include_docs to behave incorrectly or outright fail. GET /database/4cdee83a118ea1cf3050b1d006144d46 returns { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "10-df4bf65a6104ea240f100c30d3cb245d", "foo": "bar" } GET /database/4cdee83a118ea1cf3050b1d006144d46?open_revs=all returns [ { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "10-df4bf65a6104ea240f100c30d3cb245d", "foo": "bar" } }, { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "8-eea5e36daee12acd79a127abf36f7720", _deleted: true } }, { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "9-2cead1e4c813a4f0d10a9bc4aa28bfda", _deleted: true } }, { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "7-c3b44f004660caa496804409089b53d9", _deleted: true } }, { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "6-52e978041bb324d19e01a2ac5a243702", _deleted: true } }, { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "5-761bf28c6989f0fde41bdd5732c33159", _deleted: true } }, { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "4-abb005cf4b2d2dd12880a33af1e7066e", _deleted: true } }, { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "3-233e4624e620ec1c8b66f21a051832f8", _deleted: true } }, { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "10-55f0cdf9dd95ed230b733a2c826c842c", _deleted: true } }, { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "11-264c9d6c249ba2fc9b13df35cb447fd7", _deleted: true } }, { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "9-2cead1e4c813a4f0d10a9bc4aa28bfda", _deleted: true } }, { "ok": { "_id": "4cdee83a118ea1cf3050b1d006144d46", "_rev": "2-9f2df19059d9a460a12740a63a4d95e9", _deleted: true } } ] GET /database/4cdee83a118ea1cf3050b1d006144d46?rev=10-df4bf65a6104ea240f100c30d3cb245d returns { "error": "not_found", "reason": "missing" } -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1149) Documentation of list functions doesn't explain how to use the 0.9 style
[ https://issues.apache.org/jira/browse/COUCHDB-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13031701#comment-13031701 ] James Howe commented on COUCHDB-1149: - Maybe someone could more completely document all the functions available in lists then. > Documentation of list functions doesn't explain how to use the 0.9 style > > > Key: COUCHDB-1149 > URL: https://issues.apache.org/jira/browse/COUCHDB-1149 > Project: CouchDB > Issue Type: Improvement > Components: Documentation >Affects Versions: 0.9 >Reporter: James Howe >Priority: Minor > Labels: list_function > > http://wiki.apache.org/couchdb/Formatting_with_Show_and_List#Listing_Views_with_CouchDB_0.9 > describes a style of list functions, but not where they should go in a > design document. Putting them under the "lists" key results in them being > invoked as the 0.10 versions. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1149) Documentation of list functions doesn't explain how to use the 0.9 style
[ https://issues.apache.org/jira/browse/COUCHDB-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13031674#comment-13031674 ] James Howe commented on COUCHDB-1149: - The question was asked under the misapprehension that the 0.9 API was still available. The reason I wanted to use them is that it appeared the old version allows row-by-row streaming to the client, avoiding a large performance penalty of having to buffer all the results before sending. > Documentation of list functions doesn't explain how to use the 0.9 style > > > Key: COUCHDB-1149 > URL: https://issues.apache.org/jira/browse/COUCHDB-1149 > Project: CouchDB > Issue Type: Improvement > Components: Documentation >Affects Versions: 0.9 >Reporter: James Howe >Priority: Minor > Labels: list_function > > http://wiki.apache.org/couchdb/Formatting_with_Show_and_List#Listing_Views_with_CouchDB_0.9 > describes a style of list functions, but not where they should go in a > design document. Putting them under the "lists" key results in them being > invoked as the 0.10 versions. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1045) Replication reports "missing" for docs which exist and are accesible
[ https://issues.apache.org/jira/browse/COUCHDB-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13031117#comment-13031117 ] James Howe commented on COUCHDB-1045: - COUCHDB-1154 - Same symptoms of an underlying cause? > Replication reports "missing" for docs which exist and are accesible > > > Key: COUCHDB-1045 > URL: https://issues.apache.org/jira/browse/COUCHDB-1045 > Project: CouchDB > Issue Type: Bug > Components: Replication >Affects Versions: 1.0.2 >Reporter: Rachel Willmer > > In one namespace we have, which has been migrated from 0.11.2 (and possibly > from 0.9 before that), we have errors like these showing up in the log file > when continuous replication is started. > {noformat} > [Thu, 27 Jan 2011 11:56:02 GMT] [error] [<0.467.0>] Replicator: error > accessing doc uid_103172924375609852 at > http://kvn1.back.incubator.telhc.local:15984/spaces/, reason: > {"missing":"30-08f26139287d260e33299aa8caa65ea8"} > [Thu, 27 Jan 2011 11:56:02 GMT] [error] [<0.466.0>] Replicator: error > accessing doc uid_104478515691512449 at > http://kvn1.back.incubator.telhc.local:15984/spaces/, reason: > {"missing":"1-3c494a705f5eb472ba7b12a32be8555c"} > [Thu, 27 Jan 2011 11:56:02 GMT] [error] [<0.465.0>] Replicator: error > accessing doc uid_6042240 at > http://kvn1.back.incubator.telhc.local:15984/spaces/, reason: > {"missing":"12-47695f5504aff0c0049cf34befb60bcc"} > {noformat} > But, if you ask directly for those docs, they exist and are returned. > In all cases I've tried, the _rev given as "missing" was the current one. > They are returned both with and without the _rev option in the URL -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1154) Revisions returned by GET, but not include_docs
Revisions returned by GET, but not include_docs --- Key: COUCHDB-1154 URL: https://issues.apache.org/jira/browse/COUCHDB-1154 Project: CouchDB Issue Type: Bug Components: Database Core Affects Versions: 1.0.2 Reporter: James Howe We have a cluster of many couches replicating between each other, and are using a view to find conflicts for resolution. However, running the view with include_docs returns doc: null for some id/rev pairs. These revisions can be retrieved using a simple GET, and appear as expected in the open_revs list. map: function(doc) { if (doc._conflicts) { emit(doc._id, {_id: doc._id, _rev: doc._rev}); doc._conflicts.forEach(function(rev) { emit(doc._id, {_id: doc._id, _rev: rev}); }); } } -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (COUCHDB-1149) Documentation of list functions doesn't explain how to use the 0.9 style
[ https://issues.apache.org/jira/browse/COUCHDB-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Howe reopened COUCHDB-1149: - > Documentation of list functions doesn't explain how to use the 0.9 style > > > Key: COUCHDB-1149 > URL: https://issues.apache.org/jira/browse/COUCHDB-1149 > Project: CouchDB > Issue Type: Improvement > Components: Documentation >Affects Versions: 0.9 >Reporter: James Howe >Priority: Minor > Labels: list_function > > http://wiki.apache.org/couchdb/Formatting_with_Show_and_List#Listing_Views_with_CouchDB_0.9 > describes a style of list functions, but not where they should go in a > design document. Putting them under the "lists" key results in them being > invoked as the 0.10 versions. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1149) Documentation of list functions doesn't explain how to use the 0.9 style
[ https://issues.apache.org/jira/browse/COUCHDB-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13029289#comment-13029289 ] James Howe commented on COUCHDB-1149: - I don't know the answer, so I can't. > Documentation of list functions doesn't explain how to use the 0.9 style > > > Key: COUCHDB-1149 > URL: https://issues.apache.org/jira/browse/COUCHDB-1149 > Project: CouchDB > Issue Type: Improvement > Components: Documentation >Affects Versions: 0.9 >Reporter: James Howe >Priority: Minor > Labels: list_function > > http://wiki.apache.org/couchdb/Formatting_with_Show_and_List#Listing_Views_with_CouchDB_0.9 > describes a style of list functions, but not where they should go in a > design document. Putting them under the "lists" key results in them being > invoked as the 0.10 versions. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1149) Documentation of list functions doesn't explain how to use the 0.9 style
Documentation of list functions doesn't explain how to use the 0.9 style Key: COUCHDB-1149 URL: https://issues.apache.org/jira/browse/COUCHDB-1149 Project: CouchDB Issue Type: Improvement Components: Documentation Affects Versions: 0.9 Reporter: James Howe Priority: Minor http://wiki.apache.org/couchdb/Formatting_with_Show_and_List#Listing_Views_with_CouchDB_0.9 describes a style of list functions, but not where they should go in a design document. Putting them under the "lists" key results in them being invoked as the 0.10 versions. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira