[jira] Updated: (COUCHDB-465) Produce sequential, but unique, document id's
[ https://issues.apache.org/jira/browse/COUCHDB-465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Joseph Davis updated COUCHDB-465: -- Attachment: couch_uuids.patch Refactored a bit. couch_uuid_generator - couch_uuids sequential - hybrid (this was mostly playing, thoughts?) random is back to default and should stay that way call to random:uniform() - crypto:rand_uniform() (someone forgot to call random:seed(), tsk tsk :) [uuid_generator] - [uuids] in config added short description of choices next to option probably an unneccessary optimization to allow for couch_uuids:new(N) to return N ids. added simple checks to the uuids.js Futon tests Pulled in a contribution by Bob Dionne for etap tests. All calls to couch_util:new_uuid() are replaced with couch_uuuids:random() We should make the prefix length a configuration parameter I think. We might also think about adding Brian's algorithm for more diversity as well. Produce sequential, but unique, document id's - Key: COUCHDB-465 URL: https://issues.apache.org/jira/browse/COUCHDB-465 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Attachments: couch_uuids.patch, uuid_generator.patch Currently, if the client does not specify an id (POST'ing a single document or using _bulk_docs) a random 16 byte value is created. This kind of key is particularly brutal on b+tree updates and the append-only nature of couchdb files. Attached is a patch to change this to a two-part identifier. The first part is a random 12 byte value and the remainder is a counter. The random prefix is rerandomized when the counter reaches its maximum. The rollover in the patch is at 16 million but can obviously be changed. The upshot is that the b+tree is updated in a better fashion, which should lead to performance benefits. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-465) Produce sequential, but unique, document id's
[ https://issues.apache.org/jira/browse/COUCHDB-465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Joseph Davis updated COUCHDB-465: -- Attachment: couch_uuids.patch Added the utc_random algorithm. Updated tests to use it. hybrid - sequential unoptimized the calling to return a list for code cleanliness. Produce sequential, but unique, document id's - Key: COUCHDB-465 URL: https://issues.apache.org/jira/browse/COUCHDB-465 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Attachments: couch_uuids.patch, uuid_generator.patch Currently, if the client does not specify an id (POST'ing a single document or using _bulk_docs) a random 16 byte value is created. This kind of key is particularly brutal on b+tree updates and the append-only nature of couchdb files. Attached is a patch to change this to a two-part identifier. The first part is a random 12 byte value and the remainder is a counter. The random prefix is rerandomized when the counter reaches its maximum. The rollover in the patch is at 16 million but can obviously be changed. The upshot is that the b+tree is updated in a better fashion, which should lead to performance benefits. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-465) Produce sequential, but unique, document id's
[ https://issues.apache.org/jira/browse/COUCHDB-465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Joseph Davis updated COUCHDB-465: -- Attachment: (was: couch_uuids.patch) Produce sequential, but unique, document id's - Key: COUCHDB-465 URL: https://issues.apache.org/jira/browse/COUCHDB-465 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Attachments: couch_uuids.patch, uuid_generator.patch Currently, if the client does not specify an id (POST'ing a single document or using _bulk_docs) a random 16 byte value is created. This kind of key is particularly brutal on b+tree updates and the append-only nature of couchdb files. Attached is a patch to change this to a two-part identifier. The first part is a random 12 byte value and the remainder is a counter. The random prefix is rerandomized when the counter reaches its maximum. The rollover in the patch is at 16 million but can obviously be changed. The upshot is that the b+tree is updated in a better fashion, which should lead to performance benefits. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-465) Produce sequential, but unique, document id's
[ https://issues.apache.org/jira/browse/COUCHDB-465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745356#action_12745356 ] Antony Blakey commented on COUCHDB-465: --- I remember a discussion about assigning a UUID to a database to allow external view providesr to more reliably associate indexing artifacts with a given db instance (given that names can be duplicated over a create/delete/create boundary). I suggested that this could be used in document ids (and revids) and the response was that this was a privacy risk because the originating database's identity would leak into the peer group. It seems that this proposal has the same characteristic because you can identify the common source of a set of documents. Is this no longer a concern? If not, then how about revisiting the idea of assigning each database a UUID on creation, and using that UUID as a document identity prefix? Produce sequential, but unique, document id's - Key: COUCHDB-465 URL: https://issues.apache.org/jira/browse/COUCHDB-465 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Attachments: couch_uuids.patch, uuid_generator.patch Currently, if the client does not specify an id (POST'ing a single document or using _bulk_docs) a random 16 byte value is created. This kind of key is particularly brutal on b+tree updates and the append-only nature of couchdb files. Attached is a patch to change this to a two-part identifier. The first part is a random 12 byte value and the remainder is a counter. The random prefix is rerandomized when the counter reaches its maximum. The rollover in the patch is at 16 million but can obviously be changed. The upshot is that the b+tree is updated in a better fashion, which should lead to performance benefits. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-465) Produce sequential, but unique, document id's
[ https://issues.apache.org/jira/browse/COUCHDB-465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745358#action_12745358 ] Robert Newson commented on COUCHDB-465: --- I like most of Paul's changes though I thought we'd agreed on IRC to change the default to sequential and I'd still like to see that happen. I would also like to see a way to detect very quick delete/create scenarios, though I don't know if database uuids are the only solution there. If a global _changes feed would emit deleted and created events for databases in the correct order, then couchdb-lucene could work correctly without database uuids. Antony's suggestion of a fourth algorithm, where the prefix is completely static, is simple enough to add. This patch allows the deployer to decide how much he cares about predictability and server origin, so I don't see a reason not to add it. It is distinct from the sequential algorithm, though. The prefix there is only used for around 8000 ids and is then never reused, there is also no correlation between prefix and origin server. Produce sequential, but unique, document id's - Key: COUCHDB-465 URL: https://issues.apache.org/jira/browse/COUCHDB-465 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Attachments: couch_uuids.patch, uuid_generator.patch Currently, if the client does not specify an id (POST'ing a single document or using _bulk_docs) a random 16 byte value is created. This kind of key is particularly brutal on b+tree updates and the append-only nature of couchdb files. Attached is a patch to change this to a two-part identifier. The first part is a random 12 byte value and the remainder is a counter. The random prefix is rerandomized when the counter reaches its maximum. The rollover in the patch is at 16 million but can obviously be changed. The upshot is that the b+tree is updated in a better fashion, which should lead to performance benefits. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (COUCHDB-477) Add database uuid's
Add database uuid's --- Key: COUCHDB-477 URL: https://issues.apache.org/jira/browse/COUCHDB-477 Project: CouchDB Issue Type: New Feature Reporter: Robert Newson Add a uuid to db_header to distinguish different databases that have the same name (for example, by deleting and creating the same named database). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-477) Add database uuid's
[ https://issues.apache.org/jira/browse/COUCHDB-477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Newson updated COUCHDB-477: -- Attachment: 0001-add-uuid-to-database-on-creation-return-it-in-db_in.patch Add uuid=nil to #db_header, initialize it to a random uuid on creation, return it in db_info blob. Add database uuid's --- Key: COUCHDB-477 URL: https://issues.apache.org/jira/browse/COUCHDB-477 Project: CouchDB Issue Type: New Feature Reporter: Robert Newson Attachments: 0001-add-uuid-to-database-on-creation-return-it-in-db_in.patch Add a uuid to db_header to distinguish different databases that have the same name (for example, by deleting and creating the same named database). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-477) Add database uuid's
[ https://issues.apache.org/jira/browse/COUCHDB-477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bob Dionne updated COUCHDB-477: --- Comment: was deleted (was: I tested this all against a clean checkout of trunk and it looks good. The new algorithm is faster on inserts[1] but interestingly for single inserts is makes for a larger db pre-compact. After compaction the db is smaller by a factor of 3 I thought we leave the new sequentially random as the default? Paul, is the concern just what's advertised to users? Algorithms based on the system clock I think can be problematic as they assume all machines have the correct time. [1] http://gist.github.com/170982) Add database uuid's --- Key: COUCHDB-477 URL: https://issues.apache.org/jira/browse/COUCHDB-477 Project: CouchDB Issue Type: New Feature Reporter: Robert Newson Attachments: 0001-add-uuid-to-database-on-creation-return-it-in-db_in.patch Add a uuid to db_header to distinguish different databases that have the same name (for example, by deleting and creating the same named database). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-465) Produce sequential, but unique, document id's
[ https://issues.apache.org/jira/browse/COUCHDB-465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745402#action_12745402 ] Robert Newson commented on COUCHDB-465: --- The utc_random has that issue which is why it's not the default. I don't think it would make sense to choose utc_random unless you had already planned to keep your clocks pretty accurately synced. Brian's point, I'm implying, is that since he *can* arrange that, then if the ids used the clock time he'll get a nice ordering for identifiers for free. Anyone running couchdb in a data center could feasibly arrange for ntp synchronization, so I think it's a nice option for those that understand the consequences. Produce sequential, but unique, document id's - Key: COUCHDB-465 URL: https://issues.apache.org/jira/browse/COUCHDB-465 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Attachments: couch_uuids.patch, uuid_generator.patch Currently, if the client does not specify an id (POST'ing a single document or using _bulk_docs) a random 16 byte value is created. This kind of key is particularly brutal on b+tree updates and the append-only nature of couchdb files. Attached is a patch to change this to a two-part identifier. The first part is a random 12 byte value and the remainder is a counter. The random prefix is rerandomized when the counter reaches its maximum. The rollover in the patch is at 16 million but can obviously be changed. The upshot is that the b+tree is updated in a better fashion, which should lead to performance benefits. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (COUCHDB-478) Modify couch_httpd_auth:get_user to read from [admins] in .ini config and fall back to using database
Modify couch_httpd_auth:get_user to read from [admins] in .ini config and fall back to using database - Key: COUCHDB-478 URL: https://issues.apache.org/jira/browse/COUCHDB-478 Project: CouchDB Issue Type: Improvement Reporter: Jason Davies Fix For: 0.10 In the future we will make this pluggable (and orderable) but for now I think it is a sensible default to make oauth and cookie auth respect [admins] in the .ini as well as reading users from the database. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-478) Modify couch_httpd_auth:get_user to read from [admins] in .ini config and fall back to using database
[ https://issues.apache.org/jira/browse/COUCHDB-478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Davies updated COUCHDB-478: - Component/s: HTTP Interface Modify couch_httpd_auth:get_user to read from [admins] in .ini config and fall back to using database - Key: COUCHDB-478 URL: https://issues.apache.org/jira/browse/COUCHDB-478 Project: CouchDB Issue Type: Improvement Components: HTTP Interface Reporter: Jason Davies Fix For: 0.10 Attachments: oauth_ini_users.patch In the future we will make this pluggable (and orderable) but for now I think it is a sensible default to make oauth and cookie auth respect [admins] in the .ini as well as reading users from the database. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-465) Produce sequential, but unique, document id's
[ https://issues.apache.org/jira/browse/COUCHDB-465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745429#action_12745429 ] Robert Newson commented on COUCHDB-465: --- Bob, I read your gist and the pre-compact size for sequential is smaller than for random (43,144,791 vs 52,517,614) and post-compaction was smaller for sequential too (2,023,522 vs 2,506,850). Perhaps you read the numbers the wrong way around? Or did I? :) I also note that most of this compaction is from not batching as you'll have a 4k footer block for each write. With ?batch=ok and sequential (and no deletions) I find compaction to be only a marginal reduction of space. Produce sequential, but unique, document id's - Key: COUCHDB-465 URL: https://issues.apache.org/jira/browse/COUCHDB-465 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Attachments: couch_uuids.patch, uuid_generator.patch Currently, if the client does not specify an id (POST'ing a single document or using _bulk_docs) a random 16 byte value is created. This kind of key is particularly brutal on b+tree updates and the append-only nature of couchdb files. Attached is a patch to change this to a two-part identifier. The first part is a random 12 byte value and the remainder is a counter. The random prefix is rerandomized when the counter reaches its maximum. The rollover in the patch is at 16 million but can obviously be changed. The upshot is that the b+tree is updated in a better fashion, which should lead to performance benefits. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-465) Produce sequential, but unique, document id's
[ https://issues.apache.org/jira/browse/COUCHDB-465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745438#action_12745438 ] Bob Dionne commented on COUCHDB-465: Robert, sorry for the confusion, there are 4 tests there, tests 2 and 4 use the old random and the new sequential random codes respectively. So I'm comparing: precompact: 50020852 postcompact: 6099042 to precompact: 52517614 postcompact: 2506850 That's why I used the phrase sequentially random. Sequential in these tests (1 and 3) means 1,2,3 I agree about the batching, the diffs are almost noise with a batch size of 1000, which is what the compactor uses. Cheers, Bob Produce sequential, but unique, document id's - Key: COUCHDB-465 URL: https://issues.apache.org/jira/browse/COUCHDB-465 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Attachments: couch_uuids.patch, uuid_generator.patch Currently, if the client does not specify an id (POST'ing a single document or using _bulk_docs) a random 16 byte value is created. This kind of key is particularly brutal on b+tree updates and the append-only nature of couchdb files. Attached is a patch to change this to a two-part identifier. The first part is a random 12 byte value and the remainder is a counter. The random prefix is rerandomized when the counter reaches its maximum. The rollover in the patch is at 16 million but can obviously be changed. The upshot is that the b+tree is updated in a better fashion, which should lead to performance benefits. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-465) Produce sequential, but unique, document id's
[ https://issues.apache.org/jira/browse/COUCHDB-465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745458#action_12745458 ] Paul Joseph Davis commented on COUCHDB-465: --- For Bob's comment on times: The documentation for Erlang's now() function guarantees that its monotonically increasing so there's no need for me to force the suffix to be random then +1 as it'll already be ordered and the extra randomness will help prevent spurious conflicts when replicating. Yes, clocks can go out of sync but its not critical to have them in sync, and its a non-standard option. And adding a comment in the ini file about time syncing would be just fine. Regarding the sequential default, I don't remember anyone convincing me to make it something other than random. Though I may have just forgotten that conversation. I fear that any algorithm beyond completely random is a performance hack and I think that we should force people to consider the consequences when using one. Produce sequential, but unique, document id's - Key: COUCHDB-465 URL: https://issues.apache.org/jira/browse/COUCHDB-465 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Attachments: couch_uuids.patch, uuid_generator.patch Currently, if the client does not specify an id (POST'ing a single document or using _bulk_docs) a random 16 byte value is created. This kind of key is particularly brutal on b+tree updates and the append-only nature of couchdb files. Attached is a patch to change this to a two-part identifier. The first part is a random 12 byte value and the remainder is a counter. The random prefix is rerandomized when the counter reaches its maximum. The rollover in the patch is at 16 million but can obviously be changed. The upshot is that the b+tree is updated in a better fashion, which should lead to performance benefits. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-478) Modify couch_httpd_auth:get_user to read from [admins] in .ini config and fall back to using database
[
https://issues.apache.org/jira/browse/COUCHDB-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745594#action_12745594
]
Elliot Murphy commented on COUCHDB-478:
---
I think this patch has a typo:
+ {section: oauth_token_users,
+ key: bar, value: testamdmin},
amdmin instead of admin
Modify couch_httpd_auth:get_user to read from [admins] in .ini config and
fall back to using database
-
Key: COUCHDB-478
URL: https://issues.apache.org/jira/browse/COUCHDB-478
Project: CouchDB
Issue Type: Improvement
Components: HTTP Interface
Reporter: Jason Davies
Fix For: 0.10
Attachments: oauth_ini_users.patch
In the future we will make this pluggable (and orderable) but for now I think
it is a sensible default to make oauth and cookie auth respect [admins] in
the .ini as well as reading users from the database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: reduce_limit error
On Tue, Aug 18, 2009 at 2:34 AM, Brian Candlerb.cand...@pobox.com wrote: On Sun, Aug 16, 2009 at 05:55:56PM +0100, Robert Newson wrote: You can turn it off (caveat emptor); ; Changing reduce_limit to false will disable reduce_limit. ; If you think you're hitting reduce_limit with a good reduce function, ; please let us know on the mailing list so we can fine tune the heuristic. [query_server_config] reduce_limit = true I think that it should be a limit in bytes, not a true/false. I have also had cases where I generate a limited-sized reduce value which is a bit larger than 200 bytes. Oh I see that now - returning the first item in the list makes more sense. As far as a byte limit for the threshold, that would be entirely possible and not that hard to patch. I'd love to see a patch along these lines. I like the suggestion to make the threshold configurable instead of just an on/off switch. Chris -- Chris Anderson http://jchrisa.net http://couch.io
Re: Apache sub-projects
On Tue, Aug 18, 2009 at 2:55 AM, Jan Lehnardtj...@apache.org wrote: On 18 Aug 2009, at 09:10, Bernd Fondermann wrote: On Tue, Aug 18, 2009 at 02:38, Jan Lehnardtj...@apache.org wrote: Hi Paul, snip/ Related: - Do we want to foster plugins, extensions and other infrastructure software or do we want to rely on the non CouchDB open source world to come up with them? I think, that's the real question: What software does the CouchDB project want to provide to its users as (a) product(s). If you have the answer to that, you can still decide how to organize it and how to call it. For example, Apache Lucene provides Lucene (a programming framework) and Solr (a ready-to-go server). Both a complimentary. More important, they nurse each other with new feature. This ain't no one way street. Thanks, that's what I was trying to say. If there is software which is really important in the CouchDB ecosystem, I'd try to take them (code + people) onboard. +1 Thanks for all the feedback. I'm still digesting it and waiting to hear what a few more of the committers think. I think this discussion hasn't swayed my opinion about bringing on the sub-projects, but it does raise interesting questions about the CouchDB-Lounge - Erlang Lounge transformation (that are independent of the sub-project question, but still interesting...) Chris Cheers Jan -- -- Chris Anderson http://jchrisa.net http://couch.io
[jira] Commented: (COUCHDB-478) Modify couch_httpd_auth:get_user to read from [admins] in .ini config and fall back to using database
[
https://issues.apache.org/jira/browse/COUCHDB-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745604#action_12745604
]
Elliot Murphy commented on COUCHDB-478:
---
When running tests in futon against this patch applied to an svn snapshot of
svn rev 805910 on ubuntu karmic I get two test failures:
in the replication tests:
{message:docB is
null,fileName:http://127.0.0.1:5984/_utils/script/couch_test_runner.js,lineNumber:53,stack:;([object
Object],[object
Object])@http://127.0.0.1:5984/_utils/script/couch_test_runner.js:53\u000a((void
0))@http://127.0.0.1:5984/_utils/script/couch_test_runner.js:66\u000arun(-1)@http://127.0.0.1:5984/_utils/script/couch_test_runner.js:78\u000a}
and in the oauth tests:
# Assertion failed: xhr.status == expectedCode
# Assertion failed: xhr.status == expectedCode
# Assertion failed: xhr.status == expectedCode
# Assertion failed: xhr.status == expectedCode
# Assertion failed: xhr.status == expectedCode
# Assertion failed: xhr.status == expectedCode
# Assertion failed: xhr.status == expectedCode
# Assertion failed: xhr.status == expectedCode
Modify couch_httpd_auth:get_user to read from [admins] in .ini config and
fall back to using database
-
Key: COUCHDB-478
URL: https://issues.apache.org/jira/browse/COUCHDB-478
Project: CouchDB
Issue Type: Improvement
Components: HTTP Interface
Reporter: Jason Davies
Fix For: 0.10
Attachments: oauth_ini_users.patch
In the future we will make this pluggable (and orderable) but for now I think
it is a sensible default to make oauth and cookie auth respect [admins] in
the .ini as well as reading users from the database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-415) View sorting
[ https://issues.apache.org/jira/browse/COUCHDB-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745609#action_12745609 ] Chris Anderson commented on COUCHDB-415: I'm not that into this patch, because it will require complex optimizations to work on view queries with more than a trivial number of rows. Otherwise we'll asking the Erlang VM to store the entirety of the view query in memory, and that's not feasible. Footrest looks like it's coming along, now that it has the ability to use another CouchDB database as a scratch pad. I also don't think this blocks the user-definable collations. Those don't suffer from the same memory usage problems, as they are applied one row at a time when the rows are indexed. View sorting Key: COUCHDB-415 URL: https://issues.apache.org/jira/browse/COUCHDB-415 Project: CouchDB Issue Type: Wish Affects Versions: 0.9 Reporter: Nils Breunese I'd like to be able to sort the output of a view. I have a view that generates a list of event objects with start and end timestamps (and some other data). I emit this data using the end timestamp as the key, because that is what I want to use for my ?startkey=... query. But I'd like the results to be sorted by the start timestamp. I understand this is not currently possible using regular CouchDB views. The CouchDB Users mailinglist suggested to use couchdb-lucene, as that does offer the option to sort on any indexed field. I'm trying that route now, but it feels like overkill to add a full-text indexer to CouchDB just to be able to sort the output of my view. Some soundbites from the IRC channel when I brought this up: rnewson breun: jan___: in this case the data to sort on is known at map time, so an extension to emit to support multiple keys (with separate btrees) would do it. not sure how much effort that is, though. rnewson sorting at query time feels infeasible. jan rnewson: I think davisp had something like that working back in the day. i.e. alternate sort oder. davisp? rnewson jan___: I missed that, but it sounds reasonable at emit time. davisp I think its this one: http://github.com/davisp/couchdb-old-patches/tree/view-sort I would love to have view sorting native in CouchDB. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-465) Produce sequential, but unique, document id's
[ https://issues.apache.org/jira/browse/COUCHDB-465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745619#action_12745619 ] Chris Anderson commented on COUCHDB-465: I think the work that has been done has allayed my concerns. I'd suggest making the sequentialish-randomy one the default. I think db uuids are their own issue, and are needed for lots of reasons. Produce sequential, but unique, document id's - Key: COUCHDB-465 URL: https://issues.apache.org/jira/browse/COUCHDB-465 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Attachments: couch_uuids.patch, uuid_generator.patch Currently, if the client does not specify an id (POST'ing a single document or using _bulk_docs) a random 16 byte value is created. This kind of key is particularly brutal on b+tree updates and the append-only nature of couchdb files. Attached is a patch to change this to a two-part identifier. The first part is a random 12 byte value and the remainder is a counter. The random prefix is rerandomized when the counter reaches its maximum. The rollover in the patch is at 16 million but can obviously be changed. The upshot is that the b+tree is updated in a better fashion, which should lead to performance benefits. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-477) Add database uuid's
[ https://issues.apache.org/jira/browse/COUCHDB-477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745620#action_12745620 ] Chris Anderson commented on COUCHDB-477: this patch looks good to me. +1 on applying it. Add database uuid's --- Key: COUCHDB-477 URL: https://issues.apache.org/jira/browse/COUCHDB-477 Project: CouchDB Issue Type: New Feature Reporter: Robert Newson Attachments: 0001-add-uuid-to-database-on-creation-return-it-in-db_in.patch Add a uuid to db_header to distinguish different databases that have the same name (for example, by deleting and creating the same named database). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-465) Produce sequential, but unique, document id's
[ https://issues.apache.org/jira/browse/COUCHDB-465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bob Dionne updated COUCHDB-465: --- Attachment: 041-uuid-gen.t 041-uuid-gen-utc.ini 041-uuid-gen-seq.ini etaps to test the new couch_uuids work of rnewson and davisp. Produce sequential, but unique, document id's - Key: COUCHDB-465 URL: https://issues.apache.org/jira/browse/COUCHDB-465 Project: CouchDB Issue Type: Improvement Reporter: Robert Newson Attachments: 041-uuid-gen-seq.ini, 041-uuid-gen-utc.ini, 041-uuid-gen.t, couch_uuids.patch, uuid_generator.patch Currently, if the client does not specify an id (POST'ing a single document or using _bulk_docs) a random 16 byte value is created. This kind of key is particularly brutal on b+tree updates and the append-only nature of couchdb files. Attached is a patch to change this to a two-part identifier. The first part is a random 12 byte value and the remainder is a counter. The random prefix is rerandomized when the counter reaches its maximum. The rollover in the patch is at 16 million but can obviously be changed. The upshot is that the b+tree is updated in a better fashion, which should lead to performance benefits. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: svn commit: r804427 - in /couchdb/trunk: etc/couchdb/default.ini.tpl.in share/www/script/test/delayed_commits.js src/couchdb/couch_db.erl src/couchdb/couch_httpd_db.erl
On Thu, Aug 20, 2009 at 12:52:27PM -0700, Chris Anderson wrote: e.g. a PUT request could specify max_wait=2000 (if not specified, use a default value from the ini file). Subsequent requests could specify their own max_wait params, and a full commit would occur when the earliest of these times occurs. max_wait=0 would then replace the x-couch-full-commit: header, which seems like a bit of a frig to me anyway. I like the approach you've outlined here. I think the default could eventually be to allow CouchDB to be adaptive for high throughput, but we'd have to write that code first. Once we have all this, batch=ok could still be meaningful, when the client wants to be able to send a bunch of serial writes to CouchDB and is fine with the reduced guarantees that 202 Accepted gives. Yes, I agree that batch=ok makes sense if it means just give me a 202 response straight away, and the data will probably make it to disk some time in the future.
Re: View Performance (was Re: The 1.0 Thread)
Thanks, Dmitry. I've sent the update to github now. On Fri, Jul 10, 2009 at 03:32:21AM +0300, Dmitry Unkovsky wrote: P.S. and, github is fine, I cannot find a password I need to 'git push' ... by the way, I've checked it just now, they pretty fast send you a token for lost password, if it's the case. Or, to 'git push' from command line, you should add your public ssh key to your github account at https://github.com/account Add another public key, there are instructions for that - http://github.com/guides/providing-your-ssh-key In short, I did something like $ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (~/.ssh/id_rsa): enter Enter passphrase (empty for no passphrase): enter passphrase or hit enter Enter same passphrase again: enter passphrase again Your identification has been saved in ~/.ssh/id_rsa. Your public key has been saved in ~/.ssh/id_rsa.pub. The key fingerprint is: aa:aa: then vim ~/.ssh/config add Host github.com User github-username Port 22 Hostname github.com IdentityFile ~/.ssh/id_rsa TCPKeepAlive yes IdentitiesOnly yes then in local repository $git remote add origin g...@github.com:oilcrayons/around-codemirror.git after that I'm able to push to github from local repo with just a line - $git push
