[jira] [Created] (COUCHDB-2513) Improve CouchDB rebar build scripts
Brian Mitchell created COUCHDB-2513: --- Summary: Improve CouchDB rebar build scripts Key: COUCHDB-2513 URL: https://issues.apache.org/jira/browse/COUCHDB-2513 Project: CouchDB Issue Type: Improvement Security Level: public (Regular issues) Components: Build System Reporter: Brian Mitchell I've worked on improving the rebar build process over the past week after having many issues on different machines which all required editing of various source controlled files in order to get things to build. The effort focusses on simplifying most of the rebar specific configuration. In most cases this means moving things out of scripts and into vanilla config files to make it clear what the defaults are. The other part is handling build flags for ports which include separate flags for each port properly to allow the user to pass in new configuration via environment variables. Finally, I have a user.config file which can be placed in the couch directory which can be generated by configure scripts. This allows merging in persistent values for building (for example, I often have non-standard icu4c paths so I'll ensure those are put into this file). There is a bit more work to be done cleaning up the configure script side of things but you can find a work in progress over at this git repo: https://github.com/strmpnk/couchdb-couch. I've also got more experimental changes at https://github.com/strmpnk/couchdb though I've not split out the build specific changes into a branch yet. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (COUCHDB-2506) Fix couch_att merge
Brian Mitchell created COUCHDB-2506: --- Summary: Fix couch_att merge Key: COUCHDB-2506 URL: https://issues.apache.org/jira/browse/COUCHDB-2506 Project: CouchDB Issue Type: Bug Security Level: public (Regular issues) Components: Database Core, HTTP Interface, Replication Reporter: Brian Mitchell This code was originally factored out of most of the system but it seems like some of the original code paths have been reverted to their old state. This code should be ported back to couch_att and replaced with calls into couch_att's code. It shouldn't be too hard but it does touch multiple components. As such it will require simultaneous pull requests across a few repositories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2063) Return current document with 409 response
[ https://issues.apache.org/jira/browse/COUCHDB-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901708#comment-13901708 ] Brian Mitchell commented on COUCHDB-2063: - We could pick the winning revision but that does dig that hole a little deeper. Maybe instead of true, you could provide the type of result like winning, all_conflicts, and siblings. > Return current document with 409 response > - > > Key: COUCHDB-2063 > URL: https://issues.apache.org/jira/browse/COUCHDB-2063 > Project: CouchDB > Issue Type: Bug > Security Level: public(Regular issues) >Reporter: Isaac Z. Schlueter > > You do a PUT, and it doesn't have the current rev. > So then you do a GET, to get the current rev. > Then you re-try your PUT. > Please make the second request unnecessary. Just send the current doc in the > 409 response body, unless the user opts-out with a `?send_current=false` or > something. > There are almost no examples of cases where you'd do a PUT and *not* want to > immediately GET the doc on a 409. The only case I could think of would be an > in-place modifying follower where you don't care about a 409, because it > means that another change is coming in the stream anyway. Are there any > others? Even in those cases, you could just set the opt-out flag to not get > the current data in the response. > The only harm in doing this is that outdated apps will still do the > (now-unnecessary) GET. That's not so bad. They'll keep working. > Systems with a single write-master and multiple read-slaves, however, will be > able to leverage this to great effect, rather than relying on cache-busting > query params and other kludges. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: [PROPOSAL] new underscore namespacing
On Tue, Dec 3, 2013 at 10:35 AM, Volker Mische wrote:
> I personally would prefer to have the meta information completely
> separate from the document. I know there have been discussion in the
> past to even have them separate in the backend (but that's not the point
> of this proposal).
>
> So the API for the view function could change to `function(doc, meta)`.
> This way you could store in your document whatever you like.
This is convenient in some cases but I really do like how CouchDB
documents are self-contained and self-representing. Having to rely on
an outside context to know things like identity and revision seems
unfortunate.
Most cases I've interacted with CouchDB would require at least that
minimal _id and _rev. Separating them might seem okay if you just look
at a map function with two arguments but it gets messier if you look
at client code in the applications that use CouchDB. Many do things
like:
my_doc = db.get('doc')
...
db.put(my_doc)
Juggling this context separately will require extra work or at least
force CouchDB clients to reinvent some way to wrap up and store
metadata. My experience writing client code against Riak, which
externalizes this metadata, was not nearly as clean as working with
CouchDB since everything needed some sort of wrapper where CouchDB
could many times get away with just a simple value.
On the side of internals, there is certainly some work to be done to
lift things in and out of this special field, but it could be done w/o
forcing clients to live with their world split in two.
Brian.
Re: [PROPOSAL] new underscore namespacing
On Tue, Dec 3, 2013 at 9:01 AM, Benjamin Young wrote: > Hi all, > > Recently the "doc._*" reservation has been causing me trouble when pulling > in "arbitrary" JSON from various sources that also use the underscore > prefixed names for things (HAL [1], vnd.error [2], other APIs). I've also > hit the wall several times when trying to import filesystem contents > (Sphinx, ghpages, and the like) that use _* prefixing for their "special > folders." I've usually stored such data inside nested objects. Our top-level namespace is getting muddy which not only affects our use of other formats but other systems that would want to integrate CouchDB data in them as well. I can imagine the sort of metadata that needs to be stored will only grow with plugins. > As such, I'd like to propose the following: > 1. Begin storing new reserved terms in doc._.* (rather than doc._*). > - this gives developers one object to look into for the meta-data about a > doc > - you can see the scope creep of our current doc._* best in the replicator > status messages. > - doc._ replication_* would become doc._.replication.* Very nice. It makes a clear separation between system-controlled and user-controlled name spaces. Avoiding the topic of whether or not metadata should be embedded or not, this seems like an improvement. > 2. Move "magic" API endpoints under "/_/" term as well (for the sake of > attachments. > - _design/doc would stay the same > - but the child endpoints would live under "_design/doc/_/*" > - _design/doc/_/view/by_date > - _design/doc/_/list/by_date/ul > - _design/doc/_/rewrite We do have many other places where we use special names like _changes, _all_dbs, _uuids &c. While this does mirror the document's special _ namespace, it makes it less consistent with everything else we put in the URL. I'm not sure what else we are really buying here outside of attachments with odd names. > I realize these are extreme API shifts, and would need to wait for CouchDB > 2.0. All the more reason to find a way to get plugins working well so there can be a playground for ideas before such a serious version jump is made. Some of these are trickier since we don't really expose proper APIs for manipulating some of these but it's probably worth looking at still. > The first steps this direction would be to put new reserved word keys into a > "doc._.*" namespace going forward. Closer to the "cut over" for 2.0 > duplicates of the existing keys (doc._id, doc._rev, especially) could also > live at their new underscore prefixed names (doc._.id, doc._.rev) which > would give devs a chance to migrate code and content. I wonder if the API could be versioned using HTTP headers. The CouchDB host could set a default version and clients could request specific versions. I've played with versioned vendor media types but those get complicated and confuse far too many HTTP agents. A simple header like Couch-API-Version: 2 might be a place to start. > Thanks for considering this, > Benjamin Thank you for sharing.
Re: Plugin Registry (Was Summary of IRC meeting in #couchdb-meeting)
On Thu, Nov 14, 2013 at 1:52 PM, till wrote: > > > On Thursday 14 November 2013 at 18:28, Simon Metson wrote: > >> Hey, >> >> > > I agree, src/couch_plugins/README.md (http://README.md) has a quasi >> > > roadmap that I want to work >> > > towards. The only edit for now would be that I’d like a registry that >> > > can be >> > > update independently from CouchDB releases. Would you be interested in >> > > helping >> > > out with that? Or anything else around plugins? >> > > >> > >> > >> > I'm definitely interested to hear more. I'd love to help out with >> > getting a common place for contributed couchdb things to be indexed >> > and published. Plugins seem like an ideal testing ground. How do you >> > imagine a plugin registry working? >> > >> >> >> I’m interested in this too. Can we pick up stuff from npm? >> Cheers >> Simon >> >> > > Please don’t. For starters, a more or less static registry would be awesome. > Not databased. I'm curious how static really avoids the database problem. Are you saying it'd be better to just pull down a know good URL and that never changes? Beyond that, I'm not sure we need to use CouchDB just like NPM does but it is pretty nice. Maybe fewer document mutations would help (splitting plugin releases into their own documents could be a good idea for those running local mirrors). > Then minor nitpicks like: > > - enforce semantic versions +1. We should expect plugins to follow the same conventions that CouchDB does. > - don’t allow people to re-upload releases Yes. We could avoid this partly by making releases more like immutable documents. > - make it easy to mirror it Replication is a natural fit here. There might be an argument for allowing automated installation via plain-old-HTTP or filesystem mirrors as well for the cases where people don't want to have to have a separate database running just to setup their own CouchDB nodes with private plugins (chicken-egg like situations emerge). > - some kind of signature for plugins/verification +1.
Plugin Registry (Was Summary of IRC meeting in #couchdb-meeting)
> >> # 4. plans for 1.6 release # > >> 19:15:31 [Kxepal]: btw. any plans for 1.6 release? > >> 19:15:46 [awenkhh]: when is 1.6 release scheduled for? > >> 19:15:46 [Kxepal]: one feature we have: stable Fauxton and replacement > >> Futon withit. any other goals? > >> 19:16:37 [Kxepal]: awenkhh: theoretically, there is release per one > >> month schedule, but practically month or two > >> 19:16:57 [Kxepal]: so it might be "holidays" release > >> 19:17:04 [awenkhh]: ;-) > >> 19:17:04 [Humbedooh]: a christmas couch :) > >> 19:17:18 [Kxepal]: yea! > >> 19:17:18 [Humbedooh]: Manger 1.6 > >> 19:17:18 [nslater]: awenkhh: we should kick it off in one month > >> 19:17:25 [nslater]: but completing it may take longer because hols > >> 19:17:34 [jan]: I’d like to work more on plugins and get the node > >> query server improved > > > > Should have this somewhere here: > >> 19:22:46 [strmpnk]: jan: any milestones for that in mind or just > >> general progress? > > > > I do think we should make sure experimental code doesn’t dwindle in the > > tree, not that these new additions are at all old. Having some intermediate > > milestones before we make these things stable would be interesting. > > > > For the plugins, it’d be good to have some contributed plugins users can > > easily activate to test the system. > > I agree, src/couch_plugins/README.md has a quasi roadmap that I want to work > towards. The only edit for now would be that I’d like a registry that can be > update independently from CouchDB releases. Would you be interested in helping > out with that? Or anything else around plugins? I'm definitely interested to hear more. I'd love to help out with getting a common place for contributed couchdb things to be indexed and published. Plugins seem like an ideal testing ground. How do you imagine a plugin registry working?
Re: Summary of IRC meeting in #couchdb-meeting, Wed Nov 6 19:09:32 2013
Thanks for the meeting log. I’ll note my comments were lagging behind from the channel. I’ll annotate inline. Brian. On Nov 6, 2013, at 15:13 , Alexander Shorin wrote: > Members present: jan, strmpnk, awenkhh, nslater, Dedalo, > chewbranca, Humbedooh, Kxepal > > > Meeting summary: > > > 1. Preface > 2. Welcome Andy Wenk > 3. 1.5. release > a. https://blogs.apache.org/couchdb/entry/apache_couchdb_1_5_0 > (Kxepal, 19:12:54) > 4. plans for 1.6 release > a. get a “couchdb architecture” diagram into the docs (jan, 19:26:03) > b. collaborate with benoitc to document CouchDB internals > (architecture, components, relations, services etc.) (Kxepal, > 19:27:09) > 5. l10n > a. https://translate.apache.org/projects/CouchDB/ (Kxepal, 19:31:29) > b. l...@couch.apache.org (Kxepal, 19:31:44) > c. jan to look into pootle/jenkins integration (jan, 19:35:19) > d. added awenkhh to jenkins (jan, 19:46:14) > 6. CouchDB conference > a. http://conf.couchdb.org/ (Kxepal, 19:52:10) > 7. Hackathon > a. http://www.couchhack.org/ (Kxepal, 19:53:16) > b. https://github.com/refuge/rcouch (Kxepal, 19:57:12) > > > Actions: > > > - get a “couchdb architecture” diagram into the docs (jan, 19:26:03) > - collaborate with benoitc to document CouchDB internals > (architecture, components, relations, services etc.) (Kxepal, > 19:27:09) > - jan to look into pootle/jenkins integration (jan, 19:35:19) > > IRC log follows: > > # 1. Preface # > 19:09:32 [jan]: not a live meeting, but about to start :) > 19:09:58 [jan]: First: Welcome awenkhh as a comitter :) > > > # 2. Welcome Andy Wenk # > 19:10:16 [Humbedooh]: welcome! \o/ > 19:10:16 [Kxepal]: awenkhh: welcome! (: > 19:10:18 [Dedalo]: welcome! > 19:10:39 [chewbranca]: ACTION cheers for awenkhh > 19:10:49 [awenkhh]: thank you all - it's a pleasure to be here ;-) > 19:11:35 [Kxepal]: wish you relax with CouchDB, us and have success in > all your ideas (: > 19:11:44 [awenkhh]: and would like to thank Noah for setting everything up !!! > 19:12:13 [awenkhh]: Kxepal: I will ;-) > 19:12:39 [Kxepal]: (; > > > # 3. 1.5. release # > Kxepal: https://blogs.apache.org/couchdb/entry/apache_couchdb_1_5_0 > 19:13:18 [Kxepal]: Finally. Dirkjan. Made it. HORRAY!(: > 19:13:39 [nslater]: three cheers for djc > 19:13:49 [Humbedooh]: *cheer* *cheer* *cheer* > 19:13:55 [chewbranca]: great work everyone!! > 19:14:38 [Kxepal]: Fauxton is awesome and impressive, but hope that > others experimental features will get stable flag in 1.6 (: > > > # 4. plans for 1.6 release # > 19:15:31 [Kxepal]: btw. any plans for 1.6 release? > 19:15:46 [awenkhh]: when is 1.6 release scheduled for? > 19:15:46 [Kxepal]: one feature we have: stable Fauxton and replacement > Futon withit. any other goals? > 19:16:37 [Kxepal]: awenkhh: theoretically, there is release per one > month schedule, but practically month or two > 19:16:57 [Kxepal]: so it might be "holidays" release > 19:17:04 [awenkhh]: ;-) > 19:17:04 [Humbedooh]: a christmas couch :) > 19:17:18 [Kxepal]: yea! > 19:17:18 [Humbedooh]: Manger 1.6 > 19:17:18 [nslater]: awenkhh: we should kick it off in one month > 19:17:25 [nslater]: but completing it may take longer because hols > 19:17:34 [jan]: I’d like to work more on plugins and get the node > query server improved Should have this somewhere here: > 19:22:46 [strmpnk]: jan: any milestones for that in mind or just > general progress? I do think we should make sure experimental code doesn’t dwindle in the tree, not that these new additions are at all old. Having some intermediate milestones before we make these things stable would be interesting. For the plugins, it’d be good to have some contributed plugins users can easily activate to test the system. > 19:17:51 [awenkhh]: ok > 19:18:36 [awenkhh]: I had in mind that it's 3 months so then I would > have propsed to take the first L10n steps ... but 1 month is too near > 19:18:38 [Kxepal]: btw..I have found comment from 1.5 release news. > not sure if it important, but I found it interesting: > 19:18:44 [Kxepal]: > Erlang, JavaScript, now Node.js. I think the end > of this project will be that it will tear to shreds because of > internal contradictions. There must be very serious reasons to use it > in such state. Should have this follow it: > 19:29:04 [strmpnk]: Do we have any evidence that people are genuinely > confused on this point? It just seems like a distraction to worry > about such statements. I’m not sure there is a good reason we should worry about commentary about how CouchDB is built and focus people on the ease of use and make sure that installations are always smooth. AFAICT, the project delivers on these points. > 19:19:23 [jan]: heh, this is a bit uninformed I’d say > 19:19:25 [Humbedooh]: same could be said about Apache in general, when > we branched out to include Java projects - but here we are, still > alive and kic
[jira] [Created] (COUCHDB-1483) Update handlers allow documents with empty ids to be created
Brian Mitchell created COUCHDB-1483:
---
Summary: Update handlers allow documents with empty ids to be
created
Key: COUCHDB-1483
URL: https://issues.apache.org/jira/browse/COUCHDB-1483
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 1.2, 1.3
Environment: I've tested this on multiple versions including 1.2.0 on
linux and a build from git master.
Reporter: Brian Mitchell
Priority: Blocker
If an update document returns a document without an _id field it will create or
attempt to update a document with an id of "".
To reproduce simply invoke an update handler which does a "return [{}, 'oops']".
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Apache CouchDB 1.2.0 release, third round
After some extended rebuilding (forgot to build against the 0.9.8 OpenSSL) I've got make check passing consistently and the browser tests all green [0]. Setup: OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5. +1 [0] I had one run where the view_compaction test failed on this assertion: "resp.view_index.disk_size < disk_size_before_compact". Rerunning cleared it up. On Saturday, March 24, 2012 at 13:39 , Noah Slater wrote: > Hello, > > I would like call a vote for the Apache CouchDB 1.2.0 release, third round. > > (I am calling this the "Haters Gon Hate" release.) > > Changes since last round: > > * COUCHDB-1424 fixed, thank you Bob Dionne > * COUCHDB-1426 fixed, thank you Benoît Chesneau (and Paul, and others) > * Performance regressions fixed, thank you Robert Newson > * R15B patch landed, thank you Jan Lehnardt > * COUCHDB-1445 fixed, thanks Randal Leeds > > We encourage the whole community to download and test these > release artefacts so that any critical issues can be resolved before the > release is made. Everyone is free to vote on this release, so get stuck in! > > We are voting on the following release artifacts: > > http://people.apache.org/~nslater/dist/1.2.0/ > > > These artifacts have been built from the following tree-ish in Git: > > 654768d32bae3639756c18b200582093aa51f349 > > > Please follow the test procedure before voting: > > http://wiki.apache.org/couchdb/Test_procedure > > > Thank you. > > Happy voting, > > N
[jira] [Commented] (COUCHDB-1424) make check hangs when compiling with R15B
[ https://issues.apache.org/jira/browse/COUCHDB-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217016#comment-13217016 ] Brian Mitchell commented on COUCHDB-1424: - Actually, 076-file-compression.t seems to hang for me when run alone and in sequence during make check for R15B on OS X. > make check hangs when compiling with R15B > - > > Key: COUCHDB-1424 > URL: https://issues.apache.org/jira/browse/COUCHDB-1424 > Project: CouchDB > Issue Type: Bug > Components: Test Suite >Affects Versions: 1.2, 1.3 >Reporter: Jan Lehnardt > > make check hangs when running under R15B. For me it is 160-vhosts.t where > execution stops, but if I recall correctly others have reported other tests. > The crux here is that running the tests individually succeeds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Apache CouchDB 1.2.0 release, second round
+1 (checks passed on Arch Linux 3.2.6 64bit, R14B04, Spidermonkey 1.8.5) On Wednesday, February 22, 2012 at 23:39 , Brian Mitchell wrote: > Same result as last vote on OS X 10.7.3 using Erlang OTP R15B and > spidermonkey 1.8.5 . `make check` hangs on 076-file-compression.t and passes > in the browser as long as I'm in private browsing mode or have the cache > disabled. I'll do a test on Linux tomorrow with R14B04 before I +1. > > Brian. > > On Wednesday, February 22, 2012 at 18:28 , Noah Slater wrote: > > > Hello, > > > > I would like call a vote for the Apache CouchDB 1.2.0 release, second round. > > > > We encourage the whole community to download and test these > > release artifacts so that any critical issues can be resolved before the > > release is made. Everyone is free to vote on this release, so get stuck in! > > > > We are voting on the following release artifacts: > > > > http://people.apache.org/~nslater/dist/1.2.0/ > > > > > > These artifacts have been built from the following tree-ish in Git: > > > > 4cd60f3d1683a3445c3248f48ae064fb573db2a1 > > > > > > Please follow the test procedure before voting: > > > > http://wiki.apache.org/couchdb/Test_procedure > > > > > > Thank you. > > > > Happy voting, > > > > N >
Re: [VOTE] Apache CouchDB 1.2.0 release, second round
Same result as last vote on OS X 10.7.3 using Erlang OTP R15B and spidermonkey 1.8.5 . `make check` hangs on 076-file-compression.t and passes in the browser as long as I'm in private browsing mode or have the cache disabled. I'll do a test on Linux tomorrow with R14B04 before I +1. Brian. On Wednesday, February 22, 2012 at 18:28 , Noah Slater wrote: > Hello, > > I would like call a vote for the Apache CouchDB 1.2.0 release, second round. > > We encourage the whole community to download and test these > release artifacts so that any critical issues can be resolved before the > release is made. Everyone is free to vote on this release, so get stuck in! > > We are voting on the following release artifacts: > > http://people.apache.org/~nslater/dist/1.2.0/ > > > These artifacts have been built from the following tree-ish in Git: > > 4cd60f3d1683a3445c3248f48ae064fb573db2a1 > > > Please follow the test procedure before voting: > > http://wiki.apache.org/couchdb/Test_procedure > > > Thank you. > > Happy voting, > > N
Re: Roadmap for the 1.3.0 release
Has there been any discussion around BigCouch integration strategies? It seems like it would fit the bill for the next undertaking on the general couch side. Does anyone from Cloudant have a suggestion for the timeline here? Any other work from mobile builds and the like might be interesting to support. Were there any interesting changes to pull in from the mobile and embedded device ports? Also, I'm not sure where the view engine refactoring work ended up… I'll look at JIRA but maybe there are specific follow-ups to be made here. Finally, for our included javascript libraries and Futon runtime, are we going to replace or update anything here? I'd imagine a newer jQuery could be an advantage for those doing CouchApps. Brian. On Tuesday, February 14, 2012 at 7:28 , Noah Slater wrote: > So far we have: > > https://issues.apache.org/jira/browse/COUCHDB-1410
Re: [VOTE] Apache CouchDB 1.2.0 release, first round
Seems it was chopped off as my outbox has it attached. Check here: https://gist.github.com/1785525 On Thursday, February 9, 2012 at 21:09 , Noah Slater wrote: > Missing your attachment. > > On Fri, Feb 10, 2012 at 1:57 AM, Brian Mitchell (mailto:binar...@gmail.com)> wrote: > > > On Thursday, February 9, 2012 at 20:38 , Brian Mitchell wrote: > > > > On Thursday, February 9, 2012 at 20:26 , Noah Slater wrote: > > > > Brian, were you able to follow the test procedure? > > > > > > Yes. OS X 10.7.3 with R14B04. It also seems to work on R15B after some > > quick tests (running full test suite in a moment). > > > > On make check I get a hang on test/etap/076-file-compression.t for R15B. > > Not sure what's up. > > > > Output of that test is attached.
Re: [VOTE] Apache CouchDB 1.2.0 release, first round
On Thursday, February 9, 2012 at 20:38 , Brian Mitchell wrote: > On Thursday, February 9, 2012 at 20:26 , Noah Slater wrote: > > Brian, were you able to follow the test procedure? > > > > > > > > > Yes. OS X 10.7.3 with R14B04. It also seems to work on R15B after some quick > tests (running full test suite in a moment). > On make check I get a hang on test/etap/076-file-compression.t for R15B. Not sure what's up. Output of that test is attached.
Re: [VOTE] Apache CouchDB 1.2.0 release, first round
On Thursday, February 9, 2012 at 20:26 , Noah Slater wrote: > Brian, were you able to follow the test procedure? > > Yes. OS X 10.7.3 with R14B04. It also seems to work on R15B after some quick tests (running full test suite in a moment).
Re: [VOTE] Apache CouchDB 1.2.0 release, first round
I've not had any issues with running on that change set. Looking through this release form 1.1, I'd say it puts the project in a much improved state. +1 for release. On Thursday, February 9, 2012 at 19:52 , Noah Slater wrote: > Hello, > > I would like call a vote for the Apache CouchDB 1.2.0 release, first round. > > We encourage the whole community to download and test these release > artifacts so that any critical issues can be resolved before the release is > made. Everyone is free to vote on this release, so get stuck in! > > We are voting on the following release artifacts: > > http://people.apache.org/~nslater/dist/1.2.0/ > > > These artifacts have been built from the following tree-ish in Git: > > cea48f36d8b088e3fd1c64e2aa1b9d9c742a3b71 > > > Please follow the test procedure before voting: > > http://wiki.apache.org/couchdb/Test_procedure > > > Thank you. > > Happy voting, > > N
Re: doc permission mask proposal
On Wednesday, April 27, 2011 at 6:48 PM, Jan Lehnardt wrote: > On 27 Apr 2011, at 15:43, Randall Leeds wrote: > > I think the answer is actually "yes". If you can see the design > > document you can see everything the view emits, even if it came from a > > document you can't view.Hm, I was thinking that the view updater would > > match the design doc acl against the doc acl when the view is created and > > exclude it if it doesn't match up for reads. I agree in this case. I think it'd be much more valuable to have a general read access policy on all documents (_uid/_gid) and allow views to break these rules. Each view could be tagged with a _uid/_gid. For writes, I'd use existing validations for regular writes and possibly allow update functions to be tagged with a _uid/_gid as well. _all_docs and _changes would have to be filtered on read unfortunately. This allows users to craft more refined access policies in their design documents rather than with overly-complex special attributes. CouchDB needs only to put the foundation for a feature like this, not do all the work. It does make writing design document a high privilege for any database but it's no different now. If we wanted to emulate a more fine grained access pattern, I'd encourage the developer to create more specific design documents and/or use a proxy server. Brian.
[jira] [Commented] (COUCHDB-1120) Snappy compression (databases, view indexes) + keeping doc bodies as ejson binaries
[
https://issues.apache.org/jira/browse/COUCHDB-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020343#comment-13020343
]
Brian Mitchell commented on COUCHDB-1120:
-
I've found the patch very useful in my limited testing. Right now I have a
rather large dataset (10's of TB to start) that I'm going to be dealing with
and the compression is an extremely valuable improvement (CPU time being
important but a good trade-off in my measurement). I'd be happy to work on
getting more testing done on this patch if there are points to evaluate more
closely.
I don't have any points for review on the code but it certainly worked. Paul
makes important points that might help this move to production worthy status
(C++ exceptions bringing things down is definitely not acceptable).
> Snappy compression (databases, view indexes) + keeping doc bodies as ejson
> binaries
>
>
> Key: COUCHDB-1120
> URL: https://issues.apache.org/jira/browse/COUCHDB-1120
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
> Environment: trunk
>Reporter: Filipe Manana
>Assignee: Filipe Manana
>
> The branch at:
> https://github.com/fdmanana/couchdb/compare/snappy
> Is an experiment which adds snappy compression to database files and view
> index files. Snappy is a very fast compressor/decompressor developed by and
> used by Google [1] - even for small data chunks like 100Kb it can be 2 orders
> of magnitude faster then zlib or Erlang's term_to_binary compression level 1.
> Somewhere at [1] there are benchmark results published by Google that compare
> against zlib's deflate, Erlang's term_to_binary compression, lzo, etc.
> Even small objects like database headers or btree nodes, still get smaller
> after compressing them with snappy, see the shell session at [2].
> Besides the compression, this branch also keeps the document bodies
> (#doc.body fields) as binaries (snappy compressed ejson binaries) and only
> converts them back to ejson when absolutely needed (done by
> couch_doc:to_json_obj/2 for e.g.) - this is similar to COUCHDB-1092 - but the
> bodies are EJSON compressed binaries and doesn't suffer from the same issue
> Paul identified before (which could be fixed without many changes) - on reads
> we decompress and still do the binary_to_term/1 + ?JSON_ENCODE calls as
> before.
> It also prepares the document summaries before sending the documents to the
> updater, so that we avoid copying EJSON terms and move this task outside of
> the updater to add more parallelism to concurrent updates.
> I made some tests, comparing trunk before and after the JSON parser NIF was
> added, against this snappy branch.
> I created databases with 1 000 000 documents of 4Kb each. The document
> template is this one: http://friendpaste.com/qdfyId8w1C5vkxROc5Thf
> The databases have this design document:
> {
> "_id": "_design/test",
> "language": "javascript",
> "views": {
> "simple": {
> "map": "function(doc) { emit(doc.data5.float1, [doc.strings[2],
> doc.strings[10]]); }"
> }
> }
> }
> == Results with trunk ==
> database file size after compaction: 7.5 Gb
> view index file size after compaction: 257 Mb
> ** Before JSON nif:
> $ time curl
> 'http://localhost:5985/trunk_db_1m/_design/test/_view/simple?limit=1'
> {"total_rows":100,"offset":0,"rows":[
> {"id":"0632-d25d-49c6-9b4e-e038b78ff97d","key":76.572,"value":["jURcBZ0vrJcmf2roZUMzZJQoTsKZDIdj7KhO7itskKvM80jBU9","fKYYthv8iFvaYoFoYZyB"]}
> ]}
> real 58m28.599s
> user 0m0.036s
> sys 0m0.056s
> ** After JSON nif:
> fdmanana 12:45:55 /opt/couchdb > time curl
> 'http://localhost:5985/trunk_db_1m/_design/test/_view/simple?limit=1'
> {"total_rows":100,"offset":0,"rows":[
> {"id":"0632-d25d-49c6-9b4e-e038b78ff97d","key":76.572,"value":["jURcBZ0vrJcmf2roZUMzZJQoTsKZDIdj7KhO7itskKvM80jBU9","fKYYthv8iFvaYoFoYZyB"]}
> ]}
> real 51m14.738s
> user 0m0.040s
> sys 0m0.044s
> == Results with the snappy branch ==
> database file size after compaction: 3.2 Gb (vs 7.5 Gb on trunk)
> view index file size after compaction: 100 Mb (vs 257 Mb on trunk)
> ** Before JSON nif:
> $ time
Re: Introducing Bram Neijt
On Tue, Oct 26, 2010 at 10:48, Jan Lehnardt wrote: > As for per-doc auth: It is very hard to get right and probably > against the nature of CouchDB. I'm not saying we shouldn't try > to solve it, but we need to be aware of the impact. > > I remember Damien saying that Notes did get per-doc auth, but > it wasn't a good solution and it sucked ever since. I don't > think anybody here wants that :) I'm currently looking for a clear explanation of what Notes did online but I just hit lots of manuals with very little help. If someone has a pointer here, I'd love to look into this. > The biggest problem here are views, the reduced kind. I kind of feel like views output something different enough from the original document that you should be able to tag each "document" that the view outputs in a similar manner. It'd also be interesting to consider view specific security defaults in a similar way to database security. That still means that maps with a reduce on it would not compute based on roles... but that the output would have to aggregate things in a way which would produce the desired result. This basically means, it'd be up to the view writer... not keeping them separate. In most of those cases, I see separate databases as a better solution than clever view partitioning per user. Other questions: Can the document security attributes add or only subtract from the database security defaults? Will we need a special category at the database level to note the types of users allowed to apply custom document access control or is the writer's group enough? Do we need both read and write or just read since we have other validation tools in place? Are there performance/efficiency concerns here? Should we expand HEAD responses to include more information relating to security? > While that is conceptually rather easy, you are basically creating > a view for each user. This may work for small amounts of data, > but not large, and many users. Exactly. I think multiple databases serves the purpose best here. If you need to box data up you can replicate to by-user-filtered databases or just keep separate databases to begin with. In the end I think there are some elegant API's already in place that seem like they'd be easy to transport as long as you don't make weak assumptions of how security will apply (i.e. view results). Right now I think a good route might be to experiment with transplanting the db/_security API as a special document attribute. It has simple semantics and could share the session system that the database code paths use. It also avoids requiring JS to run any code on requests. There might be problems but I can't say we're better off not trying something like this. Do others have ideas of how to take this work on? Maybe a prototype could be built using a node.js proxy first. Brian.
Re: Using rebar to install couchdb
couch_* seems a little repetitive. Why not core/, http/, etc? On Oct 14, 2010, at 18:52, Robert Dionne wrote: > +1 also > > I think the convention is > > ./apps/couch_core/ebin > ./apps/couch_core/src > ./apps/couch_core/include > ./apps/couch_core/priv > ./apps/couch_http/ebin > > > rather than ./src/ > > I like the idea of still using the existing build, which is awesome, and have > it feed into rebar so we can make use of reltools, etc, and templates for > parameterizing the various .ini files > > doing it after the next release will be a good time to break everything > > On Oct 14, 2010, at 4:03 PM, Robert Newson wrote: > >> Paul, >> >> Brilliant writeup and proposal. I'd like to see all those things >> happen pretty much as you said. Cleaning the cycles out will be much >> easier once things are broken out in that style. >> >> +1 >> >> B. >> >> On Thu, Oct 14, 2010 at 7:54 PM, Paul Davis >> wrote: >>> On Wed, Oct 13, 2010 at 5:23 PM, Benoit Chesneau >>> wrote: In an attempt to start some merging with cloudant I would like to start by using rebar in our install process. Like i see it, we could continue to use autotools to create the rebar.config files and other templates an then rebar for the final build and dependencies management. This changes as noticed by @davisp also imply we make our tree a little more OTP compliant. I would like to start this work asap. Thoughts ? - benoit >>> >>> So there's a couple issues at hand here which seem to be motivated by >>> the desire to start using tools like rebar. >>> >>> Our current source tree is not compliant with some of the basic >>> Erlang/OTP conventions. This is both bad technically and socially. >>> Technically, it prevents us from easily integrating tools like rebar >>> that would help advanced users with things like making Erlang reltools >>> packages. Socially, it doesn't reflect well on us to members of the >>> Erlang community that may have otherwise become contributors. All >>> languages have a standard package layout and Erlang is no different. >>> >>> The current CouchDB Erlang app has grown considerably. There's been >>> general consensus that we need to start splitting it up into smaller >>> applications that encompass specific functionality. There's been a bit >>> of effort in this direction, but its such a major change to source >>> file location it needs to have a community consensus to really start >>> working on seriously. >>> >>> I don't think we should focus directly on the issue of integrating >>> rebar. It should definitely be a goal, but not at the cost of our >>> current situation. Noah Slater has maintained an excellent build >>> system for us as is shown by the number of people building CouchDB >>> from source and the number of packages available. While I have argued >>> with him on numerous occasions about details, I have come to the >>> conclusion that it is not possible for him to be wrong. I personally >>> attribute this to the fact that he's most likely an advanced robot >>> from the future. That said, Noah has voiced concerns to various ideas >>> and we should make sure that any of his concerns are fully addressed. >>> >>> We should attempt to make sure that any tool support doesn't morph >>> into tool requirement. For instance, I think we should make sure that >>> its possible to keep compiling CouchDB without rebar and not come to >>> rely on it. >>> >>> While I'd be more than happy to start in on this and handle all of the >>> build system refactoring to make this happen, I'm not going to start >>> until there's a community consensus on what needs to be done. There >>> are a couple paths that I could see us taking to make this happen. We >>> could just make the current source tree be rebar compatible and figure >>> out the build system to do the optional rebar build or we could also >>> take this chance to split the source code into multiple applications. >>> Personally, I'd prefer to take this opportunity to organize the code >>> with multiple erlang apps. >>> >>> Too get the conversation rolling here's a first pass at a new app proposal: >>> >>> etap: >>> >>> Nick Gerakines now releases etap as a single .erl file that can be >>> dropped into the test directory. This app should be removed in favor >>> of that method. >>> >>> erlang-oauth: >>> >>> Should be renamed to just oauth. That erlang- prefix has bugged me >>> fore entirely too long. >>> >>> mochiweb, ibrowse, oauth: >>> >>> Refactored to use standard src, include, ebin, priv directories to >>> be OTP compliant. This results in directories like >>> >>> ./src/$APP/ebin >>> ./src/$APP/incldue >>> ./src/$APP/priv >>> ./src/$APP/src >>> >>> couchdb: >>> >>> Each proposed app will be structured as described above. Proposed apps: >>> >>> couch_core: The core Erlang modules for storing docs and managing >>> "internal infrast
Re: JavaScript 1.8 Features in SpiderMonkey (Please?)
On Mon, Mar 23, 2009 at 14:44, Mikeal Rogers wrote: > MacPorts is at 1.7 > > http://spidermonkey.darwinports.com/ > > I can't tell if apt is at 1.8 or 1.9 > > http://packages.debian.org/sid/libmozjs1d > http://packages.debian.org/sid/libnspr4-0d > > FreeBSD Ports is at 1.7 > > http://www.freebsd.org/cgi/cvsweb.cgi/ports/lang/spidermonkey/ I would imagine using anything but 1.7 would require inclusion of the "vendorized" spidermonkey sources somehow. I'm not sure what the policy is for that in an Apache project repository since it is under a non Apache license. Brian.
