[jira] [Created] (COUCHDB-1252) A way to have views return _deleted documents
A way to have views return _deleted documents - Key: COUCHDB-1252 URL: https://issues.apache.org/jira/browse/COUCHDB-1252 Project: CouchDB Issue Type: New Feature Components: JavaScript View Server Affects Versions: 1.0.3, 1.1 Reporter: James Howe Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown. I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086238#comment-13086238 ] Noah Slater commented on COUCHDB-1012: -- Making our own script would be super trivial. If we go down that route, there's no point in also using pkg-config. Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1250) Start accepting pull requests via github...
[ https://issues.apache.org/jira/browse/COUCHDB-1250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noah Slater updated COUCHDB-1250: - Description: ... this system is a bear for simple documentation fixes - something this community really needs! Summary: Start accepting pull requests via github... (was: Start accepting pull requests via github - this system is a bear for simple documentation fixes - something this community really needs!) Start accepting pull requests via github... --- Key: COUCHDB-1250 URL: https://issues.apache.org/jira/browse/COUCHDB-1250 Project: CouchDB Issue Type: Wish Reporter: Mike McKay ... this system is a bear for simple documentation fixes - something this community really needs! -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Configuration Load Order
On 16 Aug 2011, at 23:03, Randall Leeds wrote: Agreed also. We still have a question about load and save order. One idea would be to track the .ini file from whence an option came. If an option comes from a local.ini or local.d/ file it could be updated in place. If it comes from a default.ini or default.d/ file, updates should be placed in local.ini. This would make the most sense to me. I think our previous discussion holds on this. CouchDB should not be modifying default.ini, default.d/*, local.ini, or local.d/* at all. Anything CouchDB needs to write back to the config files should be put in runtime.ini, or runtime.d/* only. Sorry, I couldn't help giving the whole generated thing a new name while I was at it. In this case, it means that someone might put a password hash in local.ini, and CouchDB would later write out an updated password hash to runtime.ini, but that is fine. The runtime.ini file is loaded after local.ini, so as long as it is in place, the setting there would override any previous values. To be honest, getting back to a previous sub-thread of this thread, if CouchDB is needing to write back at all, that might be a good indication that the config value should be held in CouchDB in the first place. If you look at how MySQL uses its config files, for example, you see that all the values there are values that the database needs BEFORE IT INITIALISES and is ready to serve requests. Anything else is kept within the MySQL instance itself. If we were going to remove anything from the ini files that isn't needed for database initialisation, what would we loose? I am guessing the users and passwords would go, which would obviate this issue. MySQL keeps its user information in MySQL. I would also be in favor of enforcing a load order that supports a directory structure like: local.d/ 010-stuff.ini 020-others.ini We don't need to ship anything like that by default. I think right now we take the load directories on the command line, no? It'd be nice if the order of resolution within those directories was well specified. This is how the system is designed. The code that loads files from a directory is: for file in $1/*.ini; do if [ -r $file ]; then _add_config_file $file fi done This should list files in some alphanum sort order, as demonstrated: nslater@tumbolia:~$ echo test/* test/b test/c test/d test/e test/f test/h test/i test/j This means that with an example full config directory, the load order should look like: default.ini default.d/001_foo.ini default.d/002_bar.ini local.ini local.d/001_foo.ini local.d/002_bar.ini runtime.ini
Re: Configuration Load Order
On 16 Aug 2011, at 23:07, Robert Newson wrote: nice idea to have a separate htpasswd (-like) file. Passwords are special, let's treat them accordingly. Just to clarify. The proposal you're commenting on only suggest a separate password script that generates hashes which are then put into the regular ini files.
Re: Configuration Load Order
Oh, that's not what I'm +1 for then. I specifically mean that passwords are never held in any .ini file, but instead in a separate password file that contains nothing but passwords, and never plaintext passwords either. It is updated with a script which takes username and password, crunches it, and writes it to the file. I used 'htpasswd' as shorthand for all of the above but clearly was too terse, sorry. I quite like your proposal too, though, where only bootstrapping (database_dir, etc) are in config and the rest can be read from a special _config database (thus eliminating any manual alterations from outside _config). B. On 17 August 2011 12:29, Noah Slater nsla...@apache.org wrote: On 16 Aug 2011, at 23:07, Robert Newson wrote: nice idea to have a separate htpasswd (-like) file. Passwords are special, let's treat them accordingly. Just to clarify. The proposal you're commenting on only suggest a separate password script that generates hashes which are then put into the regular ini files.
Re: Configuration Load Order
On 17 Aug 2011, at 02:04, Jason Smith wrote: The American quip goes: the professor who never even ran for dog catcher presumes to tell the president how to do his job. Developers who spend all day in ./utils/run pontificate about good daemon behavior in an OS or distribution. Just to clear up the ad hominem before I continue. Most of your arguments are directed comments I have made. But of course, I built the original system, and used my experience as a Debian developer to inform my decisions about best practice. I've also been a sysadmin for a number of companies, and in fact, I have never used ./utils/run for anything, so… My proposal is already implemented. Now I say promote HTTP config (Futon) over .ini files when possible. Integrators, packagers, and advanced sysadmins can attack the .ini files just as before. The more I've reflected on this thread, the more I think that the ini files should be for pre-initialisation settings. Conversely, anything post-initialisation should probably be moved to a special CouchDB database. So it seems we are in agreement here. Now we are invoking use cases of versioning the config, and auditing it. Wow! In many setups, the basic text file configuration of a system is versioned and audited. Think again of the MySQL daemon configuration versus the users and permissions. Debian packages themselves are, of course, a great example of this. Each package ships with it a default configuration that gets the daemon up and running. It would be much harder to do this if that process involved starting CouchDB and then communicating with the database. It would, in the very least, involve writing a custom tool that speaks to CouchDB. For the basics, you want to be able to just move a file to the correct part of a filesystem. Config files that change themselves are bizarre and scary. Agreed. Admins, passwords, and non-boostrappy configuration over HTTP seems more Couch-like, more of the web, and more relaxed. Agreed. Take a MySQL admin, or an admin of Drupal, Wordpress, Moodle, Joomla, or pretty much any big PHP application. Tell them this: You have to get CouchDB up in the first place. So you edit some config files. Once it's up, you connect with your client/browser. It assumes you are an admin, and you complete installation over that interface. They would respond: Yeah, sure. Agreed. I do not buy the misbehaving Couch scenario. Firstly, how common is that? After installation and confirmation, daemons get pretty stable. If a misconfiguration totally destroys the couch, well, they are still plain text. As before, load emacs and go for it! The argument was that if daemon configuration lives inside the daemon, and you mess it up, then you wont be able to run the daemon to edit the daemon configuration. As long as there is a separation of concerns, this shouldn't be a problem. You shouldn't be able to hose you CouchDB installation through the configuration options hosted within CouchDB. Think of it in terms of a managed hosting environment. As the service provider, you want to be able to let your customers configure users and permissions and URL handlers, etc. But you don't, necessarily, want them to be messing about with the daemon options. Finally, I am basically happy with the Couch config. It's quirky but not too bad. I only hope to share a fresh perspective: the viewpoint of people for whom couch is just another daemon, like MySQL or httpd or cron. Why is it quirky? Luckily, I have been building the system from that perspective from the start!
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086309#comment-13086309 ] Benoit Chesneau commented on COUCHDB-1012: -- the script is already done and attached to this patch. So what are we waiting for? Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: The replicator needs a superuser mode
On Aug 16, 2011, at 11:37 PM, Jason Smith wrote: 2. Replication is a whole worldview, adding ?force=true breaks that worldview Replication is not a worldview, it's a mechanism by which documents are transferred between databases. I think that's the crux of our disagreement. Cheers, Adam
Re: Configuration Load Order
On Wed, Aug 17, 2011 at 8:24 PM, Noah Slater nsla...@apache.org wrote: On 17 Aug 2011, at 02:04, Jason Smith wrote: The American quip goes: the professor who never even ran for dog catcher presumes to tell the president how to do his job. Developers who spend all day in ./utils/run pontificate about good daemon behavior in an OS or distribution. Just to clear up the ad hominem before I continue. Most of your arguments are directed comments I have made. But of course, I built the original system, and used my experience as a Debian developer to inform my decisions about best practice. I've also been a sysadmin for a number of companies, and in fact, I have never used ./utils/run for anything, so… Sorry, I'll cut that out on the dev@ list. While I addressed many of your points later on, the motivation to chime in came directly from Randall's post. Out of the blue comes discussion of couchctl, and password management tools, and config file convention-over-configuration. My feeling is make configuration simpler than it is today. Looks like you and I are in broad agreement on the details. Thanks for the feedback! I do not buy the misbehaving Couch scenario. Firstly, how common is that? After installation and confirmation, daemons get pretty stable. If a misconfiguration totally destroys the couch, well, they are still plain text. As before, load emacs and go for it! The argument was that if daemon configuration lives inside the daemon, and you mess it up, then you wont be able to run the daemon to edit the daemon configuration. As long as there is a separation of concerns, this shouldn't be a problem. You shouldn't be able to hose you CouchDB installation through the configuration options hosted within CouchDB. Think of it in terms of a managed hosting environment. As the service provider, you want to be able to let your customers configure users and permissions and URL handlers, etc. But you don't, necessarily, want them to be messing about with the daemon options. I have no difficulties thinking of it in terms of a managed hosting environment :) and I wrote config_whitelist because users were changing their listen address and port, taking their couches offline. Totally agree. Finally, I am basically happy with the Couch config. It's quirky but not too bad. I only hope to share a fresh perspective: the viewpoint of people for whom couch is just another daemon, like MySQL or httpd or cron. Why is it quirky? The .ini file that edits itself. There are probably some servers out there which do that, but as you know, sysadmins expect /etc files to stay put! Many use CVS to manage them, as you noted. Hence, I like the generated.ini idea because as a sysadmin, during peacetime (couch is running fine) it is just an opaque file blob I need to back up. During wartime (couch is crashed/misconfigured), I can dig into it with emacs and troubleshoot. -- Iris Couch
Re: The replicator needs a superuser mode
On 17 Aug 2011, at 11:06, Benoit Chesneau wrote: Philosophy apart, dump and restore could be indeed useful to bootstrap db, make plain backup/restore strategies, exchange dbs over a disk/mem card without any couch installed etc. Yep, but in my mind this should live outside CouchDB's HTTP API. A dump and restore tool that lived on the command line, like the Subversion hotcopy stuff is the first thing that springs to mind. Or PostgreSQL's pgdump tool, or whatever. But as far as I understand the current file format, you should be able to just rsync the .couch files while the database is running.
Re: Configuration Load Order
On 17 Aug 2011, at 15:07, Jason Smith wrote: The .ini file that edits itself. There are probably some servers out there which do that, but as you know, sysadmins expect /etc files to stay put! Many use CVS to manage them, as you noted. Hence, I like the generated.ini idea because as a sysadmin, during peacetime (couch is running fine) it is just an opaque file blob I need to back up. During wartime (couch is crashed/misconfigured), I can dig into it with emacs and troubleshoot. Gotcha. :)
The _security object should be versioned
I believe the _security object should be versioned in order to ease synchronization of the object between databases. This proposal is motivated (unsurprisingly) by BigCouch, which typically stores multiple copies of each database in a multi-master configuration. When the _security object is written in BigCouch the update is issued to all available shards. We run into problems if an update is issued while some shards are unavailable, because we don't know how to synchronize the divergent copies once all the shards are back online. In my head I see us representing the _security object using a #full_doc_info, just as we would a document. Unlike documents the _security object (or a pointer to it) would still be written in the header of the database for fast access during request processing. I haven't quite decided what I think the API should look like, e.g. whether the full document API (including attachments?) should be supported. Regards, Adam
Re: Configuration Load Order
distilled from IRC chat A separate password file as described above, but can only be updated thus; # couchdb --set-password admin Password: foo Password updated. # Whether this password is written to a separate file or to a future _config database is immaterial, but separate file is proposed for the first implementation. In the case of the latter, the issue of locking yourself out is raised, so we propose adding; # couchdb --admin-party Apache CouchDB version (LogLevel=info) is starting. Apache CouchDB has started in admin party mode! Time to be scared! [info] [0.31.0] Apache CouchDB has started on http://127.0.0.1:5984/ # B. On 17 August 2011 15:16, Noah Slater nsla...@apache.org wrote: On 17 Aug 2011, at 15:07, Jason Smith wrote: The .ini file that edits itself. There are probably some servers out there which do that, but as you know, sysadmins expect /etc files to stay put! Many use CVS to manage them, as you noted. Hence, I like the generated.ini idea because as a sysadmin, during peacetime (couch is running fine) it is just an opaque file blob I need to back up. During wartime (couch is crashed/misconfigured), I can dig into it with emacs and troubleshoot. Gotcha. :)
Re: Configuration Load Order
On 18 August 2011 00:47, Robert Newson rnew...@apache.org wrote: Oh, that's not what I'm +1 for then. I specifically mean that passwords are never held in any .ini file, but instead in a separate password file that contains nothing but passwords, and never plaintext passwords either. It is updated with a script which takes username and password, crunches it, and writes it to the file. I used 'htpasswd' as shorthand for all of the above but clearly was too terse, sorry. +1 to htpasswd. self-mutilation is not a good look contravenes principle of least surprise. I quite like your proposal too, though, where only bootstrapping (database_dir, etc) are in config and the rest can be read from a special _config database (thus eliminating any manual alterations from outside _config). B. +0. I support minimal ini entries for bootstrapping. Personally when admining stuff I used svn; storing in couch makes that a script instead of a text file. It's already easy for people to lose DB access by fiddling with rewrite rules; we need to ensure its reversable and intuitive in case of self-inflicted muppet moments. I'm not convinced the coolness of having everything in a _config.couch outweighs the convenience of vi on .ini. A+ Dave On 17 August 2011 12:29, Noah Slater nsla...@apache.org wrote: On 16 Aug 2011, at 23:07, Robert Newson wrote: nice idea to have a separate htpasswd (-like) file. Passwords are special, let's treat them accordingly. Just to clarify. The proposal you're commenting on only suggest a separate password script that generates hashes which are then put into the regular ini files.
Re: The _security object should be versioned
On Wed, Aug 17, 2011 at 9:18 PM, Adam Kocoloski kocol...@apache.org wrote:
I believe the _security object should be versioned in order to ease
synchronization of the object between databases. This proposal is motivated
(unsurprisingly) by BigCouch, which typically stores multiple copies of each
database in a multi-master configuration. When the _security object is
written in BigCouch the update is issued to all available shards. We run
into problems if an update is issued while some shards are unavailable,
because we don't know how to synchronize the divergent copies once all the
shards are back online.
In my head I see us representing the _security object using a #full_doc_info,
just as we would a document. Unlike documents the _security object (or a
pointer to it) would still be written in the header of the database for fast
access during request processing.
I haven't quite decided what I think the API should look like, e.g. whether
the full document API (including attachments?) should be supported. Regards,
In the spirit of re-using existing working components:
How do you feel about migrating to a blessed _local/security document?
Maybe its latest version could be cached in the header for speed?
Pros:
* Couch gets (conceptually) simpler rather than more complex
* It's versioned, you get full doc semantics
* It doesn't replicate, but 3rd-party tools can pseudo-replicate it as needed
* Design documents can enforce policies: if(doc._id == _local/security
doc.members.length == 0) throw {forbidden:This database may never
be public}
Eagerly awaiting a list of cons :)
Since the _security object is conveniently glued to databases but not
replicated, I have been tempted to overload it to store per-database
security settings. There is the CORS discussion, but even simple
stuff:
{ admins: {names:[...], roles:[...]}
, members: {names:[...], roles:[...]}
, read_only: true
, create_only: true
, delete_only: true
, changes_only_allowed_on:thursday
}
// validate_doc_update
function(newDoc, oldDoc, userCtx, secObj) {
// read_only, create_only, delete_only obviously aren't true at the
// same time, just making a concise example.
// Read-only example
if(secObj.read_only)
throw {forbidden: This database is read-only};
// Create-only example
if(oldDoc secObj.create_only)
throw {forbidden: This database is create-only};
// Delete-only example
if(!newDoc._deleted secObj.delete_only)
throw {forbidden: This database is delete-only};
// Changes only allowed on Thursday left as an exercise.
}
However I have not done that because I'm not sure if it is with the
grain in CouchDB. Maybe these are best swallowed up into the ddoc
itself. (But then when I replicate, now I have to quickly update that
ddoc for that database's security environment.)
--
Iris Couch
Re: The _security object should be versioned
On Aug 17, 2011, at 10:35 AM, Jason Smith wrote:
On Wed, Aug 17, 2011 at 9:18 PM, Adam Kocoloski kocol...@apache.org wrote:
I believe the _security object should be versioned in order to ease
synchronization of the object between databases. This proposal is motivated
(unsurprisingly) by BigCouch, which typically stores multiple copies of each
database in a multi-master configuration. When the _security object is
written in BigCouch the update is issued to all available shards. We run
into problems if an update is issued while some shards are unavailable,
because we don't know how to synchronize the divergent copies once all the
shards are back online.
In my head I see us representing the _security object using a
#full_doc_info, just as we would a document. Unlike documents the _security
object (or a pointer to it) would still be written in the header of the
database for fast access during request processing.
I haven't quite decided what I think the API should look like, e.g. whether
the full document API (including attachments?) should be supported. Regards,
In the spirit of re-using existing working components:
How do you feel about migrating to a blessed _local/security document?
Maybe its latest version could be cached in the header for speed?
Pros:
* Couch gets (conceptually) simpler rather than more complex
* It's versioned, you get full doc semantics
* It doesn't replicate, but 3rd-party tools can pseudo-replicate it as needed
* Design documents can enforce policies: if(doc._id == _local/security
doc.members.length == 0) throw {forbidden:This database may never
be public}
Eagerly awaiting a list of cons :)
The only trouble I have with _local/security is that _local documents are
represented using #doc records instead of #full_doc_info records. As such,
they have no support for MVCC. If we managed to get into a situation where the
copies of the _local/security document had really diverged (instead of one copy
simply missing some updates), we'd have a difficult time merging those
conflicting versions.
On the other hand, the challenges I'm having with _security happen with any
_local document, too. Perhaps the right approach is to upgrade the storage
format for the local tree to use #full_doc_info and store the _security object
as a _local document like you suggest (cached on opening the database, of
course).
Since the _security object is conveniently glued to databases but not
replicated, I have been tempted to overload it to store per-database
security settings. There is the CORS discussion, but even simple
stuff:
{ admins: {names:[...], roles:[...]}
, members: {names:[...], roles:[...]}
, read_only: true
, create_only: true
, delete_only: true
, changes_only_allowed_on:thursday
}
// validate_doc_update
function(newDoc, oldDoc, userCtx, secObj) {
// read_only, create_only, delete_only obviously aren't true at the
// same time, just making a concise example.
// Read-only example
if(secObj.read_only)
throw {forbidden: This database is read-only};
// Create-only example
if(oldDoc secObj.create_only)
throw {forbidden: This database is create-only};
// Delete-only example
if(!newDoc._deleted secObj.delete_only)
throw {forbidden: This database is delete-only};
// Changes only allowed on Thursday left as an exercise.
}
However I have not done that because I'm not sure if it is with the
grain in CouchDB. Maybe these are best swallowed up into the ddoc
itself. (But then when I replicate, now I have to quickly update that
ddoc for that database's security environment.)
I have no qualms about adding that sort of information to _security. Regards,
Adam
Re: Configuration Load Order
On Wed, Aug 17, 2011 at 9:22 PM, Robert Newson rnew...@apache.org wrote: distilled from IRC chat A separate password file as described above, but can only be updated thus; # couchdb --set-password admin Password: foo Password updated. What problem is this solving exactly? This thread started because you edit foo.ini and subsequent changes go to bar.ini. That foo.ini happens to hold plaintext passwords instead of, say, TCP nodelay only underscores the problem. But plaintext vs. hashed passwords is a totally different matter. But regarding passwords, would you humor me and please re-state the requirements? I think it is a solution looking for a problem. Are we talking about moving *all* passwords to this file (ignoring _user doc .salt and .password_sha)? Or are we keeping those in sync now? Or is this just admin passwords? But only admins can see (hashed) passwords over HTTP. On Unix filesystems, if you have permission to read /etc/couchdb/local.ini then you very likely have permission to read /var/lib/couchdb/everything.couch, so what is the point? Regarding --set-password and couchctl, unless I am missing some serious requirement (possible), it sounds like CouchDB is poised to get much more complex soon. I spend all my free time bragging about how simple it is so that would be quite a blow to my ego. Thanks. -- Iris Couch
Re: The _security object should be versioned
On Wed, Aug 17, 2011 at 9:51 PM, Adam Kocoloski kocol...@apache.org wrote:
How do you feel about migrating to a blessed _local/security document?
Maybe its latest version could be cached in the header for speed?
Pros:
* Couch gets (conceptually) simpler rather than more complex
* It's versioned, you get full doc semantics
* It doesn't replicate, but 3rd-party tools can pseudo-replicate it as needed
* Design documents can enforce policies: if(doc._id == _local/security
doc.members.length == 0) throw {forbidden:This database may never
be public}
Eagerly awaiting a list of cons :)
The only trouble I have with _local/security is that _local documents are
represented using #doc records instead of #full_doc_info records. As such,
they have no support for MVCC.
I did not realize that, so one of the points I made was wrong. FWIW I
always assumed _local docs were normal docs, except they are
special-cased to neither replicate nor be represented in views.
Thanks for the tip!
--
Iris Couch
Re: The replicator needs a superuser mode
On Wed, Aug 17, 2011 at 9:11 PM, Noah Slater nsla...@apache.org wrote: On 17 Aug 2011, at 11:06, Benoit Chesneau wrote: Philosophy apart, dump and restore could be indeed useful to bootstrap db, make plain backup/restore strategies, exchange dbs over a disk/mem card without any couch installed etc. Yep, but in my mind this should live outside CouchDB's HTTP API. A dump and restore tool that lived on the command line, like the Subversion hotcopy stuff is the first thing that springs to mind. Or PostgreSQL's pgdump tool, or whatever. But as far as I understand the current file format, you should be able to just rsync the .couch files while the database is running. A small note, rsync copies the _security object and also _local docs. The latter are AFAIK only used by the replicator, and if you rsync to a different URL, those docs are pretty inert. It's not clear to me whether _security should travel with a database dump. It seems prudent to want to back that up. But if I restore to a different couch, it is imperative that I remember to correct the _security. The new couch (generally) has totally different user accounts and roles defined. Yet despite my initial disagreement with _dump, Adam has reminded or persuaded me (FWIW) that Couch really needs a better mechanism to clone or copy data efficiently. -- Iris Couch
Re: Configuration Load Order
Jason, The --set-password thing is to ensure there are no plaintext passwords in the first place, which eliminates the oddness of couch rewriting a plaintext pwd to a digested pwd (and putting the output in a different file). This is only for admin passwords. User passwords are held in _users as normal (though many are concerned that digested passwords are visible, and rightly so). And, as ever, holding your ego together is our primary concern but I think it can survive an improvement to how admin passwords are managed in couchdb. :) B. On 17 August 2011 15:58, Jason Smith j...@iriscouch.com wrote: On Wed, Aug 17, 2011 at 9:22 PM, Robert Newson rnew...@apache.org wrote: distilled from IRC chat A separate password file as described above, but can only be updated thus; # couchdb --set-password admin Password: foo Password updated. What problem is this solving exactly? This thread started because you edit foo.ini and subsequent changes go to bar.ini. That foo.ini happens to hold plaintext passwords instead of, say, TCP nodelay only underscores the problem. But plaintext vs. hashed passwords is a totally different matter. But regarding passwords, would you humor me and please re-state the requirements? I think it is a solution looking for a problem. Are we talking about moving *all* passwords to this file (ignoring _user doc .salt and .password_sha)? Or are we keeping those in sync now? Or is this just admin passwords? But only admins can see (hashed) passwords over HTTP. On Unix filesystems, if you have permission to read /etc/couchdb/local.ini then you very likely have permission to read /var/lib/couchdb/everything.couch, so what is the point? Regarding --set-password and couchctl, unless I am missing some serious requirement (possible), it sounds like CouchDB is poised to get much more complex soon. I spend all my free time bragging about how simple it is so that would be quite a blow to my ego. Thanks. -- Iris Couch
Re: The _security object should be versioned
MVCC for _local docs would be useful for BigCouch for the same reason
it's useful for _security docs, so your impression of what they are is
superior to reality.
B.
On 17 August 2011 16:01, Jason Smith j...@iriscouch.com wrote:
On Wed, Aug 17, 2011 at 9:51 PM, Adam Kocoloski kocol...@apache.org wrote:
How do you feel about migrating to a blessed _local/security document?
Maybe its latest version could be cached in the header for speed?
Pros:
* Couch gets (conceptually) simpler rather than more complex
* It's versioned, you get full doc semantics
* It doesn't replicate, but 3rd-party tools can pseudo-replicate it as
needed
* Design documents can enforce policies: if(doc._id == _local/security
doc.members.length == 0) throw {forbidden:This database may never
be public}
Eagerly awaiting a list of cons :)
The only trouble I have with _local/security is that _local documents are
represented using #doc records instead of #full_doc_info records. As such,
they have no support for MVCC.
I did not realize that, so one of the points I made was wrong. FWIW I
always assumed _local docs were normal docs, except they are
special-cased to neither replicate nor be represented in views.
Thanks for the tip!
--
Iris Couch
Re: Configuration Load Order
On Wednesday, August 17, 2011, Jason Smith j...@iriscouch.com wrote: On Wed, Aug 17, 2011 at 9:22 PM, Robert Newson rnew...@apache.org wrote: distilled from IRC chat A separate password file as described above, but can only be updated thus; # couchdb --set-password admin Password: foo Password updated. What problem is this solving exactly? This thread started because you edit foo.ini and subsequent changes go to bar.ini. because the biggest pb are passwords. local.ini could be then used for what it should be: local configuration updated via http or not. That foo.ini happens to hold plaintext passwords instead of, say, TCP nodelay only underscores the problem. But plaintext vs. hashed passwords is a totally different matter. But regarding passwords, would you humor me and please re-state the requirements? I think it is a solution looking for a problem. Are we talking about moving *all* passwords to this file (ignoring _user doc .salt and .password_sha)? Or are we keeping those in sync now? Or is this just admin passwords? But only admins can see (hashed) passwords over HTTP. On Unix filesystems, if you have permission to read /etc/couchdb/local.ini then you very likely have permission to read /var/lib/couchdb/everything.couch, so what is the point? be safer and more logical. passwords shouldn't be put in plain text at all. Regarding --set-password and couchctl, unless I am missing some serious requirement (possible), it sounds like CouchDB is poised to get much more complex soon. I spend all my free time bragging about how simple it is so that would be quite a blow to my ego. Thanks. what is the argument against smplicity here? - benoit
Re: The _security object should be versioned
On Wednesday, August 17, 2011, Adam Kocoloski kocol...@apache.org wrote: I believe the _security object should be versioned in order to ease synchronization of the object between databases. This proposal is motivated (unsurprisingly) by BigCouch, which typically stores multiple copies of each database in a multi-master configuration. When the _security object is written in BigCouch the update is issued to all available shards. We run into problems if an update is issued while some shards are unavailable, because we don't know how to synchronize the divergent copies once all the shards are back online. In my head I see us representing the _security object using a #full_doc_info, just as we would a document. Unlike documents the _security object (or a pointer to it) would still be written in the header of the database for fast access during request processing. I haven't quite decided what I think the API should look like, e.g. whether the full document API (including attachments?) should be supported. Regards, Adam I was about to open an issue in bigcouch about that :) +1 for the change then. btw how is handle the security object during the replication? i would be for a #full_doc_info object. Also it could be renamed as _meta. I would really like to see a way to add meta data. so we could eventually annotate a db or add any oyther metadata. As i read the code it is directly added to the db header. how would it work herecwould having one revision in the header be enough? or do you expect to keep one pointer to the latest revision in the header? benoit
[jira] [Commented] (COUCHDB-1252) A way to have views return _deleted documents
[ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086395#comment-13086395 ] Paul Joseph Davis commented on COUCHDB-1252: We could fairly easily add a include_deleted option to the view options parameter the same we do for include_design. A way to have views return _deleted documents - Key: COUCHDB-1252 URL: https://issues.apache.org/jira/browse/COUCHDB-1252 Project: CouchDB Issue Type: New Feature Components: JavaScript View Server Affects Versions: 1.1, 1.0.3 Reporter: James Howe Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown. I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: The _security object should be versioned
On Wednesday, August 17, 2011, Robert Newson rnew...@apache.org wrote:
MVCC for _local docs would be useful for BigCouch for the same reason
it's useful for _security docs, so your impression of what they are is
superior to reality.
B.
would also simplify the code imo. but do you mean their would also be a way
to even replicate local docs with some options?
- benoit
On 17 August 2011 16:01, Jason Smith j...@iriscouch.com wrote:
On Wed, Aug 17, 2011 at 9:51 PM, Adam Kocoloski kocol...@apache.org
wrote:
How do you feel about migrating to a blessed _local/security document?
Maybe its latest version could be cached in the header for speed?
Pros:
* Couch gets (conceptually) simpler rather than more complex
* It's versioned, you get full doc semantics
* It doesn't replicate, but 3rd-party tools can pseudo-replicate it as
needed
* Design documents can enforce policies: if(doc._id == _local/security
doc.members.length == 0) throw {forbidden:This database may never
be public}
Eagerly awaiting a list of cons :)
The only trouble I have with _local/security is that _local documents
are represented using #doc records instead of #full_doc_info records. As
such, they have no support for MVCC.
I did not realize that, so one of the points I made was wrong. FWIW I
always assumed _local docs were normal docs, except they are
special-cased to neither replicate nor be represented in views.
Thanks for the tip!
--
Iris Couch
Re: The replicator needs a superuser mode
I think that the operations of replication and backing up are quite different. Although some are using the replication features for backing up, I tend to think of replication as an operation taking place between two nodes that do not necessarily trust one another. If what you are proposing is a special privilege given to the admin party, then I do not have much of an issue with this, since administrators already have intimate access to the server. However, the concept of creating a new replicator role, which would supersede the validation functions is another thing. In applications that must ensure that some document types have a given structure, opening the door to a user (and here I assume a user that attempts a replication from a different node, not a local administrator performing a back up) to work around the validation function is probably a bad idea. If the validation function could not be counted on, it would really affect the way an application must be written. JP On 11-08-16 02:40 PM, Adam Kocoloski wrote: Hi Jean-Pierre, I'm not quite sure I follow that line of reasoning. A user with _admin privileges on the database can easily remove any validation functions prior to writing today. In my proposal skipping validation would require _admin rights and an explicit opt-in on a per-request basis. What are you trying to guard against with those validation functions? Best, Adam On Aug 16, 2011, at 2:29 PM, Jean-Pierre Fiset wrote: I understand the issue brought by Adam since in our CouchDb application, there is a need to have a replicator role and the validation functions skip most of the tests if the role is set for the current user. On the other hand, at the current time, I am not in favour of making super users for the sake of replication. Although it might solve the particular problem stated, it removes the ability for a design document to enforce some invariant properties of a database. Since there is already a way to allow a replicator to perform any changes (role + proper validation function), I do not see the need for this change. Since the super replicator user removes the ability that a database has to protect the consistency of its data, and that there does not seem to be a work-around, I would rather not see this change pushed to CouchDb. JP On 11-08-16 10:26 AM, Adam Kocoloski wrote: One of the principal uses of the replicator is to make this database look like that one. We're unable to do that in the general case today because of the combination of validation functions and out-of-order document transfers. It's entirely possible for a document to be saved in the source DB prior to the installation of a ddoc containing a validation function that would have rejected the document, for the replicator to install the ddoc in the target DB before replicating the other document, and for the other document to then be rejected by the target DB. I propose we add a role which allows a user to bypass validation, or else extend that privilege to the _admin role. We should still validate updates by default and add a way (a new qs param, for instance) to indicate that validation should be skipped for a particular update. Thoughts? Adam
Re: The _security object should be versioned
No, _local docs shouldn't replicate. The value of MVCC in _local docs
is to allow reconciliation in copies of the same database, where
redundancy is being added under the covers.
Can you start a separate thread for your _meta proposal? It's a
interesting thought but it's out of scope for both threads you've
raised it in.
B.
On 17 August 2011 16:41, Benoit Chesneau bchesn...@gmail.com wrote:
On Wednesday, August 17, 2011, Robert Newson rnew...@apache.org wrote:
MVCC for _local docs would be useful for BigCouch for the same reason
it's useful for _security docs, so your impression of what they are is
superior to reality.
B.
would also simplify the code imo. but do you mean their would also be a way
to even replicate local docs with some options?
- benoit
On 17 August 2011 16:01, Jason Smith j...@iriscouch.com wrote:
On Wed, Aug 17, 2011 at 9:51 PM, Adam Kocoloski kocol...@apache.org
wrote:
How do you feel about migrating to a blessed _local/security document?
Maybe its latest version could be cached in the header for speed?
Pros:
* Couch gets (conceptually) simpler rather than more complex
* It's versioned, you get full doc semantics
* It doesn't replicate, but 3rd-party tools can pseudo-replicate it as
needed
* Design documents can enforce policies: if(doc._id == _local/security
doc.members.length == 0) throw {forbidden:This database may never
be public}
Eagerly awaiting a list of cons :)
The only trouble I have with _local/security is that _local documents
are represented using #doc records instead of #full_doc_info records. As
such, they have no support for MVCC.
I did not realize that, so one of the points I made was wrong. FWIW I
always assumed _local docs were normal docs, except they are
special-cased to neither replicate nor be represented in views.
Thanks for the tip!
--
Iris Couch
Re: Configuration Load Order
On Wed, Aug 17, 2011 at 10:22 PM, Robert Newson rnew...@apache.org wrote: Jason, The --set-password thing is to ensure there are no plaintext passwords in the first place, which eliminates the oddness of couch rewriting a plaintext pwd to a digested pwd (and putting the output in a different file). Thanks for the clarification. If you can read a plaintext password from an .ini file, then you can hit the HTTP API as the admin and make changes to the couch. So that is privilege escalation. To answer Benoit's question, it is simpler to tell admins to use the HTTP API (or Futon) to create the admin account. The password is stored *somewhere* under the hood. IMHO it is less simple to add a command-line tool as a requirement (or worse, as an alternative option) to deploy Couch. -- Iris Couch
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086408#comment-13086408 ] Paul Joseph Davis commented on COUCHDB-1012: Looking at both of these, I quite prefer the couch-config path over the pkg-config. couch-config offers a lot more settings and the patch is a lot more sane. I don't see why the pkg-config script is introducing new top level directories into the build system and so on. Only minor thing, the couch-config script uses -? for to display usage. I've not seen ? used for usage info since DOS, that's not some sort of shell script standard I've never seen is it? Even if it is, -h and --help should be supported as well. Other than that, I'd like to have Noah glance at that shell script as well. It looks pretty straight forward to me but he's the shell script portability expert. Other than that, I'd say it looks good to ship. Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Configuration Load Order
On Wednesday, August 17, 2011, Jason Smith j...@iriscouch.com wrote: On Wed, Aug 17, 2011 at 10:22 PM, Robert Newson rnew...@apache.org wrote: Jason, The --set-password thing is to ensure there are no plaintext passwords in the first place, which eliminates the oddness of couch rewriting a plaintext pwd to a digested pwd (and putting the output in a different file). Thanks for the clarification. If you can read a plaintext password from an .ini file, then you can hit the HTTP API as the admin and make changes to the couch. So that is privilege escalation. To answer Benoit's question, it is simpler to tell admins to use the HTTP API (or Futon) to create the admin account. The password is stored *somewhere* under the hood. IMHO it is less simple to add a command-line tool as a requirement (or worse, as an alternative option) to deploy Couch. -- it all depends if you admin via a console. couchctl set-password username is a way easier than curl -XPUT http://blah/_users -D... -H... . at the end if you are a good admin you will write this script. providing useful helpere don't break the kiss way here. benoît
Re: Configuration Load Order
It's also the sort of thing one would expect in a Debian postinst via Dialog. B. On 17 August 2011 17:04, Benoit Chesneau bchesn...@gmail.com wrote: On Wednesday, August 17, 2011, Jason Smith j...@iriscouch.com wrote: On Wed, Aug 17, 2011 at 10:22 PM, Robert Newson rnew...@apache.org wrote: Jason, The --set-password thing is to ensure there are no plaintext passwords in the first place, which eliminates the oddness of couch rewriting a plaintext pwd to a digested pwd (and putting the output in a different file). Thanks for the clarification. If you can read a plaintext password from an .ini file, then you can hit the HTTP API as the admin and make changes to the couch. So that is privilege escalation. To answer Benoit's question, it is simpler to tell admins to use the HTTP API (or Futon) to create the admin account. The password is stored *somewhere* under the hood. IMHO it is less simple to add a command-line tool as a requirement (or worse, as an alternative option) to deploy Couch. -- it all depends if you admin via a console. couchctl set-password username is a way easier than curl -XPUT http://blah/_users -D... -H... . at the end if you are a good admin you will write this script. providing useful helpere don't break the kiss way here. benoît
Re: The replicator needs a superuser mode
On Aug 17, 2011, at 11:46 AM, Jean-Pierre Fiset wrote: I think that the operations of replication and backing up are quite different. Although some are using the replication features for backing up, I tend to think of replication as an operation taking place between two nodes that do not necessarily trust one another. That's one possible use case for replication, but hardly the only one. Anyway, if you don't trust the replication then I certainly hope the replication doesn't use credentials that map to _admin powers on your database. If the replication doesn't have _admin powers it cannot bypass validation. If what you are proposing is a special privilege given to the admin party, then I do not have much of an issue with this, since administrators already have intimate access to the server. However, the concept of creating a new replicator role, which would supersede the validation functions is another thing. Yes, I probably should have picked one approach and stuck with it. Either way, my intent was that a replicator could bypass validation only if an admin had given it credentials that mapped to a powerful role (possibly _admin), *and* if the admin had explicitly asked for the replicator to bypass validation. In applications that must ensure that some document types have a given structure, opening the door to a user (and here I assume a user that attempts a replication from a different node, not a local administrator performing a back up) to work around the validation function is probably a bad idea. That's not going to happen, unless you granted the user this really powerful role. Don't do that. If the validation function could not be counted on, it would really affect the way an application must be written. Understood, I'm certainly not asking for the replicator to bypass validations in general. Cheers, Adam
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086455#comment-13086455 ] Noah Slater commented on COUCHDB-1012: -- Oh, I had missed the patch. Comments: Please rename the script to couchenv. Please change the help output to be more helpful: The $basename command runs the %package_name% environment tool. When %package_name% is installed, certain environment variables are set. This script will display the current values of those variables. The exit status is 0 for success or 1 for failure. Please support --V, --version for printing version information. Please support -h, --help for printing help information. Please remove the -? option. Please make the wording of the help output start with: -h display a short help message and exit -V display version information and exit This is essential to main consistency between the output of the scripts we ship. Please change the wording of the help output the match this. As an example: --logfile Display log file path Please change the flags to mirror the variable name precisely: --couchlibdir --couchincludedir --couchebindir --dbdir --viewdir --couchprivlibdir --bindir --confdir --urifile --logfile Please add the new script to the dist_man1_MANS variable so that we build a man page for it. Please check that the new -h, -V output produces a sensible looking man page. Compare this with the man page for the couched command. I would recommend keeping with the variable names as they exist: So from this: +dbdir=%localstatelibdir% To this: +localstatelibdir=%localstatelibdir% As with the help output, it might seem like shortening them helps. I would argue that given sufficient knowledge of either CouchDB or Autotools, the originals make more sense. I know what the localstatelibdir is, because I have worked with that variable so much. The same is not necessarily true for dbdir. We should keep with the existing values throughout the code and our external interfaces to that code. Other than that, this is exactly how I would have implemented it. Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: The replicator needs a superuser mode
We've strayed onto the topic of backup/restore/import/export and someone mentioned pg_dump, so I'll toss this out there. As a long time user of PostgreSQL and their import/export tools, I'd definitely suggest having a look at what options have evolved in their tool too get a feel for what people might want from a couch tool in the long run. Dumping users, schema, data, privileges, blobs, etc. or disabling triggers, etc. are all options specified at the command line. Many of these options could be mapped to couch concepts such as design docs, security objects, attachments, validation processing, etc. pg_dump is also careful to dump things in the order required for proper import. For couch, this might mean dumping out in such a way that future imports would see data loaded before design documents, etc. Whether couchdb winds up with a command line tool or an authenticated URL with parameters and possibly automation is an interesting question too. The former could well be built on the latter. FYI. Below is the pg_dump command line help from PostgreSQL 8.4 for reference. I hope it's helpful. -- Amos --- pg_dump dumps a database as a text file or to other formats. Usage: pg_dump [OPTION]... [DBNAME] General options: -f, --file=FILENAME output file name -F, --format=c|t|p output file format (custom, tar, plain text) -i, --ignore-version proceed even when server version mismatches pg_dump version -v, --verboseverbose mode -Z, --compress=0-9 compression level for compressed formats --help show this help, then exit --versionoutput version information, then exit Options controlling the output content: -a, --data-only dump only the data, not the schema -b, --blobs include large objects in dump -c, --clean clean (drop) schema prior to create -C, --createinclude commands to create database in dump -d, --inserts dump data as INSERT commands, rather than COPY -D, --column-insertsdump data as INSERT commands with column names -E, --encoding=ENCODING dump the data in encoding ENCODING -n, --schema=SCHEMA dump the named schema(s) only -N, --exclude-schema=SCHEMA do NOT dump the named schema(s) -o, --oids include OIDs in dump -O, --no-owner skip restoration of object ownership in plain text format -s, --schema-only dump only the schema, no data -S, --superuser=NAMEspecify the superuser user name to use in plain text format -t, --table=TABLE dump the named table(s) only -T, --exclude-table=TABLE do NOT dump the named table(s) -x, --no-privileges do not dump privileges (grant/revoke) --disable-dollar-quotingdisable dollar quoting, use SQL standard quoting --disable-triggers disable triggers during data-only restore --use-set-session-authorization use SESSION AUTHORIZATION commands instead of ALTER OWNER commands to set ownership Connection options: -h, --host=HOSTNAME database server host or socket directory -p, --port=PORT database server port number -U, --username=NAME connect as specified database user -W, --password force password prompt (should happen automatically) If no database name is supplied, then the PGDATABASE environment variable value is used. Report bugs to pgsql-b...@postgresql.org. --- On 2011-08-17, at 10:11 AM, Noah Slater wrote: On 17 Aug 2011, at 11:06, Benoit Chesneau wrote: Philosophy apart, dump and restore could be indeed useful to bootstrap db, make plain backup/restore strategies, exchange dbs over a disk/mem card without any couch installed etc. Yep, but in my mind this should live outside CouchDB's HTTP API. A dump and restore tool that lived on the command line, like the Subversion hotcopy stuff is the first thing that springs to mind. Or PostgreSQL's pgdump tool, or whatever. But as far as I understand the current file format, you should be able to just rsync the .couch files while the database is running.
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086457#comment-13086457 ] Noah Slater commented on COUCHDB-1012: -- That example should read: --logfile display logfile and exit Additionally, remove this: +urifile=%localstaterundir%/couch.uri +logfile=%localstatelogdir%/couch.log And instead have: +localstaterundir=%localstaterundir% And then use this for the help output: --localstaterundir display localstaterundir and exit Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Configuration Load Order
To be clear, is the suggestion that this script simply be a wrapper around updating the password via CouchDB's HTTP API? Having gone back and forth on this thread for the past few days, I think it makes sense to keep user info in CouchDB, and not in the ini files. If a user manages to lock themselves out of the server, they can start it with --admin-party and disable the auth system while they fix things up. On 17 Aug 2011, at 17:09, Robert Newson wrote: It's also the sort of thing one would expect in a Debian postinst via Dialog. B. On 17 August 2011 17:04, Benoit Chesneau bchesn...@gmail.com wrote: On Wednesday, August 17, 2011, Jason Smith j...@iriscouch.com wrote: On Wed, Aug 17, 2011 at 10:22 PM, Robert Newson rnew...@apache.org wrote: Jason, The --set-password thing is to ensure there are no plaintext passwords in the first place, which eliminates the oddness of couch rewriting a plaintext pwd to a digested pwd (and putting the output in a different file). Thanks for the clarification. If you can read a plaintext password from an .ini file, then you can hit the HTTP API as the admin and make changes to the couch. So that is privilege escalation. To answer Benoit's question, it is simpler to tell admins to use the HTTP API (or Futon) to create the admin account. The password is stored *somewhere* under the hood. IMHO it is less simple to add a command-line tool as a requirement (or worse, as an alternative option) to deploy Couch. -- it all depends if you admin via a console. couchctl set-password username is a way easier than curl -XPUT http://blah/_users -D... -H... . at the end if you are a good admin you will write this script. providing useful helpere don't break the kiss way here. benoît
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086461#comment-13086461 ] Paul Joseph Davis commented on COUCHDB-1012: I think Noah's got a good point on accessing the original variables but we should keep the aliases. For people *not* versed in internals or the build system localstatelibdir is about as WTF as possible compared to dbdir. I would say maybe list the user friendly versions first, then the build system names second (though keeping -V and -h first). I'd disagree on changing the name to couchenv though. In my experience its a pretty consistent pattern for these scripts to be named $projectname-config. Changing it seems like it'll just be more WTF fodder. Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086465#comment-13086465 ] Noah Slater commented on COUCHDB-1012: -- Providing aliases for these environment variables is just going to muddy the water and potentially confuse people more. If you look at the output. The values here should precisely match the output from ./configure --help, which I did not think of checking before. Let me check that right now, as I write this very sentence. This is the output: Fine tuning of the installation directories: --bindir=DIRuser executables [EPREFIX/bin] --sbindir=DIR system admin executables [EPREFIX/sbin] --libexecdir=DIRprogram executables [EPREFIX/libexec] --sysconfdir=DIRread-only single-machine data [PREFIX/etc] --sharedstatedir=DIRmodifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] --libdir=DIRobject code libraries [EPREFIX/lib] --includedir=DIRC header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] --datarootdir=DIR read-only arch.-independent data root [PREFIX/share] --datadir=DIR read-only architecture-independent data [DATAROOTDIR] --infodir=DIR info documentation [DATAROOTDIR/info] --localedir=DIR locale-dependent data [DATAROOTDIR/locale] --mandir=DIRman documentation [DATAROOTDIR/man] --docdir=DIRdocumentation root [DATAROOTDIR/doc/apache-couchdb] --htmldir=DIR html documentation [DOCDIR] --dvidir=DIRdvi documentation [DOCDIR] --pdfdir=DIRpdf documentation [DOCDIR] --psdir=DIR ps documentation [DOCDIR] I contend that the output of this script should be: --bindiruser executables [EPREFIX/bin] --sbindir system admin executables [EPREFIX/sbin] --libexecdirprogram executables [EPREFIX/libexec] --sysconfdirread-only single-machine data [PREFIX/etc] --sharedstatedirmodifiable architecture-independent data [PREFIX/com] --localstatedir modifiable single-machine data [PREFIX/var] --libdirobject code libraries [EPREFIX/lib] --includedirC header files [PREFIX/include] --oldincludedir C header files for non-gcc [/usr/include] --datarootdir read-only arch.-independent data root [PREFIX/share] --datadir read-only architecture-independent data [DATAROOTDIR] --infodir info documentation [DATAROOTDIR/info] --localedir locale-dependent data [DATAROOTDIR/locale] --mandirman documentation [DATAROOTDIR/man] --docdirdocumentation root [DATAROOTDIR/doc/apache-couchdb] --htmldir html documentation [DOCDIR] --dvidirdvi documentation [DOCDIR] --pdfdirpdf documentation [DOCDIR] --psdir ps documentation [DOCDIR] I would also update the help description to read: The $basename command runs the %package_name% environment tool. When %package_name% is installed, certain environment variables are set. This script will display the current values of those variables. See the %package_name% ./configure --help output for more information. The exit status is 0 for success or 1 for failure. Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086469#comment-13086469 ] Noah Slater commented on COUCHDB-1012: -- Oh, forgot to mention, I don't think it matters that we don't necessarily use all of these variables for anything. It maintains consistency with out build system that sets them in the first place. Also, from these, and knowing where CouchDB puts files, you can work out where CouchDB specific sub-directories are. If you can't work out how to get from localstatedir to the place CouchDB keeps it's .couch files, you're not the intended audience for this script. Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086483#comment-13086483 ] Paul Joseph Davis commented on COUCHDB-1012: 1. We don't have to alias everything. 2. You have to be kidding me that you expect people to puzzle over which of those variables is where .couch files are stored. There are a couple common use cases that are motivating this tool: * Where the hell are the config files? * Where the hell are my db files? * Where the hell is that URI file? * Where the hell do I put beam files? These are answers people are looking for when they get frustrated. The last thing people should have to do is work out where CouchDB specific sub-directories are. I'm biblically familiar with our build system and I can't name every combination of variables and subpaths off the top of my head. Bottom line: I *can* work these things out but I shouldn't have to. I want this tool so that I don't have to recall that localstatedir where where the one things are while the other bits are in `couch-config -n --localstatelibdir`/couchdb/erlang/couchdb-`couchdb -n --version`/ebin/. Then again maybe I'm just slow and that's more clear than --dbdir. Two related notes: We should add a -n to affect the echo adding a newline (useful for script substitution as above). couchebindir and couchincludedir should be replaced with --erllibdir. There's no reason for ebin dir. The include directory is only useful for compiling other code. After install this is setup as a proper ERL_LIBS directory structure so -include_lib(couchdb/include/couch_db.hrl). should work with --erllibdir alone. Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Configuration Load Order
It makes sense to perform the PUT to _config, yes, though I was thinking it just updated the file. As long as the PUT in_ config for passwords explicitly writes to a dedicated password file, not any of the .ini files. Or something. B. On 17 August 2011 18:15, Noah Slater nsla...@apache.org wrote: To be clear, is the suggestion that this script simply be a wrapper around updating the password via CouchDB's HTTP API? Having gone back and forth on this thread for the past few days, I think it makes sense to keep user info in CouchDB, and not in the ini files. If a user manages to lock themselves out of the server, they can start it with --admin-party and disable the auth system while they fix things up. On 17 Aug 2011, at 17:09, Robert Newson wrote: It's also the sort of thing one would expect in a Debian postinst via Dialog. B. On 17 August 2011 17:04, Benoit Chesneau bchesn...@gmail.com wrote: On Wednesday, August 17, 2011, Jason Smith j...@iriscouch.com wrote: On Wed, Aug 17, 2011 at 10:22 PM, Robert Newson rnew...@apache.org wrote: Jason, The --set-password thing is to ensure there are no plaintext passwords in the first place, which eliminates the oddness of couch rewriting a plaintext pwd to a digested pwd (and putting the output in a different file). Thanks for the clarification. If you can read a plaintext password from an .ini file, then you can hit the HTTP API as the admin and make changes to the couch. So that is privilege escalation. To answer Benoit's question, it is simpler to tell admins to use the HTTP API (or Futon) to create the admin account. The password is stored *somewhere* under the hood. IMHO it is less simple to add a command-line tool as a requirement (or worse, as an alternative option) to deploy Couch. -- it all depends if you admin via a console. couchctl set-password username is a way easier than curl -XPUT http://blah/_users -D... -H... . at the end if you are a good admin you will write this script. providing useful helpere don't break the kiss way here. benoît
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086498#comment-13086498 ] Noah Slater commented on COUCHDB-1012: -- Okay, how about this: Options: -h display a short help message and exit -V display version information and exit Common directories: --bindiruser executables [EPREFIX/bin] --sbindir system admin executables [EPREFIX/sbin] --libexecdirprogram executables [EPREFIX/libexec] --sysconfdirread-only single-machine data [PREFIX/etc] --sharedstatedirmodifiable architecture-independent data [PREFIX/com] --localstatedir modifiable single-machine data [PREFIX/var] --libdirobject code libraries [EPREFIX/lib] --includedirC header files [PREFIX/include] --oldincludedir C header files for non-gcc [/usr/include] --datarootdir read-only arch.-independent data root [PREFIX/share] --datadir read-only architecture-independent data [DATAROOTDIR] --infodir info documentation [DATAROOTDIR/info] --localedir locale-dependent data [DATAROOTDIR/locale] --mandirman documentation [DATAROOTDIR/man] --docdirdocumentation root [DATAROOTDIR/doc/apache-couchdb] --htmldir html documentation [DOCDIR] --dvidirdvi documentation [DOCDIR] --pdfdirpdf documentation [DOCDIR] --psdir ps documentation [DOCDIR] CouchDB directories: --localconfdir configuration data [SYSCONFDIR/$package_identifier] --localdatadir runtime data [DATADIR/$package_identifier] --localdocdir documentation data [DATADIR/doc/$package_identifier] --locallibdir library data [LIBDIR/$package_identifier] --localstatelibdir database data [LOCALSTATEDIR/lib/$package_identifier] --localstatelogdir log data [LOCALSTATEDIR/log/$package_identifier] --localstaterundir daemon data [LOCALSTATEDIR/run/$package_identifier] Note: the above has been correctly formatted for fixed width display, and $variables will be replaced by the script. I think that should be enough. We list all of the default Autoconf stuff, as well as listing the most important variable set specifically for CouchDB. I don't think we should include options such as --uri-file or whatever. This duplicates this information, as it is only stored in one other place in the source, so it would quickly rot. Also, the tiny amount of space we have in the output of this tool is not enough to document these files. That is what the documentation proper is for. This is a tool to allow people already familiar with what they are looking for to print out values. The -n output you suggest is not needed: $ echo foo `echo bar` baz foo bar baz This would change your example usage of the script to this: echo `couch-config -locallibdir`/ebin/ Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086506#comment-13086506 ] Noah Slater commented on COUCHDB-1012: -- Oh, it WAS formatted. Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086510#comment-13086510 ] Paul Joseph Davis commented on COUCHDB-1012: You're just yanking my chain right? Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086533#comment-13086533 ] Randall Leeds commented on COUCHDB-1012: I'm in favor of couch-config over couchenv. *-config is the convention I'm familiar with. I'd also like to ship a couch.m4, but that's a separate ticket and I'll work with Noah to do that. I wonder if it's better not to include the database, view, log directories. I'd prefer the minimal set of things required for _developers_, and in particular things that can't be changed through CouchDB's configuration system (because things that can possibly make the output of couch-config wrong if they are changed). Feel free to tell me I'm crazy. On the one hand, most people won't change these settings in their config, so they could stay in the couch-config tool and likely not report the wrong path. On the other hand, most people won't need to read these settings for plugin development, so maybe they're better left out for the sake of simplicity. I'm mostly tempted to leave out the db directory to force plugin developers to initialize any special dbs they might need when their plugin is loaded. I think it reduces the likelihood of certain surprises, such as a user deleting a plugin's configuration database by mistake or a plugin author accidentally creating their plugin's config db with root permissions and couch can't read it, etc... Otherwise, everything being discussed above sounds reasonable. Seems like you all have it under control. Ship it. Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: The _security object should be versioned
On Wed, Aug 17, 2011 at 08:48, Robert Newson rnew...@apache.org wrote:
No, _local docs shouldn't replicate. The value of MVCC in _local docs
is to allow reconciliation in copies of the same database, where
redundancy is being added under the covers.
To be clear, _local docs do obey MVCC semantics, but do not keep a rev tree
and thus do not support merging divergent histories. Multiple versions are
not kept available even when they're in conflict. They have a _rev and they
do get rejected with conflicts, you just can't access ?new_edits=false style
merging.
I'm in favor of giving them a full rev tree.
Can you start a separate thread for your _meta proposal? It's a
interesting thought but it's out of scope for both threads you've
raised it in.
B.
On 17 August 2011 16:41, Benoit Chesneau bchesn...@gmail.com wrote:
On Wednesday, August 17, 2011, Robert Newson rnew...@apache.org wrote:
MVCC for _local docs would be useful for BigCouch for the same reason
it's useful for _security docs, so your impression of what they are is
superior to reality.
B.
would also simplify the code imo. but do you mean their would also be a
way
to even replicate local docs with some options?
- benoit
On 17 August 2011 16:01, Jason Smith j...@iriscouch.com wrote:
On Wed, Aug 17, 2011 at 9:51 PM, Adam Kocoloski kocol...@apache.org
wrote:
How do you feel about migrating to a blessed _local/security
document?
Maybe its latest version could be cached in the header for speed?
Pros:
* Couch gets (conceptually) simpler rather than more complex
* It's versioned, you get full doc semantics
* It doesn't replicate, but 3rd-party tools can pseudo-replicate it
as
needed
* Design documents can enforce policies: if(doc._id ==
_local/security
doc.members.length == 0) throw {forbidden:This database may never
be public}
Eagerly awaiting a list of cons :)
The only trouble I have with _local/security is that _local documents
are represented using #doc records instead of #full_doc_info records. As
such, they have no support for MVCC.
I did not realize that, so one of the points I made was wrong. FWIW I
always assumed _local docs were normal docs, except they are
special-cased to neither replicate nor be represented in views.
Thanks for the tip!
--
Iris Couch
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086605#comment-13086605 ] Noah Slater commented on COUCHDB-1012: -- @Paul, nope? Which bit? Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Randall Leeds Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Leeds reassigned COUCHDB-1012: -- Assignee: Noah Slater (was: Randall Leeds) Take it home, Noah. Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Noah Slater Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1012) Utility to help plugin developers manage paths
[ https://issues.apache.org/jira/browse/COUCHDB-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086658#comment-13086658 ] Paul Joseph Davis commented on COUCHDB-1012: @Randall, You make a good point about switching configurations away from the install time defaults. I'm not entirely certain that I care too much though. If someone is at the level that they're swapping their lib or db directories then they can hang out in the corner. As to remove the db directory to force developers to do something funny I don't see it. I mean, creating a database at install time? Not even we do that. @Noah, Look at the list of variables you have. Do we even actually install anything in any of these directories? Maybe the doc directory gets some stuff. But seriously, dvi, pdf, and ps directories? --sbindir system admin executables [EPREFIX/sbin] --libexecdir program executables [EPREFIX/libexec] --sharedstatedir modifiable architecture-independent data [PREFIX/com] --infodir info documentation [DATAROOTDIR/info] --localedir locale-dependent data [DATAROOTDIR/locale] --docdir documentation root [DATAROOTDIR/doc/apache-couchdb] --htmldir html documentation [DOCDIR] --dvidir dvi documentation [DOCDIR] --pdfdir pdf documentation [DOCDIR] --psdir ps documentation [DOCDIR] And then in the CouchDB section: --localconfdir configuration data [SYSCONFDIR/$package_identifier] --localdatadir runtime data [DATADIR/$package_identifier] --localdocdir documentation data [DATADIR/doc/$package_identifier] --locallibdir library data [LIBDIR/$package_identifier] --localstatelibdir database data [LOCALSTATEDIR/lib/$package_identifier] --localstatelogdir log data [LOCALSTATEDIR/log/$package_identifier] --localstaterundir daemon data [LOCALSTATEDIR/run/$package_identifier] What in the world does local mean here? Do you seriously expect users that aren't autotools experts to have any clue what this stuff means? Consider this as compared to: --config-dir configuration directory --db-dir database directory --static-dir static asset directory --doc-dir documentation directory --erl-libs-dir Erlang library directory --log-dir log directory --pid-file daemon PID file --uri-file daemon sockets file Its almost pointless to even include a description line for these variable names. And why aren't the CouchDB specific directories first? How often do you expect someone to want to know about the DVI or PDF directory before the db, log, or erl-libs directory? I'm not saying we have to get rid of the dvi and ps directories. Its quite possible they'll get used at some point but the user friendly alias options should be first. Otherwise you force every user to learn and remember what locallibdir is compared to localdatadir compared to localstaterundir. What happened to the old Noah that loved and cared about our users? The Noah that would treat our users like a small furry rabbit to cuddle. Furry rabbits like this little guy: http://28.media.tumblr.com/tumblr_lq2sexkiya1qzp2x4o1_500.jpg Utility to help plugin developers manage paths -- Key: COUCHDB-1012 URL: https://issues.apache.org/jira/browse/COUCHDB-1012 Project: CouchDB Issue Type: New Feature Components: Build System Reporter: Randall Leeds Assignee: Noah Slater Fix For: 1.2 Attachments: 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-add-couch-config-file-used-to-ease-the-build-of-plug.patch, 0001-support-pkg-config-for-plugins-COUCHDB-1012.patch Developers may want to write plugins (like GeoCouch) for CouchDB. Many hooks in the configuration system allow loading arbitrary Erlang modules to handle various internal tasks, but currently there is no straightforward and portable way for developers of these plugins to discover the location of the CouchDB library files. Two options that have been proposed are to use pkg-config or install a separate script that could be invoked (e.g. as couch-config --erl-libs) to discover important CouchDB installation paths. As far as I know the loudest argument against pkg-config is lack of support for Windows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1246) CouchJS process spawned and not killed on each Reduce Overflow Error
[
https://issues.apache.org/jira/browse/COUCHDB-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Joseph Davis updated COUCHDB-1246:
---
Attachment: COUCHDB-1246.patch
Cleaned up the test. Everything passes for me locally.
CouchJS process spawned and not killed on each Reduce Overflow Error
Key: COUCHDB-1246
URL: https://issues.apache.org/jira/browse/COUCHDB-1246
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Affects Versions: 1.1
Environment: Linux Debian Squeeze
[query_server_config]
reduce_limit = true
os_process_limit = 25
Reporter: Michael Newman
Attachments: COUCHDB-1246.patch, COUCHDB-1246.patch, categories,
os_pool_trunk.patch, os_pool_trunk.patch, os_pool_trunk.patch
Running the view attached results in a reduce_overflow_error. For each
reduce_overflow_error a process of /usr/lib/couchdb/bin/couchjs
/usr/share/couchdb/server/main.js starts running. Once this gets to 25, which
is the os_process_limit by default, all views result in a server error:
timeout {gen_server,call,[couch_query_servers,{get_proc,javascript}]}
As far as I can tell, these processes and the non-response from the views
will continue until couch is restarted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1246) CouchJS process spawned and not killed on each Reduce Overflow Error
[
https://issues.apache.org/jira/browse/COUCHDB-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086769#comment-13086769
]
Paul Joseph Davis commented on COUCHDB-1246:
I swear to christ I clicked the for contribution button. Not that it matters.
Also, how does one delete attachments?
CouchJS process spawned and not killed on each Reduce Overflow Error
Key: COUCHDB-1246
URL: https://issues.apache.org/jira/browse/COUCHDB-1246
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Affects Versions: 1.1
Environment: Linux Debian Squeeze
[query_server_config]
reduce_limit = true
os_process_limit = 25
Reporter: Michael Newman
Attachments: COUCHDB-1246.patch, COUCHDB-1246.patch, categories,
os_pool_trunk.patch, os_pool_trunk.patch, os_pool_trunk.patch
Running the view attached results in a reduce_overflow_error. For each
reduce_overflow_error a process of /usr/lib/couchdb/bin/couchjs
/usr/share/couchdb/server/main.js starts running. Once this gets to 25, which
is the os_process_limit by default, all views result in a server error:
timeout {gen_server,call,[couch_query_servers,{get_proc,javascript}]}
As far as I can tell, these processes and the non-response from the views
will continue until couch is restarted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1246) CouchJS process spawned and not killed on each Reduce Overflow Error
[
https://issues.apache.org/jira/browse/COUCHDB-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Joseph Davis updated COUCHDB-1246:
---
Attachment: COUCHDB-1246.patch
Finally found the delete attachments button. It was like a scavenger hunt. Only
it sucked.
CouchJS process spawned and not killed on each Reduce Overflow Error
Key: COUCHDB-1246
URL: https://issues.apache.org/jira/browse/COUCHDB-1246
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Affects Versions: 1.1
Environment: Linux Debian Squeeze
[query_server_config]
reduce_limit = true
os_process_limit = 25
Reporter: Michael Newman
Attachments: COUCHDB-1246.patch, categories, os_pool_trunk.patch,
os_pool_trunk.patch, os_pool_trunk.patch
Running the view attached results in a reduce_overflow_error. For each
reduce_overflow_error a process of /usr/lib/couchdb/bin/couchjs
/usr/share/couchdb/server/main.js starts running. Once this gets to 25, which
is the os_process_limit by default, all views result in a server error:
timeout {gen_server,call,[couch_query_servers,{get_proc,javascript}]}
As far as I can tell, these processes and the non-response from the views
will continue until couch is restarted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1246) CouchJS process spawned and not killed on each Reduce Overflow Error
[
https://issues.apache.org/jira/browse/COUCHDB-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Joseph Davis updated COUCHDB-1246:
---
Attachment: (was: COUCHDB-1246.patch)
CouchJS process spawned and not killed on each Reduce Overflow Error
Key: COUCHDB-1246
URL: https://issues.apache.org/jira/browse/COUCHDB-1246
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Affects Versions: 1.1
Environment: Linux Debian Squeeze
[query_server_config]
reduce_limit = true
os_process_limit = 25
Reporter: Michael Newman
Attachments: COUCHDB-1246.patch, categories, os_pool_trunk.patch,
os_pool_trunk.patch, os_pool_trunk.patch
Running the view attached results in a reduce_overflow_error. For each
reduce_overflow_error a process of /usr/lib/couchdb/bin/couchjs
/usr/share/couchdb/server/main.js starts running. Once this gets to 25, which
is the os_process_limit by default, all views result in a server error:
timeout {gen_server,call,[couch_query_servers,{get_proc,javascript}]}
As far as I can tell, these processes and the non-response from the views
will continue until couch is restarted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1246) CouchJS process spawned and not killed on each Reduce Overflow Error
[
https://issues.apache.org/jira/browse/COUCHDB-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Joseph Davis updated COUCHDB-1246:
---
Attachment: (was: COUCHDB-1246.patch)
CouchJS process spawned and not killed on each Reduce Overflow Error
Key: COUCHDB-1246
URL: https://issues.apache.org/jira/browse/COUCHDB-1246
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Affects Versions: 1.1
Environment: Linux Debian Squeeze
[query_server_config]
reduce_limit = true
os_process_limit = 25
Reporter: Michael Newman
Attachments: COUCHDB-1246.patch, categories, os_pool_trunk.patch,
os_pool_trunk.patch, os_pool_trunk.patch
Running the view attached results in a reduce_overflow_error. For each
reduce_overflow_error a process of /usr/lib/couchdb/bin/couchjs
/usr/share/couchdb/server/main.js starts running. Once this gets to 25, which
is the os_process_limit by default, all views result in a server error:
timeout {gen_server,call,[couch_query_servers,{get_proc,javascript}]}
As far as I can tell, these processes and the non-response from the views
will continue until couch is restarted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1246) CouchJS process spawned and not killed on each Reduce Overflow Error
[
https://issues.apache.org/jira/browse/COUCHDB-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086826#comment-13086826
]
Filipe Manana commented on COUCHDB-1246:
Thanks Paul, looks good to me.
CouchJS process spawned and not killed on each Reduce Overflow Error
Key: COUCHDB-1246
URL: https://issues.apache.org/jira/browse/COUCHDB-1246
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Affects Versions: 1.1
Environment: Linux Debian Squeeze
[query_server_config]
reduce_limit = true
os_process_limit = 25
Reporter: Michael Newman
Attachments: COUCHDB-1246.patch, categories, os_pool_trunk.patch,
os_pool_trunk.patch, os_pool_trunk.patch
Running the view attached results in a reduce_overflow_error. For each
reduce_overflow_error a process of /usr/lib/couchdb/bin/couchjs
/usr/share/couchdb/server/main.js starts running. Once this gets to 25, which
is the os_process_limit by default, all views result in a server error:
timeout {gen_server,call,[couch_query_servers,{get_proc,javascript}]}
As far as I can tell, these processes and the non-response from the views
will continue until couch is restarted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-1246) CouchJS process spawned and not killed on each Reduce Overflow Error
[
https://issues.apache.org/jira/browse/COUCHDB-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Filipe Manana resolved COUCHDB-1246.
Resolution: Fixed
Fix Version/s: 1.2
1.1.1
Applied to trunk and branch 1.1.x
CouchJS process spawned and not killed on each Reduce Overflow Error
Key: COUCHDB-1246
URL: https://issues.apache.org/jira/browse/COUCHDB-1246
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Affects Versions: 1.1
Environment: Linux Debian Squeeze
[query_server_config]
reduce_limit = true
os_process_limit = 25
Reporter: Michael Newman
Fix For: 1.1.1, 1.2
Attachments: COUCHDB-1246.patch, categories, os_pool_trunk.patch,
os_pool_trunk.patch, os_pool_trunk.patch
Running the view attached results in a reduce_overflow_error. For each
reduce_overflow_error a process of /usr/lib/couchdb/bin/couchjs
/usr/share/couchdb/server/main.js starts running. Once this gets to 25, which
is the os_process_limit by default, all views result in a server error:
timeout {gen_server,call,[couch_query_servers,{get_proc,javascript}]}
As far as I can tell, these processes and the non-response from the views
will continue until couch is restarted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
