[jira] Commented: (COUCHDB-650) Add "now" option for ?since parameter on _changes
[ https://issues.apache.org/jira/browse/COUCHDB-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832869#action_12832869 ] Randall Leeds commented on COUCHDB-650: --- "The problem with since=now is that it can lead to missed updates." This statement is true only under the assumption that the client cares about old updates, which may not be the case. I submit that this patch is non-essential (hence "Improvement"). It is always possible to query /db to get the current seq. However, in my head I see the potential for since=now to become a common usage pattern that avoids an extra GET. My particular use case involves a clustering scenario (specifically, Lounge) in which I am streaming _changes notifications from all shards and returning an opaque token that encodes the seq number for every shard. When the redundancy configuration changes the server has a few options when encountering a ?since request that doesn't entirely specify where to begin listening on every shard. 1) The server uses ?since=0 on the shards of which the client has no knowledge. Clearly not okay with a large history. 2) The server issues a /db request to the unspecified shard in order to start from "now". I was just trying to create a shortcut for the now case, which would simplify the logic of the server. After I received reasonably enthusiastic feedback from those present on IRC I wrote the quick patch. Again, non-essential. If there's a clear non-inclusion policy toward non-essential features like this, I understand if you refuse it. As I see it, it doesn't complicate the API significantly because it reuses the ?since parameter. I wonder now how many people would actually find this useful. We probably shouldn't dwell on something so small for too long. So, Chris, I'm happy for you to just make the call. Feel free to close as wont-fix or apply as you see fit. > Add "now" option for ?since parameter on _changes > - > > Key: COUCHDB-650 > URL: https://issues.apache.org/jira/browse/COUCHDB-650 > Project: CouchDB > Issue Type: Improvement > Components: HTTP Interface >Reporter: Randall Leeds >Priority: Minor > Attachments: > 0001-add-since-now-option-to-continuous-catch-edge-cases-.patch > > > As per a discussion on IRC with mikeal, jan_, whartung, and rnewson: > Having _changes?since="now" allows clients to listen to only new changes > without making two requests (one to extract seq from /db, and one _changes). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-650) Add "now" option for ?since parameter on _changes
[ https://issues.apache.org/jira/browse/COUCHDB-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832859#action_12832859 ] Chris Anderson commented on COUCHDB-650: The problem with since=now is that it can lead to missed updates. There should be a ticket to make view responses include the view_seq and to expose doc._local_seq via a GET parameter. It's much better to know what seq your state is current as of, than to miss updates. > Add "now" option for ?since parameter on _changes > - > > Key: COUCHDB-650 > URL: https://issues.apache.org/jira/browse/COUCHDB-650 > Project: CouchDB > Issue Type: Improvement > Components: HTTP Interface >Reporter: Randall Leeds >Priority: Minor > Attachments: > 0001-add-since-now-option-to-continuous-catch-edge-cases-.patch > > > As per a discussion on IRC with mikeal, jan_, whartung, and rnewson: > Having _changes?since="now" allows clients to listen to only new changes > without making two requests (one to extract seq from /db, and one _changes). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Auth Roadmap
On 11 Feb 2010, at 22:10, Chris Anderson wrote: >> >> 2) ACLs / Security Object >> >> I couldn't originally think of a reason the validation funs would need >> to see the per-db admins / readers lists. Brian has a use case for >> this. Also, I think I can accomplish this feature while also >> simplifying the implementation. I'd like to make it so each db has an >> /_security object that contains admins and readers (in their current >> form, but as fields on the object, not as separate object at different >> URLs). This entire object would be made available to the validation >> functions. > > This is done and in trunk. I plan to backport this to the 0.11.x > branch unless there are objections. Go for it. It looks like you got rid of some extra code there too :) Cheers Jan -- http://couch.io/
Re: Auth Roadmap
On Tue, Feb 9, 2010 at 2:52 PM, Chris Anderson wrote: > Devs, > > I've been getting a lot of feedback about the authentication & > authorization work that I did over the holidays and over the last few > weeks. There are also some enhancements I've been thinking about for a > while. Here's a quick list of what I see as the important things to > do. I'm not concerned here with releases / feature freeze etc as in my > opinion CouchDB development is expected to continue even after we > reach 1.0. > > 1) Extensible password storage. > > Thanks Brian Candler for the links to the OpenLDAP style of storage. I > think we should do this asap so we don't have to worry about backwards > compatibility with the current storage mechanism until the end of > time. The relevant message: > http://permalink.gmane.org/gmane.comp.db.couchdb.devel/7588 I'm helping Filipe Manana with an implementation of this, which will be backwards compatible with existing admin passwords (stored in config). We won't try to be backwards compatible with old _users dbs (it should be simple to write an upgrade script if you have crucial data). This doesn't need to block 0.11 but it could go in 0.11.1 as it'd be nice to get it out there for people who want better crypto. > > 2) ACLs / Security Object > > I couldn't originally think of a reason the validation funs would need > to see the per-db admins / readers lists. Brian has a use case for > this. Also, I think I can accomplish this feature while also > simplifying the implementation. I'd like to make it so each db has an > /_security object that contains admins and readers (in their current > form, but as fields on the object, not as separate object at different > URLs). This entire object would be made available to the validation > functions. This is done and in trunk. I plan to backport this to the 0.11.x branch unless there are objections. > > 3) More system roles. > > Brian also mentioned something about _user and _anon roles which could > be applied to the userCtx automatically. This would be handy in both > per-db access control and in validation functions. This will be a bit > harder to implement as it touches more of the codebase. I'm also > uneasy about these roles as they raise the burden for implementors of > pluggable authentication modules. > > Going forward we maybe want to add a _replicator role (or maybe that's > a horrible idea). We should also think about making it possible for > _admins to interact with the database without the _admin role. They > could trigger admin actions with something like sudo. I want to put > this off for now, because it's complicated and worse-case scenario is > people don't realize they need to "sudo" to get things done, and come > away thinking CouchDB is fighting with them. we can punt on this for 1.0, it won't break backward compat to add it later > > 4) _temp_views and _all_dbs (maybe more) > > Regardless of whether the above is done, for 1.0 we should clean up > any bugs like these. > Please file tickets (especially with JavaScript test cases) if you find anything that needs fixing for 1.0. These are important bugs to fix but they don't need to block 0.11 > 5) drop box > this can happen after 1.0 Thanks, Chris -- Chris Anderson http://jchrisa.net http://couch.io
[jira] Updated: (COUCHDB-650) Add "now" option for ?since parameter on _changes
[ https://issues.apache.org/jira/browse/COUCHDB-650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Leeds updated COUCHDB-650: -- Attachment: 0001-add-since-now-option-to-continuous-catch-edge-cases-.patch This patch adds the requested functionality. Additionally, I took care of some edge cases while I was in there. 1) ?feed=continuous&descending=true now throws an error 2) numeric ?since will clamp to [0, db update_seq] (makes last_seq sane in the out of range cases) Adds tests for descending=true conflict and since="now". > Add "now" option for ?since parameter on _changes > - > > Key: COUCHDB-650 > URL: https://issues.apache.org/jira/browse/COUCHDB-650 > Project: CouchDB > Issue Type: Improvement > Components: HTTP Interface >Reporter: Randall Leeds >Priority: Minor > Attachments: > 0001-add-since-now-option-to-continuous-catch-edge-cases-.patch > > > As per a discussion on IRC with mikeal, jan_, whartung, and rnewson: > Having _changes?since="now" allows clients to listen to only new changes > without making two requests (one to extract seq from /db, and one _changes). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (COUCHDB-650) Add "now" option for ?since parameter on _changes
Add "now" option for ?since parameter on _changes - Key: COUCHDB-650 URL: https://issues.apache.org/jira/browse/COUCHDB-650 Project: CouchDB Issue Type: Improvement Components: HTTP Interface Reporter: Randall Leeds Priority: Minor As per a discussion on IRC with mikeal, jan_, whartung, and rnewson: Having _changes?since="now" allows clients to listen to only new changes without making two requests (one to extract seq from /db, and one _changes). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Auth Roadmap
On Thu, Feb 11, 2010 at 08:32:49AM -0800, Chris Anderson wrote:
> To be clear, I'm not suggesting this at all.
>
> It'd be more like (pardon my earlier accidental _underscores):
>
> {
> "readers":{
> "names":["foo","bar"],
> "roles":["baz", "_replicator", "doctor"]
> },
> "admins":{
> "names":["jan","brian"],
> "roles":["support", (_admin is an implied member)]
> },
> "other_security_stuff":{...}
> }
Oh I see. When you replicate, you give the credentials for the remote host,
but perhaps the local side should pick up a _replicator role. (Or perhaps
not, if it runs with the credentials of the user who started the
replication)
I can imagine "readers" splitting in future though: an indirect reader
capability which can access _show/_list/_update but nothing else would be
able to enforce controls at the document and view row level, since those
points all have access to userCtx.
Regards,
Brian.
Re: Auth Roadmap
On Thu, Feb 11, 2010 at 4:20 AM, Brian Candler wrote:
>> I think what you say has merit, but it doesn't jibe with my
>> understanding of our implementation in a logical way. I'm ready to
>> ship the basic programming model we have - it maps cleanly onto the
>> underlying infrastructure (less abstraction can be a good thing).
>
> With respect, I think we're actually talking about the same thing :-)
>
> You've already said that it may make sense to lump _readers and _admins into
> _security. If so, then I think this is what you would get:
>
> {
> "_readers":{
> "names":["foo","bar"],
> "roles":["baz"]
> },
> "_admins":{
> "names":["jan","brian"],
> "roles":["support"]
> },
> "other_security_stuff":{...}
> }
>
> You've also suggested adding some more granular capabilities, such as
> _replicators and create-only (dropbox). Then _security would become:
>
> {
> "_readers":{
> "names":["foo","bar"],
> "roles":["baz"]
> },
> "_admins":{
> "names":["jan","brian"],
> "roles":["support"]
> },
> "_replicators":{
> "names":["cron"],
> "roles":["baz"],
> },
> "_creators":{
> "names":["foo"],
> "roles":["_anon"],
> },
> "other_security_stuff":{...}
> }
>
>
To be clear, I'm not suggesting this at all.
It'd be more like (pardon my earlier accidental _underscores):
{
"readers":{
"names":["foo","bar"],
"roles":["baz", "_replicator", "doctor"]
},
"admins":{
"names":["jan","brian"],
"roles":["support", (_admin is an implied member)]
},
"other_security_stuff":{...}
}
_replicator, etc wouldn't go in as top level keys with special
meaning. They'd be just another role like doctor or foo. I actually
can't think of a time you'd put _replicator into the ACLs. It'd be
more likely something checked by the validation function. (Eg only
allow docs to be written if doc.author == userCtx.name, unless we are
replicating and the replication was triggered with both _admin and
_replicator roles.) The more I think about it the more of a bad idea I
think a _replicator role could potentially be.
As far as drop box, I think it'd just be a boolean flag ("allow anyone
to write to this database even if they aren't in the readers list")
which should be easy to add in the future without changing any of the
existing stuff.
There's no need for a _creators list as that can be implemented in the
validation function.
> Now, the top-level keys there are exactly what I called "rights", and would
> be enforced in erlang by check_is_admin, check_is_reader,
> check_is_replicator etc.
Something like check_is_replicator can be written in javascript and be
part of the validation function.
>
> You can keep the data structure exactly like that. All I'm offering is that
> you *could* turn the structure on its head like this:
>
> {
> "names":{
> "foo":["_reader","_creator"],
> "bar":["_reader"],
> "jan":["_admin"],
> "brian":["_admin"],
> "cron":["_replicator"]
> },
> "roles:{
> "baz":["_reader","_replicator"],
> "support":["_admin"],
> "_anon":["_creator"]
> },
> "other_security_stuff":{...}
> }
>
> I prefer this format because all the rights for one user/role are in a
> single place; consequently it's easier to remove all access for a user.
>
> The other benefit is that Futon can also display application-specific rights
> without being confused by other stuff at the top level of the _security
> object; and, if extra couchdb rights are added, Futon doesn't need to
> change.
>
> For example, it would be hard for Futon to display the extra 'doctors'
> rights from this:
>
> {
> "_readers":{
> "names":["foo","bar"],
> "roles":["baz"]
> },
> "_admins":{
> "names":["jan","brian"],
> "roles":["support"]
> },
> "doctors":{
> "names":["foo","jim"],
> },
> "hmac_key":"abc12345",
> }
>
> ("doctors" having significance in validate_doc_update and _show/_list)
> because it wouldn't know which of "doctors" and "hmac_key" should be
> displayed as an ACL. But if you restructure it as
>
> {
> "names":{
> "foo":["_reader","doctor"],
> "bar":["_reader"],
> "jan":["_admin"],
> "jim":["doctor"],
> "brian":["_admin"],
> },
> "roles:{
> "baz":["_reader"],
> "support":["_admin"]
> },
> "hmac_key":"abc12345"
> }
>
> then futon can show that jim is a doctor, and allow the doctor right to be
> added to other users.
>
> The functionality is ultimately the same though, so I'm not wedded to the
> second format.
>
> Regards,
>
> Brian.
>
--
Chris Anderson
http://jchrisa.net
http://couch.io
Re: Auth Roadmap
> I think what you say has merit, but it doesn't jibe with my
> understanding of our implementation in a logical way. I'm ready to
> ship the basic programming model we have - it maps cleanly onto the
> underlying infrastructure (less abstraction can be a good thing).
With respect, I think we're actually talking about the same thing :-)
You've already said that it may make sense to lump _readers and _admins into
_security. If so, then I think this is what you would get:
{
"_readers":{
"names":["foo","bar"],
"roles":["baz"]
},
"_admins":{
"names":["jan","brian"],
"roles":["support"]
},
"other_security_stuff":{...}
}
You've also suggested adding some more granular capabilities, such as
_replicators and create-only (dropbox). Then _security would become:
{
"_readers":{
"names":["foo","bar"],
"roles":["baz"]
},
"_admins":{
"names":["jan","brian"],
"roles":["support"]
},
"_replicators":{
"names":["cron"],
"roles":["baz"],
},
"_creators":{
"names":["foo"],
"roles":["_anon"],
},
"other_security_stuff":{...}
}
Now, the top-level keys there are exactly what I called "rights", and would
be enforced in erlang by check_is_admin, check_is_reader,
check_is_replicator etc.
You can keep the data structure exactly like that. All I'm offering is that
you *could* turn the structure on its head like this:
{
"names":{
"foo":["_reader","_creator"],
"bar":["_reader"],
"jan":["_admin"],
"brian":["_admin"],
"cron":["_replicator"]
},
"roles:{
"baz":["_reader","_replicator"],
"support":["_admin"],
"_anon":["_creator"]
},
"other_security_stuff":{...}
}
I prefer this format because all the rights for one user/role are in a
single place; consequently it's easier to remove all access for a user.
The other benefit is that Futon can also display application-specific rights
without being confused by other stuff at the top level of the _security
object; and, if extra couchdb rights are added, Futon doesn't need to
change.
For example, it would be hard for Futon to display the extra 'doctors'
rights from this:
{
"_readers":{
"names":["foo","bar"],
"roles":["baz"]
},
"_admins":{
"names":["jan","brian"],
"roles":["support"]
},
"doctors":{
"names":["foo","jim"],
},
"hmac_key":"abc12345",
}
("doctors" having significance in validate_doc_update and _show/_list)
because it wouldn't know which of "doctors" and "hmac_key" should be
displayed as an ACL. But if you restructure it as
{
"names":{
"foo":["_reader","doctor"],
"bar":["_reader"],
"jan":["_admin"],
"jim":["doctor"],
"brian":["_admin"],
},
"roles:{
"baz":["_reader"],
"support":["_admin"]
},
"hmac_key":"abc12345"
}
then futon can show that jim is a doctor, and allow the doctor right to be
added to other users.
The functionality is ultimately the same though, so I'm not wedded to the
second format.
Regards,
Brian.
