Re: Auth Roadmap
On Fri, Feb 12, 2010 at 7:10 AM, Chris Anderson jch...@apache.org wrote:
On Tue, Feb 9, 2010 at 2:52 PM, Chris Anderson jch...@apache.org wrote:
Devs,
I've been getting a lot of feedback about the authentication
authorization work that I did over the holidays and over the last few
weeks. There are also some enhancements I've been thinking about for a
while. Here's a quick list of what I see as the important things to
do. I'm not concerned here with releases / feature freeze etc as in my
opinion CouchDB development is expected to continue even after we
reach 1.0.
1) Extensible password storage.
Thanks Brian Candler for the links to the OpenLDAP style of storage. I
think we should do this asap so we don't have to worry about backwards
compatibility with the current storage mechanism until the end of
time. The relevant message:
http://permalink.gmane.org/gmane.comp.db.couchdb.devel/7588
I'm helping Filipe Manana with an implementation of this, which will
be backwards compatible with existing admin passwords (stored in
config). We won't try to be backwards compatible with old _users dbs
(it should be simple to write an upgrade script if you have crucial
data). This doesn't need to block 0.11 but it could go in 0.11.1 as
it'd be nice to get it out there for people who want better crypto.
Actually I think I wouild prefer a json object so we don't have to
parse string a such
{
auth-type: brypt-sha1,
key: ...
}
and in ini
[crypt-handlers]
{ bcrypt-sha1, couch_httpd_auth, brcrypt_sha1_encode,
brcrypt_sha1_decode }
Somethink like it. Which is basically the same as having all in one
string but is more jsonful. Also it remove the need to parse the
string or do more pattern matching than needed.
Is there any ticket open about it ?
- benoît
Re: Auth Roadmap
Benoit,
The .ini file suggestion seems a good idea, as it allows for pluggable
password hash validation. Now we're thinking about OpenLDAP schemes, but
tomorrow someone might suggest or require something else.
About the JSON object, containing the hash scheme in an attribute and the
hash in a different one, I'm not 100% sure about it. Usually the scheme,
password hash and used salt (if any) travel in the same string. Take a look
at OpenLDAP:
$ slappasswd -s 'teste_password123' -h '{SSHA}'
{SSHA}Pmnj6Jg3CBccqLuYttGMfTYh1GKMLLyC
$ slappasswd -s 'teste_password123' -h '{SMD5}'
{SMD5}ojYaLtvJga+4hJqMSgi34JGUV/8=
and at RFC 3112 ( http://www.ietf.org/rfc/rfc3112.txt ) -
SHA1$RzqH67DY3uQ=$atAcDs1eS+IJwPy7V4UDXEoBrDI=
I would prefer having a .ini file with:
[couch_httpd_auth]
password_hash_scheme = -hashed- ; the default scheme to use for hashing
passwords
password_validators= {couch_httpd_auth, hashed_password_validator},
{couch_httpd_auth, openldap_password_validator}
[password_hash_creators]
; scheme = {module, function}
-hashed- = {couch_httpd_auth, hash_password}
{SSHA} = {couch_httpd_auth, openldap_hash_password}
{SMD5} = {couch_httpd_auth, openldap_hash_password}
To validate a user supplied password, we would execute each
password_validator in turn until one of them succeeds. Exactly like we do
now with the authentication_handlers.
Each password validator would receive 2 paremeters: the user supplied
password and the password hash previously computed.
Each password hash creator would receive 2 parameters: the scheme and the
password to hash.
In a _users doc, we would drop the attribute salt and rename
password_sha to password_hash.
What do you think?
On Fri, Feb 12, 2010 at 1:20 PM, Benoit Chesneau bchesn...@gmail.comwrote:
On Fri, Feb 12, 2010 at 7:10 AM, Chris Anderson jch...@apache.org wrote:
On Tue, Feb 9, 2010 at 2:52 PM, Chris Anderson jch...@apache.org
wrote:
Devs,
I've been getting a lot of feedback about the authentication
authorization work that I did over the holidays and over the last few
weeks. There are also some enhancements I've been thinking about for a
while. Here's a quick list of what I see as the important things to
do. I'm not concerned here with releases / feature freeze etc as in my
opinion CouchDB development is expected to continue even after we
reach 1.0.
1) Extensible password storage.
Thanks Brian Candler for the links to the OpenLDAP style of storage. I
think we should do this asap so we don't have to worry about backwards
compatibility with the current storage mechanism until the end of
time. The relevant message:
http://permalink.gmane.org/gmane.comp.db.couchdb.devel/7588
I'm helping Filipe Manana with an implementation of this, which will
be backwards compatible with existing admin passwords (stored in
config). We won't try to be backwards compatible with old _users dbs
(it should be simple to write an upgrade script if you have crucial
data). This doesn't need to block 0.11 but it could go in 0.11.1 as
it'd be nice to get it out there for people who want better crypto.
Actually I think I wouild prefer a json object so we don't have to
parse string a such
{
auth-type: brypt-sha1,
key: ...
}
and in ini
[crypt-handlers]
{ bcrypt-sha1, couch_httpd_auth, brcrypt_sha1_encode,
brcrypt_sha1_decode }
Somethink like it. Which is basically the same as having all in one
string but is more jsonful. Also it remove the need to parse the
string or do more pattern matching than needed.
Is there any ticket open about it ?
- benoît
--
Filipe David Manana,
fdman...@gmail.com
PGP key - http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xC569452B
Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men.
Re: Auth Roadmap
I think what you say has merit, but it doesn't jibe with my
understanding of our implementation in a logical way. I'm ready to
ship the basic programming model we have - it maps cleanly onto the
underlying infrastructure (less abstraction can be a good thing).
With respect, I think we're actually talking about the same thing :-)
You've already said that it may make sense to lump _readers and _admins into
_security. If so, then I think this is what you would get:
{
_readers:{
names:[foo,bar],
roles:[baz]
},
_admins:{
names:[jan,brian],
roles:[support]
},
other_security_stuff:{...}
}
You've also suggested adding some more granular capabilities, such as
_replicators and create-only (dropbox). Then _security would become:
{
_readers:{
names:[foo,bar],
roles:[baz]
},
_admins:{
names:[jan,brian],
roles:[support]
},
_replicators:{
names:[cron],
roles:[baz],
},
_creators:{
names:[foo],
roles:[_anon],
},
other_security_stuff:{...}
}
Now, the top-level keys there are exactly what I called rights, and would
be enforced in erlang by check_is_admin, check_is_reader,
check_is_replicator etc.
You can keep the data structure exactly like that. All I'm offering is that
you *could* turn the structure on its head like this:
{
names:{
foo:[_reader,_creator],
bar:[_reader],
jan:[_admin],
brian:[_admin],
cron:[_replicator]
},
roles:{
baz:[_reader,_replicator],
support:[_admin],
_anon:[_creator]
},
other_security_stuff:{...}
}
I prefer this format because all the rights for one user/role are in a
single place; consequently it's easier to remove all access for a user.
The other benefit is that Futon can also display application-specific rights
without being confused by other stuff at the top level of the _security
object; and, if extra couchdb rights are added, Futon doesn't need to
change.
For example, it would be hard for Futon to display the extra 'doctors'
rights from this:
{
_readers:{
names:[foo,bar],
roles:[baz]
},
_admins:{
names:[jan,brian],
roles:[support]
},
doctors:{
names:[foo,jim],
},
hmac_key:abc12345,
}
(doctors having significance in validate_doc_update and _show/_list)
because it wouldn't know which of doctors and hmac_key should be
displayed as an ACL. But if you restructure it as
{
names:{
foo:[_reader,doctor],
bar:[_reader],
jan:[_admin],
jim:[doctor],
brian:[_admin],
},
roles:{
baz:[_reader],
support:[_admin]
},
hmac_key:abc12345
}
then futon can show that jim is a doctor, and allow the doctor right to be
added to other users.
The functionality is ultimately the same though, so I'm not wedded to the
second format.
Regards,
Brian.
Re: Auth Roadmap
On Thu, Feb 11, 2010 at 4:20 AM, Brian Candler b.cand...@pobox.com wrote:
I think what you say has merit, but it doesn't jibe with my
understanding of our implementation in a logical way. I'm ready to
ship the basic programming model we have - it maps cleanly onto the
underlying infrastructure (less abstraction can be a good thing).
With respect, I think we're actually talking about the same thing :-)
You've already said that it may make sense to lump _readers and _admins into
_security. If so, then I think this is what you would get:
{
_readers:{
names:[foo,bar],
roles:[baz]
},
_admins:{
names:[jan,brian],
roles:[support]
},
other_security_stuff:{...}
}
You've also suggested adding some more granular capabilities, such as
_replicators and create-only (dropbox). Then _security would become:
{
_readers:{
names:[foo,bar],
roles:[baz]
},
_admins:{
names:[jan,brian],
roles:[support]
},
_replicators:{
names:[cron],
roles:[baz],
},
_creators:{
names:[foo],
roles:[_anon],
},
other_security_stuff:{...}
}
To be clear, I'm not suggesting this at all.
It'd be more like (pardon my earlier accidental _underscores):
{
readers:{
names:[foo,bar],
roles:[baz, _replicator, doctor]
},
admins:{
names:[jan,brian],
roles:[support, (_admin is an implied member)]
},
other_security_stuff:{...}
}
_replicator, etc wouldn't go in as top level keys with special
meaning. They'd be just another role like doctor or foo. I actually
can't think of a time you'd put _replicator into the ACLs. It'd be
more likely something checked by the validation function. (Eg only
allow docs to be written if doc.author == userCtx.name, unless we are
replicating and the replication was triggered with both _admin and
_replicator roles.) The more I think about it the more of a bad idea I
think a _replicator role could potentially be.
As far as drop box, I think it'd just be a boolean flag (allow anyone
to write to this database even if they aren't in the readers list)
which should be easy to add in the future without changing any of the
existing stuff.
There's no need for a _creators list as that can be implemented in the
validation function.
Now, the top-level keys there are exactly what I called rights, and would
be enforced in erlang by check_is_admin, check_is_reader,
check_is_replicator etc.
Something like check_is_replicator can be written in javascript and be
part of the validation function.
You can keep the data structure exactly like that. All I'm offering is that
you *could* turn the structure on its head like this:
{
names:{
foo:[_reader,_creator],
bar:[_reader],
jan:[_admin],
brian:[_admin],
cron:[_replicator]
},
roles:{
baz:[_reader,_replicator],
support:[_admin],
_anon:[_creator]
},
other_security_stuff:{...}
}
I prefer this format because all the rights for one user/role are in a
single place; consequently it's easier to remove all access for a user.
The other benefit is that Futon can also display application-specific rights
without being confused by other stuff at the top level of the _security
object; and, if extra couchdb rights are added, Futon doesn't need to
change.
For example, it would be hard for Futon to display the extra 'doctors'
rights from this:
{
_readers:{
names:[foo,bar],
roles:[baz]
},
_admins:{
names:[jan,brian],
roles:[support]
},
doctors:{
names:[foo,jim],
},
hmac_key:abc12345,
}
(doctors having significance in validate_doc_update and _show/_list)
because it wouldn't know which of doctors and hmac_key should be
displayed as an ACL. But if you restructure it as
{
names:{
foo:[_reader,doctor],
bar:[_reader],
jan:[_admin],
jim:[doctor],
brian:[_admin],
},
roles:{
baz:[_reader],
support:[_admin]
},
hmac_key:abc12345
}
then futon can show that jim is a doctor, and allow the doctor right to be
added to other users.
The functionality is ultimately the same though, so I'm not wedded to the
second format.
Regards,
Brian.
--
Chris Anderson
http://jchrisa.net
http://couch.io
Re: Auth Roadmap
On Thu, Feb 11, 2010 at 08:32:49AM -0800, Chris Anderson wrote:
To be clear, I'm not suggesting this at all.
It'd be more like (pardon my earlier accidental _underscores):
{
readers:{
names:[foo,bar],
roles:[baz, _replicator, doctor]
},
admins:{
names:[jan,brian],
roles:[support, (_admin is an implied member)]
},
other_security_stuff:{...}
}
Oh I see. When you replicate, you give the credentials for the remote host,
but perhaps the local side should pick up a _replicator role. (Or perhaps
not, if it runs with the credentials of the user who started the
replication)
I can imagine readers splitting in future though: an indirect reader
capability which can access _show/_list/_update but nothing else would be
able to enforce controls at the document and view row level, since those
points all have access to userCtx.
Regards,
Brian.
Re: Auth Roadmap
On Tue, Feb 9, 2010 at 2:52 PM, Chris Anderson jch...@apache.org wrote: Devs, I've been getting a lot of feedback about the authentication authorization work that I did over the holidays and over the last few weeks. There are also some enhancements I've been thinking about for a while. Here's a quick list of what I see as the important things to do. I'm not concerned here with releases / feature freeze etc as in my opinion CouchDB development is expected to continue even after we reach 1.0. 1) Extensible password storage. Thanks Brian Candler for the links to the OpenLDAP style of storage. I think we should do this asap so we don't have to worry about backwards compatibility with the current storage mechanism until the end of time. The relevant message: http://permalink.gmane.org/gmane.comp.db.couchdb.devel/7588 I'm helping Filipe Manana with an implementation of this, which will be backwards compatible with existing admin passwords (stored in config). We won't try to be backwards compatible with old _users dbs (it should be simple to write an upgrade script if you have crucial data). This doesn't need to block 0.11 but it could go in 0.11.1 as it'd be nice to get it out there for people who want better crypto. 2) ACLs / Security Object I couldn't originally think of a reason the validation funs would need to see the per-db admins / readers lists. Brian has a use case for this. Also, I think I can accomplish this feature while also simplifying the implementation. I'd like to make it so each db has an /_security object that contains admins and readers (in their current form, but as fields on the object, not as separate object at different URLs). This entire object would be made available to the validation functions. This is done and in trunk. I plan to backport this to the 0.11.x branch unless there are objections. 3) More system roles. Brian also mentioned something about _user and _anon roles which could be applied to the userCtx automatically. This would be handy in both per-db access control and in validation functions. This will be a bit harder to implement as it touches more of the codebase. I'm also uneasy about these roles as they raise the burden for implementors of pluggable authentication modules. Going forward we maybe want to add a _replicator role (or maybe that's a horrible idea). We should also think about making it possible for _admins to interact with the database without the _admin role. They could trigger admin actions with something like sudo. I want to put this off for now, because it's complicated and worse-case scenario is people don't realize they need to sudo to get things done, and come away thinking CouchDB is fighting with them. we can punt on this for 1.0, it won't break backward compat to add it later 4) _temp_views and _all_dbs (maybe more) Regardless of whether the above is done, for 1.0 we should clean up any bugs like these. Please file tickets (especially with JavaScript test cases) if you find anything that needs fixing for 1.0. These are important bugs to fix but they don't need to block 0.11 5) drop box this can happen after 1.0 Thanks, Chris -- Chris Anderson http://jchrisa.net http://couch.io
Re: Auth Roadmap
On 11 Feb 2010, at 22:10, Chris Anderson wrote: 2) ACLs / Security Object I couldn't originally think of a reason the validation funs would need to see the per-db admins / readers lists. Brian has a use case for this. Also, I think I can accomplish this feature while also simplifying the implementation. I'd like to make it so each db has an /_security object that contains admins and readers (in their current form, but as fields on the object, not as separate object at different URLs). This entire object would be made available to the validation functions. This is done and in trunk. I plan to backport this to the 0.11.x branch unless there are objections. Go for it. It looks like you got rid of some extra code there too :) Cheers Jan -- http://couch.io/
Re: Auth Roadmap
On Wed, Feb 10, 2010 at 12:24:26AM +0100, Benoit Chesneau wrote:
I've read all the thread, and I'm not conviced all readers and admins
should be in one doc. List could be long also it would require to
check if one already exists some stuff like it. Why not putting all in
their docs and making a view on it ?
I'd prefer a real doc or docs too.
But if we have to have an opaque non-scalable non-document object, it would
be better to have one of these things rather than three of them.
3) More system roles.
Brian also mentioned something about _user and _anon roles which could
be applied to the userCtx automatically. This would be handy in both
per-db access control and in validation functions. This will be a bit
harder to implement as it touches more of the codebase. I'm also
uneasy about these roles as they raise the burden for implementors of
pluggable authentication modules.
Everyone could be _anon instead of _admin though.
Maybe _anon isn't even needed, because they will have name:null.
But you'd want a way of saying any logged-in user, which could be as
simple as adding a fixed role like _user to everyone who has a non-null
name. And if you're going to do that, you might as well add _anon role to
everyone who has a null name. Sounds easy to me, but I've not tried coding
it yet.
We should also think about making it possible for
_admins to interact with the database without the _admin role. They
could trigger admin actions with something like sudo.
That's where having a real document would win, because you could use
validate_doc_update for this logic. Such logic already exists in the _users
database, to prevent non-admins from adding roles to themselves.
Personally I think that if a system admin is a role, then a database admin
should be a role, and a database reader should be a role (and by extension,
you could have a system reader role)
Suppose the per-database _security doc or object looked like this:
{
names:{
brian:[_admin], # brian is database-level admin
jan:[_reader],
bob:[_reader,foo] # foo role has local significance
}
roles:{
support:[_reader]# anyone with support role can read this DB
}
}
You'd merge the roles from the userCtx with these roles to get the effective
set of roles for this user when accessing this particular database.
Just to be sure everything is one user db ? Just want to make sure
it's possible to replicate all authentification for a node or even a
cluster.
That's another reason for doing this in docs. Either make _security a real
doc within each database, which can replicate along with it; or put the
authorizations into the _users database (which is easy with
database-specific roles).
The only objection I've heard to the latter idea is that if a database is
filesystem-copied or renamed, the authorizations won't move with it.
Authorizing against a database uuid rather than database name would solve
that.
Also: given that bob on db1 could be a different person than bob on db2,
it seems dangerous to replicate or copy the authorizations from one database
to another independently of the users database.
5) drop box
If we had the concept of DBs that you could write to but not read
from, it would make read-controlled _users dbs compatible with new
user signups. This would be not that fun to implement, but potentially
useful for lots of other things too.
+1 on this.
Interesting idea - also for logging databases, audit databases etc. Need to
think about the semantics.
Possibly POST to the database itself could have its own role, and it would
ignore _rev so it could only be used to create new docs. But still some
other mechanism would be required for users to change their own passwords.
One idea is to split _reader into two; one role would permit access to
_show, _list and _update whilst blocking all direct access.
Regards,
Brian.
Re: Auth Roadmap
On Wed, Feb 10, 2010 at 2:12 AM, Brian Candler b.cand...@pobox.com wrote:
On Wed, Feb 10, 2010 at 12:24:26AM +0100, Benoit Chesneau wrote:
I've read all the thread, and I'm not conviced all readers and admins
should be in one doc. List could be long also it would require to
check if one already exists some stuff like it. Why not putting all in
their docs and making a view on it ?
I'd prefer a real doc or docs too.
But if we have to have an opaque non-scalable non-document object, it would
be better to have one of these things rather than three of them.
3) More system roles.
Brian also mentioned something about _user and _anon roles which could
be applied to the userCtx automatically. This would be handy in both
per-db access control and in validation functions. This will be a bit
harder to implement as it touches more of the codebase. I'm also
uneasy about these roles as they raise the burden for implementors of
pluggable authentication modules.
Everyone could be _anon instead of _admin though.
Maybe _anon isn't even needed, because they will have name:null.
But you'd want a way of saying any logged-in user, which could be as
simple as adding a fixed role like _user to everyone who has a non-null
name. And if you're going to do that, you might as well add _anon role to
everyone who has a null name. Sounds easy to me, but I've not tried coding
it yet.
We should also think about making it possible for
_admins to interact with the database without the _admin role. They
could trigger admin actions with something like sudo.
That's where having a real document would win, because you could use
validate_doc_update for this logic. Such logic already exists in the _users
database, to prevent non-admins from adding roles to themselves.
Personally I think that if a system admin is a role, then a database admin
should be a role, and a database reader should be a role (and by extension,
you could have a system reader role)
Suppose the per-database _security doc or object looked like this:
{
names:{
brian:[_admin], # brian is database-level admin
jan:[_reader],
bob:[_reader,foo] # foo role has local significance
}
roles:{
support:[_reader] # anyone with support role can read this DB
}
}
The problem with this approach, imho, is that currently users have the
same set of roles in every db. That is, your userCtx doesn't change
depending on the db you are accessing. I can see how adding that
capability increases flexibility. But it's the sort of flexibility
that I see having a nasty complexity tax. Eg: currently the /_session
resource is server-wide. As soon as some roles are available on
certain dbs and not others, we need to make it /db/_session which is
misleading as you don't actually log into dbs, you log into the
server. The consequences of per-db roles being applied to users would
ripple through the system as complexity in all sorts of places.
The thing you've got above I'd do as:
jan and bob both have foobar in their list of roles in their
documents in the _users db.
the security object looks like:
{
readers: {
names : [],
roles : [foobar, support],
}
admins : {
names : [brian]
roles : []
}
whatever else...
}
I think we don't want this object to replicate, as moving data around
shouldn't have a chance of affecting ACLs on the target db.
This is why it is not a document. (It could be a _local document and
have similar semantics, but O(log n) read cost instead of O(1) read
cost. This is why we use a special member.)
You'd merge the roles from the userCtx with these roles to get the effective
set of roles for this user when accessing this particular database.
Just to be sure everything is one user db ? Just want to make sure
it's possible to replicate all authentification for a node or even a
cluster.
That's another reason for doing this in docs. Either make _security a real
doc within each database, which can replicate along with it; or put the
authorizations into the _users database (which is easy with
database-specific roles).
The only objection I've heard to the latter idea is that if a database is
filesystem-copied or renamed, the authorizations won't move with it.
Authorizing against a database uuid rather than database name would solve
that.
Also: given that bob on db1 could be a different person than bob on db2,
it seems dangerous to replicate or copy the authorizations from one database
to another independently of the users database.
5) drop box
If we had the concept of DBs that you could write to but not read
from, it would make read-controlled _users dbs compatible with new
user signups. This would be not that fun to implement, but potentially
useful for lots of other things too.
+1 on this.
Interesting idea - also for logging databases, audit databases etc. Need to
think about the semantics.
Possibly POST to the database itself could have its own
Re: Auth Roadmap
On Wed, Feb 10, 2010 at 2:27 PM, Brian Candler b.cand...@pobox.com wrote: On Wed, Feb 10, 2010 at 07:26:00AM -0800, Chris Anderson wrote: The problem with this approach, imho, is that currently users have the same set of roles in every db. That is, your userCtx doesn't change depending on the db you are accessing. I can see how adding that capability increases flexibility. But it's the sort of flexibility that I see having a nasty complexity tax. Eg: currently the /_session resource is server-wide. As soon as some roles are available on certain dbs and not others, we need to make it /db/_session which is misleading as you don't actually log into dbs, you log into the server. Yes this could be confusing, especially adding db-roles to system-roles. Maybe it's clearer to think of it as groups and rights: * groups are what your user is a member of (in your global _session). * rights are what you can do in a particular database. * rights within a db can be assigned to individual users or to groups. * The _admin group picks up _admin rights automatically on all dbs. I think what you say has merit, but it doesn't jibe with my understanding of our implementation in a logical way. I'm ready to ship the basic programming model we have - it maps cleanly onto the underlying infrastructure (less abstraction can be a good thing). I think the kinds of apps you want to write can work in the current model, and I hope the trade off for less abstraction (eg no _rev on the _security object) in favor of simplicity will turn out to be the right choice. I'd like to help you find an architecture that works for your app. Maybe db(s) per user + filtered _admin controlled replication, which also maps really cleanly to usage patterns. Thanks for all the feedback. Regards, Brian. -- Chris Anderson http://jchrisa.net http://couch.io
