I want to close the gaps which were mentioned previously in this thread with observability as these are valid questions. I think Derek Chen-Becker / Claude Warren wanted to know more about the auditing / observability here (2).
To expose the information about the violations, the CEP will integrate with Diagnostics framework by emitting a diagnostic event when warning or failure is emitted (this is already in place for all guardrails so password guardrail will do the same), and we will also make sure that the violations are visible in Audit logs. If this is all OK for people and there are no other questions, I would like to vote on that CEP soonish. (1) https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=228494146#CEP24:Passwordvalidation/generation-Observability (2) https://lists.apache.org/thread/0xm8ooshmg9l5jzgrfyko8sn1rchpwgc Thanks On Wed, Jun 12, 2024 at 11:25 PM Bernardo Botella < conta...@bernardobotella.com> wrote: > +1 on Francisco’s comments. TCM is a general feature that a lot of other > things will benefit from, and the fact that this CEP is one of those that > will benefit shouldn’t block it from moving forward. > > > On Jun 11, 2024, at 11:16 PM, Francisco Guerrero <fran...@apache.org> > wrote: > > > > Stefan, thanks for moving this CEP forward. This CEP brings a lot of > value > > to Cassandra without needing to wait for TCM. I can see how a > misconfigured > > node can be problematic, but the issue is not something introduced in > > this CEP, and it affects many other features in Cassandra. I think it > needs to > > be addressed separately. > > > > Mature database offerings have functionality that is proposed in your > CEP such > > as password strength, and preventing usage of previously used passwords. > > > > I'm looking forward to see what shape this CEP takes in the coming weeks, > > and also looking forward to the pull request when it lands. > > > > I think we can even extend this concept to MutualTLS authentication where > > we can impose certain restrictions on certificates. I recently > contributed > > https://issues.apache.org/jira/browse/CASSANDRA-18951 to Cassandra to > > add restrictions to the allowed certificate validity period. We can > consider having > > CEP-24 as a pluggable way to configure restrictions that are not > necessarily > > just scoped for passwords, but more generally to other authentication > methods. > > > > Best, > > - Francisco > > > > On 2024/06/07 17:58:34 Štefan Miklošovič wrote: > >> Hi Shailaja, > >> > >> thanks for taking a look at this. > >> > >> That was indeed just an example we can change. It was more about showing > >> what might be possible in the future, nothing is set in stone yet, as > the > >> last sentence "this is not the part of the initial implementation" > explains. > >> > >> When it comes to these very specific features you mentioned, I feel like > >> this is very "business specific" and I do not want to "pollute" > Cassandra > >> system tables unnecessarily. It was a long time ago since I was writing > >> that CEP and it made sense to me back than to have a table for previous > >> passwords but then I started to reconsider it because I do not know > about > >> any database out there which would offer something similar (correct me > if I > >> am wrong) plus I start to question its actual benefit for a database > user. > >> We are not trying to mimic the behavior of a website after all. More to > it, > >> the password rotation itself is quite a topic and there are opinions > that > >> password should not be actually rotated at all. Hence I think that it is > >> not the role of Cassandra to define how passwords are going to be > rotated, > >> with what frequency etc. Let's just keep it simple and let's just > enforce > >> the password strength itself. > >> > >> More to this CEP in general, after I read in the other thread about > CEP-42 > >> that Dinesh does not consider TCM to be a hard requirement for this CEP > and > >> he finds it very useful already, I think I will consolidate what I have > and > >> I will remove TCM part of that in order to make it happen sooner. > >> > >> I think I made a mistake by waiting for config in TCM but it was only > with > >> good intentions - to provide a comprehensive feature without any > >> compromises. It seems to me that providing a well rounded config in TCM > + > >> guardrails in TCM was too much for me to handle and it would take way > more > >> time than I anticipated and it will be better if this is a more > iterative > >> process. I think that based on where I am with the implementation of > >> guardrails in TCM (POC is basically done) it is more or less just a > coding > >> exercise to integrate it into general config in TCM once config in TCM > is > >> introduced. > >> > >> I think I will restructure the current CEP-24 a little bit and I will > move > >> more optional features into possible extensions in the future in order > to > >> keep the core functionality at the minimum in order to reason about it > more > >> easily. I will try to get back to this in the upcoming weeks and I will > >> eventually start a voting thread. > >> > >> Regards > >> > >> > >> On Fri, Jun 7, 2024 at 6:00 PM <shailajako...@icloud.com> wrote: > >> > >>> Hi Stefan, > >>> > >>> Thanks for the CEP, sounds great. Regarding > >>> > >>> If we were about to make this even harder to bypass, we may say that > >>> password can be changed once per day, for example (anytime for a > >>> superuser). Since we have "created" column which is of type timeuuid, > we > >>> would check this table and see if there was some password already set > that > >>> day or not and fail the request eventually. This is not the part of the > >>> initial implementation. > >>> > >>> Allowing password change only once a day would be too restrictive and > may > >>> create chaos for users. For example, I am trying to file a tax return > on > >>> the last day of deadline, I forgot the password I had set last year, > now > >>> changed it. Assume I forgot the password I just set either due to an > >>> unclear/faulty website or due to my bad memory with stress to file tax > >>> returns on the last day. In that case either I should be able to > change the > >>> password again or reset the password. > >>> > >>> To reuse the same password, I should change the password atleast 5 > times, > >>> i.e, we can allow changing password 4 times a day and after that we can > >>> provide a reset option which generates a random password. In my > experience > >>> most of the websites/applications allow changing password at least 3 > times > >>> or 5 times a day and locks the account if attempted again. We can > still use > >>> the reset option to unlock the account. > >>> > >>> I am hoping applications can tune this restriction using custom > >>> guardrails, as per their requirement. Am I right? > >>> > >>> Thanks, > >>> Shailaja > >>> > >>> > >>> On Jun 1, 2024, at 10:42 AM, Miklosovic, Stefan via dev < > >>> dev@cassandra.apache.org> wrote: > >>> > >>> I feel like this thread deserves an update. > >>> > >>> This CEP was put in a dormant state because there was one quite > >>> substantial flaw, that is that if a node is misconfigured in such a way > >>> that it would accept weaker passwords than other nodes in a cluster, it > >>> would not be safe. The security of such solution would be as safe as > the > >>> weakiest configuration of a node from a cluster. > >>> > >>> The correct answer to this problem was / is transactional guardrails. I > >>> was waiting for TCM to appear in trunk to implement this for year and a > >>> half and we are finally there (1) which I am very excited about. > >>> > >>> What transactional guardrails are doing is that each CQL mutation to a > >>> respective guardrails virtual table (which is mutable) will commit a > >>> transfromation into TCM log. That in turn means that this > configuration is > >>> propagated to whole cluster and survives restarts etc. That also means > that > >>> we are configuring any guardrail by one CQL statement for whole > cluster in > >>> persistent manner which I would say is quite powerful and time / cost > >>> saving from techops / devops point of view, especially on a very large > >>> scale. > >>> > >>> You can do something like this > >>> > >>> UPDATE system_guardrails.flags SET value = false where name = > >>> 'simplestrategy'; > >>> > >>> and this will be commited into TCM, everything replayed on restart, > same > >>> for whole cluster ... you got the idea. Hence, similarly, you can > commit > >>> configuration for a password validator and it will be same across whole > >>> cluster as well. > >>> > >>> This solution received quite positive feedback and it was suggested > that > >>> we should actually commit into TCM all configuration which is meant to > be > >>> same for each node. > >>> > >>> I stopped with the introduction of more general "config in TCM" > solution > >>> as there seems to be entities in this space which are trying to come up > >>> with that (that is the vibe I am getting) hence I am currently in kind > of a > >>> limbo and half-way there. > >>> > >>> Let's see what happens next, I just want to highlight that the next > course > >>> of action will most probably be the introduction of transactional > >>> configuration until this one can finally be integrated with that too. > >>> Currently, there is one missing configuration property to be > transactional > >>> - default_keyspace_rf - because it is used by one of guardrails too. > This > >>> leads to more general "config in TCM" case which we have not dealt > with yet. > >>> > >>> Branch with transactional guardrails is in (2). > >>> > >>> (1) https://issues.apache.org/jira/browse/CASSANDRA-19593 > >>> (2) > >>> > https://github.com/instaclustr/cassandra/tree/CEP-24-with-generator-tcm > >>> > >>> ________________________________________ > >>> From: Miklosovic, Stefan <stefan.mikloso...@netapp.com> > >>> Sent: Monday, December 19, 2022 14:24 > >>> To: dev@cassandra.apache.org > >>> Subject: Re: [Discuss] CEP-24 Password validation and generation > >>> > >>> NetApp Security WARNING: This is an external email. Do not click links > or > >>> open attachments unless you recognize the sender and know the content > is > >>> safe. > >>> > >>> > >>> > >>> > >>> One not-so-obvious consequence of the configuration of password > validator > >>> - since it is based on guardrails - is that if there is a cluster of 50 > >>> nodes and we change a configuration (in runtime) in one node, it needs > to > >>> be done for all remaining 49. We need to be sure that the > configuration is > >>> same for all nodes because if we do not configure one node the way we > want, > >>> all it takes to pass the (less secure) validation is to create > passwords > >>> while being logged on that node. I think that something similar was > done to > >>> memtables CEP and there was some additional discussion about that - > the way > >>> how it is configured - it is in yaml and not in schema so it is only > >>> node-specific, right? (not saying it is wrong, I just noticed that > there > >>> was additional discussion questioning that approach which was further > >>> clarified). However when it comes to security, I think it should be as > >>> robust as possible. > >>> > >>> I am not completely sure what to do here. It would be great to have > some > >>> "distributed configuration" otherwise all I can do is to mimic this > >>> behavior by a table, similarly as system_auth.roles is done for > passwords, > >>> for example. However, I feel like it should be more robust and I think > that > >>> in the future there might be more cases when we need to have the > >>> configuration distributed like this. > >>> > >>> However, I am fine to proceed with my original plan when community > thinks > >>> that the current approach is enough. > >>> > >>> ________________________________________ > >>> From: Claude Warren, Jr via dev <dev@cassandra.apache.org> > >>> Sent: Wednesday, October 19, 2022 10:58 > >>> To: dev@cassandra.apache.org > >>> Subject: Re: [Discuss] CEP-24 Password validation and generation > >>> > >>> NetApp Security WARNING: This is an external email. Do not click links > or > >>> open attachments unless you recognize the sender and know the content > is > >>> safe. > >>> > >>> > >>> > >>> Just to clarify, I have no objections to the current plan. > >>> > >>> On Thu, Oct 13, 2022 at 2:56 PM Claude Warren, Jr < > claude.war...@aiven.io > >>> <mailto:claude.war...@aiven.io>> wrote: > >>> I am not familiar with the Diagnostics framework but it sounds like it > >>> would satisfy the need. Thanks for pointing it out. I will dive into > it > >>> to get an understanding of how it works. > >>> > >>> On Thu, Oct 13, 2022 at 1:52 PM Miklosovic, Stefan < > >>> stefan.mikloso...@netapp.com<mailto:stefan.mikloso...@netapp.com>> > wrote: > >>> Hi Claude, > >>> > >>> we may also integrate with Diagnostics framework Cassandra already > ships. > >>> I would say this better suits to your requirements for observability. > I am > >>> not sure to what degree you are familiar with Diagnostics though. To > give > >>> you a better picture, events are fired and external observers (in the > >>> framework called "subscribers") would be notified about the internal > >>> accordingly. As of now, observers / subscribers are meant to integrate > with > >>> JMX through which these events flow. > >>> > >>> Do you think Diagnostics events would satisfy your needs? > >>> > >>> Regards > >>> > >>> ________________________________________ > >>> From: Claude Warren, Jr via dev <dev@cassandra.apache.org<mailto: > >>> dev@cassandra.apache.org>> > >>> Sent: Thursday, October 13, 2022 14:43 > >>> To: dev@cassandra.apache.org<mailto:dev@cassandra.apache.org> > >>> Subject: Re: [Discuss] CEP-24 Password validation and generation > >>> > >>> NetApp Security WARNING: This is an external email. Do not click links > or > >>> open attachments unless you recognize the sender and know the content > is > >>> safe. > >>> > >>> > >>> > >>> The only difference I see is that I see observability (observer) as > being > >>> a way to retrieve (or be notified about) data used within a process. > >>> Logging on the other hand, is a preservation of a state discovered in > an > >>> observable object. Observability can drive logging but it can also > drive > >>> aggregate statistics in grafana, and things like that. > >>> > >>> My reading of the CEP-3 is that it is intended to provide system-wide > soft > >>> and hard limits, it is not an observability framework. It makes sense > for > >>> the validator to implement CEP-3 but I think that an observability > >>> interface is required as well. > >>> > >>> On Thu, Oct 13, 2022 at 12:36 PM Miklosovic, Stefan < > >>> stefan.mikloso...@netapp.com<mailto:stefan.mikloso...@netapp.com > ><mailto: > >>> stefan.mikloso...@netapp.com<mailto:stefan.mikloso...@netapp.com>>> > wrote: > >>> Hi Claude, > >>> > >>> all you say makes sense to me. I do not see any discrepancies. It will > be > >>> logged as discussed already. > >>> > >>> The complexity of password validation is partly covered by the library > we > >>> want to use (Passay). It will inform you in a very detailed manner > when it > >>> comes to what violations of a policy there are. We are not going to > invent > >>> a wheel here, fortunately. > >>> > >>> Terminology you used - "observer" - is Guardrail itself (CEP-3). It > will > >>> be the one doing reporting e.g by logging and returning warnings / > errors, > >>> if any, back to user who executed that query. > >>> > >>> The approach we took indeed can also be extended in such a way that it > >>> would be possible to know what was the last time a password was > changed for > >>> some user. This is the direct consequence of us having a table of > previous > >>> password for checking that a user is not reusing them. There is a > timestamp > >>> column specified here (1) if you check the schema of that table > closely so > >>> to answer "when was the password changed lastly" is rather easy to > know - > >>> "select created from system_auth.previous passwords where role = > 'stefan' > >>> limit 1" > >>> > >>> To your requirements: > >>> A simple implementation of the validator that performs series of > >>> configurable tests against the password would probably be sufficient > for > >>> the validation > >>> > >>> Sure, this is configurable, by either implementing a custom validator > if > >>> you find the default one insufficient or configuring the default one > >>> accordingly. > >>> > >>> "A simple implementation of the observer that logs the messages Jeff > >>> suggested would probably be sufficient." > >>> > >>> Yes, no problem with logging from Guardrail directly. > >>> > >>> (1) > >>> > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-24%3A+Password+validation+and+generation#CEP24:Passwordvalidationandgeneration-Validationofanewpasswordagainstpreviouspasswords > >>> > >>> Regards > >>> > >>> ________________________________________ > >>> From: Claude Warren, Jr <claude.war...@aiven.io<mailto: > >>> claude.war...@aiven.io><mailto:claude.war...@aiven.io<mailto: > >>> claude.war...@aiven.io>>> > >>> Sent: Thursday, October 13, 2022 12:50 > >>> To: Miklosovic, Stefan > >>> Cc: dev@cassandra.apache.org<mailto:dev@cassandra.apache.org><mailto: > >>> dev@cassandra.apache.org<mailto:dev@cassandra.apache.org>> > >>> Subject: Re: [Discuss] CEP-24 Password validation and generation > >>> > >>> NetApp Security WARNING: This is an external email. Do not click links > or > >>> open attachments unless you recognize the sender and know the content > is > >>> safe. > >>> > >>> > >>> > >>> I think we might be in violent agreement here. > >>> > >>> The point I was trying to make is that the rules for valid passwords > are > >>> many and varied. I have worked at places where they wanted to know the > >>> time since the last password change, this was used to prevent the rapid > >>> change of password to get back to the original one (I think 5 was the > >>> example earlier). Anyway, the point was, identify the information > >>> necessary from the system to fulfill the rules we think of (so far > this is > >>> the new password, a list of old passwords, and the time of the last > >>> password change) and call a validator plugin passing it the new > password, > >>> list of passwords, date of last change, and an observer instance. > >>> > >>> The validator implementation will verify the instance and report any > >>> issues to the observer and return true/false and potentially a user > message. > >>> > >>> Any logging is attached to the observer, any reporting to grafana or > >>> similar reporting is attached to the observer. > >>> > >>> A simple implementation of the validator that performs series of > >>> configurable tests against the password would probably be sufficient > for > >>> the validation > >>> A simple implementation of the observer that logs the messages Jeff > >>> suggested would probably be sufficient. > >>> > >>> Both would allow much more complex validation and/or reporting as > >>> necessary. > >>> > >>> On Thu, Oct 13, 2022 at 9:26 AM Miklosovic, Stefan < > >>> stefan.mikloso...@netapp.com<mailto:stefan.mikloso...@netapp.com > ><mailto: > >>> stefan.mikloso...@netapp.com<mailto:stefan.mikloso...@netapp.com > >><mailto: > >>> stefan.mikloso...@netapp.com<mailto:stefan.mikloso...@netapp.com > ><mailto: > >>> stefan.mikloso...@netapp.com<mailto:stefan.mikloso...@netapp.com>>>> > >>> wrote: > >>> Hi Claude, > >>> > >>> you said: "I don't know the govt spec. but there is a US govt security > >>> level where you are not allowed to inform the user why the login > failed." > >>> > >>> I do not think this is the case. Nobody is going to inform a user with > >>> existing role in the db why he failed to log in, when it comes to this > CEP > >>> (is not it actually already in place? CQLSH says your username / > password > >>> combo is invalid on login already) This CEP has nothing to do with it. > >>> > >>> What we have in mind, I think, it is more about informing him about the > >>> details when the password he tries to set (upon role creation) or > change > >>> (via role alteration), is not valid, based on the policy. > >>> > >>> I reckon that what Jeff simply wants to see is a log if such change was > >>> successful or not. Lets repeat here what Jeff would like to see: > >>> > >>> "Password changed for user X, complying with policies (reuse, > complexity, > >>> entropy)" > >>> "ERROR: Password rejected due to policy violation (reuse)" > >>> "ERROR: Password rejected due to policy violation (complexity)" > >>> "ERROR: Password rejected due to policy violation (entropy)" > >>> > >>> This is a generalized version of what we already have in place in CEP, > we > >>> have there information like: > >>> > >>> Password must be 10 or more characters in length. Password must > contain 2 > >>> or more uppercase characters. Password matches 3 of 4 character rules, > but > >>> 4 are required. > >>> Password matches one of 5 previous passwords. > >>> Password must be 12 or more characters in length > >>> > >>> Now, I have to admit that the information we provide above, in > contrast of > >>> what Jeff mentioned, is quite verbose. It is questionable whether we > should > >>> be so specific or whether more generalized version is enough. > >>> > >>> Maybe two versions of the logs would be the most appropriate - ours > (more > >>> detailed) would be returned to a user in cqlsh as a warning / error > after > >>> unsuccessful query execution but the messages Jeff mentioned would be > >>> written in system logs via slf4j. So we would be detailed for a user > but > >>> general for auditing purposes. > >>> > >>> Do you think this makes sense to you all? I think this is want you > said, > >>> more or less, in your middle paragraph, just formulated differently. > >>> > >>> I agree with Jackson with the password meter e-mail. After all, if > >>> somebody really wants that to happen, since our solution is pluggable, > >>> people can implement their own password-meter-based solution if they > find > >>> it necessary. > >>> > >>> To fail a password when it is reused (or found among previous n). I am > on > >>> the edge here. I understand what Josh is telling, that we can go just > so > >>> far when it comes to prevent people from doing wrong things, maybe > >>> increasing the password history to 20 last passwords would be enough. > >>> Anyway, I plan to make this historical password verification optional > so it > >>> might be turned on / off on demand. > >>> > >>> Finally, when it comes to password dictionaries. This might be > included in > >>> the CEP but I would keep it out for the very first implementation and > it > >>> can be finished afterwards in some other commit. I do not find it > >>> absolutely necessary to do it right now. > >>> > >>> Regards, > >>> > >>> Stefan > >>> > >>> ________________________________________ > >>> From: Claude Warren, Jr via dev <dev@cassandra.apache.org<mailto: > >>> dev@cassandra.apache.org><mailto:dev@cassandra.apache.org<mailto: > >>> dev@cassandra.apache.org>><mailto:dev@cassandra.apache.org<mailto: > >>> dev@cassandra.apache.org><mailto:dev@cassandra.apache.org<mailto: > >>> dev@cassandra.apache.org>>>> > >>> Sent: Thursday, October 13, 2022 9:44 > >>> To: dev@cassandra.apache.org<mailto:dev@cassandra.apache.org><mailto: > >>> dev@cassandra.apache.org<mailto:dev@cassandra.apache.org>><mailto: > >>> dev@cassandra.apache.org<mailto:dev@cassandra.apache.org><mailto: > >>> dev@cassandra.apache.org<mailto:dev@cassandra.apache.org>>> > >>> Subject: Fwd: [Discuss] CEP-24 Password validation and generation > >>> > >>> NetApp Security WARNING: This is an external email. Do not click links > or > >>> open attachments unless you recognize the sender and know the content > is > >>> safe. > >>> > >>> > >>> > >>> > >>> I managed not to send this to the mailaing list... > >>> > >>> > >>> I don't know the govt spec. but there is a US govt security level where > >>> you are not allowed to inform the user why the login failed. > >>> > >>> > >>> It seems to me that there are 2 intertwined components being discussed. > >>> > >>> 1) A component to perform a user password change capability > >>> > >>> 2) A plugable validation component. > >>> > >>> 3) A pluggable observability component. > >>> > >>> Without a validation component all passwords are valid and provides > user > >>> messages for failures. Validation receives the new password and some > >>> list of old passwords as arguments. Validation returns a structure > >>> comprising the success/failure, the user message, internal result, > >>> internal result message. > >>> > >>> The observability implementations could log the results, send counts to > >>> Grafana, etc. If there is no observer then no results are presented. > >>> > >>> > >>> Alternatively the validation could accept the observability component > as > >>> an argument and pass the internal result and internal result message > >>> directly to the observability component. > >>> > >>> > >>> > >>> > >> > >
