[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344947#comment-17344947
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

{quote}going though this, this still does not have password obfuscation as 
[~vinaykumarcse] mentioned (together with [~eanujwa]).
{quote}
That is correct [~stefan.miklosovic], CASSANDRA-16669 to track the same.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Normal
>  Labels: fqltool
> Fix For: 4.0, 4.0-alpha1
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Miklosovic (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338977#comment-17338977
 ] 

Stefan Miklosovic commented on CASSANDRA-12151:
---

We observe passwords to be present in logs.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Normal
>  Labels: fqltool
> Fix For: 4.0, 4.0-alpha1
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Miklosovic (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338797#comment-17338797
 ] 

Stefan Miklosovic commented on CASSANDRA-12151:
---

Hi,

going though this, this still does not have password obfuscation as 
[~vinaykumarcse] mentioned (together with [~eanujwa]).

Is this something we are ok to go with in 4.0? Double checking this to be sure. 
I think we should obfuscate it.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Normal
>  Labels: fqltool
> Fix For: 4.0, 4.0-alpha1
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Laxmikant Upadhyay (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17030404#comment-17030404
 ] 

Laxmikant Upadhyay commented on CASSANDRA-12151:


Alpha-3 is about to release (voting has been done) . Regarding post alpha 
release you can track by referring to details in mail from Joshua regarding 
current 4.0 release status.

[https://mail.google.com/mail/u/0/#search/label%3Acassandra+alpha/FMfcgxwGDDhhxLTfQtPlFsBhfXKNQSnB]

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Normal
>  Labels: fqltool
> Fix For: 4.0
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Dhawal Mody (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17030119#comment-17030119
 ] 

Dhawal Mody commented on CASSANDRA-12151:
-

[~laxmikant99] and team - when do you think cassandra 4.0 post-alpha will be 
released? 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Normal
>  Labels: fqltool
> Fix For: 4.0
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Laxmikant Upadhyay (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16800383#comment-16800383
 ] 

Laxmikant Upadhyay commented on CASSANDRA-12151:


[~Dhawal Mody] Audit logging is a new feature is Cassandra hence it will not be 
back-ported in 3.x. The release date of 4.0 has not been announced yet.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Normal
>  Labels: fqltool
> Fix For: 4.0
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Dhawal Mody (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799223#comment-16799223
 ] 

Dhawal Mody commented on CASSANDRA-12151:
-

[~jasobrown] [~eperott] - we are using 3.11.3 in production and we could use 
this option too - any chance it will be backported in Cassandra 3.11 ? Also 
when will Cassandra 4.0 release to the public? 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Normal
>  Labels: fqltool
> Fix For: 4.0
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622014#comment-16622014
 ] 

Per Otterström commented on CASSANDRA-12151:


[~mehdic], if you're interested we have created an audit-log plug-in for 
Cassandra 3.0 and 3.11 which we open sourced a while back.

[https://github.com/Ericsson/ecaudit]

There is a feature gap, but we're working to close that as much as possible.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
>  Labels: fqltool
> Fix For: 4.0
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596703#comment-16596703
 ] 

Jason Brown commented on CASSANDRA-12151:
-

[~mehdic] No, as it is a new feature, it goes into trunk/4.0. Further, 3.7 is 
an unsupported version, so even if we did consider a backport, it would have to 
go into the 3.11 series.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
>  Labels: fqltool
> Fix For: 4.0
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Mehdi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589219#comment-16589219
 ] 

Mehdi commented on CASSANDRA-12151:
---

We are using Cassandra 3.7 and we could use this audit feature. Is there a way 
we can backport this patch to 3.7?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.0
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16472417#comment-16472417
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


Pushed a commit at a6bf9c53531 to let users know that they can't expect any 
prepared statement values in the logs. Thanks for your contribution, 
[~vinaykumarcse].

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.0
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471856#comment-16471856
 ] 

Jason Brown commented on CASSANDRA-12151:
-

Ugh, I forgot we did this, having -D jvm args for all the (directory) things. 
I'm not a fan, but apparently this is a thing we do. sgtm.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471316#comment-16471316
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

{quote}
Why do you want this to be inconsistent? What is the argument for allowing 
this one field, of all the audit log config, to have an additional mechanism 
for configuration?
{quote}

Operators set the logging directory different from data directory, consistent 
across the nodes and consolidated location for all logs (debug logs, system 
logs, gc logs and audit logs), hence followed the same convention that is 
followed for configuring other log locations and without extra step to 
configure audit log location. However, considering everyone might not follow 
the same or operator thinks it is additional/ different mechanism for auditlog, 
this location can also be configured using yaml along with other auditlog 
configs if they would like to, they don't necessarily need to follow the 
convention of setting log location via an environment variable, they can choose 
to set it via yaml too. We are consolidating all logs here by managing/ 
controlling the location via an environment variable, so we end up using the 
environment variable. It just avoids yet another location(directory) 
configuration to manage. Hopefully, this gives the background.



> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468713#comment-16468713
 ] 

Jason Brown commented on CASSANDRA-12151:
-

bq. Idea is to be inconsistent with rest of the project's log configuration 
process by setting them via -D jvm arg,

*Why* do you want this to be inconsistent? What is the argument for allowing 
this one field, of all the audit log config, to have an additional mechanism 
for configuration? 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468526#comment-16468526
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

Hi Jason,

Yes, we can set audit_logs_dir either from -D jvm arg or from 
{{cassandra.yaml}} - {{audit_logging_options::audit_logs_dir}}. Idea is to be 
inconsistent with rest of the project's log configuration process by setting 
them via   {{-D jvm arg}}, hence naming is kept inconsistent with logs - 
{{cassandra.logdir.audit}}. I have also updated the documentation for the same.

{quote}Operators cannot, in an obvious manner, adjust the following 
AuditLogOptions options: 
audit_logs_dir/block/max_queue_weight/max_log_size/roll_cycle. They are 
basically hidden yaml options: you can set them, but you have to know they 
exist. I understand these are more advanced options, but we should document 
them either in the yaml or the doc page.
{quote}
Yes, they intentionally kept there for advanced configurations. Agree with you 
on documenting them. Updated the documentation for this.

 
||*Branch*||*PR*||*uTest*||
|[trunk_CASSANDRA-12151|https://github.com/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151]|[PR
 for 
trunk|https://github.com/vinaykumarchella/cassandra/pull/2/commits]|[!https://circleci.com/gh/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151.svg?style=svg!|https://circleci.com/gh/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151]|

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468457#comment-16468457
 ] 

Jason Brown commented on CASSANDRA-12151:
-

On the whole, we're almost there. I've done a last-pass review, fixed a few 
last dangling items (see below), and pushed a commit up to my branch. Just a 
few last questions:
 - Do we need the -D jvm arg to set {{AuditLogOptions#audit_logs_dir}}? This 
kinda comes out of nowhere and matches nothing else in the audit log config.
 - Operators cannot, in an obvious manner, adjust the following 
{{AuditLogOptions}} options: 
audit_logs_dir/block/max_queue_weight/max_log_size/roll_cycle. They are 
basically hidden yaml options: you can set them, but you have to know they 
exist. I understand these are more advanced options, but we should document 
them either in the yaml or the doc page.

what i've done in latest commit:
 - cleaned up yaml comments, and point users to the audit_log docs; else, it's 
gonna real messy real quick
 - trivial clean ups: NEWS.txt, formatting, white space, comments, removed 
unused methods, fixed test method names in {{AuditLoggerTest}}
 - reworked {{AuditLogFilter#create()}} to eliminate garbage collection of all 
the {{HashSet}} s, which then we pass to {{ImmutableSet#of}}
 - Added path checking in {{AuditLogManager}} to ensure when enabling either 
FQL or audit logging, the path doesn't conflict with the other. This was easy 
enough to do for the {{BinLog}}-related {{IAuditLogger}}, but I have no idea 
how to get the directory or file from logback (when using {{FileAuditLogger}}). 
Since that's a 'spare' implementation, I've punted for now.
 - clarified {{AuditLogEntry#timestamp}} to allow {{Builder}} users to 
explicitly set the value. This allows the FQL paths to be more in tune with 
their original code (where it would get {{System#currentTimeMillis()}} instead 
of using {{queryStartNanoTime}}).

Tests running.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468212#comment-16468212
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

Thanks, [~jasobrown] for cleanup and fixing [~eperott] comments.
{quote}When using logback as backend, would it make sense to mark audit records 
with a specific appender name such as "AUDIT" rather than 
"FileAuditLoggerAppender". That way we can easily tell regular log messages 
from audit log messages.
{quote}
Yes, certainly. However, AuditLog feature does not ship with appender 
configurations. I see that "FileAuditLoggerAppender" is being referenced in the 
documentation, have updated and pushed.
{quote}On a similar topic, rather than creating the AuditLogEntryCategory type, 
the mapping in AuditLogEntryType and the kespace/scope of (I)AuditLogContext, 
would it make sense to use the existing Permission type (SELECT, MODIFY, 
CREATE...) and IResource (Data, Role, Function...). We could create a new 
resource type to represent Connections (like connection/native, 
connection/thrift, connection/jmx) which could be used for managing white-lists 
for authentication.
{quote}
I don't think it is a good idea to piggyback on Permission type and IResource 
to get the AuditLogType, that makes those 2 features tightly bound and it seems 
like a hack rather than cleaner implementation. Also, binding them tightly 
makes future extensions on those features tough to manage and we end up 
separating eventually. So not sure, if that is a good idea to piggyback on 2 
other different features to get the AuditLog needs. 
\\
{quote}Sure, I understand we seek to close this ticket. I'm just a bit 
concerned with the timing. If this ticket is merged as is and we take a cut for 
4.0, then I assume we will have to stick to this way of configure audit logs 
for some time.
{quote}
CQL grammar for managing audit log configurations is an interesting idea, 
considering the changes needed at this point, hierarchical and composite 
requirements that come with it, I agree with @Jason on exploring as a followup. 
Please feel free to create followup JIRA on this.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468087#comment-16468087
 ] 

Jason Brown commented on CASSANDRA-12151:
-

pushed a few trivial clean up patches (includes some of the things [~eperott] 
reported), and running dtests and utests:

||12151||
|[branch|https://github.com/jasobrown/cassandra/tree/trunk_CASSANDRA-12151]|
|[utests & 
dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/trunk_CASSANDRA-12151]|
||

bq. should we create some dtests for this as well?

I thought about that,as well. However, as this feature is not distributed, we 
can test all the functionality via unit tests. In fact, [~vinaykumarcse] 
already has a bunch of unit tests already in this patch. I'm going to give this 
patch one last major review, and will see if we need more tests.


> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467079#comment-16467079
 ] 

Per Otterström commented on CASSANDRA-12151:


bq. This is an intersting idea. However, I'm reticent to introduce CQL grammar 
changes this late into this ticket. As a followup, I think this could be worth 
exploring.

Sure, I understand we seek to close this ticket. I'm just a bit concerned with 
the timing. If this ticket is merged as is and we take a cut for 4.0, then I 
assume we will have to stick to this way of configure audit logs for some time.

bq. While there's some conceptual overlap, I don't think it's a good idea to 
try to force the concepts of 'audit logging' and 'auth' onto one another here.

I believe it depends on the use case for doing audits. In some cases it would 
make perfect sense to have these two concepts in sync. E.g. I want to grant a 
user to have SELECT permission on a specific table. Then I would grant the same 
user to have NOAUDIT for SELECT on the very same table. Shoudl the user try to 
read somethign else, or write to that table, and audit entry would be created. 
In a case like this it is very helpful for the admin to use the same concepts 
for these two "permissions".

To verify accuracy, should we create some dtests for this as well?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466880#comment-16466880
 ] 

Jason Brown commented on CASSANDRA-12151:
-

bq. Would it not make sense to maintain audit whitelist permissions next to all 
other permissions in Cassandra?

This is an intersting idea. However, I'm reticent to introduce CQL grammar 
changes this late into this ticket. As a followup, I think this could be worth 
exploring.

bq. would it make sense to use the existing Permission type (SELECT, MODIFY, 
CREATE...) and IResource (Data, Role, Function...)?

While there's some conceptual overlap, I don't think it's a good idea to try to 
force the concepts of 'audit logging' and 'auth' onto one another here. 

[~eperott]'s other comments I agree with.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466440#comment-16466440
 ] 

Per Otterström commented on CASSANDRA-12151:


I've been following this ticket with some interest. Spent some time to review 
the patch, didn't try things out yet.

When using logback as backend, would it make sense to mark audit records with a 
specific appender name such as "AUDIT" rather than "FileAuditLoggerAppender". 
That way we can easily tell regular log messages from audit log messages.

Having audit include/exclude filters in the yaml file seem a bit impractical 
when you want to update it. I believe there are quite a few similarities with 
the permissions-on-resources model in Cassandra. Would it not make sense to 
maintain audit whitelist permissions next to all other permissions in 
Cassandra? Example "GRANT NOAUDIT ON KEYSPACE myks TO myuser". I've been 
experimenting with a similar approach (storing white-lists in the role option 
field) in our internal audit log plugin and it looks promising.

On a similar topic, rather than creating the AuditLogEntryCategory type, the 
mapping in AuditLogEntryType and the kespace/scope of (I)AuditLogContext, would 
it make sense to use the existing Permission type (SELECT, MODIFY, CREATE...) 
and IResource (Data, Role, Function...). We could create a new resource type to 
represent Connections (like connection/native, connection/thrift, 
connection/jmx) which could be used for managing white-lists for authentication.

Why always mute audit logs for the system keyspaces? IMO, we should make less 
assumptions on the use cases and let this be configurable like all other 
keyspaces.

The AuditLogManager contain a few redundant null checks and 
isAuditintEnabled()-checks.

The BinLogAuditLogger declare "LoggerFactory.getLogger(FullQueryLogger.class)"

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465523#comment-16465523
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

Hi [~jasobrown],

Thanks for the changes. Yes, extracting `BinLogAuditLogger` is a cleaner way to 
do this.

Updated changeset with minor fixes to pass tests, moved the {{category}} in 
AuditLogEntryType to enum.

Filtered unintended messages to FQL.
{quote}Why did you remove the reloadFilters functionality?
{quote}
Updated documentation regarding "ReloadFilters" functionality.

Yes, circleci utests are green, trying to get paid account to run dtests. 

\\

||*Branch*||*PR*||*uTest*||
|[trunk_CASSANDRA-12151|https://github.com/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151]|[PR
 for 
trunk|https://github.com/vinaykumarchella/cassandra/pull/2/commits]|[!https://circleci.com/gh/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151.svg?style=svg!|https://circleci.com/gh/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151]|

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458567#comment-16458567
 ] 

Jason Brown commented on CASSANDRA-12151:
-

After rereading this patch, I've merged the FQL and AuditLog code paths as they 
do the same thing from the outside: record user-initiated events. To that end, 
I've moved the management of {{FullQueryLogger}} under {{AuditLogManager}}, 
thus making {{AuditLogManager}} the hub for logging these actions. Further, 
instead of {{BinAuditLogger}} inherit from {{FullQueryLogger}} (which never 
felt quite right), I've pulled the shared componentry into a base class 
({{BinLogAuditLogger}}), and {{BinAuditLogger}} / {{FullQueryLogger}} inherit 
from that.

I refactored {{AuditLogEntry}} to use the builder pattern for construction as 
all the {{static #getLogEntry()}} with the {{set*()}} methods was rather 
confusing. Further, we're using the builder pattern elsewhere in the code. 
{{#toString()}} is typically used for debugging, so I'd prefer a different 
method name - I used {{#getLogString()}}. If you want the debug dump to have 
the same output, you can have {{toString()}} call that renamed function.

We need an extra filter in {{AuditLogManager#log(AuditLogEntry)}} to make sure 
we don't send unintended messages to FQL. Can you figure out what that is? I'm 
not sure if we can simply use the entry's {{AuditLogEntryType}} or the type's 
category.

Why did you remove the reloadFilters functionality? 

I feel like the {{category}} in AuditLogEntryType should be an enum, as well. 
wdyt?

I haven't looked at the circleci results yet for utests/dtests, but let's make 
sure this is right path forward first.


> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456004#comment-16456004
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

Hi [~jasobrown],

Implemented {{nodetool enableauditlog}} and {{disableauditlog}} commands, 
updated documentation for the same. Fixed whitespaces in rest of the changeset, 
also fixed the tests. 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446557#comment-16446557
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

Thanks, [~jasobrown], for reviewing it and making changes, sure I will take 
care of tests. 

{quote}
 I've asked Vinay to add some nodetool commands for enabling/disabling auditlog 
as well (or, at least, to seriously think about it).
{quote}
I agree, will add nodetool commands for enabling/disabling auditlog to help 
improve the usability at node level. 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446271#comment-16446271
 ] 

Jason Brown commented on CASSANDRA-12151:
-

Thanks to [~djoshi3] and [~spo...@gmail.com] for doing the first passes of 
review here, I'll take it the rest of the way.

[~vinaykumarcse] and I spoke offline about some things, and on the whole we are 
in the right direction. As I started reviewing the patch, the whitespace 
inconsistencies nagged at me, so I went ahead and fixed those, as well as some 
random code comments. Further, I made {{AuditLogFilters}} atomic and final, and 
no longer a singleton; a volatile reference is maintained in 
{{AuditLogManager}}. I also moved getLogEntry methods from {{AuditLogManager}} 
to {{AuditLogEntry}} as they seemed more apropos there. Code available 
[here|https://github.com/jasobrown/cassandra/tree/trunk_CASSANDRA-12151].
 


> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16443377#comment-16443377
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

Hi [~spo...@gmail.com],

Sorry, was away from work for last few days. If we are filtering users, to log 
any of these failures/ failed attempts, we can include {{system}} user in 
include filters.
{quote}question regarding how we should go on with prepared statements. Cause I 
can't see how we can get away with not logging these, if we want to address any 
compliance or security use cases.
{quote}
As you also suggested on this thread previously, either filtering based on log 
entry types or ParameterizedClass for logger should address this concern. I 
will open followup JIRAs as soon as we close this PR to keep the changeset 
moderate at this moment. 

Sounds like a plan?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16435587#comment-16435587
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


I must have missed that log statement when filtering by user. Its a bit strange 
that failed login attempts don't get logged, if you filter by a particular 
user. But that's probably more a minor issue compared to the the question 
regarding how we should go on with prepared statements. Cause I can't see how 
we can get away with not logging these, if we want to address any compliance or 
security use cases.

Any ideas?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16435109#comment-16435109
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

Hi [~spo...@gmail.com],

Updated the PR with comments provided. {{SYSTEM_USER}} is used in our internal 
backport for thrift code path, I removed it on trunk. I intended to use 
{{IAuditLogger.error()}} initially in error scenarios, but I ended up using 
{{IAuditLogger.log()}}, hence removed it, if any use case comes up, we can add 
it back. Fixed and added more test cases for {{AuditLogFilter.isFiltered()}} to 
address the scenarios you requested.
{quote}Username will not be provided for failed authentications
{quote}
Below is the sample audit statement on failed authentications, username is 
provided as part of the audit log. Are you referring to something else?


{code:java}
INFO [Native-Transport-Requests-1] 2018-04-12 00:37:20,284 
FileAuditLogger.java:35 - 
user:system|host:127.0.0.1:7000|source:/127.0.0.1|port:64481|timestamp:1523518640284|type:LOGIN_ERROR|category:AUTH|operation:LOGIN
 FAILURE; Provided username vchella and/or password are incorrect
{code}


> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431899#comment-16431899
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


The native transport integration for diagnostic events is basically just 
pushing events one by one over the control connection to the subscribed client. 
It's not really designed as a generic, fully scalable server-side data push 
solution. There are also no delivery guarantees, as it's really just supposed 
for debugging and analysis and not for implementing any control instances on 
top if it. The use case I have in mind is to have 1-2 clients subscribing to 
some kind of event, either ad-hoc or constantly running in the background. But 
I don't really see any use case for having a large fanout of e.g. compaction 
events. For that, the solution proposed in CASSANDRA-13459 should be 
sufficient. But we should probably discuss further details there, as it's 
slightly off-topic for this ticket.
{quote}I think specifying a shell script is probably OK although if someone 
specifies the script we should run it immediately once Chronicle rolls the 
file. Also if the script is specified we probably shouldn't delete artifacts.
{quote}
I've created a new ticket (CASSANDRA-14373) for this, as it's not strictly an 
auditing feature.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431296#comment-16431296
 ] 

Ariel Weisberg commented on CASSANDRA-12151:


Adding a way for clients to subscribe to events is great, but there are couple 
of questions that brings up. How does backpressure work? We have to have a 
convincing story for what happens so that slow or unavailable clients can't 
consume unbounded memory.

The other nice to have is what happens if a client disconnects and reconnects? 
Can it get the events that it missed? This ties into being able to consume on 
disk artifacts like the BinLog. Do the diagnostic events have enough 
information in them that you can tell where in the stream of events for a 
particular node you are? Just enough to facilitate at least once delivery with 
duplicates dropped.

Are diagnostic events always really going to be used for diagnostic purposes? 
I'm just questioning the name. Maybe it should just be SUBSCRIBE and maybe only 
some of the more problematic things should be locked behind config options. And 
then we have to think about what happens with large numbers of subscribers 
although for V1 it could just be a sharp edge.

In terms of using the BinLog as more of a store of record. We need to figure 
out how crashes and restarts are going to be handled. I don't recall exactly 
what happens, but I suspect that chronicle is just going to leave behind the 
old files and start a new one for appending so we need to come back and process 
the old files on startup. I think specifying a shell script is probably OK 
although if someone specifies the script we should run it immediately once 
Chronicle rolls the file. Also if the script is specified we probably shouldn't 
delete artifacts.

As these things start to get more complex how are we going to change the YAML 
and fqltool to be the ri

Also this entire change set is starting to get really large.
* Generating whole new classes of events, multiple targets for logging
* Modifications to the existing BinLog to add new capabilities like using it as 
a store of record, running a users provided script
* Wire protocol changes and a new way to subscribe to have the server push stuff

Would be nice to discuss and review these in more isolation. GL Dinesh!

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Nate McCall (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431160#comment-16431160
 ] 

Nate McCall commented on CASSANDRA-12151:
-

{quote}I've now managed to update CASSANDRA-13668 by implementing IAuditLogger 
and expose audit events as diagnostic events via native transport to subscribed 
clients. Went pretty much as expected and seems to work fine.
{quote}
[~spo...@gmail.com] That's a fantastic idea! Thank you. 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430448#comment-16430448
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


I've now managed to update CASSANDRA-13668 by implementing IAuditLogger and 
expose audit events as diagnostic events via native transport to subscribed 
clients. Went pretty much as expected and seems to work fine.

Smaller issues I've came across:
 * AuditLogUtil doesn't look very useful. DEFAULT_SOURCE is only used once, 
SYSTEM_USER not at all.
 * Do we really want to keep IAuditLogger.error()? Please provide a description 
of the log/error semantics in context with audit logging for possible 
subclasses, or get rid of error().
 * AuditLogFilter.isFiltered(): proposed logic will ignore includeSet if 
excludeSet is provided (may not make sense to do so, but not strictly forbidden 
by cassandra.yaml either), e.g. exclude(A), include(A,B,C) should only have B,C 
pass

There are also a couple of limitations:
 * Username will not be provided for failed authentications
 * Bound values will not get logged for prepared statements

I haven't found a quick way to work around these, but being able to avoid the 
audit log by using prepared statements, is something we have to address. It's 
probably not going to be that much of an issue for my use case logging ad-hoc 
commands for regular users, once we have CASSANDRA-8303 and can disable 
prepared statement for them. But for logging all activity for application 
users, I don't know.

 
 [~laxmikant99]
{quote}Can we have a configurable property like exitOnAuditFailure ? This 
fulfills requirement of strict auditing .. I mean in case auditing fails for a 
db operation, then the operation should not get executed.
{quote}
Any logging to the BinLogger should block by default. But it doesn't exit the 
JVM.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Laxmikant Upadhyay (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16428102#comment-16428102
 ] 

Laxmikant Upadhyay commented on CASSANDRA-12151:


Can we have a configurable property like exitOnAuditFailure ? This fulfills 
requirement of strict auditing .. I mean in case auditing fails db operation 
should not get executed. 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427318#comment-16427318
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

Thank you [~spo...@gmail.com]

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426915#comment-16426915
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


I was just giving the BinLogger a try and it looks like the way the bin log 
files are created, should make it possible to archive them before they get 
deleted. The most simple way would probably to run a rsync job for pulling the 
latest chronicle bin logs from the nodes. The timestamp based naming of the 
files should make this trivial.

Another option would be to allow the user to configure an archival script that 
will be executed in {{BinLog.onReleased(cycle, file)}} for every deleted bin 
log, just as we do in {{CommitLogArchiver}}. WDYT [~aweisberg]?

I'm going to write a custom implementation tomorrow and will get back with some 
more feedback.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422918#comment-16422918
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

[~spo...@gmail.com],
{quote}We should not provide solutions that we know will have significant 
performance issues on busy production systems. I don’t mind keeping the 
FileAuditLogger class, as it’s really trivial. But please remove references to 
it from cassandra.yaml and the corresponding entries in logback.xml (don’t like 
to have an empty audit.log file on all nodes either). Let’s nudge users to use 
to BinAuditLogger right away as the recommended solution.
{quote}
Yes, I took care of this. Removed FileAuditLogger references from 
cassandra.yaml and also code defaults
{quote}If you don’t think adding an option like include_auditlog_types should 
be necessary, that’s fine. But then let me use my own implementation for 
filtering logs, if my requirements are different. This means that I’d have to 
be able to specify additional parameters (e.g. custom filtering options) along 
with the class name for my implementation. So I'd suggest to use 
ParameterizedClass for the logger in cassandra.yaml to make that easily 
possible.

Looking at IAuditLogger and thinking about how to filter log events makes me a 
bit worried about the design in general there. We keep generating AuditLogEntry 
instances and create unnecessary garbage, even if we’re only interested in some 
specific entry types. Maybe we should move filtering either into the 
IAuditLogger implementation or make it possible to use a custom AuditLogFilter 
as well (IAuditLogger.getLogFilter() ?).
{quote}
Agreed, just to be clear from my previous reply - I am not against adding a new 
filter (include_auditlog_types) or extending the interface, I would like to get 
the initial simple version out and iterate on it rather than increasing the 
changeset to be too heavy.
{quote}Just looking at AuditLogFilter makes me also think that the isFiltered 
logic should be reconsidered, as null values will cause entries to always pass, 
even if the include set is not empty.
{quote}
Sure, considered null entries also. Now, {{isFiltered}} logic takes care of 
null values when input set is not empty. 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Dinesh Joshi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16420963#comment-16420963
 ] 

Dinesh Joshi commented on CASSANDRA-12151:
--

[~spo...@gmail.com] The {{BinLog}} does what I suggested. It deletes old files 
once you exceed the allocated storage quota. From my reading, it seems that it 
segments logs on the basis of time (defaults to hourly).

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16420389#comment-16420389
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


We should not provide solutions that we know will have significant performance 
issues on busy production systems. I don’t mind keeping the FileAuditLogger 
class, as it’s really trivial. But please remove references to it from 
cassandra.yaml and the corresponding entries in logback.xml (don’t like to have 
an empty audit.log file on all nodes either). Let’s nudge users to use to 
BinAuditLogger right away as the recommended solution.

If you don’t think adding an option like include_auditlog_types should be 
necessary, that’s fine. But then let me use my own implementation for filtering 
logs, if my requirements are different. This means that I’d have to be able to 
specify additional parameters (e.g. custom filtering options) along with the 
class name for my implementation. So I'd suggest to use ParameterizedClass for 
the logger in cassandra.yaml to make that easily possible.

Looking at IAuditLogger and thinking about how to filter log events makes me a 
bit worried about the design in general there. We keep generating AuditLogEntry 
instances and create unnecessary garbage, even if we’re only interested in some 
specific entry types. Maybe we should move filtering either into the 
IAuditLogger implementation or make it possible to use a custom AuditLogFilter 
as well (IAuditLogger.getLogFilter() ?).

Just looking at AuditLogFilter makes me also think that the isFiltered logic 
should be reconsidered, as null values will cause entries to always pass, even 
if the include set is not empty.
{quote}How do you suggest we approach this problem? We cannot keep the audit 
log files around indefinitely. Perhaps we can specify a disk quota for audit 
log files in the configuration? We can expose this setting in cassandra.yaml?
{quote}
How does this work with BinLogger? I haven't looked at that part in detail yet. 
Does it overwrite old entries automatically? How would you suggest users should 
archive logs from there?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Dinesh Joshi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16420323#comment-16420323
 ] 

Dinesh Joshi commented on CASSANDRA-12151:
--

Hey [~vinaykumarcse], thank you for making the code changes. Minor nit on the 
whitespacing but other than that LGTM.


Hey [~spo...@gmail.com], 
{quote}Rotating out log files by simply deleting them is probably also not what 
you'd expect from a auditing solution.
{quote}
How do you suggest we approach this problem? We cannot keep the audit log files 
around indefinitely. Perhaps we can specify a disk quota for audit log files in 
the configuration? We can expose this setting in cassandra.yaml?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419652#comment-16419652
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

[~spo...@gmail.com] ,
{quote}I'd not use the logback logger for query logging in production. As shown 
recently in CASSANDRA-14318, the performance impact is significant and your 
benchmark results show that as well. Rotating out log files by simply deleting 
them is probably also not what you'd expect from an auditing solution.
{quote}
Agreed, yes, BinAuditLogger is the default for obvious reasons based on my perf 
numbers, but if any users are willing to use log back based AuditLogger thereby 
willing to compromise on performance while benefiting on easy readability, they 
can by changing the yaml configuration.
{quote}NIT: check logger.isEnabled before toString in FileAuditLogger
{quote}
FileAuditLogger::log method is called only when AuditLogger is enabled, do you 
think the redundant check is needed inside the IAuditLogger implementations as 
well?
{quote}But maybe we can keep the logback logger for some simple auth related 
logging and make it also useful for users who would not enable auditing in 
first place. How about enabling the FileAuditLogger by default and let it 
create an auth.log, which would log all failed login attempts? Maybe add a 
comment on how to enable successful attempts as well.
{quote}
Logging just auth failure attempts can be achieved by either the design idea 
that you brought up or new feature to filter {{include_auditlog_types}}. But 
not sure, if everyone would like or need auth failures attempts by default.

 

I also see good design ideas in terms of {{multiple implementations in 
parallel}} and feature requests {{include_auditlog_types}}, however, do you 
agree with me that it is a {{nice to have}} but not {{must to have}} feature 
considering the scope of the first iteration, to keep the initial 
implementation simple, iterate faster and changeset smaller? If you do, I can 
create followup JIRAs for the ideas that you proposed.

 

Hi [~chovatia.jayd...@gmail.com] .
{quote}Currently if audit is enabled then it will log all types of statements 
DDL|DML|DCL, people will have different requirements for logging (say only log 
DCL OR DCL + DDL etc.) given DML would generate huge volume of data, so I think 
we need to give better control to users as to what they want to log
{quote}
Correct, the current design enables users to have control of what to log since 
the use cases could be different (eg., DCL or DCL+DDL etc.,) using yaml 
configurations
{quote}probably at what frequency as well.
{quote}
I don't know as a database when it promises for audit logging for compliance if 
it does based on some probability. However, if you have such one of use case 
(e.g., audit logging for debugging needs instead of compliance needs), you can 
implement {{IAuditLogger}} or extend one of existing loggers to have 
{{FrequencyBasedAuditLogger}} which logs based on some probability. 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jaydeepkumar Chovatia (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419564#comment-16419564
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-12151:
---

Hi [~vinaykumarcse]

Currently if audit is enabled then it will log all types of statements 
{{DDL|DML|DCL}}, people will have different requirements for logging (say only 
log {{DCL }} OR  {{DCL + DDL}} etc.) given {{DML}} would generate huge volume 
of data, so I think we need to give better control to users as to what they 
want to log and probably at what frequency as well.

Jaydeep

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16418918#comment-16418918
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


I'd not use the logback logger for query logging in production. As shown 
recently in CASSANDRA-14318, the performance impact is significant and your 
benchmark results show that as well. Rotating out log files by simply deleting 
them is probably also not what you'd expect from a auditing solution. But maybe 
we can keep the logback logger for some simple auth related logging and make it 
also useful for users who would not enable auditing in first place. How about 
enabling the {{FileAuditLogger}} by default and let it create an {{auth.log}}, 
which would log all failed login attempts? Maybe add a comment on how to enable 
successful attempts as well.

Looks like this won't be possible with the current {{included_categories}} 
filtering, which is based on the DDL, DML, .. categories. I'd suggest to make 
the filter work for both the category and actual AuditLogEntryType (SELECT, 
UPDATE, DELETE,..).

Full query logging should be done using the BinLogger or a custom 
implementation. It would be nice to be able to use mutliple implementations in 
parallel, in case we want to enable FileAuditLogger by default.

NIT: check logger.isEnabled before toString in FileAuditLogger

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416952#comment-16416952
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

Hi [~djoshi3] [~jolynch],

Thanks for the review, implemented review suggestions provided and pushed those 
changes in PR. 

\\

||[branch|https://github.com/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151]||
|[PR for trunk|https://github.com/vinaykumarchella/cassandra/pull/2/commits]|
|[circleci|https://circleci.com/gh/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151]|

\\

{quote}Initialization can be simplified like this - excludedUsers = 
ImmutableSet.of(DatabaseDescriptor.getAuditLoggingOptions().excluded_users.split(","));
{quote}
{{loadFilters()}} are designed so that filter values can be loaded during the 
startup as well as JMX call to reload the filters
{quote}AuditLogFilter:: isFiltered - You can return directly at line 137 and 
get rid of isExcluded variable You can get rid of isIncluded as well and return 
at line 148. Let the control fall through to line 153 and it will return false. 
Furthermore, If you check the sets during initialization and ensure that 
they're mutually exclusive, your logic simplifies to if 
(includedSet.contains(object)) return false; else if 
(excludedSet.contains(object)}) return true;
{quote}

I went with the suggestions provided in former part to simplify the logic in 
{{isFiltered}} rather than mutual exclusion at initialization part.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Dinesh Joshi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16412272#comment-16412272
 ] 

Dinesh Joshi commented on CASSANDRA-12151:
--

Hey [~vinaykumarcse], 
 # {{AuditLogEntry.host}} - this should be final. You should use 
{{FBUtilities::getBroadcastAddressAndPort}} instead for completeness sake.
 # {{AuditLogEntry}} - variables {{source, srcPort, user, type}} could be made 
final
 # {{AuditLogEntry}} - line 47 Do you need to create a new Date object for each 
entry? How about using {{System.currentTimeMillis()}} instead?
 # {{AuditLogFilter.logger}} - this needs to be private
 # {{AuditLogFilter::loadFilters}} - consider using {{private volatile 
ImmutableSet}} instead of {{AtomicReferences}} to mutable sets. The class 
invariant should be that once constructed none of the sets should be null.
 # Initialization can be simplified like this - {{excludedUsers = 
ImmutableSet.of(DatabaseDescriptor.getAuditLoggingOptions().excluded_users.split(","));}}
 # Make {{AuditLogFilter}} constructor private
 # {{AuditLogFilter}}'s constructor - Do not call a public overridable method 
{{loadFilters}} in a constructor. See: 
[https://wiki.sei.cmu.edu/confluence/display/java/MET05-J.+Ensure+that+constructors+do+not+call+overridable+methods]
 # {{AuditLogFilter:: isFiltered}} - You can return directly at line 137 and 
get rid of {{isExcluded}} variable You can get rid of {{isIncluded}} as well 
and return at line 148. Let the control fall through to line 153 and it will 
return false. Furthermore, If you check the sets during initialization and 
ensure that they're mutually exclusive, your logic simplifies to {{if 
(includedSet.contains(object)) return false; else if 
(excludedSet.contains(object)}) return true;}}
 # {{AuditLogManager.logger}} - this needs to be private
 # {{AuditLogManager::logError}} - get rid of this method as it is not used 
anywhere
 # {{AuditLogUtil.DEFAULT_SOURCE}} - this should be {{0.0.0.0/0}} for IPv4 and 
{{::/0}} for IPv6. This are used to represent "unknown or unspecified" 
addresses.
 # {{IAuditLogContext.AuditLogContext}} - please make all the class variables 
final.
 # {{CreateTypeStatement, AlterTypeStatement, DropTypeStatement}} all have 
{{getStringTypeName}}. Since it's not being used anywhere except inside the 
class, you can get rid of it.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Joseph Lynch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410693#comment-16410693
 ] 

Joseph Lynch commented on CASSANDRA-12151:
--

I took a look at the patch and it looks good to me. Comments left on the PR, 
but I like that the interface is pluggable and future implementations can do 
what they like (percentage based, remote/network logs, different output 
formats, etc).

Should we do some dtests and ccm multi node testing locally?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403146#comment-16403146
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

[~djoshi3]

Implemented all the code reviews comments provided in this JIRA thread as well 
as Github PR. Except one below

{quote}
Consider refactoring your code to add a netty handler that invokes an auditing 
interface. The advantage of this approach would be that, when audit logging is 
disabled, you can take this handler out of the netty pipeline. This way there 
is zero performance impact when the audit is disabled. You can define a 
IAuditLogger interface that has sufficient contextual information to log all 
queries. This will help make the audit logging implementation pluggable.
{quote}

I am creating a follow-up JIRA to discuss the more details on this.

On a high level, this changeset includes following changes

# Extended and reused FullQueryLogger in logging audit events
# Combined and Simplified FQL and AuditLog entry points in the request path
# AuditLogEntryType::allStatementsMap - Instead of creating an explicit map of 
statements, type of statement is being added to the actual class itself. This 
makes new statements easy to manage
# AuditLogFilter::loadFilters - Simplified filter loading logic, easy to add 
new filters if needed
# CQL query auditing can now be filtered on user level.
# Added documentation in the doc folder
# Removed ConsistencyLevel in logging details
# Added more test cases
# Implemented code review comments provided in this JIRA as well as Github PR

\\

||[branch|https://github.com/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151]||
|[PR for trunk|https://github.com/vinaykumarchella/cassandra/pull/2/commits]|
|[circleci|https://circleci.com/gh/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151]|

\\

We ran cassandra stress test with this patch and attached stress test results. 
Here is the high level summary

Note: Below tests are run on AWS i2.2xl instance.
\\
{{cass-stress cmd: write n=100 -rate threads=10 -graph 
file=CASSANDRA_12151-benchmark.html}}
||WRITE - Test Suite||Throughput||Latency Mean||Latency 95th||Latency 99th||
|trunk|13,925 op/s|0.7 ms|1.1 ms|1.7 ms|
|CASSANDRA-12151:Disabled AuditLog|14,422 op/s|0.7 ms|1.1 ms|1.6 ms|
|CASSANDRA-12151:FQL based AuditLog with Sync|13,372 op/s|0.7 ms|1.2 ms|1.7 ms|
|CASSANDRA-12151:FQL based AuditLog with Async|12,908 op/s|0.8 ms|1.2 ms|1.9 ms|
|CASSANDRA-12151:SLF4j based AuditLog|10,520 op/s|0.9 ms|1.6 ms|2.4 ms|
\\
{{cass-stress cmd: mixed n=100 -rate threads=10 -graph 
file=CASSANDRA_12151-benchmark.html}}
||MIXED - Test Suite||Throughput||Latency Mean||Latency 95th||Latency 99th||
|trunk|12,939 op/s [READ: 6,494 op/s, WRITE: 6,444 op/s]|0.7 ms [READ: 0.8 ms, 
WRITE: 0.7 ms]|1.2 ms [READ: 1.3 ms, WRITE: 1.2 ms]|1.7 ms [READ: 1.8 ms, 
WRITE: 1.7 ms]|
|CASSANDRA-12151: Disabled AuditLog|12,840 op/s [READ: 6,421 op/s, WRITE: 6,419 
op/s]|0.8 ms [READ: 0.8 ms, WRITE: 0.7 ms]|1.2 ms [READ: 1.3 ms, WRITE: 1.2 
ms]|1.8 ms [READ: 1.8 ms, WRITE: 1.7 ms]|
|CASSANDRA-12151: FQL based AuditLog with Sync|10,932 op/s [READ: 5,452 op/s, 
WRITE: 5,481 op/s]|0.9 ms [READ: 1.0 ms, WRITE: 0.8 ms]|1.5 ms [READ: 1.6 ms, 
WRITE: 1.4 ms]|2.3 ms [READ: 2.4 ms, WRITE: 2.1 ms]|
|CASSANDRA-12151: FQL based AuditLog with Async|11,146 op/s [READ: 5,565 op/s, 
WRITE: 5,581 op/s]|0.9 ms [READ: 0.9 ms, WRITE: 0.8 ms]|1.5 ms [READ: 1.5 ms, 
WRITE: 1.4 ms]|2.2 ms [READ: 2.2 ms, WRITE: 2.1 ms]|
|CASSANDRA-12151: SLF4j based AuditLog|9,764 op/s [READ: 4,883 op/s, WRITE: 
4,882 op/s]|1.0 ms [READ: 1.0 ms, WRITE: 1.0 ms]|1.7 ms [READ: 1.7 ms, WRITE: 
1.6 ms]|2.5 ms [READ: 2.6 ms, WRITE: 2.4 ms]|

\\

Looking at the results, with AuditLog feature disabled, there appears to be no 
measurable difference in performance. FQL appears to have little or no overhead 
in WRITE only workloads, and a minor overhead in MIXED workload. SLF4J appears 
to have minor regressions in both workloads (with mixed slightly worse).

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that 

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389656#comment-16389656
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


To further summarize, let me quote [~jolynch]'s list of collected use cases 
once more and add another use case highlighted with bold letters.
{quote} # Logging for security
 # Logging for business accounting compliance (e.g. SOX).
 # Logging for monetary transaction compliance (e.g. PCI).
 # *Logging for data protection compliance (GDPR)*.
 # Logging for replay later (e.g. for correctness testing)
 # Logging for debugging{quote}
Why is user auditing relevant to comply with GDPR regulations? [Art. 
30|https://gdpr-info.eu/art-30-gdpr/] mandates that organizations must maintain 
records of processing activities. As already mentioned, this doesn't have to 
take place in Cassandra directly and can often be done more effectively at 
application level. But if data is manipulated manually, or say you have some 
small adhoc tools that change data but won't produce valid logs, you need to be 
able to enable auditing logging for these cases, to be able to document how 
data was changed.
 It's also required by [art. 32|https://gdpr-info.eu/art-32-gdpr/] that any 
natural person must only process personal data as instructed, ie. you need to 
have an activity log to be able to prove that this is actually the case.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388931#comment-16388931
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

Thanks everyone for your inputs. I am trying to summarize the discussion here 
so that I can take action items and either incorporate in this patch or create 
separate JIRAs to track.

1. Simple and incremental approach
2. Reuse the BinLog/ Chronicle work that is put in CASSANDRA-13983 so that we 
don't duplicate efforts and achieve asynchronous, efficient logging
3. Provide context needed in AuditLogger interfaces so that we could have 
various implementations based on logging needs (e.g., more information to log, 
etc.,)
4. Filter/ whitelist users for AuditLog events.
5. Pluggable component that we can fit into the client-facing netty pipeline
6. Code review comments from Dinesh and Jaydeepkumar 
7. Auditlog to log percentage of queries instead of logging every query

I am currently working on #2, #3, #4 and #6. I will gather more data on #5, #7 
and create separate JIRAs as needed. To satisfy #1, we are trying best to get 
the simple version(simple/ basic configs) out, take feedback and improve on in 
later versions/ patches.

I am also planning to stress test this patch and publish numbers with and 
without AuditLog so that user knows the cost of this feature.

Short story: Everyone wants C* audit logging, but we also know it has a 
non-trivial cost in terms of performance (not a free lunch). The debate is 
around what level of details should go in the log, what should be configurable, 
which can be addressed with #1 approach (incremental approach). However, 
design(Interface level details) feedback and redundant 
efforts(BinLog/Chronicle) are being addressed in this patch.

Hope this summary helps. Please let me know if I am missing something important.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jaydeepkumar Chovatia (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387303#comment-16387303
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-12151:
---

As per this [PR|https://github.com/vinaykumarchella/cassandra/pull/2] every 
single query will be audited once audit is enabled, can we make this selective 
to not audit all the queries but percentage of queries as people will have 
different requirements for audit query frequency?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386288#comment-16386288
 ] 

Ariel Weisberg commented on CASSANDRA-12151:


I think logging more things to more formats, and making what is logged more 
configurable are great additions. I'm -1 on having two approaches to logging 
database access to a file that are redundant.

I don't really care what implementation we go with just that there aren't two. 
You could completely remove the existing FQL since we haven't shipped it and 
replace it with something with a similar high performance asynchronous binary 
logging option as well as the text based one you think is best for audit logs. 
Or you can iterate on the existing FQL functionality to get what you want from 
it.

Chronicle has a human and machine readable text format. Switching between the 
two is a flag. You could use the existing full query log plumbing to write one 
of those if you wanted. You can also convert binary logs to human readable logs 
using fqltool. You can also tail the full query log in human readable format 
using fqltool. I think binary vs text is a bit of a red herring.

Some things a new implementation needs to preserve is support for asynchronous 
logging, bounded memory utilization for queueing, and a reasonably efficient 
approach to appending to a file from multiple threads. It should be possible 
for a beefy node to log all the operations a node is executing and process them 
using a single core if configured to use binary format.



> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386137#comment-16386137
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


[~eanujwa] wrote:
{quote}If auditing for a database application user is only at application 
level, what would happen in case direct queries are executed on the database 
using the the application user credentials (using tools such as cqlsh)?
{quote}
If the operator decided not to enable audit logging for the application users, 
nothing would get logged. If someone steals the credentials, none of the 
activities would show up in the logs. If you think that this is unacceptable, 
then you can enable auditing for the app user as well, which I would not, but 
it’s really up to you.

[~jolynch] wrote:
{quote}Doesn't the category filter adequately achieve this (you could exclude 
DML or QUERY)? Do we need per user query logging when there is already per user 
permissions limiting their access to the database in the first place?
{quote}
Although I understand the motivation behind the query categories from a 
developer perspective, it’s probably a feature that is not as easy to 
understand and handled by operators. Let alone managers and compliance 
officers, who lack the technical understanding for discussing which categories 
needs to be enabled for which tables. Once you’ll notice that you have to limit 
the audit logging due to the performance impact, it’s going to be a rather 
painful discussion how categories and tables should be configured in this 
regard. I’d rather give ops people more simpler options that are easy to 
understand and reason about, when discussed between ops and non-technical 
stakeholders. Offering “all or nothing” audit logging on user basis would fit 
this requirements for me (at least for CQL, login events could be a different 
story). 

I'd also really like to avoid luring people into the idea that Cassandra would 
be a one-stop solution for audit logging and you don't have to implement your 
own logging on application level anymore, since Cassandra does handle this for 
you. It probably won't, as logging all statements for applications will be too 
much of a performance impact. Now the question for me is to offer operators 
options to optimize their way out of this situation, or simply tell people that 
it's not possible to handle this use-case (audit logging for application users 
on high-load systems) in Cassandra. I'd clearly go with the later, as many 
users think that Cassandra already puts too much of a burden on operators.

 
{quote}While I agree use case #1 (security) does not require this, use cases #2 
and #3 very much do. For #2 or similar you typically have to prove that only 
authorized applications manipulated the database and a typical way to do that 
is to produce query logs showing that only trusted application IP addresses and 
specific credentials made DML statements (but QUERY is less important). For #3 
the requirements are even greater, e.g. you may have to be able to prove that 
user data was not exfiltrated at all, requiring auditing of QUERY statements. 
Yes it's higher overhead but if you can turn it off with the category filters I 
think it's fine don't you?
{quote}
Can you really “prove” unauthorised data manipulation did *not* happen through 
the audit logs? Say I have an application that is updating a credit score and 
now I use that user to do an update manually, how would you ever notice that 
from the audit logs? My manual statement looks just like the million others 
executed by the app. IP based access restrictions should be in place anyways, 
that’s also not something that would necessarily give you additional clues.

As was already suggested, let’s take an incremental approach that will allow us 
to introduce a simple audit logging that can be understood by both technical 
and non-technical people and configured in little time. If any external 
compliance authority raises some concerns, e.g. that DCL statements need to be 
handled differently from queries, we can take care of that. But any 90% 
solutions would still be a big win over what we currently have.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be ab

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Joseph Lynch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384516#comment-16384516
 ] 

Joseph Lynch commented on CASSANDRA-12151:
--

There are a lot of competing desires in this ticket, and I want to firmly +1 an 
incremental approach where we get the basic interface in and start adding new 
implementations or configuration options in follow up items. If I understand 
the comments above we're trying to solve the following all in one ticket:
 # Logging for security
 # Logging for business accounting compliance (e.g. SOX).
 # Logging for monetary transaction compliance (e.g. PCI).
 # Logging for replay later (e.g. for correctness testing)

These require different implementations because they require different 
tradeoffs, and they may not all get done in this ticket. For example, 
CASSANDRA-13983 is focused on use case #4 and appears to use a highly optimized 
binary format which is lower overhead but does not meet requirements for #1/2/3 
and requires custom parsers (you can't just hand the file to the auditors), 
whereas the patch [~vinaykumarcse] has provided I believe aims more for #1/2/3. 
Generally #2/3 require guarantees of logging if a query was attempted, 
regardless of it succeeded, and #2/3 generally require a lot more context than 
#1/#4 do. I think it would be great if in this ticket we can commit the basic 
interface and starting points of the configuration options (e.g. is it 
controlled through the cassandra.yaml or table options), and we can work on 
improving performance, configuration flexibility, etc in follow up tickets.

[~jasobrown] wrote:
{quote}I agree with [~djoshi3] and [~jjordan]: this functionality should really 
leverage the existing behavior of FQL (CASSANDRA-13893). There is no need to 
create a parallel or duplicate set of behaviors, unless it's completely 
warranted - and I have heard no arguments here that it is.
{quote}
Do you think the patch creates a parallel or duplicate set of behaviors? It 
provides a different query logging implementation that makes different 
tradeoffs and targets a different use case, but I think everyone is agreeing 
that we can unify the two behind one interface (we just have to make sure that 
interface has enough query context for all use cases which might be tough as 
FQL really doesn't need all the session context like user info but #1,2,3 do).

[~eanujwa] wrote:
{quote}If you are logging the exact query with all the values in case of 
regular queries (not prepared), then how would logging bind values of a 
prepared statement becomes a security concern?
{quote}
Generally speaking secure applications exclusively use prepared statements as 
simple statements are vulnerable to injection. Also, if you're using audit 
logging for PCI (or even SOX) the data in DML could easily be sensitive (e.g. 
credit cards or user's names), which you probably want to avoid by default. It 
could certainly be an option though.

[~spo...@gmail.com] wrote:
{quote}Usually you'll see two kind of users on production systems: privileged 
users and application users. Auditing privileged users (admins or developers) 
will almost always make sense, in order to be able to detect unauthorized 
access and data manipulation. There's only a limited amount of statements to 
log, as these will be executed manually. It also shouldn't matter which 
keyspaces or tables are access by the users; he is either monitored or not.
{quote}
Doesn't the category filter adequately achieve this (you could exclude DML or 
QUERY)? Do we need per user query logging when there is already per user 
permissions limiting their access to the database in the first place?
{quote}However, auditing queries of application users has a very limited 
security and data privacy benefit, but adds a great deal of load to the 
database. Those queries will be automatically generated by the application and 
there will be no way to tell if the query or statement was authorized, as you 
don't know on behalf of whom it was executed. Any auditing functionality for 
these operations must therefor take place at application level.
{quote}
While I agree use case #1 (security) does not require this, use cases #2 and #3 
very much do. For #2 or similar you typically have to prove that only 
authorized applications manipulated the database and a typical way to do that 
is to produce query logs showing that only trusted application IP addresses and 
specific credentials made DML statements (but QUERY is less important). For #3 
the requirements are even greater, e.g. you may have to be able to prove that 
user data was not exfiltrated at all, requiring auditing of QUERY statements. 
Yes it's higher overhead but if you can turn it off with the category filters I 
think it's fine don't you?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Dinesh Joshi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383854#comment-16383854
 ] 

Dinesh Joshi commented on CASSANDRA-12151:
--

Hi [~eanujwa] - I did review the design. As you can see there are different 
perspectives on the Audit functionality - granularity, type etc. Personally, 
defining an interface for Auditing would help keep things generic. For example, 
based on your needs you could, in theory, have an Audit logger implementation 
that logs at the keyspace or table level or even make more sophisticated 
decisions. It frees up everyone to define the Audit implementation to their 
specific needs. Bear in mind that you do not want to degrade performance for 
users that do not use Auditing capabilities.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Anuj Wadehra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383690#comment-16383690
 ] 

Anuj Wadehra commented on CASSANDRA-12151:
--

[~djoshi3] Can you review the design first and close open points? Once the 
design is closed, code review can be done for the revised patch.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Anuj Wadehra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383682#comment-16383682
 ] 

Anuj Wadehra commented on CASSANDRA-12151:
--

[~djoshi3] Can you review the design in detail and close open points?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Anuj Wadehra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383680#comment-16383680
 ] 

Anuj Wadehra commented on CASSANDRA-12151:
--

[~spo...@gmail.com] My comments:
{quote}Any auditing functionality for these operations must therefor take place 
at application level.
{quote}
If auditing for a database application user is only at application level, what 
would happen in case direct queries are executed on the database using the the 
application user credentials (using tools such as cqlsh)?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Anuj Wadehra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383672#comment-16383672
 ] 

Anuj Wadehra commented on CASSANDRA-12151:
--

[~vinaykumarcse] My comments are as follows:
{quote}I like the idea of keeping the auditing configurations to be simple at 
keyspace level
{quote}
I think enabling/disabling auditing should be at table level rather than 
keyspace level. Reasoning-Keyspaces are designed based on data replication 
needs rather than auditing needs. 
Thus, a keyspace k1 may have two tables with same replication needs but 
different auditing requirements e.g. t1 with sensitive data which must be 
audited and t2 with some temporary data which need not be audited. With 
keyspace level auditing, users will end up doing unnecessary auditing and 
cluttering of audit logs.
{quote}if we are managing any configurations at table level, config file might 
not be a good place, table property?
{quote}
Agree. We can have default auditing configuration for tables in cassandra.yaml 
(say enable_audit=true/false, categories=DDL,DCL etc.,extended=true/false etc.) 
and table level properties to override the default auditing options in yaml. 
This way we will not clutter cassandra.yaml with tables to be audited, there 
would not be any additional configuration file and Cassandra configuration 
files would not have any application schema awareness.
{quote}All the usecases that I see here are Auditing at the cluster or no 
auditing, but not specific to user.
{quote}
If a Cassandra role defined for an application user has limited permissions, 
you may not want to audit everything from the application and clutter the 
auditing logs. 
But If you have a single powerful application user for database, the feature 
may not be that useful as the application user may be misused and you may 
easily bypass the application and directly login to cqlsh. I dont think 
Cassandra has an authentication mechanism to identify and restrict direct login 
for specific application users/roles. In case of direct login using an 
application user, you will log the operation but will not be able to fix the 
responsibility of the operation. More thoughts?
{quote}This would be a security concern to keep the actual data in logs
{quote}
If you are logging the exact query with all the values in case of regular 
queries (not prepared), then how would logging bind values of a prepared 
statement becomes a security concern?
{quote}
If we think logging individual statement in the batch might not come for free, 
I would rather approach it differently in case of batch, we just limit it to 
log only statement types and its count or just the prepared statement ids?
{quote}
Can we only log unique query strings in a batch to restrict IO? In case of a 
batch with prepared statements and extended configuration set to true, audit 
log format could be:

Q1: INSERT INTO t1(variable1,variable2) values(?,?)
Q2: INSERT INTO t2(variable1,variable2,variable3) values(?,?,?)

Q1 [bind_value1,bind_value2]
Q2 [bind_value1,bind_value2,bind_value3]
Q1 [bind_value1,bind_value2]
Q1 [bind_value1,bind_value2]
 
If separate log statements are used, UUID may be associated with the batch 
(like it is done in your patch)
{quote}Why do you think logging CL is required? Is CL adding any value for the 
auditor?
{quote}
What are your thoughts on this?

 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383606#comment-16383606
 ] 

Jason Brown commented on CASSANDRA-12151:
-

I agree with [~djoshi3] and [~jjordan]: this functionality should really 
leverage the existing behavior of FQL (CASSANDRA-13893). There is no need to 
create a parallel or duplicate set of behaviors, unless it's completely 
warranted - and I have heard no arguments here that it is.

Further, I like [~djoshi3]'s idea about adding a pluggable component that we 
can fit into the client-facing netty pipeline. It allows somebody to go bananas 
adding audit checks/details, and yet leaves untouched the standard behavior 
that 99.9% of users expect.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383488#comment-16383488
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


[~vinaykumarcse] wrote:

bq. Seems like a complicated configuration, would like to understand the use 
cases more here and see if anyone else needs this functionality. All the 
usecases that I see here are Auditing at the cluster or no auditing, but not 
specific to user. I would love to hear if there are any other users with this 
usecase.


Usually you'll see two kind of users on production systems: privileged users 
and application users. Auditing privileged users (admins or developers) will 
almost always make sense, in order to be able to detect unauthorized access and 
data manipulation. There's only a limited amount of statements to log, as these 
will be executed manually. It also shouldn't matter which keyspaces or tables 
are access by the users; he is either monitored or not.

However, auditing queries of application users has a very limited security and 
data privacy benefit, but adds a great deal of load to the database. Those 
queries will be automatically generated by the application and there will be no 
way to tell if the query or statement was authorized, as you don't know on 
behalf of whom it was executed. Any auditing functionality for these operations 
must therefor take place at application level. Eg. a help desk tool, which is 
used by a support employee to access personal data of a customer in Cassandra, 
must keep an activity log for that directly. It doesn't make sense to log 
queries for the generic help desk tool Cassandra user on the database side. 
Therefor we need a way to enable CQL query auditing on user level.



> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Dinesh Joshi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383281#comment-16383281
 ] 

Dinesh Joshi commented on CASSANDRA-12151:
--

Hi [~vinaykumarcse], I have gone over your patch. Here is my feedback -
 # {{AuditLogEntryType::allStatementsMap}} - Instead of creating an explicit 
map of statements consider adding the type of statement in the actual class 
itself. That way when new statements are introduced, you don't need to keep 
updating this map.
 # {{AuditLogFilter::loadFilters}} - Consider adding checks here to make sure 
{{includedKeyspacesList}} and {{excludedKeyspacesList}} do not have the same 
keyspace specified. This will help simplifying your logic in 
{{AuditLogFilter::isFiltered}}. Also consider lowercasing / uppercasing while 
storing the keyspace and category names as you are doing a case insensitive 
match in {{isFiltered}}.
 # {{AuditLogFilter}} consider using {{Set}} instead of 
{{List}} for {{includedKeyspacesList}}, {{excludedKeyspacesList}}, 
{{excludedCategoriesList}}, {{includedCategoriesList}}. Keyspace names and 
Categories should be unique.
 # {{AuditLogFilter::isFiltered}} can be simplified and made more performant if 
you use {{Set}} instead of {{List}}.
 # It would be great if you can update the docs in the source tree.

On the implementation design, it would be great to see if you can extend 
-CASSANDRA-13983-. It does pretty much what you're doing except logging the 
authorization failures.

Consider refactoring your code to add a netty handler that invokes an auditing 
interface. The advantage of this approach would be that, when audit logging is 
disabled, you can take this handler out of the netty pipeline. This way there 
is zero performance impact when audit is disabled. You can define a 
IAuditLogger interface that has sufficient contextual information to log all 
queries. This will help make the audit logging implementation pluggable.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382483#comment-16382483
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

I like the idea of keeping the auditing configurations to be simple at keyspace 
level, as you mentioned specifying the configurations at table level gets too 
messy to manage. Also adding new configurations as on when adding new tables 
would be tough to manage and operate. Regarding maintaining separate files for 
audit log configurations makes it difficult for the C* operators, sidecars, 
config management systems to manage several configuration files. It would 
become maintenance overhead to manage configuration files per feature. So I 
would keep it simple and manage configurations via a single file in 
cassandra.yaml. Even maintaining separate config file does not reduce the 
overhead of adding/ removing table level audit log configurations. IMO, if we 
are managing any configurations at table level, config file might not be a good 
place, table property? 

 
{quote}Configuration of whitelisted users and separate auditing configuration 
for each user
{quote}
seems like a complicated configuration, would like to understand the use cases 
more here and would like to see if anyone else needs this functionality. All 
the usecases that I see here, either Audit the cluster or not, but not specific 
to user, I would love to hear if there are any other users with this usecase.
{quote}Auditing bind values of prepared statements and its configuration
{quote}
This would be a security concern to keep the actual data in logs, these logs 
could be shipped over the wire to an archival system for log rotation, in which 
case data goes over the wire as a plain text and all the effort of securing the 
C* cluster are not helpful at that point. Audit log should not become a 
security hole in C* design.
{quote}Logging every statement in a Batch separately may have significant 
performance hit.
{quote}
Merging the entire batch of statements to a single log line would be a 
performance hit or might hit the limitations of message sizes in logger or any 
such implementations. Considering the performance of storing the entire batch 
of statements and logging them might not be a great idea for GC reasons as 
well, there is always a tradeoff between small working set size vs one large 
working set in terms of low garbage collection overhead, allocator overhead. If 
we think logging individual statement in the batch might not come for free, I 
would rather approach it differently in case of batch, we just limit it to log 
only statement types and its count or just the prepared statement ids? We also 
need to keep the performance overhead that we are adding to the request path 
with AuditLog.

{quote} Password Obfuscation for DCL Queries
{quote}
Yes, Agreed - Password Obfuscation for DCL Queries shall be fixed.
{quote}Are you planning to evaluate and implement a Chronicle Queue variant 
similar to CASSANDRA-13983?
{quote}
Yes, there is certainly a possible option to provide one more minimal 
performance impact implementation for IAuditLogger, and we can consider binlog. 
I will take a look at it next week and create a follow-up JIRA as needed.

 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Anuj Wadehra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381505#comment-16381505
 ] 

Anuj Wadehra commented on CASSANDRA-12151:
--

[~vinaykumarcse] I have gone through your patch.

Some high level review comments on the patch:
 # Why do you think logging CL is required? Is CL adding any value for the 
auditor?
 # Don’t you think we should have separate configuration file for auditing 
rather than cassandra.yaml? —there are applications with considerable number of 
tables. As per the proposed design, user shall be able to individually specify 
tables which must be audited and this would clutter the cassandra.yaml file.
 # Logging every statement in a Batch separately may have significant 
performance hit. Can we just log once per Batch and make sure that all 
operations in the batch are included in that one log statement?
 # Are you planning to evaluate and implement a Chronicle Queue variant similar 
to CASSANDRA-13983?

The patch lacks following features from the design document:
 # Configuration of whitelisted/application users and separate auditing 
configuration for that.
 # Configuration of tables to be audited with/without regular expressions e.g. 
ks1.*,*.table1 etc.
 # Auditing bind values of prepared statements and its configuration
 # Password Obfuscation for DCL Queries

[~jasobrown] [~vinaykumarcse] We can develop auditing feature (Cassandra code) 
incrementally. In the first iteration, [~vinaykumarcse] can contribute his 
patch and then our team shall contribute the remaining 4 features. We are 
already developing an auditing plugin for Cassandra which aligns with the 
proposed design and we can port these remaining features from the plugin to the 
main Cassandra code.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16379130#comment-16379130
 ] 

Jeremiah Jordan commented on CASSANDRA-12151:
-

One of the goals should be recording queries with  minimal impact on workloads, 
which was also a goal of CASSANDRA-13983, so I would think some re-use might be 
a good idea rather than coming up with a new way of doing that.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Anuj Wadehra
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Anuj Wadehra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16379076#comment-16379076
 ] 

Anuj Wadehra commented on CASSANDRA-12151:
--

Thanks for the review comments !

[~vinaykumarcse]  I will go through your patch and share my comments.

[~jjordan] I had a look at  -CASSANDRA-13983-  The use cases are quite 
different. Yes we can have a chronicle-queue variant of Audit logger like 
[~vinaykumarcse] said but I think we should start with a simple logger 
implementation unless we have real good reasons to go with chronicle-queue for 
Audit logging.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Anuj Wadehra
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377534#comment-16377534
 ] 

Vinay Chella commented on CASSANDRA-12151:
--

Hi [~eanujwa]  [~jasobrown],

I’m excited to see the design document and it looks good to us!

Netflix had a similar requirement recently for our internal 2.1 clusters and we 
implemented a simple version (no query categories, etc…) for sox auditing. As 
your design is very close to what we implemented, just a few differently named 
classes for the most part, can we work together on the trunk 
[patchset|https://github.com/vinaykumarchella/cassandra/pull/2] to add the 
missing components from your design? Alternatively, we could take an 
incremental approach, review what we have on the trunk branch of the simple 
version and get it committed and then add in some of the more advanced features 
next. I believe this patch follows the design goals that you put together.

Please review and let me know if you have any questions or concerns about the 
first iteration. If folks are interested in the 3.x/2.x branches I can put 
those up on my github as well.

[~jhb]
{quote}I just have one question, do you think enabling/updating/disabling audit 
require a node restart?
{quote}
The posted patch allows online auditlog enable/disable or filter updates via 
JMX.

[~jjordan]
{quote}You should take a look at the infrastructure added in CASSANDRA-13983 
for query logging
{quote}
Yes, we looked and that certainly looks interesting, perhaps this design allows 
us to use it as another implementation of {{IAuditLogger}}?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Anuj Wadehra
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377259#comment-16377259
 ] 

Jeremiah Jordan commented on CASSANDRA-12151:
-

You should take a look at the infrastructure added in CASSANDRA-13983 for query 
logging

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Anuj Wadehra
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377175#comment-16377175
 ] 

Michael Shuler commented on CASSANDRA-12151:


Title: {{s/3.x/4.x/}}

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Anuj Wadehra
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jacques-Henri Berthemet (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376850#comment-16376850
 ] 

Jacques-Henri Berthemet commented on CASSANDRA-12151:
-

Proposed design looks good to me. I think using a file is better than a table 
as a rogue user may just truncate/drop/overwrite the table.

I just have one question, do you think enabling/updating/disabling audit 
require a node restart?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Anuj Wadehra
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Anuj Wadehra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16375459#comment-16375459
 ] 

Anuj Wadehra commented on CASSANDRA-12151:
--

Please find attached the Design proposal for "Database Auditing" feature in 
Cassandra. The solution suggests file-based audit logs rather than storing 
audit logs in database. I request everyone to review the proposal and share 
their feedback so that we can incorporate comments and go ahead with the 
implementation.

[^DesignProposal_AuditingFeature_ApacheCassandra_v1.docx]

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Anuj Wadehra
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Anuj Wadehra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16326295#comment-16326295
 ] 

Anuj Wadehra commented on CASSANDRA-12151:
--

[~jasobrown] Yes. Our team is keen to work on this ticket. I will be sharing 
the "Proposal" soon. For now, you can assign the JIRA to me.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322253#comment-16322253
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


I took a different approach with CASSANDRA-13668 by proposing to send auditing 
events to external subscribers that would be able to store or analyze them 
further. It's a different approach from storing auditing events in a local or 
distributed keyspace, as implemented in this ticket. My assumption was 
basically that raising alerts on certain events would be easier and more 
efficient to implement, if consumed by an external client, compared to polling 
a table. But I'm curious how any discussion as part of this ticket will turn 
out and what the different requirements are.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
> Fix For: 4.x
>
> Attachments: 12151.txt
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1636#comment-1636
 ] 

Jason Brown commented on CASSANDRA-12151:
-

[~eanujwa] No one is currently assigned to the ticket, and [~stefan setyadi]'s 
last comments are from Aug 2016. Thus I suspect nobody is working on it at the 
moment. Are you interested in working on it, or just requesting the feature?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
> Fix For: 4.x
>
> Attachments: 12151.txt
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Anuj Wadehra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322076#comment-16322076
 ] 

Anuj Wadehra commented on CASSANDRA-12151:
--

There is no update since May,2017. Is anyone working on implementing auditing 
in Cassandra ? 

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
> Fix For: 4.x
>
> Attachments: 12151.txt
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[nhanpt14 (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15938440#comment-15938440
 ] 

nhanpt14 commented on CASSANDRA-12151:
--

Could we have Data Auditing in Apache Cassandra like DataStax Enterprise?
I hope it will be support in next release.


> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
> Fix For: 3.11.x
>
> Attachments: 12151.txt
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[stefan setyadi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401929#comment-15401929
 ] 

stefan setyadi commented on CASSANDRA-12151:


After some discussion with my team, we were thinking of the idea of being able 
to configure the log to each keyspace or table.
And so I propose:
# Cassandra will still log meta-changes of the server
# We can configure it to log every query (update/read) which targets a specific 
keyspace or table. For our use case, we need to log it even knowing that there 
will be some performance penalty.

all in all, the ability to configure which query will be logged will be helpful 
in cases where
* the table is important, such as employee data, financial information, etc
* or where we know the table won't be accessed as much

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
> Fix For: 3.x
>
> Attachments: 12151.txt
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[stefan setyadi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401669#comment-15401669
 ] 

stefan setyadi commented on CASSANDRA-12151:


okay so first of all, my initial use case was for the audit log to be reviewed 
and used to detect intrusion. At first I was thinking of logging the queries so 
it could be used to detect malicious insert/read.

I admit now that in hindsight, I didn't have a clear idea of how big the scale 
of the operations were. You're probably right and we shouldn't log every 
insert/read query.
I agree it is still useful to know any meta-changes but currently I have no 
clear picture of how the audit log will be used.
+1 on the user login idea and the byteman.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
> Fix For: 3.x
>
> Attachments: 12151.txt
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399584#comment-15399584
 ] 

Jonathan Ellis commented on CASSANDRA-12151:


Remember that people almost always use Cassandra to drive applications at 
scale, not to do interactive analytics.  I can't see that logging 100,000 ops 
per second of the same ten queries is going to add much value.  I don't want to 
load that gun for people to blow their feet off with...

Generally auditing is most useful to see "who *changed* what" not "who *asked 
for* what."  (Again, the "who" for most of the latter is going to be "the 
application server.")  And again, it's not super useful to know that the app 
server inserted 10,000 new user accounts today, but it IS useful to know when 
Jonathan added a new column to the users table.  

(I would also include user logins as an interesting event.  This will be 
dominated by app servers still but much much less noise than logging every 
query or update.)

Besides changes over CQL, this could also include JMX changes, although there 
are so many entry points to JMX mbeans that this would be ugly to do by hand.  
Perhaps we could inject this with byteman?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
> Fix For: 3.x
>
> Attachments: 12151.txt
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[stefan setyadi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398691#comment-15398691
 ] 

stefan setyadi commented on CASSANDRA-12151:


yeah sure, sorry for going ahead and doing my own thing.

# I was thinking of global one as it would be easier to be queried. it would 
also help in case any of the server fails as we probably don't want to lose the 
logs.
# can I ask for clarification on what you meant by every query? at the very 
least we want to know the usual "who did what and when". Maybe have different 
levels/choice for which query are logged? In the end the trade off between 
performance and log is something which I'd rather leave on the hands of the 
company using it.

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
> Fix For: 3.x
>
> Attachments: 12151.txt
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397542#comment-15397542
 ] 

Jonathan Ellis commented on CASSANDRA-12151:


Can we back up a little and talk about design?

Some questions in my mind:

# Do we want a global audit log, or server-local?  If the former (easier for 
users to query), it should go in system_distributed keyspace; otherwise just in 
system (higher performance).
# Is there a use case where you'd want to log every query?  That seems like it 
would entail a prohibitive performance penalty.  I would think most users would 
be better served by logging meta-changes (adding roles, altering tables, etc)

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
> Fix For: 3.x
>
> Attachments: 12151.txt
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[stefan setyadi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15393643#comment-15393643
 ] 

stefan setyadi commented on CASSANDRA-12151:


Hi this is my first time submitting a patch to jira so tell me if there's 
anything I can improve on.
I attached a patch for feedback, do tell me if I should change my approach.

to start the logging put {noformat}audit_log: true{noformat} in cassandra.yaml

for now, a summary of what I've done:
* I created keyspace and column families to be initialize at server start.
* Right now it's also logging queries by cassandra itself.
* I didn't put the connections attempt as we can get that information from the 
clientstate when processing the statement.
* the activity type is just the class name of the statement.
* the column family value when creating keyspace is 'null' as in string value

I haven't tested this on prepared and batch statement.
any feedback is welcome, regards

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
> Fix For: 3.x
>
> Attachments: 12151.txt
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)