Hi Nikolay, I actually have a somewhat different approach that is somewhat similar to the Authorizer interface. I have updated the KIP to reflect that. I'm happy to collaborate during the implementation. I have a code change not yet published but I can publish it and we can see what's the best way to collaborate :).
Viktor On Mon, Sep 7, 2020 at 12:20 PM Nikolay Izhikov <nizhi...@apache.org> wrote: > Hello, Viktor. > > Do you want to implement the exact approach as it described in the current > KIP? > Or you have another proposal on how it has to be implemented? > > I abandoned this KIP due to lack of interest from community. > Guess we can collaborate during implementation. > > > 7 сент. 2020 г., в 13:13, Viktor Somogyi-Vass <viktorsomo...@gmail.com> > написал(а): > > > > Hi folks, > > > > It's been a few days since I last pinged and nobody replied so I assume > > that this KIP is abandoned and I can take this over (but please let me > know > > if it's not). I will keep the current version of the KIP and move it to a > > sub-page if it's ever needed. > > > > Thanks, > > Viktor > > > > On Fri, Aug 28, 2020 at 11:35 AM Viktor Somogyi-Vass < > > viktorsomo...@gmail.com> wrote: > > > >> Hi folks, > >> > >> I have a use-case and a non-trivial implementation with Apache Atlas for > >> this KIP and since this kip seems to be dormant for a while now, I'd > take > >> it over and drive it to completion if you don't mind. > >> The current state of the PoC can be found on my fork at > >> https://github.com/viktorsomogyi/kafka/tree/atlas-audit-impl. > >> > >> Viktor > >> > >> On Wed, Jan 29, 2020 at 9:32 AM Игорь Мартемьянов <ledost...@gmail.com> > >> wrote: > >> > >>> Hello, Nikolai. > >>> > >>> > >>> > >>>> Can you, please, make it more specific? > >>> > >>>> Why does a business want to have this information? > >>> > >>> It is very demanded for security department to know who/when/where > create > >>> or edit ACL settings. The same situation about topics. > >>> > >>> > >>> > >>>> What are the use-cases for it? > >>> > >>> This KIP are able to help catching some unauthorized changes of cluster > >>> resources or save history these changes. > >>> > >>> > >>> > >>>> Who will be analyzing these events and how? > >>> > >>> Events could be analyzed by some security department or developers who > >>> builds applications that can change cluster resources. > >>> > >>> > >>> > >>>> Why it’s not convenient to implement it with third-party tools? > >>> > >>> It is imposible to my mind getting detailed information about these > events > >>> through third-party tools, because we don`t have the ability to catch > >>> these > >>> events outside Kafka core. > >>> > >>> > >>> > >>>> 1. `audit` name sounds too general for me. How about `onEvent`? > >>> > >>> Thanks, good point. I`ll change the method name. > >>> > >>> > >>> > >>>> 2. Should we introduce a special marker interface for audit events? > >>> > >>>> `AuditableRequest`, for example? > >>> > >>> No, we shouldn`t. We are going to create these events if we process > >>> special > >>> request in KafkaApis class (ApiKeys.CREATE_TOPICS, > ApiKeys.DELETE_TOPICS, > >>> ApiKeys.CREATE_ACLS, ApiKeys.DELETE_ACLS, ApiKeys.DESCRIBE_ACLS, > >>> ApiKeys.ALTER_CONFIGS) > >>> > >>> > >>> > >>>>> public interface AuditEvent<T extends AbstractRequest> { > >>> > >>>>> String guid(); > >>> > >>> > >>> > >>>> Where this `guid` comes from? > >>> > >>> Guid generated automatically when the audit event is created. > >>> > >>> > >>> > >>>> Will it be the same on each node that receives an auditable event? > >>> > >>> Event is creating while request that changes cluster resource is > processed > >>> and will destroyed after the prosessing will finished on each node. > >>> > >>> > >>> > >>>> Do we have `guid` for any extensions of `AbstractRequest`? > >>> > >>> No, we don`t. > >>> > >>> > >>> > >>>> If this field is `guid` why do we format this as a String on the API > >>> level? > >>> > >>> Thanks for the point. > >>> > >>> I have changed event interface recently. > >>> > >>> > >>> > >>>> Can you, please, add a full list of proposed new configuration > >>> properties > >>> and > >>> > >>>> examples for each to clarify your intentions? > >>> > >>> Yes, I have added some examples of new configuration properties. > >>> > >>> ср, 29 янв. 2020 г., 10:23 Владимир Беруненко <vvberune...@gmail.com>: > >>> > >>>> Hi Nikolai! > >>>> > >>>>> Can you, please, make it more specific? > >>>> Why does a business want to have this information? > >>>>> What are the use-cases for it? > >>>>> Who will be analyzing these events and how? > >>>>> Why it’s not convenient to implement it with third-party tools? > >>>> > >>>> This is required by the guys from information security to detect > >>> potential > >>>> threats of violation the rules for providing access to the data layer. > >>>> Analysis in the audit system is usually based on identifying > >>>> uncharacteristic integrations. The key feature of the audit, is that > >>>> cluster administrators do not have access to modification of audit > >>> events. > >>>> Third-party tools are great for data analysis, but its not good idea > to > >>> use > >>>> it for audit events collection. > >>>> > >>>>> It’s not clear for me where and when AuditEvents will be sent? > >>>> Who will be the receiver of events? > >>>> > >>>> In my opinion, sending an audit event should be initiated when the > >>> broker > >>>> receives a request that matches the audit parameters. Each > organization > >>> has > >>>> its own receiver system, so a common interface is required that the > >>>> organization’s development team can implement to integrate with their > >>> audit > >>>> system. > >>>> > >>>> Best wishes, Vladimir > >>>> > >>>> Hello, Igor. > >>>>> > >>>>> Thanks for the KIP. > >>>>> I have a couple of comments for it: > >>>>> > >>>>>> Motivation > >>>>>> It is highly demanded in most businesses to have the ability of > >>>>> obtaining audit information in case someone changes cluster > >>> configuration > >>>>> (like creation/deletion/modify/description of any topic or ACLs). > >>>>> > >>>>> Can you, please, make it more specific? > >>>>> Why does a business want to have this information? > >>>>> What are the use-cases for it? > >>>>> Who will be analyzing these events and how? > >>>>> Why it’s not convenient to implement it with third-party tools? > >>>>> > >>>>> It’s not clear for me where and when AuditEvents will be sent? > >>>>> Who will be the receiver of events? > >>>>> > >>>>>> void audit<T extends AbstractRequest>(AuditEvent<T> event); > >>>>> > >>>>> 1. `audit` name sounds too general for me. How about `onEvent`? > >>>>> 2. Should we introduce a special marker interface for audit events? > >>>>> `AuditableRequest`, for example? > >>>>> > >>>>>> public interface AuditEvent<T extends AbstractRequest> { > >>>>>> String guid(); > >>>>> > >>>>> Where this `guid` comes from? > >>>>> Will it be the same on each node that receives an auditable event? > >>>>> Do we have `guid` for any extensions of `AbstractRequest`? > >>>>> If this field is `guid` why do we format this as a String on the API > >>>> level? > >>>>> > >>>>>> One or more of AuditExtension's implementations can be configured > >>> via > >>>>> the configuration audit.extension.classes > >>>>> > >>>>> Can you, please, add a full list of proposed new configuration > >>> properties > >>>>> and examples for each to clarify your intentions? > >>>>> > >>>>> > >>>>>> 24 янв. 2020 г., в 18:12, Alexander Dunayevsky < > >>>> a.a.dunayev...@gmail.com> > >>>>> написал(а): > >>>>>> > >>>>>> Hello Igor, > >>>>>> > >>>>>> Thanks for your KIP 🙌🏽 > >>>>>> It would be great to adopt this functionality and getting the best > >>> of > >>>>>> tracking cluster activity. > >>>>>> > >>>>>> +1 vote from me > >>>>>> > >>>>>> Cheers, > >>>>>> Alex Dunayevsky > >>>>>> > >>>>>> > >>>>>> On Fri, 24 Jan 2020, 15:35 Игорь Мартемьянов, <ledost...@gmail.com> > >>>>> wrote: > >>>>>> > >>>>>>> Motivation: > >>>>>>> > >>>>>>> > >>>>>>> *It is highly demanded in most businesses to have the ability of > >>>>> obtaining > >>>>>>> audit information in case someone changes cluster configuration > >>> (like > >>>>>>> creation/deletion/modify/description of any topic or ACLs).We may > >>> add > >>>>> this > >>>>>>> ability. Since audit requirements are so broad, it's impractical to > >>>>> support > >>>>>>> all of them.Hence we have to provide ability for users to plug > >>>> resources > >>>>>>> helping to achieve required capabilities.* > >>>>>>> > >>>>>>> > >>>>>>> > >>>>> > >>>> > >>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-567%3A+Kafka+Cluster+Audit > >>>>>>> > >>>>>>> > >>>>>>> пт, 24 янв. 2020 г., 17:29 Игорь Мартемьянов <ledost...@gmail.com > >>>> : > >>>>>>> > >>>>>>>> Hello there. > >>>>>>>> Please review this KIP. > >>>>>>>> Thanks. > >>>>>>>> > >>>>>>> > >>>>> > >>>>> > >>>> > >>> > >> > >
