[VOTE] KIP-567: Auditing in Kafka

Hi Everyone, It seems people don't have anything to add on the discussion, so I'd like to propose a vote now. In any case if you still have something to add, please feel free to write either on this thread or on the discussion. A summary of the KIP: I plan to add a simple interface called

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

someone who is being interrogated but again, I don't often do this. :) If there are no further objections I will start a voting in a few days. Viktor On Tue, Feb 2, 2021 at 1:54 PM Viktor Somogyi-Vass wrote: > Hi all, > > I have updated the interfaces. I managed to shrink the requir

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

. Viktor On Thu, Oct 29, 2020 at 5:15 PM Viktor Somogyi-Vass wrote: > Hi Tom. > > Sorry for the delay. > Answering your points: > > > Why is it necessary to introduce this interface to produce the audit > trail > > when there is logging that can already record a l

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

> I think the current state of the proposal is flexible enough to support > > use-cases where the response data is of interest to the auditor. > > This part ensures that: "... doing the auditing before sending the > response > > back ...". A

[jira] [Created] (KAFKA-10650) Use Murmur3 hashing instead of MD5 in SkimpyOffsetMap

Viktor Somogyi-Vass created KAFKA-10650: --- Summary: Use Murmur3 hashing instead of MD5 in SkimpyOffsetMap Key: KAFKA-10650 URL: https://issues.apache.org/jira/browse/KAFKA-10650 Project: Kafka

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

f the partitions. > The event-based approach sounds good - new events and fields can be added > on-demand. Do we need the same versioning strategy we use with the > requests/responses? > > Daniel > > Viktor Somogyi-Vass ezt írta (időpont: 2020. > szept. 21., H, 14:08): > > >

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

ike an awful lot of overhead, and could > introduce timing issues as well. > > Daniel > > > Viktor Somogyi-Vass ezt írta (időpont: 2020. > szept. 16., Sze, 17:17): > > > One more after-thought on your second point (AbstractRequest): the > reason I > > introduced it i

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

to pass down the AbstractRequest and the implementation can do the extraction of valuable info. So that's why I added this in the first place and I'm interested in your thoughts. On Wed, Sep 16, 2020 at 4:41 PM Viktor Somogyi-Vass wrote: > Hi Mickael, > > Thanks for reviewing the KIP. >

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

that sets the error field to None. However, with > the builder pattern, if error is not set it's null. > > 5) Should AuditInfo be an interface? > > On Mon, Sep 14, 2020 at 3:26 PM Viktor Somogyi-Vass > wrote: > > > > Hi everyone, > > > > Changed the

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

but if there are no objections until then, I'll start a vote. Viktor On Tue, Sep 8, 2020 at 9:49 AM Viktor Somogyi-Vass wrote: > Hi Everyone, > > I'd like to restart the discussion on this. Since the KIP has been > revamped I thought I'd start a new discussion thread. > > Link: > htt

[DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

Hi Everyone, I'd like to restart the discussion on this. Since the KIP has been revamped I thought I'd start a new discussion thread. Link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-567%3A+Kafka+Cluster+Audit Short summary: - Would like to introduce a new interface similar to the

Re: Backports of KAFKA-9839 onto 2.2, 2.3 and 2.4

Hi again, Would someone please have a look at these? Thanks, Viktor On Fri, Aug 28, 2020 at 10:11 AM Viktor Somogyi-Vass < viktorsomo...@gmail.com> wrote: > Hi folks, > > I have backported KAFKA-9839 > <https://issues.apache.org/jira/browse/KAFKA-9839> onto 2.2, 2.

Re: [DISCUSS] KIP-567: Kafka Cluster Audit

this KIP due to lack of interest from community. > Guess we can collaborate during implementation. > > > 7 сент. 2020 г., в 13:13, Viktor Somogyi-Vass > написал(а): > > > > Hi folks, > > > > It's been a few days since I last pinged and nobody replied so I as

Re: [DISCUSS] KIP-567: Kafka Cluster Audit

, 2020 at 11:35 AM Viktor Somogyi-Vass < viktorsomo...@gmail.com> wrote: > Hi folks, > > I have a use-case and a non-trivial implementation with Apache Atlas for > this KIP and since this kip seems to be dormant for a while now, I'd take > it over and drive it to complet

Re: [DISCUSS] KIP-567: Kafka Cluster Audit

Hi folks, I have a use-case and a non-trivial implementation with Apache Atlas for this KIP and since this kip seems to be dormant for a while now, I'd take it over and drive it to completion if you don't mind. The current state of the PoC can be found on my fork at

Backports of KAFKA-9839 onto 2.2, 2.3 and 2.4

Hi folks, I have backported KAFKA-9839 onto 2.2, 2.3 and 2.4 respectively as we detected it with a customer running 2.2. Would anyone please help me review it? The PRs are: https://github.com/apache/kafka/pull/9150

Re: [VOTE] KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override

+1 from me (non-binding), thanks for the KIP. On Mon, Jul 27, 2020 at 10:02 AM Dániel Urbán wrote: > Hello everyone, > > I'd like to start a vote on KIP-635. The KIP enhances the GetOffsetShell > tool by enabling querying multiple topic-partitions, adding new filtering > options, and adding a

Re: [ANNOUNCE] New committer: Boyang Chen

Congrats :) On Thu, Jun 25, 2020 at 12:28 AM Liquan Pei wrote: > Congrats! > > On Wed, Jun 24, 2020 at 9:42 AM Raymond Ng wrote: > > > Congrats Boyang! Look forward to more awesome contributions from you in > the > > future. > > > > Regards, > > Ray > > > > On Wed, Jun 24, 2020 at 6:07 AM

[jira] [Resolved] (KAFKA-7805) Use --bootstrap-server option for kafka-topics.sh in ducktape tests where applicable

[ https://issues.apache.org/jira/browse/KAFKA-7805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass resolved KAFKA-7805. Resolution: Fixed > Use --bootstrap-server option for kafka-topics.sh in duckt

[jira] [Created] (KAFKA-9814) Allow AdminClient.listTopics to list based on expression

Viktor Somogyi-Vass created KAFKA-9814: -- Summary: Allow AdminClient.listTopics to list based on expression Key: KAFKA-9814 URL: https://issues.apache.org/jira/browse/KAFKA-9814 Project: Kafka

Re: [DISCUSS] Apache Kafka 2.5.0 release

Hi David, There are two short JIRAs related to KIP-352 that documents the newly added metrics. Is it possible to merge them in? https://github.com/apache/kafka/pull/7434 (trunk) https://github.com/apache/kafka/pull/8127 (2.5 cherry-pick) Thanks, Viktor On Mon, Feb 24, 2020 at 7:22 PM David

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

d > this to the KIP. > Please let me know if we're good to continue with this. > > Best, > Viktor > > On Mon, Jan 20, 2020 at 5:45 PM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> wrote: > >> Hi Rajini, >> >> 1) I think we can to keep the conv

Re: [DISCUSS] KIP-550: Mechanism to Delete Stray Partitions on Broker

Hi folks, Perhaps a solution option is to only rename partitions to "whatever-topic-x.stray" when processing the LAIR and delete it with a periodic task (so not with a fixed delay but have a thread which scans and deletes them periodically). I think it has an advantage as it is a similar approach

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

, Viktor On Mon, Jan 20, 2020 at 5:45 PM Viktor Somogyi-Vass wrote: > Hi Rajini, > > 1) I think we can to keep the conventions in the tool. As an addition we > wouldn't have to retain certain characters (for creating the list). > 2) Yes, so based on 1) and this --users changes to -

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

? I believe we are deprecating ZK-based ACL > updates, so we need to add this to AdminClient? > > Regards, > > Rajini > > On Fri, Jan 17, 2020 at 3:15 PM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> > wrote: > > > Hi Jun & Richard, > > > >

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

uot;0-2". > > > > 101. The option --users "owner1,owner2" in AclCommand. Since SSL user > name > > can include comma, perhaps we could use semicolon as the separator. > > > > Jun > > > > On Wed, Jan 15, 2020 at 2:11 AM Viktor Somogyi-Vass < &g

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

Congrats to you guys, it's a great accomplishment! :) On Wed, Jan 15, 2020 at 10:20 AM David Jacot wrote: > Congrats! > > On Wed, Jan 15, 2020 at 12:00 AM James Cheng wrote: > > > Congrats Colin, Vahid, and Manikumar! > > > > -James > > > > > On Jan 14, 2020, at 10:59 AM, Tom Bentley wrote: >

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

Hey folks, bumping this again as KIP freeze is nearing and I hope to get this into the next release. We need only one binding vote. Thanks, Viktor On Thu, Jan 9, 2020 at 1:56 PM Viktor Somogyi-Vass wrote: > Bumping this in the hope of a vote or additional feedback. > > Viktor > >

Re: [VOTE] KIP-555: An admin tools proposal to accomplish the deprecation of zookeeper access that is direct

+1 (non-binding) Viktor On Wed, Jan 15, 2020 at 10:27 AM Manikumar wrote: > +1 (binding) > > Thanks for the KIP. > > On Wed, Jan 15, 2020 at 2:48 AM Gwen Shapira wrote: > > > +1 (binding, re-vote) > > > > On Tue, Jan 14, 2020 at 11:23 AM Colin McCabe > wrote: > > > > > > Hi all, > > > > > >

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

Bumping this in the hope of a vote or additional feedback. Viktor On Tue, Dec 3, 2019 at 1:07 PM Viktor Somogyi-Vass wrote: > Hi Folks, > > I'd like to bump this once more in the hope of a binding vote or any > additional feedback. > > Thanks, > Viktor > > On F

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

the throttle and further limit the non-ISR "original" replicas from > catching up? > > Best, > Stanislav > > On Tue, Dec 10, 2019 at 9:09 AM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> > wrote: > > > This config will only be applied to those replica

Re: [DISCUSS] KIP-435: Internal Partition Reassignment Batching

revision. There will probably be pain points with the new API, but > it's not completely clear what they will be yet. > > best, > Colin > > > On Fri, Dec 6, 2019, at 08:55, Viktor Somogyi-Vass wrote: > > Hi Colin, > > > > Thanks for the honest answer. As a bottom l

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

ne > could say that it's a configuration error. > > Do we have specific scenarios that would be solved by the proposed change? > > Ismael > > On Fri, Dec 6, 2019 at 2:26 AM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> > wrote: > > > Thanks for

Re: [DISCUSS] KIP-435: Internal Partition Reassignment Batching

signed, then add it to the NR > >3. If size(NR) < min.insync.replicas then take > min(min.insync.replicas, > >reassignment.parallel.replica.count) - size(NR) replicas from FTR > >4. Otherwise take as many replica as > >*reassignment.parallel.replica.count* allows &

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

Dec 4, 2019, 4:31 AM Viktor Somogyi-Vass > wrote: > > > Thanks for the notice Ismael, KAFKA-4313 fixed this issue indeed. I've > > updated the KIP. > > > > Viktor > > > > On Tue, Dec 3, 2019 at 3:28 PM Ismael Juma wrote: > > > > > Hi Viktor

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

ffic is not throttled. > > Ismael > > On Thu, Oct 24, 2019 at 5:38 AM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> > wrote: > > > Hi People, > > > > I've created a KIP to improve replication quotas by handling reassignment > > related throttling a

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

ation. But that assumes >> that we would set the replication.throttled.rate much lower than what the >> broker could handle. >> >> Perhaps the best approach would be to denote how much the broker can >> handle >> (total.replication.throttle.rate) and then allow only up

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

Hi Folks, I'd like to bump this once more in the hope of a binding vote or any additional feedback. Thanks, Viktor On Fri, Oct 25, 2019 at 2:24 PM Viktor Somogyi-Vass wrote: > Hi All, > > Would like to bump this in the hope of one binding vote (or any additional > feedback)

Re: [DISCUSS] KIP-548 Add Option to enforce rack-aware custom partition reassignment execution

Hi Satish, Couple of questions/suggestions: 1. You say that when you execute the planned reassignment then it would throw an error if the generated reassignment doesn't comply with the rack-aware requirement. Opposed to this: why don't you have the --generate option to generate a rack-aware

[jira] [Created] (KAFKA-9167) Implement a broker to controller request channel

Viktor Somogyi-Vass created KAFKA-9167: -- Summary: Implement a broker to controller request channel Key: KAFKA-9167 URL: https://issues.apache.org/jira/browse/KAFKA-9167 Project: Kafka

[jira] [Created] (KAFKA-9166) Implement MetadataFetch API

Viktor Somogyi-Vass created KAFKA-9166: -- Summary: Implement MetadataFetch API Key: KAFKA-9166 URL: https://issues.apache.org/jira/browse/KAFKA-9166 Project: Kafka Issue Type: Sub-task

Re: [ANNOUNCE] New committer: Mickael Maison

Congrats Mickael!! :) On Fri, Nov 8, 2019 at 1:24 PM Satish Duggana wrote: > Congratulations Mickael!! > > On Fri, Nov 8, 2019 at 2:50 PM Rajini Sivaram > wrote: > > > > Congratulations, Mickael, well deserved!! > > > > Regards, > > > > Rajini > > > > On Fri, Nov 8, 2019 at 9:08 AM David Jacot

[jira] [Created] (KAFKA-9154) ProducerId generation should be managed by the Controller

Viktor Somogyi-Vass created KAFKA-9154: -- Summary: ProducerId generation should be managed by the Controller Key: KAFKA-9154 URL: https://issues.apache.org/jira/browse/KAFKA-9154 Project: Kafka

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

Hi Elmahdi, I've added the JIRA to the KIP (and also below) where you can track the progress (but more subtask will come as the current ones don't represent the full work to be done). https://issues.apache.org/jira/browse/KAFKA-9119 Viktor On Tue, Nov 5, 2019 at 5:15 PM Elmahdi FRID wrote: >

Re: [DISCUSS] KIP-544: Make metrics exposed via JMX configurable

Hi Xavier, That's certainly an option, however it does not solve the problem for our users that still rely on JMX integration to collect metrics. Absolutely. We already provide the ability to write reporter plugins via the MetricsReporter interface. And rather than building a separate HTTP

Re: [VOTE] KIP-543: Expand ConfigCommand's non-ZK functionality

> > > > best, > > Colin > > > > > > On Thu, Oct 31, 2019, at 07:20, Viktor Somogyi-Vass wrote: > > > Hi Brian, > > > > > > I was just asking it because of curiosity although I do think it would > be > > > useful but it sho

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

lication.throttle.rate) and then allow only up to N% of that go > towards reassignments (reassignment.throttled.rate) in a best-effort way > (preferring replication traffic). That sounds tricky to implement though > Interested to hear what others think > > Best, > Stanislav >

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

mpatibility - should we ensure that > the old `*.replication.throttled.rate` and `*.throttled.replicas` still > apply to reassigning traffic if set? We could have the new config take > precedence, but still preserve backwards compatibility. > > Thanks, > Stanislav > > On Thu, Oct 24, 2019 at 1:38

[jira] [Created] (KAFKA-9125) GroupMetadataManager and TransactionStateManager should use metadata cache instead of zkClient

Viktor Somogyi-Vass created KAFKA-9125: -- Summary: GroupMetadataManager and TransactionStateManager should use metadata cache instead of zkClient Key: KAFKA-9125 URL: https://issues.apache.org/jira/browse

[jira] [Reopened] (KAFKA-9124) Topic creation should be done by the controller

[ https://issues.apache.org/jira/browse/KAFKA-9124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass reopened KAFKA-9124: > Topic creation should be done by the control

[jira] [Resolved] (KAFKA-9124) Topic creation should be done by the controller

[ https://issues.apache.org/jira/browse/KAFKA-9124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass resolved KAFKA-9124. Resolution: Invalid Opened by mistake. > Topic creation should be d

[jira] [Created] (KAFKA-9124) Topic creation should be done by the controller

Viktor Somogyi-Vass created KAFKA-9124: -- Summary: Topic creation should be done by the controller Key: KAFKA-9124 URL: https://issues.apache.org/jira/browse/KAFKA-9124 Project: Kafka

Re: [VOTE] KIP-543: Expand ConfigCommand's non-ZK functionality

ter, but as of now, you're the > first to voice it. > > Thanks, > Brian > > On Wed, Oct 30, 2019 at 5:01 AM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> > wrote: > > > Hi Brian, > > > > Have you thought about having one letter abbreviations like '

[jira] [Created] (KAFKA-9118) LogDirFailureHandler shouldn't use Zookeeper

Viktor Somogyi-Vass created KAFKA-9118: -- Summary: LogDirFailureHandler shouldn't use Zookeeper Key: KAFKA-9118 URL: https://issues.apache.org/jira/browse/KAFKA-9118 Project: Kafka Issue

Re: [VOTE] KIP-543: Expand ConfigCommand's non-ZK functionality

Hi Brian, Have you thought about having one letter abbreviations like '-b' for --brokers etc.? Thanks, Viktor On Tue, Oct 29, 2019 at 5:39 PM Brian Byrne wrote: > Hello all, > > I'd like to call a vote on KIP-543: Expand ConfigCommand's non-ZK > functionality, linked here:

[jira] [Created] (KAFKA-9116) Improve navigation on the kafka site

Viktor Somogyi-Vass created KAFKA-9116: -- Summary: Improve navigation on the kafka site Key: KAFKA-9116 URL: https://issues.apache.org/jira/browse/KAFKA-9116 Project: Kafka Issue Type

Re: [DISCUSS] KIP-544: Make metrics exposed via JMX configurable

Hey Xavier, Thanks for the explanation. A follow-up question, maybe to list in the future work section as it's somewhat parallel to this KIP: have you thought about implementing a REST reporter for metrics? In my opinion it would be useful as you can write it so that you directly query the

Re: [DISCUSS] KIP-544: Make metrics exposed via JMX configurable

Hi Xavier, How would the practical application look like if this was implemented? Would monitoring agents switch between the whitelist and blacklist periodically if they wanted to monitor every metrics? I think we should make some usage recommendations. Thanks, Viktor On Sun, Oct 27, 2019 at

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

t;>> make it easier for you and other people. Looking forward to > >>> your > >>> > >> > feedback! > >>> > >> > >>>> > >>> > >> > >>>> > >>> > >>

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

Hi All, Would like to bump this in the hope of one binding vote (or any additional feedback). Thanks, Viktor On Wed, Sep 18, 2019 at 5:25 PM Viktor Somogyi-Vass wrote: > Hi All, > > Harsha, Ryanne: thanks for the vote! > > I'd like to bump this again as today is the

[DISCUSS] KIP-542: Partition Reassignment Throttling

Hi People, I've created a KIP to improve replication quotas by handling reassignment related throttling as a separate case with its own configurable limits and change the kafka-reassign-partitions tool to use these new configs going forward. Please have a look, I'd be happy to receive any

Re: [DISCUSS] KIP-506: Allow setting SCRAM password via Admin interface

tore and once the > cluster is up and running those credentials would be added to the real > store (the metadata log, or a topic) and the file deleted. I think this is > orthogonal to KIP-506 as it stands right now. > > Kind regards, > > Tom > > > On Mon, Oct 21, 20

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

to remove the remote.log.storage.enable config? Thanks, Viktor On Thu, Oct 24, 2019 at 10:37 AM Viktor Somogyi-Vass < viktorsomo...@gmail.com> wrote: > Hi Jun & Harsha, > > I think it would be beneficial to at least provide one simple reference > implementation (file s

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

> > > > > on > > > > > > access > > > > > > > > > frequency/recency? For example, when a remote segment is > > accessed, > > > > > > it could > > > > > > > > > be first fetched to disk and then read from there. I > sup

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

gt; in out of local disk , hence the reason we didn't add to alternatives > > > > considered :). > > > > > > > > > > > > > > > > Thanks, > > > > Harsha > > > > > > > > > > > > > > > > > > >

[jira] [Resolved] (KAFKA-8081) Flaky Test TopicCommandWithAdminClientTest#testDescribeUnderMinIsrPartitions

[ https://issues.apache.org/jira/browse/KAFKA-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass resolved KAFKA-8081. Assignee: Viktor Somogyi-Vass Resolution: Fixed > Flaky T

Re: [DISCUSS] KIP-506: Allow setting SCRAM password via Admin interface

Hey Tom, Bringing over the conversation from KIP-500 and also adding my observations. Let me capture them in points. 1. Perhaps I'm a bit aggressive in introducing new features but I'm not sure if we need sasl.scram.password.change.enabled. As I understand it controls whether your can use the

[jira] [Created] (KAFKA-9059) Implement MaxReassignmentLag

Viktor Somogyi-Vass created KAFKA-9059: -- Summary: Implement MaxReassignmentLag Key: KAFKA-9059 URL: https://issues.apache.org/jira/browse/KAFKA-9059 Project: Kafka Issue Type

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

der-replicated partition. Otherwise, it will > > add a new URP. > > > > best, > > Colin > > > > > > On Wed, Oct 16, 2019, at 11:26, Ismael Juma wrote: > > > If a broker dies and loses the disk, is it replication or reassignment? > > > > > >

[jira] [Created] (KAFKA-9049) TopicCommandWithAdminClientTest should use mocks

Viktor Somogyi-Vass created KAFKA-9049: -- Summary: TopicCommandWithAdminClientTest should use mocks Key: KAFKA-9049 URL: https://issues.apache.org/jira/browse/KAFKA-9049 Project: Kafka

[jira] [Created] (KAFKA-9016) Warn when log dir stopped serving replicas

Viktor Somogyi-Vass created KAFKA-9016: -- Summary: Warn when log dir stopped serving replicas Key: KAFKA-9016 URL: https://issues.apache.org/jira/browse/KAFKA-9016 Project: Kafka Issue

[jira] [Created] (KAFKA-9015) Unify metric names

Viktor Somogyi-Vass created KAFKA-9015: -- Summary: Unify metric names Key: KAFKA-9015 URL: https://issues.apache.org/jira/browse/KAFKA-9015 Project: Kafka Issue Type: Improvement

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

Hi All, During the code review it came up that we shouldn't count replication bytes together with reassignment bytes so they count to a different metrics. This is a change in the semantics of ReplicationBytesInPerSec and ReplicationBytesOutPerSec metrics but since we plan to separate reassignment

Re: Status of user Admin API for quotas

Hi Tom, Colin, I abandoned my KIP because at that time I haven't had time unfortunately to continue working on it (KIP-248) and I wanted to rework it to remove the java based client since the admin commands use Scala. Since at that time it mostly had consensus on the design I think it might be a

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

non-binding > > Ryanne > > On Mon, Sep 16, 2019, 5:11 PM Harsha Ch wrote: > > > +1 (binding). Thanks for the KIP Viktor > > > > Thanks, > > > > Harsha > > > > On Mon, Sep 16, 2019 at 3:02 AM, Viktor Somogyi-Vass < > >

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

:) ) Thanks, Viktor On Wed, Aug 21, 2019 at 1:44 PM Manikumar wrote: > Hi, > > +1 (binding). > > Thanks for the updated KIP. LGTM. > > Thanks, > Manikumar > > > > On Tue, Aug 6, 2019 at 3:14 PM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> > wrote:

Re: [DISCUSS] Apache Kafka 2.4.0 release

Hi Manikumar, Can we please also include KIP-434? https://cwiki.apache.org/confluence/display/KAFKA/KIP-434%3A+Add+Replica+Fetcher+and+Log+Cleaner+Count+Metrics It has been accepted and there is already a pull request under review. Thanks, Viktor On Fri, Sep 6, 2019 at 9:59 AM Manikumar wrote:

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

Bumping this in the hope I can get more feedback and/or votes. Thanks, Viktor On Tue, Sep 3, 2019 at 1:49 PM Gabor Somogyi wrote: > +1 (non-binding) I've had a deeper look and this would be a good addition > to Spark. > > > On Thu, Aug 15, 2019 at 6:19 PM Viktor Somogyi-Vass

Re: [DISCUSS] KIP-435: Internal Partition Reassignment Batching

Hey Folks, I think I'll open a vote early next week about this if there are no more feedback. Thanks, Viktor On Fri, Aug 9, 2019 at 5:25 PM Viktor Somogyi-Vass wrote: > Hey Stanislav, > > I reiterated on the current algorithm and indeed it would change the order > of re

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

for the extra round :). Thanks, Viktor On Wed, Aug 14, 2019 at 2:40 PM Viktor Somogyi-Vass wrote: > Sorry, reading my email the second time I probably wasn't clear. > So basically the concept is that there is a user who can add other users > as resources (such as userB and userC) prior to

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

o create "UserA can > create tokens for UserB, UserC" association. > > Thanks, > > On Tue, Aug 13, 2019 at 3:37 PM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> > wrote: > > > Hi Manikumar, > > > > Yea, I just brought u

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

t;UserA can create tokens for > any > > user". > > I am thinking, can we change the permissions like "UserA can create > tokens > > for UserB, UserC"? > > > > Thanks, > > > > > > > > > > > > > > On Fri, Aug 9, 2019 a

Re: [VOTE] KIP-497: Add inter-broker API to alter ISR

Hey Jason, +1 from me too. One note though: since it's a new protocol we could perhaps rename CurrentZkVersion to something like "IsrEpoch" or "IsrVersion". I think that'd reflect its purpose better. Best, Viktor On Wed, Aug 7, 2019 at 8:37 PM Jason Gustafson wrote: > Hi All, > > I'd like to

Re: [DISCUSS] KIP-435: Internal Partition Reassignment Batching

. Let me know what you think. Viktor On Thu, Jul 11, 2019 at 7:16 PM Viktor Somogyi-Vass wrote: > Hi Stan, > > I meant the following (maybe rare) scenario - we have replicas [1, 2, 3] on > a lot of partitions and the user runs a massive rebalance to change them > all to [3, 2,

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

ot;User". What do you think? > > > Thanks, > Manikumar > > > On Wed, Jul 31, 2019 at 2:26 PM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> > wrote: > > > Hi Folks, > > > > I'm starting a vote on this. > > > > Viktor >

Re: Alternative of poll(0) without pulling records

Hey Jungtaek, Thanks for your interest, sometimes I also think such an API would be a good thing. I don't see any strong reasons neither in KIP-288 nor in KIP-266 why such an API shouldn't be created, so go ahead with it, although you'll need to create a short KIP for this as the KafkaConsumer

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

Hi All, Bumping this, I'd be happy to get some additional feedback and/or votes. Thanks, Viktor On Wed, Jul 31, 2019 at 11:04 AM Viktor Somogyi-Vass < viktorsomo...@gmail.com> wrote: > Hi All, > > I'd like to start a vote on this KIP. > > https://cwiki.apache.org/conflue

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

Hey Colin, I think this is a long-awaited KIP, thanks for driving it. I'm excited to see this in Kafka once. I collected my questions (and I accept the "TBD" answer as they might be a bit deep for this high level :) ). 1.) Are there any specific reasons for the Controller just periodically

[VOTE] KIP-373: Allow users to create delegation tokens for other users

Hi All, I'd like to start a vote on this KIP. https://cwiki.apache.org/confluence/display/KAFKA/KIP-373%3A+Allow+users+to+create+delegation+tokens+for+other+users To summarize it: the proposed feature would allow users (usually superusers) to create delegation tokens for other users. This is

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

Hi Folks, I'm starting a vote on this. Viktor On Thu, Jun 27, 2019 at 12:02 PM Viktor Somogyi-Vass < viktorsomo...@gmail.com> wrote: > Hi Folks, > > I took over this issue from Manikumar. Recently another motivation have > been raised in Spark for this (SPARK-28173) and I t

Re: [DISCUSS] KIP-435: Internal Partition Reassignment Batching

; config has a bit > of a different meaning when used in the context of a real reassignment > (where data moves from the leader to N more replicas) versus in the > preferred election switch (where all we need is two lightweight > LeaderAndIsr requests), so I am not entirely convinced we may w

Re: [DISCUSS] KIP-435: Internal Partition Reassignment Batching

> reading, whereas some developers might want to skim through the change to > get a quick sense. It also improves readability but that's my personal > opinion. > - could you better clarify how a reassignment step is different from the > currently existing algorithm? maybe laying both

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

Hey Colin, So in my understanding this is how the controller handles a reassignment in a simple scenario: 1. When an AlterPartitionReassignmentRequest arrives it updates the partitions' ZK data in /brokers/topics/[topic]/partitions/[partitionId]/state with targetReplicas 2. Sends out LeaderAndIsr

[DISCUSS] KIP-435: Internal Partition Reassignment Batching

Hi All, I've renamed my KIP as its name was a bit confusing so we'll continue it in this thread. The previous thread for record: https://lists.apache.org/thread.html/0e97e30271f80540d4da1947bba94832639767e511a87bb2ba530fe7@%3Cdev.kafka.apache.org%3E A short summary of the KIP: In case of a vast

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

a way of knowing what target replicas are currently > being worked on. So maybe we'll have to add a field to the structures > returned by listPartitionReassignments. > > best, > Colin > > > On Wed, Jun 26, 2019, at 06:20, Viktor Somogyi-Vass wrote: > > Hey Colin, > > > &g

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

Hi Folks, I took over this issue from Manikumar. Recently another motivation have been raised in Spark for this (SPARK-28173) and I think it'd be great to continue this task. I updated the KIP and will wait for a few days to get some feedback then proceed for the vote. Thanks, Viktor On Tue,

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

> > best, > Colin > > > On Tue, Jun 25, 2019, at 08:09, Viktor Somogyi-Vass wrote: > > Hi All, > > > > I have added another improvement to this, which is to limit the parallel > > leader movements. I think I'll soon (maybe late this week or early next)

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

Hi All, I have added another improvement to this, which is to limit the parallel leader movements. I think I'll soon (maybe late this week or early next) start a vote on this too if there are no additional feedback. Thanks, Viktor On Mon, Apr 29, 2019 at 1:26 PM Viktor Somogyi-Vass wrote

[jira] [Created] (KAFKA-8566) Force Topic Deletion

Viktor Somogyi-Vass created KAFKA-8566: -- Summary: Force Topic Deletion Key: KAFKA-8566 URL: https://issues.apache.org/jira/browse/KAFKA-8566 Project: Kafka Issue Type: Improvement

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

e: > > > > > Thanks Viktor, proposed metrics are really useful to monitor > > replication > > > status on brokers. > > > > > > +1 (non-binding) > > > > > > On Thu, Jun 6, 2019 at 2:05 AM Colin McCabe > > wrote: > &

<    1   2   3   >