Jenkins build is back to normal : kafka-1.1-jdk7 #245

See

Build failed in Jenkins: kafka-2.0-jdk8 #229

See Changes: [jason] KAFKA-7959; Delete leader epoch cache files with old message format -- [...truncated 2.50 MB...] org.apache.kafka.streams.TopologyTest > shouldNotAddNull

Re: [VOTE] KIP-430 - Return Authorized Operations in Describe Responses

Hi Rajini, Thanks for the explanations. On Fri, Feb 22, 2019, at 11:59, Rajini Sivaram wrote: > Hi Colin, > > Thanks for the review. Sorry I meant that an array of INT8's, each of which > is an AclOperation code will be returned. I have clarified that in the KIP. Do you think it's worth conside

Jenkins build is back to normal : kafka-2.2-jdk8 #29

See

[jira] [Resolved] (KAFKA-7959) Clear/delete epoch cache if old message format is in use

[ https://issues.apache.org/jira/browse/KAFKA-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7959. Resolution: Fixed Fix Version/s: 2.0.2 > Clear/delete epoch cache if old message for

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Colin, CIL, Thanks! Yaodong On Fri, Feb 22, 2019 at 10:56 AM Colin McCabe wrote: > Hi Yaodong, > > KIP-422 says that it would be good if "applications [could] leverage the > unified KafkaAdminClient to manage their user/client configurations, > instead of the direct dependency on Zookeeper

Re: [DISCUSS] KIP-433: Provide client API version to authorizer

Hi Gwen, Thank you for the quick feedback! It's a good point that broker configuration can be dynamic and is more convenient. Technically, anything inside the authorizer can also be dynamic. For example, the SimpleAclAuthorizer in Kafka stores ACLs in Zookeeper, which can be dynamically changed w

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

Love it. Thanks. On Mon, Feb 18, 2019 at 2:00 PM Mateusz Zakarczemny wrote: > > Hi all, > > I have created a KIP to support additional message fields in console > consumer: > KIP-431 - Support of printing additional ConsumerRecord fields in > DefaultMessageFormatter >

Re: [DISCUSS] KIP-433: Provide client API version to authorizer

Link, for convenience: https://cwiki.apache.org/confluence/display/KAFKA/KIP-433%3A+Provide+client+API+version+to+authorizer I actually prefer the first rejected alternative (add a configuration). While you are right that configuration is inherently less flexible, putting the logic in the authoriz

[DISCUSS] KIP-433: Provide client API version to authorizer

Build failed in Jenkins: kafka-trunk-jdk8 #3409

See Changes: [github] KAFKA-7672: Restoring tasks need to be closed upon task suspension -- [...truncated 2.31 MB...] org.apache.kafka.streams.test.OutputVerifierTest > s

Jenkins build is back to normal : kafka-trunk-jdk11 #308

See

[jira] [Created] (KAFKA-7986) distinguish the logging from different ZooKeeperClient instances

Jun Rao created KAFKA-7986: -- Summary: distinguish the logging from different ZooKeeperClient instances Key: KAFKA-7986 URL: https://issues.apache.org/jira/browse/KAFKA-7986 Project: Kafka Issue Typ

[jira] [Created] (KAFKA-7987) a broker's ZK session may die on transient auth failure

Jun Rao created KAFKA-7987: -- Summary: a broker's ZK session may die on transient auth failure Key: KAFKA-7987 URL: https://issues.apache.org/jira/browse/KAFKA-7987 Project: Kafka Issue Type: Improve

Re: [VOTE] KIP-430 - Return Authorized Operations in Describe Responses

Hi Colin, Thanks for the review. Sorry I meant that an array of INT8's, each of which is an AclOperation code will be returned. I have clarified that in the KIP. All permitted operations will be returned from the set of supported operations on each resource. This is regardless of whether the acce

Build failed in Jenkins: kafka-trunk-jdk8 #3408

See Changes: [bbejeck] KAFKA-7492 : Updated javadocs for aggregate and reduce methods returning [junrao] KAFKA-7864; validate partitions are 0-based (#6246) -- [...trunc

[jira] [Created] (KAFKA-7985) Cleanup AssignedTasks / AbstractTask logic

Guozhang Wang created KAFKA-7985: Summary: Cleanup AssignedTasks / AbstractTask logic Key: KAFKA-7985 URL: https://issues.apache.org/jira/browse/KAFKA-7985 Project: Kafka Issue Type: Improvem

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Yaodong, KIP-422 says that it would be good if "applications [could] leverage the unified KafkaAdminClient to manage their user/client configurations, instead of the direct dependency on Zookeeper." But the KIP doesn't talk about any changes to KafkaAdminClient. Instead, the only changes p

Re: [VOTE] KIP-430 - Return Authorized Operations in Describe Responses

Hi Rajini, Thanks for the KIP! The KIP specifies that "Authorized operations will be returned as [an] INT8 consistent with [the] AclOperation used in ACL requests and responses." But there may be more than one AclOperation that is applied to a given resource. For example, a principal may hav

[jira] [Created] (KAFKA-7984) Do not rebuild leader epochs on segments that do not support it

Stanislav Kozlovski created KAFKA-7984: -- Summary: Do not rebuild leader epochs on segments that do not support it Key: KAFKA-7984 URL: https://issues.apache.org/jira/browse/KAFKA-7984 Project: Ka

[jira] [Created] (KAFKA-7983) supporting replication.throttled.replicas in dynamic broker configuration

Jun Rao created KAFKA-7983: -- Summary: supporting replication.throttled.replicas in dynamic broker configuration Key: KAFKA-7983 URL: https://issues.apache.org/jira/browse/KAFKA-7983 Project: Kafka

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

Hi Viktor,  Thanks for reading and provide feedbacks on KIP-236.  For reassignments, one can generate a json for new assignments and another json with "original" assignments for rollback purpose.  In production cluster, from our experience, we need to submit the reassignments in batches with

[jira] [Resolved] (KAFKA-7492) Explain `null` handling for reduce and aggregate

[ https://issues.apache.org/jira/browse/KAFKA-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck resolved KAFKA-7492. Resolution: Fixed Thanks asutosh for the contribution! > Explain `null` handling for reduce and ag

[jira] [Resolved] (KAFKA-7864) AdminZkClient.validateTopicCreate() should validate that partitions are 0-based

[ https://issues.apache.org/jira/browse/KAFKA-7864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-7864. Resolution: Fixed Fix Version/s: 2.3.0 Merged the PR to trunk. > AdminZkClient.validateTopicCreate(

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

Hi Viktor, There is already KIP-236 for the same feature and George made a PR for this as well. Lets consolidate these two discussions. If you have any cases that are not being solved by KIP-236 can you please mention them in that thread. We can address as part of KIP-236. Thanks, H

Re: [DISCUSS] KIP-434: Add Replica Fetcher and Log Cleaner Count Metrics

Hey Viktor, First off, thanks for the KIP! I think that it is almost always a good idea to have more metrics. Observability never hurts. In regards to the LogCleaner: * Do we need to know log-cleaner-thread-count? That should always be equal to "log.cleaner.threads" if I'm not mistaken. * log-cle

Re: [DISCUSS] KIP-432: Additional Broker-Side Opt-In for Default, Unsecure SASL/OAUTHBEARER Implementation

That's an interesting approach - it looks good to me at first glance. Should we consider creating a new KIP to initiate a proper discussion? People might miss the discussion here due to the KIP title On Thu, Feb 21, 2019 at 6:52 PM Ron Dagostino wrote: > My gut says a building block that takes a

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

Read through the KIP and I have one comment: It seems like it is not looking strictly for cancellation but also implements rolling back to the original. I think it'd be much simpler to generate a reassignment json on cancellation that contains the original assignment and start a new partition reas

[DISCUSS] KIP-435: Incremental Partition Reassignment

Hi Folks, I've created a KIP about an improvement of the reassignment algorithm we have. It aims to enable partition-wise incremental reassignment. The motivation for this is to avoid excess load that the current replication algorithm implicitly carries as in that case there are points in the algo

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

I've published the above mentioned KIP here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-435%3A+Incremental+Partition+Reassignment Will start a discussion about it soon. On Fri, Feb 22, 2019 at 12:45 PM Viktor Somogyi-Vass < viktorsomo...@gmail.com> wrote: > Hi Folks, > > I also have a

[jira] [Created] (KAFKA-7982) ConcurrentModificationException and Continuous warnings "Attempting to send response via channel for which there is no open connection"

Abhi created KAFKA-7982: --- Summary: ConcurrentModificationException and Continuous warnings "Attempting to send response via channel for which there is no open connection" Key: KAFKA-7982 URL: https://issues.apache.org/jira/

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

Hi Folks, I also have a pending active work on the incremental partition reassignment stuff here: https://issues.apache.org/jira/browse/KAFKA-6794 I think it would be good to cooperate on this to make both work compatible with each other. I'll write up a KIP about this today so it'll be easier to

[DISCUSS] KIP-434: Add Replica Fetcher and Log Cleaner Count Metrics

Hi All, I'd like to start a discussion about exposing count gauge metrics for the replica fetcher and log cleaner thread counts. It isn't a long KIP and the motivation is very simple: monitoring the thread counts in these cases would help with the investigation of various issues and might help in

[jira] [Created] (KAFKA-7981) Add Replica Fetcher and Log Cleaner Count Metrics

Viktor Somogyi-Vass created KAFKA-7981: -- Summary: Add Replica Fetcher and Log Cleaner Count Metrics Key: KAFKA-7981 URL: https://issues.apache.org/jira/browse/KAFKA-7981 Project: Kafka I