[jira] [Created] (KAFKA-9907) Switch default build to Scala 2.13

Ismael Juma created KAFKA-9907: -- Summary: Switch default build to Scala 2.13 Key: KAFKA-9907 URL: https://issues.apache.org/jira/browse/KAFKA-9907 Project: Kafka Issue Type: Improvement

Build failed in Jenkins: kafka-trunk-jdk11 #1378

See Changes: [github] KAFKA-9388: Refactor integration tests to always use different -- [...truncated 3.03 MB...] org.apache.kafka.streams.TopologyTestDriverTest > shoul

Jenkins build is back to normal : kafka-trunk-jdk8 #4458

See

Jenkins build is back to normal : kafka-trunk-jdk11 #1377

See

[jira] [Created] (KAFKA-9906) Is bytesSinceLastIndexEntry updated correct when LogSegment.append() is called ?

Xiang Zhang created KAFKA-9906: -- Summary: Is bytesSinceLastIndexEntry updated correct when LogSegment.append() is called ? Key: KAFKA-9906 URL: https://issues.apache.org/jira/browse/KAFKA-9906 Project: K

Re: [DISCUSSION] KIP-418: A method-chaining way to branch KStream

Hi, I have read the John's "DSL design principles" and have completely rewritten the KIP, see https://cwiki.apache.org/confluence/display/KAFKA/KIP-418%3A+A+method-chaining+way+to+branch+KStream This version includes all the previous discussion results and follows the design principles, wit

Build failed in Jenkins: kafka-trunk-jdk8 #4457

See Changes: [github] KAFKA-9863: replace the deprecated --zookeeper options in the -- [...truncated 3.01 MB...] org.apache.kafka.streams.TopologyTestDriverTest > should

[jira] [Created] (KAFKA-9905) The equals functions for generated classes should compare all fields

Colin McCabe created KAFKA-9905: --- Summary: The equals functions for generated classes should compare all fields Key: KAFKA-9905 URL: https://issues.apache.org/jira/browse/KAFKA-9905 Project: Kafka

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Hi Colin, thanks for your summary! Just one question - and I may be missing an obvious point here.. You write: "The initial broker should do authentication (who are you?) and come up with a principal name. Then it creates an envelope request, which will contain that principal name, and sends it

[jira] [Resolved] (KAFKA-9388) Flaky Test StandbyTaskCreationIntegrationTest.shouldCreateStandByTasksForMaterializedAndOptimizedSourceTables

[ https://issues.apache.org/jira/browse/KAFKA-9388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-9388. -- Fix Version/s: 2.6.0 Assignee: Guozhang Wang Resolution: Fixed > Flaky Test >

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Colin, Boyang: thanks for the updates, I agree that an EnvelopeRequest would be a less vulnerable approach than optional fields, and I'm just wondering if we would keep the EnvelopeRequest for a long time. I was thinking that, potentially if we require clients to be on newer version when talking to

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Thanks Colin for the summary! And Guozhang, regarding the future use cases, consider a scenario where there are temporary connectivity issue between controller to a fellow broker A, the controller could then leverage another healthy broker B to do a forwarding request to A in order to maintain a co

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Hi all, I guess the way I see this working is that the request gets sent from the client, to the initial broker, and then forwarded to the final broker. The initial broker should do authentication (who are you?) and come up with a principal name. Then it creates an envelope request, which will

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Hello Gwen, The purpose here is for maintaining compatibility old clients, who do not have functionality to do re-routing admin requests themselves. New clients can of course do this themselves by detecting who's the controller. Hello Colin / Boyang, Regarding the usage of the envelope, I'm cur

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

Hey Jason, thanks for the suggestions! Addressed in the KIP. On Wed, Apr 22, 2020 at 9:21 AM Jason Gustafson wrote: > +1 Just a couple small comments: > > 1. My comment about `initTransactions()` usage in the javadoc above appears > not to have been addressed. > 2. For the handling of INVALID_P

Clarification regarding multi topics implementation

Hi Team, Greetings. I have a use-case wherein I have to consume messages from multiple topics using Kafka and process it using Kafka Streams, then publish the message to multiple target topics. The example is below. Source topic A - process A - target topic A Source topic B - process B - targe

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

+1 Just a couple small comments: 1. My comment about `initTransactions()` usage in the javadoc above appears not to have been addressed. 2. For the handling of INVALID_PRODUCER_EPOCH in the produce response, would we only try to abort if the broker supports the newer protocol version? I guess it w

Re: Rocksdb Statistics

Thanks Bruno. On 2020/04/22 15:58:50, Bruno Cadonna wrote: > Hi Nagendra,> > > What you describe is a known issue:> > https://issues.apache.org/jira/browse/KAFKA-9675> > > And you also described the appropriate fix:> > https://github.com/apache/kafka/pull/8256> > > The issue is fixed an

Re: Rocksdb Statistics

Hi Nagendra, What you describe is a known issue: https://issues.apache.org/jira/browse/KAFKA-9675 And you also described the appropriate fix: https://github.com/apache/kafka/pull/8256 The issue is fixed and will be included in 2.6.0, 2.4.2, 2.5.1. I am sorry for any inconveniences. Best, Bruno

Rocksdb Statistics

I am using kafka-streams-2.4.0. My issue details : metrics.recording.level is set to DEBUG Still all the stream-state-metrics values recorded are all 0. (Ex: bytes-written-total etc.,) Changed the code in RocksDBStore class openDB method. Moved the call to maybeSetupMetricsRecorder b

[jira] [Created] (KAFKA-9904) Use ThreadLocalConcurrent to Replace Random

David Mollitor created KAFKA-9904: - Summary: Use ThreadLocalConcurrent to Replace Random Key: KAFKA-9904 URL: https://issues.apache.org/jira/browse/KAFKA-9904 Project: Kafka Issue Type: Impro

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Hey Boyang, Sorry if this was already discussed, but I didn't see this as rejected alternative: Until now, we always did client side routing - the client itself found the controller via metadata and directed requests accordingly. Brokers that were not the controller, rejected those requests. Why

Re: Retention policies question

Kafka purges old messages from both leaders and replicas. If there was a mistake in the book, can you tell me which chapter and page? We'll fix it. Gwen On Wed, Apr 22, 2020, 7:51 AM Alex Bull wrote: > Hi, Dear Kafka Developers, > > I've read 'Kafka: The Definitive Guide' by Narkhede and other

Retention policies question

Hi, Dear Kafka Developers, I've read 'Kafka: The Definitive Guide' by Narkhede and others and I have a following question. On what side topic retention policies (delete or compact) are performed? I have a guess that they work only on brokers that hold leader replica of partition. Or am I wrong ?

[jira] [Created] (KAFKA-9903) kafka ShutdownableThread judge thread isRuning status has some bug

shilin Lu created KAFKA-9903: Summary: kafka ShutdownableThread judge thread isRuning status has some bug Key: KAFKA-9903 URL: https://issues.apache.org/jira/browse/KAFKA-9903 Project: Kafka Is

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

We do use custom principals that rely on information not contained in the serialized principal and hence authorization based on serialized principals can break existing production systems. Apart from the information contained in the session principal, ACLs can also be based on host I, but I guess t

[jira] [Created] (KAFKA-9902) java client api can not completely take out the kafka-consumer-groups.sh output of information

startjava created KAFKA-9902: Summary: java client api can not completely take out the kafka-consumer-groups.sh output of information Key: KAFKA-9902 URL: https://issues.apache.org/jira/browse/KAFKA-9902

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Hi Boyang and Sönke, Regarding custom Principals, I don't think too many people do this in > practice, but in theory you can provide you own PrincipalBuilder and use > your own Principal objects that contain as much additional information as > you wish. And since these can basically be any Java ob