Build failed in Jenkins: kafka-trunk-jdk8 #3726

2019-06-14 Thread Apache Jenkins Server
See Changes: [github] KAFKA-8179: Part I, Bump up consumer protocol to v2 (#6528) -- [...truncated 5.03 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

2019-06-14 Thread Boyang Chen
Hey Mike, thanks for asking! We have shipped 85% of the features in 345 already, the remaining works are just tooling supports: https://issues.apache.org/jira/browse/KAFKA-7018 you should be able to use it in 2.3  Boyang From: Mike Freyberger Sent: Friday,

Build failed in Jenkins: kafka-trunk-jdk8 #3725

2019-06-14 Thread Apache Jenkins Server
See Changes: [github] KAFKA-8530; Check for topic authorization errors in OffsetFetch response [cmccabe] KAFKA-8488: Reduce logging-related string allocation in --

Re: [DISCUSS] KIP-470: TopologyTestDriver test input and output usability improvements

2019-06-14 Thread John Roesler
Sounds good. Thanks as always for considering my feedback! -John On Fri, Jun 14, 2019 at 12:12 PM Jukka Karvanen wrote: > > Ok, I will modify KIP Public Interface in a wiki based on the feedback. > > TestRecordFactory / ConsumerRecordFactory was used by TestInputTopic with > the version I had

Jenkins build is back to normal : kafka-trunk-jdk8 #3724

2019-06-14 Thread Apache Jenkins Server
See

Jenkins build is back to normal : kafka-2.2-jdk8 #138

2019-06-14 Thread Apache Jenkins Server
See

[jira] [Created] (KAFKA-8541) Flakey test LeaderElectionCommandTest#testTopicPartition

2019-06-14 Thread Boyang Chen (JIRA)
Boyang Chen created KAFKA-8541: -- Summary: Flakey test LeaderElectionCommandTest#testTopicPartition Key: KAFKA-8541 URL: https://issues.apache.org/jira/browse/KAFKA-8541 Project: Kafka Issue

Re: Kafka streams issue

2019-06-14 Thread Matthias J. Sax
It is correct, that Kafka Streams does not create input/output/through() topics. You should create those topics upfront. > We've *set auto.create.topics.enable=true *in our kafka >> server properties file and we'd expect that the topics get created, but the >> consumer goes into SHUTDOWN mode and

[jira] [Created] (KAFKA-8540) Improve InsertField connect transform to allow inserting processed timestamp

2019-06-14 Thread Khushal Kumthekar (JIRA)
Khushal Kumthekar created KAFKA-8540: Summary: Improve InsertField connect transform to allow inserting processed timestamp Key: KAFKA-8540 URL: https://issues.apache.org/jira/browse/KAFKA-8540

Re: [DISCUSS] KIP-470: TopologyTestDriver test input and output usability improvements

2019-06-14 Thread Jukka Karvanen
Ok, I will modify KIP Public Interface in a wiki based on the feedback. TestRecordFactory / ConsumerRecordFactory was used by TestInputTopic with the version I had with KIP456, but maybe I can merge That functionality to InputTopic or TestRecordFactory can kept non public maybe moving it to

Re: Posted a new article about Kafka Streams

2019-06-14 Thread Development
Bad link: https://medium.com/@daniyaryeralin/utilizing-kafka-streams-processor-api-and-implementing-custom-aggregator-6cb23d00eaa7 > On Jun 14, 2019, at 11:07 AM, Development wrote: > > Hello Kafka Dev community, > > I wrote an article on implementing a custom transformer using Processor API

Re: [DISCUSS] KIP-470: TopologyTestDriver test input and output usability improvements

2019-06-14 Thread John Roesler
Thanks, Jukka, Ok, I buy this reasoning. Just to echo what I think I read, you would just drop ClientRecord from the proposal, and TestRecord would stand on its own, with the same methods and properties you proposed, and the "input topic" would accept TestRecord, and the "output topic" would

[jira] [Resolved] (KAFKA-8530) Consumer should handle authorization errors in OffsetFetch

2019-06-14 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8530. Resolution: Fixed Fix Version/s: 2.4.0 > Consumer should handle authorization

Posted a new article about Kafka Streams

2019-06-14 Thread Development
Hello Kafka Dev community, I wrote an article on implementing a custom transformer using Processor API for Kafka Streams! https://medium.com/@daniyaryeralin/utilizing-kafka-streams-processor-api-and-implementing-custom-aggregation-f6a4a6c376be

[jira] [Resolved] (KAFKA-6874) Add Configuration Allowing for Optional Topology Optimization

2019-06-14 Thread Bill Bejeck (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck resolved KAFKA-6874. Resolution: Duplicate Fix Version/s: 2.0.0 Duplicate of 

Re: [DISCUSS] KIP-471: Expose RocksDB Metrics in Kafka Streams

2019-06-14 Thread Bruno Cadonna
Hi, I decided to go for the option in which metrics are exposed for each logical state store. I revisited the KIP correspondingly and added a section on how to aggregate metrics over multiple physical RocksDB instances within one logical state store. Would be great, if you could take a look and

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

2019-06-14 Thread Mike Freyberger
Hi Boyang, Thanks for the update. To what extent will KIP-345 be available in 2.3.0? Mike On 6/13/19, 5:36 PM, "Boyang Chen" wrote: Hey all, we decided to push 2 minor changes for better usability for static membership. 1. Add `group.instance.id` field to the

Re: [DISCUSS] KIP-470: TopologyTestDriver test input and output usability improvements

2019-06-14 Thread Jukka Karvanen
Hi, I am not a fan of swapping only ProducerRecord and ConsumerRecord. As a test writer point of view I do not want to care about the difference of those and that way I like to have object type which can be used to pipe records in and compare outputs. That way avoid unnecessary conversions

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-06-14 Thread Viktor Somogyi-Vass
Hi All, It's been a few days since the last vote and we have three binding votes, so the vote passed. Thank you all who voted and participated in the discussions, I'd be excited to see this in the codebase! Binding: Jason, Harsha, Colin Non-binding: Dhruvil, Stanislav, Satish, Ryanne, Andrew,