Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

On a side note, I have noticed that the several other methods in classes such as StoreChangeLogReader in Streams calls position() which causes tests to hang. It might be out of the scope of the KIP, but should I also change the methods which use position() as a callback to at the very least prevent

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

Thanks for the review Becket. About the methods beginningOffsets(), endOffsets(), ...: I took a look through the code of KafkaConsumer, but after looking through the offsetsByTimes() method and its callbacks in Fetcher, I think these methods already block for a set period of time. I know that ther

Re: [VOTE] KIP-272: Add API version tag to broker's RequestsPerSec metric

+1 (binding) The incompatibility is unfortunate, but seems unlikely to cause a problem in practice. Let's just make sure there's a note in the upgrade notes about the incompatibility when we have a PR for this. -Ewen On Fri, Mar 30, 2018 at 10:22 AM, Jun Rao wrote: > Hi, Allen, > > Thanks for

[jira] [Reopened] (KAFKA-6390) Update ZooKeeper to 3.4.11, Gradle and other minor updates

[ https://issues.apache.org/jira/browse/KAFKA-6390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reopened KAFKA-6390: > Update ZooKeeper to 3.4.11, Gradle and other minor updates > -

[jira] [Created] (KAFKA-6730) Simplify state store recovery

Matthias J. Sax created KAFKA-6730: -- Summary: Simplify state store recovery Key: KAFKA-6730 URL: https://issues.apache.org/jira/browse/KAFKA-6730 Project: Kafka Issue Type: Improvement

[jira] [Created] (KAFKA-6729) KTable should use user source topics if possible and not create changelog topic

Matthias J. Sax created KAFKA-6729: -- Summary: KTable should use user source topics if possible and not create changelog topic Key: KAFKA-6729 URL: https://issues.apache.org/jira/browse/KAFKA-6729 Pro

Re: [DISCUSS] KIP-274: Kafka Streams Skipped Records Metrics

Allrighty! The KIP is updated. Thanks again, all, for the feedback. -John On Fri, Mar 30, 2018 at 3:35 PM, John Roesler wrote: > Hey Guozhang and Bill, > > Ok, I'll update the KIP. At the risk of disturbing consensus, I'd like to > put it in the task instead of the thread so that it'll show up

Re: [DISCUSS] KIP-274: Kafka Streams Skipped Records Metrics

Hey Guozhang and Bill, Ok, I'll update the KIP. At the risk of disturbing consensus, I'd like to put it in the task instead of the thread so that it'll show up in the TopologyTestDriver metrics as well. I'm leaning toward keeping the scope where it is right now, but if others want to advocate for

Re: [DISCUSS] KIP-273 Kafka to support using ETCD beside Zookeeper

Hi, I had an offline discussion with Ismael and wanted to summarize the comments and questions he raised so we are all on the same page. The core issue is that this change adds a new public API. Since we already know that the goal for the next 1-2 years is to get rid of ZK completely. Do we want

Re: [DISCUSS] KIP-274: Kafka Streams Skipped Records Metrics

Thanks for the KIP John, and sorry for the late comments. I'm on the fence with providing a single level metrics, but I think we'll have that discussion outside of this KIP. > * maintain one skipped-record metric (could be per-thread, per-task, or > per-processor-node) with no "reason" > * introd

Re: [DISCUSS] KIP-274: Kafka Streams Skipped Records Metrics

The proposal sounds good to me. About "maintain only one level of metrics" maybe we can discuss about that separately from this KIP since that would be a larger scope of discussion. I agree that if we are going to maintain only one-level metrics it should be lowest level and we would let users to d

Re: [VOTE] KIP-272: Add API version tag to broker's RequestsPerSec metric

Hi, Allen, Thanks for the explanation. That's a great point. Could you add that to the rejected section of the KIP. +1 on this KIP. Jun On Fri, Mar 30, 2018 at 10:06 AM, Allen Wang wrote: > Hi Jun, > > I think this KIP helps to gain insight into how many clients have been > upgraded *before*

Re: [VOTE] KIP-272: Add API version tag to broker's RequestsPerSec metric

Hi Jun, I think this KIP helps to gain insight into how many clients have been upgraded *before* the we decide to upgrade the message format. In other words, users will know if they will pay down conversion cost before message format upgrade. It is about making an informed decision. To be conserva

Re: [DISCUSS] KIP-274: Kafka Streams Skipped Records Metrics

Hey Guozhang, Thanks for the reply. Regarding JMX, I can dig it. I'll provide a list in the KIP. I was also thinking we'd better start a documentation page with the metrics listed. I'd have no problem logging a warning when we skip records. On the metric front, really I'm just pushing for us to m

[jira] [Created] (KAFKA-6728) Kafka Connect Header Null Pointer Exception

Philippe Hong created KAFKA-6728: Summary: Kafka Connect Header Null Pointer Exception Key: KAFKA-6728 URL: https://issues.apache.org/jira/browse/KAFKA-6728 Project: Kafka Issue Type: Bug

Re: Expiring record(s) for topic: 30011 ms has passed since last append

Opened a new thread about this: https://lists.apache.org/thread.html/de97ef6434169336d90017db6e04238ef6805eb9658faad114630550@%3Cdev.kafka.apache.org%3E

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

Glad to see the KIP, Richard. This has been a really long pending issue. The original arguments from Jay for using config, such as max.block.ms, instead of using timeout parameters was that people will always hard code the timeout, and the hard coded timeout is rarely correct because it has to con

Build failed in Jenkins: kafka-trunk-jdk7 #3301

See Changes: [junrao] KAFKA-6630: Speed up the processing of -- [...truncated 1.93 MB...] org.apache.kafka.streams.processor.internals.assignment.StickyTaskAssignorTest >