Possible bug: KafkaConsumer.endOffsets can return partial data

Hello. When migrating to 1.1.0 I faced with changed behavior: KafkaConsumer.endOffsets can return partial data. As I see it was introduced with KAFKA-6397 (Consumer should not block setting initial positions of unavailable partitions). There is strange code at

[jira] [Created] (KAFKA-6861) Missing ')' in Kafka Streams documentation

Ravi Raj Singh created KAFKA-6861: - Summary: Missing ')' in Kafka Streams documentation Key: KAFKA-6861 URL: https://issues.apache.org/jira/browse/KAFKA-6861 Project: Kafka Issue Type: Bug

[VOTE] KIP-255: OAuth Authentication via SASL/OAUTHBEARER

Hi everyone. I would like to start the vote for KIP-255: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75968876 This KIP proposes to add the following functionality related to SASL/OAUTHBEARER: 1) Allow clients (both brokers when SASL/OAUTHBEARER is the inter-broker protocol

[jira] [Created] (KAFKA-6860) missing creating checkpoint when reinitializeStateStores with eosEnabled(true)

ko byoung kwon created KAFKA-6860: - Summary: missing creating checkpoint when reinitializeStateStores with eosEnabled(true) Key: KAFKA-6860 URL: https://issues.apache.org/jira/browse/KAFKA-6860

[jira] [Created] (KAFKA-6859) Follower should not send OffsetForLeaderEpoch for undefined leader epochs

Anna Povzner created KAFKA-6859: --- Summary: Follower should not send OffsetForLeaderEpoch for undefined leader epochs Key: KAFKA-6859 URL: https://issues.apache.org/jira/browse/KAFKA-6859 Project: Kafka

[jira] [Created] (KAFKA-6858) Log.truncateTo() may truncate to an earlier offset than requested

Jun Rao created KAFKA-6858: -- Summary: Log.truncateTo() may truncate to an earlier offset than requested Key: KAFKA-6858 URL: https://issues.apache.org/jira/browse/KAFKA-6858 Project: Kafka Issue

[jira] [Created] (KAFKA-6857) LeaderEpochFileCache.endOffsetFor() should check for UNDEFINED_EPOCH explicitly

Jun Rao created KAFKA-6857: -- Summary: LeaderEpochFileCache.endOffsetFor() should check for UNDEFINED_EPOCH explicitly Key: KAFKA-6857 URL: https://issues.apache.org/jira/browse/KAFKA-6857 Project: Kafka

Re: KIP-244: Add Record Header support to Kafka Streams

Thanks Guozhang and John for your feedback. > 1. We need to have a clear inheritance protocol of headers in our topology: > 1.a. In PAPI's context.forward() call, it should be straight-forward. > 1.b. In DSL stateless operators, it should be straight-forward. > 1.c. What about in stateful

Build failed in Jenkins: kafka-1.1-jdk7 #125

See Changes: [jason] KAFKA-6854; Handle batches deleted during log cleaning of logs with txns -- [...truncated 417.96 KB...] kafka.log.LogCleanerIntegrationTest >

Build failed in Jenkins: kafka-trunk-jdk8 #2617

See Changes: [jason] KAFKA-6854; Handle batches deleted during log cleaning of logs with txns -- [...truncated 421.01 KB...] kafka.utils.json.JsonValueTest >

Re: [DISCUSS] KIP-252: Extend ACLs to allow filtering based on ip ranges and subnets

Technically I absolutely agree with you, this would indeed create issues. If we were just talking about this KIP I think I'd argue that it is not too harsh of a requirement for users to refrain from using new features until they have fully upgraded their entire cluster. I think in that case it

[jira] [Resolved] (KAFKA-6854) Log cleaner fails with transaction markers that are deleted during clean

[ https://issues.apache.org/jira/browse/KAFKA-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-6854. Resolution: Fixed > Log cleaner fails with transaction markers that are deleted during

Re: [DISCUSS] KIP-290: Support for wildcard suffixed ACLs

Yeah, I guess that's a good point. It probably makes sense to support the prefix scheme for consumer groups and transactional IDs as well as topics. I agree that the current situation where anything goes in consumer group names and transactional ID names is not ideal. I wish we could rewind

Build failed in Jenkins: kafka-trunk-jdk8 #2616

See Changes: [github] Upgrade ZooKeeper to 3.4.12 and Scala to 2.12.6 (#4940) [ismael] KAFKA-6853: ZooKeeperRequestLatencyMs is incorrect (#4961) -- [...truncated 418.92

Build failed in Jenkins: kafka-1.1-jdk7 #124

See Changes: [ismael] KAFKA-6853: ZooKeeperRequestLatencyMs is incorrect (#4961) -- [...truncated 417.67 KB...] kafka.admin.ResetConsumerGroupOffsetTest >

Re: [DISCUSS] KIP-252: Extend ACLs to allow filtering based on ip ranges and subnets

There are still some problems with compatibility here, right? One example is if we construct a DENY ACL with an IP range and then install it. If all of our brokers have been upgraded, it will work. But if there are some that still haven't been upgraded, they will not honor the DENY ACL,

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

On Wed, May 2, 2018, at 14:54, John Roesler wrote: > Thanks Jason, > > I did find some production use cases "on the internet" that use poll(0) > *just* to join the group initially and ignore the response. I suppose the > assumption is that it'll be empty on the very first call to poll with >

[jira] [Created] (KAFKA-6856) Consumer Group hung in initialization/bad state

Todd created KAFKA-6856: --- Summary: Consumer Group hung in initialization/bad state Key: KAFKA-6856 URL: https://issues.apache.org/jira/browse/KAFKA-6856 Project: Kafka Issue Type: Bug

Re: [VOTE] #2 KIP-248: Create New ConfigCommand That Uses The New AdminClient

On Thu, May 3, 2018, at 05:11, Viktor Somogyi wrote: > @Magnus, yes that is correct. Thanks for your feedback. Updated it with > this (which might be subject to change based on the conversation with > Colin): "The changes done will be incremental in version 1, opposed to the > atomic behavior in

[jira] [Resolved] (KAFKA-6853) ResponseMetadata calculates latency incorrectly (and therefore ZooKeeperRequestLatencyMs is incorrect)

[ https://issues.apache.org/jira/browse/KAFKA-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-6853. Resolution: Fixed > ResponseMetadata calculates latency incorrectly (and therefore >

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

Hey Richard, I've updated the KIP with the changes discussed here. If you're happy with the state of it, I think that we're probably ready to call for a vote. However, I'm about to take a week off for vacation. Do you mind sending the [VOTE] message and managing the vote thread? Obviously, you

[jira] [Resolved] (KAFKA-6855) Kafka fails to start due to faulty Java version detection

[ https://issues.apache.org/jira/browse/KAFKA-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-6855. Resolution: Fixed Fix Version/s: 1.1.1 2.0.0 Thanks for the report. This

Re: [DISCUSS] KIP-252: Extend ACLs to allow filtering based on ip ranges and subnets

Hi all, I'd like to readopt this KIP, I got a bit sidetracked by other stuff after posting the initial version and discussion, sorry for that. I've added IPv6 to the KIP, but decided to forego the other scope extensions that I mentioned in my previous mail, as there are other efforts underway in

[jira] [Created] (KAFKA-6855) Kafka fails to start due to faulty Java version detection

Anders Tornblad created KAFKA-6855: -- Summary: Kafka fails to start due to faulty Java version detection Key: KAFKA-6855 URL: https://issues.apache.org/jira/browse/KAFKA-6855 Project: Kafka

[jira] [Created] (KAFKA-6854) Log cleaner fails with transaction markers that are deleted during clean

Rajini Sivaram created KAFKA-6854: - Summary: Log cleaner fails with transaction markers that are deleted during clean Key: KAFKA-6854 URL: https://issues.apache.org/jira/browse/KAFKA-6854 Project:

Re: [VOTE] #2 KIP-248: Create New ConfigCommand That Uses The New AdminClient

@Magnus, yes that is correct. Thanks for your feedback. Updated it with this (which might be subject to change based on the conversation with Colin): "The changes done will be incremental in version 1, opposed to the atomic behavior in version 0. For instance in version 0 sending an update for

[jira] [Created] (KAFKA-6853) ResponseMetadata calculates latency incorrectly (and therefore ZooKeeperRequestLatencyMs)

Fuud created KAFKA-6853: --- Summary: ResponseMetadata calculates latency incorrectly (and therefore ZooKeeperRequestLatencyMs) Key: KAFKA-6853 URL: https://issues.apache.org/jira/browse/KAFKA-6853 Project: Kafka

Re: [DISCUSS] KIP-280: Enhanced log compaction

Hi Guozhang, No worries, looking at the traffic on this project, I'm sure you have your hands full. Anyway, that proposal seems quite reasonable :- KIP is now updated to reflect those points. Are there any more topics you would like to address here? Cheers, LuisOn Wednesday, May 2,