[GitHub] kafka pull request #1622: KAFKA-3859: Fix describe group command to report v...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/1622 KAFKA-3859: Fix describe group command to report valid status when group is empty With the new consumer, when all consumers of a consumer group are stopped (i.e. the group is empty), the des

[jira] [Commented] (KAFKA-3859) Consumer group is stuck in rebalancing status

[ https://issues.apache.org/jira/browse/KAFKA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376317#comment-15376317 ] ASF GitHub Bot commented on KAFKA-3859: --- GitHub user vahidhashemian opened a pull re

Jenkins build is back to normal : kafka-0.10.0-jdk7 #148

See

[jira] [Resolved] (KAFKA-3941) Avoid applying eviction listener in InMemoryKeyValueLoggedStore

[ https://issues.apache.org/jira/browse/KAFKA-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3941. -- Resolution: Fixed Fix Version/s: 0.10.0.1 Issue resolved by pull request 1610 [https://gi

[jira] [Commented] (KAFKA-3941) Avoid applying eviction listener in InMemoryKeyValueLoggedStore

[ https://issues.apache.org/jira/browse/KAFKA-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376123#comment-15376123 ] ASF GitHub Bot commented on KAFKA-3941: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #1610: KAFKA-3941: Delay eviction listener in InMemoryKey...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1610 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Kafka Streams question

Hi! I am prototyping some code using Kafka Streams, and have a question. I need to map a stream into another (with different partition key) and join it with a table. How do I control number of partitions of the mapped stream? Thanks! Poul Costinsky Chief Architect (

[jira] [Commented] (KAFKA-3857) Additional log cleaner metrics

[ https://issues.apache.org/jira/browse/KAFKA-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376102#comment-15376102 ] ASF GitHub Bot commented on KAFKA-3857: --- Github user kiranptivo closed the pull requ

[jira] [Commented] (KAFKA-3857) Additional log cleaner metrics

[ https://issues.apache.org/jira/browse/KAFKA-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376103#comment-15376103 ] ASF GitHub Bot commented on KAFKA-3857: --- GitHub user kiranptivo reopened a pull requ

[GitHub] kafka pull request #1593: KAFKA-3857 Additional log cleaner metrics

GitHub user kiranptivo reopened a pull request: https://github.com/apache/kafka/pull/1593 KAFKA-3857 Additional log cleaner metrics Fixes KAFKA-3857 Changes proposed in this pull request: The following additional log cleaner metrics have been added. 1. num-runs:

[jira] [Commented] (KAFKA-3857) Additional log cleaner metrics

[ https://issues.apache.org/jira/browse/KAFKA-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376100#comment-15376100 ] ASF GitHub Bot commented on KAFKA-3857: --- Github user kiranptivo closed the pull requ

[GitHub] kafka pull request #1593: KAFKA-3857 Additional log cleaner metrics

Github user kiranptivo closed the pull request at: https://github.com/apache/kafka/pull/1593 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3857) Additional log cleaner metrics

[ https://issues.apache.org/jira/browse/KAFKA-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376101#comment-15376101 ] ASF GitHub Bot commented on KAFKA-3857: --- GitHub user kiranptivo reopened a pull requ

[GitHub] kafka pull request #1593: KAFKA-3857 Additional log cleaner metrics

GitHub user kiranptivo reopened a pull request: https://github.com/apache/kafka/pull/1593 KAFKA-3857 Additional log cleaner metrics Fixes KAFKA-3857 Changes proposed in this pull request: The following additional log cleaner metrics have been added. 1. num-runs:

[GitHub] kafka pull request #1593: KAFKA-3857 Additional log cleaner metrics

Github user kiranptivo closed the pull request at: https://github.com/apache/kafka/pull/1593 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-3963) Missing messages from the controller to brokers

[ https://issues.apache.org/jira/browse/KAFKA-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh resolved KAFKA-3963. Resolution: Invalid > Missing messages from the controller to brokers >

[jira] [Commented] (KAFKA-3963) Missing messages from the controller to brokers

[ https://issues.apache.org/jira/browse/KAFKA-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376065#comment-15376065 ] Maysam Yabandeh commented on KAFKA-3963: Thanks [~ijuma]. Yeah when reading the co

[jira] [Created] (KAFKA-3964) Metadata update requests are sometimes received after LeaderAndIsrRequests

Maysam Yabandeh created KAFKA-3964: -- Summary: Metadata update requests are sometimes received after LeaderAndIsrRequests Key: KAFKA-3964 URL: https://issues.apache.org/jira/browse/KAFKA-3964 Project:

[jira] [Commented] (KAFKA-3963) Missing messages from the controller to brokers

[ https://issues.apache.org/jira/browse/KAFKA-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376029#comment-15376029 ] Ismael Juma commented on KAFKA-3963: Thanks for the report. We don't take another item

[jira] [Updated] (KAFKA-3963) Missing messages from the controller to brokers

[ https://issues.apache.org/jira/browse/KAFKA-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3963: --- Fix Version/s: 0.10.1.0 > Missing messages from the controller to brokers > --

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

I think it's important to distinguish the use cases of defining new stores (somewhat rare) versus using the `store` method (very common). The strategy employed here is a common way to use generics to ensure type safety for the latter case. In the former case, there are all sorts of weird things one

[jira] [Created] (KAFKA-3963) Missing messages from the controller to brokers

Maysam Yabandeh created KAFKA-3963: -- Summary: Missing messages from the controller to brokers Key: KAFKA-3963 URL: https://issues.apache.org/jira/browse/KAFKA-3963 Project: Kafka Issue Type:

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

Yes, you get compile time errors On Wed, 13 Jul 2016 at 16:22 Damian Guy wrote: > You wont get a runtime error as you wouldn't find a store of that type. > The API would return null > > On Wed, 13 Jul 2016 at 16:22 Jay Kreps wrote: > >> But if "my-store" is not of type MyStoreType don't you sti

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

You wont get a runtime error as you wouldn't find a store of that type. The API would return null On Wed, 13 Jul 2016 at 16:22 Jay Kreps wrote: > But if "my-store" is not of type MyStoreType don't you still get a run time > error that in effect is the same as the class cast would be? Basically t

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

But if "my-store" is not of type MyStoreType don't you still get a run time error that in effect is the same as the class cast would be? Basically the question I'm asking is whether this added complexity is actually moving runtime errors to compile time errors. -Jay On Wed, Jul 13, 2016 at 4:16 P

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

You create your custom Store, i.e,: /** * An interface your custom store provides * @param * @param */ interface MyStoreType { V get(K key); void put(K key, V value); } /** * Implement your store * @param * @param */ public class MyStoreImpl implements StateStore, MyStoreType

[jira] [Commented] (KAFKA-3948) Invalid broker port in Zookeeper when SSL is enabled

[ https://issues.apache.org/jira/browse/KAFKA-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375965#comment-15375965 ] Bharat Viswanadham commented on KAFKA-3948: --- I will look in to this issue, to pr

[jira] [Assigned] (KAFKA-3948) Invalid broker port in Zookeeper when SSL is enabled

[ https://issues.apache.org/jira/browse/KAFKA-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham reassigned KAFKA-3948: - Assignee: Bharat Viswanadham > Invalid broker port in Zookeeper when SSL is enabl

[jira] [Updated] (KAFKA-3962) ConfigDef support for resource-specific configuration

[ https://issues.apache.org/jira/browse/KAFKA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-3962: --- Description: It often comes up with connectors that you want some piece of configuration that

[jira] [Created] (KAFKA-3962) ConfigDef support for resource-specific configuration

Shikhar Bhushan created KAFKA-3962: -- Summary: ConfigDef support for resource-specific configuration Key: KAFKA-3962 URL: https://issues.apache.org/jira/browse/KAFKA-3962 Project: Kafka Issue

[jira] [Work started] (KAFKA-3859) Consumer group is stuck in rebalancing status

[ https://issues.apache.org/jira/browse/KAFKA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3859 started by Vahid Hashemian. -- > Consumer group is stuck in rebalancing status > ---

[jira] [Assigned] (KAFKA-3859) Consumer group is stuck in rebalancing status

[ https://issues.apache.org/jira/browse/KAFKA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian reassigned KAFKA-3859: -- Assignee: Vahid Hashemian > Consumer group is stuck in rebalancing status > ---

[GitHub] kafka pull request #1621: MINOR: Added simple streams benchmark to system te...

GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/1621 MINOR: Added simple streams benchmark to system tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/enothereska/kafka simple-benchmark-st

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

But to avoid the cast you introduce a bunch of magic that doesn't really bring type safety, right? Or possibly I'm misunderstanding, how do I plug in a new store type and get access to it? Can you give the steps for that? -Jay On Wed, Jul 13, 2016 at 10:47 AM, Guozhang Wang wrote: > Personally

[jira] [Commented] (KAFKA-2729) Cached zkVersion not equal to that in zookeeper, broker not recovering.

[ https://issues.apache.org/jira/browse/KAFKA-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375878#comment-15375878 ] Tyler Bischel commented on KAFKA-2729: -- We are also seeing this issue in 0.10.0.0 pre

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

I agree too. Casting also makes it harder to rely on static analysis tools to detect unsafe code usage. Ismael On Wed, Jul 13, 2016 at 7:37 PM, Damian Guy wrote: > I'm in agreement with Gouzhang. Further, from a "Developer Experience" PoV > IMO we should avoid forcing developers to cast - it do

[jira] [Commented] (KAFKA-3940) Log should check the return value of dir.mkdirs()

[ https://issues.apache.org/jira/browse/KAFKA-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375774#comment-15375774 ] Ismael Juma commented on KAFKA-3940: Yes, I'd prefer if we handled this across the cod

[jira] [Commented] (KAFKA-3940) Log should check the return value of dir.mkdirs()

[ https://issues.apache.org/jira/browse/KAFKA-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375692#comment-15375692 ] Ishita Mandhan commented on KAFKA-3940: --- Hi Jim, I got a notification of your PR bec

[jira] [Closed] (KAFKA-3957) consumer timeout not being respected when kafka broker is not available

[ https://issues.apache.org/jira/browse/KAFKA-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Fumo closed KAFKA-3957. --- > consumer timeout not being respected when kafka broker is not available > --

[jira] [Resolved] (KAFKA-3957) consumer timeout not being respected when kafka broker is not available

[ https://issues.apache.org/jira/browse/KAFKA-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Fumo resolved KAFKA-3957. - Resolution: Duplicate > consumer timeout not being respected when kafka broker is not available >

[jira] [Commented] (KAFKA-3101) Optimize Aggregation Outputs

[ https://issues.apache.org/jira/browse/KAFKA-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375673#comment-15375673 ] Guozhang Wang commented on KAFKA-3101: -- This LGTM, also we want to get some informati

[jira] [Commented] (KAFKA-3957) consumer timeout not being respected when kafka broker is not available

[ https://issues.apache.org/jira/browse/KAFKA-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375644#comment-15375644 ] Vahid Hashemian commented on KAFKA-3957: Thanks for confirming. You can close this

[jira] [Commented] (KAFKA-3957) consumer timeout not being respected when kafka broker is not available

[ https://issues.apache.org/jira/browse/KAFKA-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375569#comment-15375569 ] Vincent Fumo commented on KAFKA-3957: - yes I see it. Apparently my use case is just on

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

+1 > On Jul 13, 2016, at 2:37 PM, Damian Guy wrote: > > I'm in agreement with Gouzhang. Further, from a "Developer Experience" PoV > IMO we should avoid forcing developers to cast - it doesn't make for nice > and easy to use API, introduces possible runtime errors due to invalid > casts, develop

[jira] [Commented] (KAFKA-2205) Generalize TopicConfigManager to handle multiple entity configs

[ https://issues.apache.org/jira/browse/KAFKA-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375527#comment-15375527 ] Jun Rao commented on KAFKA-2205: [~thesquelched], thanks for reporting that. Do you want t

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

I'm in agreement with Gouzhang. Further, from a "Developer Experience" PoV IMO we should avoid forcing developers to cast - it doesn't make for nice and easy to use API, introduces possible runtime errors due to invalid casts, developers need to know what they are casting to. On Wed, 13 Jul 2016 a

[jira] [Updated] (KAFKA-3941) Avoid applying eviction listener in InMemoryKeyValueLoggedStore

[ https://issues.apache.org/jira/browse/KAFKA-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3941: - Description: This is reported by [~norwood]. In {{InMemoryKeyValueLoggedStore}} we set the evicti

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

Personally I think the additional complexity of the introduced " QueryableStoreType" interface is still acceptable from a user's point of view: this is the only interface we are exposing to users, and other wrappers are all internal classes. Regarding "QueryableStoreTypes", maybe we can consider d

[jira] [Commented] (KAFKA-3929) Add prefix for underlying clients configs in StreamConfig

[ https://issues.apache.org/jira/browse/KAFKA-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375381#comment-15375381 ] Guozhang Wang commented on KAFKA-3929: -- I am sorry for the late reply, missed your re

[jira] [Commented] (KAFKA-3961) broker sends malformed response when switching from no compression to snappy/gzip

[ https://issues.apache.org/jira/browse/KAFKA-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375342#comment-15375342 ] Manikumar Reddy commented on KAFKA-3961: you can try DumpLogSegments tools to to v

[jira] [Commented] (KAFKA-3940) Log should check the return value of dir.mkdirs()

[ https://issues.apache.org/jira/browse/KAFKA-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375225#comment-15375225 ] Jim Jagielski commented on KAFKA-3940: -- Pull request created > Log should check the

[jira] [Commented] (KAFKA-3940) Log should check the return value of dir.mkdirs()

[ https://issues.apache.org/jira/browse/KAFKA-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375222#comment-15375222 ] ASF GitHub Bot commented on KAFKA-3940: --- GitHub user jimjag opened a pull request:

[GitHub] kafka pull request #1620: KAFKA-3940: Log should check the return value of d...

GitHub user jimjag opened a pull request: https://github.com/apache/kafka/pull/1620 KAFKA-3940: Log should check the return value of dir.mkdirs() You can merge this pull request into a Git repository by running: $ git pull https://github.com/jimjag/kafka KAFKA-3940 Alternativ

[jira] [Created] (KAFKA-3961) broker sends malformed response when switching from no compression to snappy/gzip

Dieter Plaetinck created KAFKA-3961: --- Summary: broker sends malformed response when switching from no compression to snappy/gzip Key: KAFKA-3961 URL: https://issues.apache.org/jira/browse/KAFKA-3961

[jira] [Work started] (KAFKA-3858) Add functions to print stream topologies

[ https://issues.apache.org/jira/browse/KAFKA-3858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3858 started by Eno Thereska. --- > Add functions to print stream topologies > --

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

Damian -- appreciate the example code and you convinced me. Agree that the class approach is better and renaming to KafkaStreamsMetadata along with renaming the API methods will address the issues I was referring to. One other thing I wanted to get people's thoughts on was the way we are proposing

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

IMO, that makes the most sense. > On Jul 12, 2016, at 5:11 PM, Ismael Juma wrote: > > Hi Damian, > > How about StreamsMetadata instead? The general naming pattern seems to > avoid the `Kafka` prefix for everything outside of `KafkaStreams` itself. > > Ismael > > On Tue, Jul 12, 2016 at 7:14 P

[jira] [Commented] (KAFKA-3858) Add functions to print stream topologies

[ https://issues.apache.org/jira/browse/KAFKA-3858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375140#comment-15375140 ] ASF GitHub Bot commented on KAFKA-3858: --- GitHub user enothereska opened a pull reque

[GitHub] kafka pull request #1619: KAFKA-3858: Basic printing of everything

GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/1619 KAFKA-3858: Basic printing of everything You can merge this pull request into a Git repository by running: $ git pull https://github.com/enothereska/kafka KAFKA-3858-print-topology Altern

[jira] [Commented] (KAFKA-2205) Generalize TopicConfigManager to handle multiple entity configs

[ https://issues.apache.org/jira/browse/KAFKA-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375073#comment-15375073 ] Scott Kruger commented on KAFKA-2205: - Looks like this isn't *quite* fixed all the way

[jira] [Resolved] (KAFKA-3887) StreamBounceTest.test_bounce and StreamSmokeTest.test_streams failing

[ https://issues.apache.org/jira/browse/KAFKA-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3887. -- Resolution: Fixed Marking as fixed per 1611. > StreamBounceTest.test_bounce and StreamSmokeTest

[jira] [Commented] (KAFKA-3960) Committed offset not set after first assign

[ https://issues.apache.org/jira/browse/KAFKA-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374813#comment-15374813 ] Alexey Romanchuk commented on KAFKA-3960: - I can not find any workaround for this

[jira] [Created] (KAFKA-3960) Committed offset not set after first assign

Alexey Romanchuk created KAFKA-3960: --- Summary: Committed offset not set after first assign Key: KAFKA-3960 URL: https://issues.apache.org/jira/browse/KAFKA-3960 Project: Kafka Issue Type: B

[jira] [Updated] (KAFKA-3959) __consumer_offsets wrong number of replicas at startup

[ https://issues.apache.org/jira/browse/KAFKA-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alban Hurtaud updated KAFKA-3959: - Description: When creating a stack of 3 kafka brokers, the consumer is starting faster than kafka

[jira] [Created] (KAFKA-3959) __consumer_offsets wrong number of replicas at startup

Alban Hurtaud created KAFKA-3959: Summary: __consumer_offsets wrong number of replicas at startup Key: KAFKA-3959 URL: https://issues.apache.org/jira/browse/KAFKA-3959 Project: Kafka Issue Ty