Build failed in Jenkins: kafka-trunk-jdk8 #998

See Changes: [jason] KAFKA-3144; Report members with no assigned partitions in -- [...truncated 6645 lines...] kafka.api.UserClientIdQuotaTest > testProducerConsumerOverrideUnthrottled STARTED

[jira] [Work started] (KAFKA-4117) Cleanup StreamPartitionAssignor behavior

[ https://issues.apache.org/jira/browse/KAFKA-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4117 started by Guozhang Wang. > Cleanup StreamPartitionAssignor behavior >

[jira] [Updated] (KAFKA-4117) Cleanup StreamPartitionAssignor behavior

[ https://issues.apache.org/jira/browse/KAFKA-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-4117: - Status: Patch Available (was: In Progress) > Cleanup StreamPartitionAssignor behavior > -

Build failed in Jenkins: kafka-trunk-jdk7 #1649

See Changes: [jason] KAFKA-3144; Report members with no assigned partitions in -- [...truncated 14356 lines...] org.apache.kafka.streams.processor.internals.StreamPartitionAssignorTest > testSu

Re: [DISCUSS] KIP-80: Kafka REST Server

Sriram, "Can the streaming platform exist without stream processing? - No. Processing stream data again is a core part of streaming platform." Yes, it can. There are no.of Stream processing frameworks out there, and they all have integration into Kafka. It doesn't need to be developed within Ka

[jira] [Updated] (KAFKA-3144) report members with no assigned partitions in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-3144: --- Resolution: Fixed Fix Version/s: 0.10.2.0 Status: Resolved (was: Patch Avail

[jira] [Commented] (KAFKA-3144) report members with no assigned partitions in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596984#comment-15596984 ] ASF GitHub Bot commented on KAFKA-3144: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #1336: KAFKA-3144: Report members with no assigned partit...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1336 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4099) Change the time based log rolling to only based on the message timestamp.

[ https://issues.apache.org/jira/browse/KAFKA-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596965#comment-15596965 ] Jiangjie Qin commented on KAFKA-4099: - [~junrao] Thanks for the explanation. I agree t

[GitHub] kafka pull request #2045: Fix for kafka-4295: kafka-console-consumer.sh does...

Github user amethystic closed the pull request at: https://github.com/apache/kafka/pull/2045 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Jenkins build is back to normal : kafka-trunk-jdk8 #997

See

Re: [VOTE] KIP-73 - Replication Quotas

Thanks for catching that and the fix as well. Makes sense to me. We should consider adding an "amendments" section to KIPs - perhaps just a link to KAFKA-4313 would suffice in this case. Thanks, Joel On Wed, Oct 19, 2016 at 7:12 PM, Jun Rao wrote: > Hi, > > While testing KIP-73, we found an i

[GitHub] kafka pull request #2053: KAFKA-4326: Refactor LogCleaner for better reuse o...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2053 KAFKA-4326: Refactor LogCleaner for better reuse of common copy/compress logic You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafk

[jira] [Commented] (KAFKA-4326) Refactor LogCleaner to remove duplicate log copying logic

[ https://issues.apache.org/jira/browse/KAFKA-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596601#comment-15596601 ] ASF GitHub Bot commented on KAFKA-4326: --- GitHub user hachikuji opened a pull request

[jira] [Comment Edited] (KAFKA-3144) report members with no assigned partitions in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596540#comment-15596540 ] Vahid Hashemian edited comment on KAFKA-3144 at 10/21/16 10:31 PM: -

[jira] [Created] (KAFKA-4333) Report consumer group coordinator id when '--list' option is used

Vahid Hashemian created KAFKA-4333: -- Summary: Report consumer group coordinator id when '--list' option is used Key: KAFKA-4333 URL: https://issues.apache.org/jira/browse/KAFKA-4333 Project: Kafka

[jira] [Commented] (KAFKA-3144) report members with no assigned partitions in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596540#comment-15596540 ] Vahid Hashemian commented on KAFKA-3144: [~hachikuji] suggested that we report the

[GitHub] kafka pull request #2052: MINOR: add list_topics command to help debug tests

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2052 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4296) LogCleaner CleanerStats swap logic seems incorrect

[ https://issues.apache.org/jira/browse/KAFKA-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596460#comment-15596460 ] ASF GitHub Bot commented on KAFKA-4296: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #2016: KAFKA-4296: Fix LogCleaner statistics rolling

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2016 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-4296) LogCleaner CleanerStats swap logic seems incorrect

[ https://issues.apache.org/jira/browse/KAFKA-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-4296. Resolution: Fixed Fix Version/s: (was: 0.10.1.1) 0.10.2.0 Issu

[GitHub] kafka pull request #2052: MINOR: add list_topics command to help debug tests

GitHub user xvrl opened a pull request: https://github.com/apache/kafka/pull/2052 MINOR: add list_topics command to help debug tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/xvrl/kafka test-add-list-topics Alternatively y

Re: [DISCUSS] KIP-80: Kafka REST Server

FWIW, Apache Kafka has evolved a lot from where it started. It did start as a messaging system. Over time we realized that that the vision for Kafka is to build a streaming platform and not just a messaging system. You can take a look at the site for more description about what comprises the stream

[jira] [Created] (KAFKA-4332) kafka.api.UserQuotaTest.testThrottledProducerConsumer transient unit test failure

Jun Rao created KAFKA-4332: -- Summary: kafka.api.UserQuotaTest.testThrottledProducerConsumer transient unit test failure Key: KAFKA-4332 URL: https://issues.apache.org/jira/browse/KAFKA-4332 Project: Kafka

[jira] [Reopened] (KAFKA-2089) MetadataTest transient failure

[ https://issues.apache.org/jira/browse/KAFKA-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao reopened KAFKA-2089: Reopening this issue. Saw the following transient failure in trunk. org.apache.kafka.clients.MetadataTest > tes

Re: [jira] [Commented] (KAFKA-4113) Allow KTable bootstrap

Great to know! Thanks Greg. Please keep us posted with any new finding you have. Guozhang On Fri, Oct 21, 2016 at 12:35 PM, Greg Fodor wrote: > I managed to track down one case where we were seeing issues with missing > data when transitioning to a new node to being a retention policy on the

Re: [jira] [Commented] (KAFKA-4113) Allow KTable bootstrap

I managed to track down one case where we were seeing issues with missing data when transitioning to a new node to being a retention policy on the topic. There is an additional case but have not been able to repro at this time. We recently fixed a problem where we were failing to properly gracefull

[jira] [Commented] (KAFKA-4099) Change the time based log rolling to only based on the message timestamp.

[ https://issues.apache.org/jira/browse/KAFKA-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596037#comment-15596037 ] Jun Rao commented on KAFKA-4099: I had two use cases of time-based rolling in mind. The fi

Re: [DISCUSS] KIP-80: Kafka REST Server

Are you saying Kafka REST is subjective but Kafka Streams and Kafka Connect are not subjective? > "there are likely places that can live without a rest proxy" There are also places that can live without Kafka Streams and Kafka Connect. Nacho On Fri, Oct 21, 2016 at 11:17 AM, Jun Rao wrote: >

Re: Kafka KIP meeting Oct 19 at 11:00am PST

I think a separate KIP is a good idea as well. Note however that potential decisions in this KIP could affect the other KIP. Nacho On Fri, Oct 21, 2016 at 10:23 AM, Jun Rao wrote: > Michael, > > Yes, doing a separate KIP to address the null payload issue for compacted > topics is a good idea.

[jira] [Created] (KAFKA-4331) Kafka Streams resetter is slow because it joins the same group for each topic

Roger Hoover created KAFKA-4331: --- Summary: Kafka Streams resetter is slow because it joins the same group for each topic Key: KAFKA-4331 URL: https://issues.apache.org/jira/browse/KAFKA-4331 Project: Ka

Re: [DISCUSS] KIP-80: Kafka REST Server

At the high level, I think ideally it makes sense to add a component to Apache Kafka if (1) it's widely needed and (2) it needs tight integration with Kafka core. For Kafka Stream, we do expect stream processing will be used widely in the future. Implementation wise, Kafka Stream only supports gett

Re: Kafka KIP meeting Oct 19 at 11:00am PST

Michael, Yes, doing a separate KIP to address the null payload issue for compacted topics is a good idea. Thanks, Jun On Fri, Oct 21, 2016 at 12:57 AM, Michael Pearce wrote: > I had noted that what ever the solution having compaction based on null > payload was agreed isn't elegant. > > Shall

Re: [DISCUSS] KIP-80: Kafka REST Server

Harsha, You seem to be saying that your only two options are to fork or duplicate the existing REST project or add it to Kafka to be able to contribute. I don't think those are the only two options. The other option is to contribute to the existing successful project--which is Apache licensed and

[jira] [Commented] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect ProcessorNode

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595593#comment-15595593 ] Frank Lyaruu commented on KAFKA-4311: - The ClassCast exception is gone, but I still se

Re: Kafka KIP meeting Oct 19 at 11:00am PST

++1 > On Oct 21, 2016, at 11:47 AM, Gwen Shapira wrote: > > I think this will be a good idea. It will separate an issue with concrete > use-case and a major pain-point from a wider discussion about architectures > and who is responsible for data formats. > > > On Fri, Oct 21, 2016 at 12:57 AM,

Re: Kafka KIP meeting Oct 19 at 11:00am PST

I think this will be a good idea. It will separate an issue with concrete use-case and a major pain-point from a wider discussion about architectures and who is responsible for data formats. On Fri, Oct 21, 2016 at 12:57 AM, Michael Pearce wrote: > I had noted that what ever the solution having

[GitHub] kafka pull request #2051: KAFKA-4311: Multi layer cache eviction causes forw...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2051 KAFKA-4311: Multi layer cache eviction causes forwarding to incorrect ProcessorNode Given a topology like the one below. If a record arriving in `tableOne` causes a cache eviction, it will trigger the

[jira] [Commented] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect ProcessorNode

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595401#comment-15595401 ] ASF GitHub Bot commented on KAFKA-4311: --- GitHub user dguy opened a pull request:

Re: [DISCUSS] KIP-80: Kafka REST Server

Here is Ignite’s - essentially idea is having different owners/maintainers per module/area that keep a check on that area. https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute On 10/21/16, 3:44 PM, "isma...@gmail.com on behalf of Ismael Juma" wrote: Hi Michael, Can

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect ProcessorNode

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Summary: Multi layer cache eviction causes forwarding to incorrect ProcessorNode (was: Multi layer cac

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Description: The two exceptions below were reported by Frank on the dev mailing list. After investigati

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Description: The two exceptions below were reported by Frank on the dev mailing list. After investigati

Re: [DISCUSS] KIP-80: Kafka REST Server

Hi Michael, Can you please share which Apache projects have a MMC? I couldn't find anything after a quick google. Ismael On Fri, Oct 21, 2016 at 7:28 AM, Michael Pearce wrote: > So from my reading essentially the first question needs to answered/and > voted on is: > > Is Apache Kafka Community

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Summary: Multi layer cache eviction causes forwarding to incorrect Processor Node (was: Mutli layer ca

[jira] [Updated] (KAFKA-4311) Mutli layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Summary: Mutli layer cache eviction causes forwarding to incorrect Processor Node (was: Exception in N

?????? [DISCUSS] KIP-68 Add a consumed log retention before log retention

Hi Mayuresh, Thanks for the reply: 1. In the log retention check schedule, the broker first find the all the consumed group which are consuming this topic, and query the commit offset of this consumed group for the topic using the OffsetFetch API. And the min commit offset is the minimal co

Jenkins build is back to normal : kafka-trunk-jdk7 #1647

See

?????? [DISCUSS] KIP-68 Add a consumed log retention before log retention

Hi Becket, It seems a good idea to have the topic to some specified groups to using the consumed log retention, default is all the consumed groups, it is more flexible. I have some comments for this : 1. What scenario is used to this configuration? 2. I think we can only support this c

[jira] [Created] (KAFKA-4330) Leader/Follower Quota Rates in JMX have ambiguous names

Ben Stopford created KAFKA-4330: --- Summary: Leader/Follower Quota Rates in JMX have ambiguous names Key: KAFKA-4330 URL: https://issues.apache.org/jira/browse/KAFKA-4330 Project: Kafka Issue Typ

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15594852#comment-15594852 ] Damian Guy commented on KAFKA-4311: --- Hi Frank, if you have the time, would you mind try

[GitHub] kafka pull request #2050: Replaced unnecessary isDefined and get on option v...

GitHub user himani1 opened a pull request: https://github.com/apache/kafka/pull/2050 Replaced unnecessary isDefined and get on option values with fold You can merge this pull request into a Git repository by running: $ git pull https://github.com/himani1/kafka refactored_code

Build failed in Jenkins: kafka-trunk-jdk8 #996

See Changes: [ismael] KAFKA-4309; Allow "pluggable" properties in KafkaService in System Tests -- [...truncated 1066 lines...] at org.gradle.internal.event.BroadcastDispatch.dispatch(Br

[jira] [Commented] (KAFKA-4309) Allow "pluggable" properties in KafkaService in System Tests

[ https://issues.apache.org/jira/browse/KAFKA-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15594582#comment-15594582 ] ASF GitHub Bot commented on KAFKA-4309: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-4309) Allow "pluggable" properties in KafkaService in System Tests

[ https://issues.apache.org/jira/browse/KAFKA-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4309. Resolution: Fixed Fix Version/s: 0.10.1.1 0.10.2.0 Issue resolved by pull

[GitHub] kafka pull request #2034: KAFKA-4309: Allow "pluggable" properties in KafkaS...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2034 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] KIP-80: Kafka REST Server

Harsha Ch wrote on 20/10/2016 22:26:53: > The Streams and Connect projects that were pushed into Kafka could > have been left in their own Github projects based on your arguments. What > about the REST API is so different that such that it should stay out of the > Kafka project? From m

Re: [ANNOUNCE] Apache Kafka 0.10.1.0 Released

Great job ! Thanks Jason and to everyone who contributed. On Fri, Oct 21, 2016 at 6:09 AM, Manikumar wrote: > Thanks Jason. Great job. > > On Fri, Oct 21, 2016 at 10:29 AM, Becket Qin wrote: > >> Congratulations! Great job, Jason. >> >> On Thu, Oct 20, 2016 at 11:57 AM, Vahid S Hashemian < >> va

[jira] [Updated] (KAFKA-4329) The order of the parameters for creating the ZkUtils object is reversed

[ https://issues.apache.org/jira/browse/KAFKA-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Wang updated KAFKA-4329: - Summary: The order of the parameters for creating the ZkUtils object is reversed (was: The parameters for

[jira] [Updated] (KAFKA-4329) The parameters for creating the ZkUtils object is reverse

[ https://issues.apache.org/jira/browse/KAFKA-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Wang updated KAFKA-4329: - Description: When creating the ZkUtils object, the parameters of zkSessionTimeOutMs and zkConnectionTimeo

[jira] [Created] (KAFKA-4329) The parameters for creating the ZkUtils object is reverse

Matt Wang created KAFKA-4329: Summary: The parameters for creating the ZkUtils object is reverse Key: KAFKA-4329 URL: https://issues.apache.org/jira/browse/KAFKA-4329 Project: Kafka Issue Type: B

[jira] [Created] (KAFKA-4328) The parameters for creating the ZkUtils object is reverse

Matt Wang created KAFKA-4328: Summary: The parameters for creating the ZkUtils object is reverse Key: KAFKA-4328 URL: https://issues.apache.org/jira/browse/KAFKA-4328 Project: Kafka Issue Type: B

Re: Kafka KIP meeting Oct 19 at 11:00am PST

I had noted that what ever the solution having compaction based on null payload was agreed isn't elegant. Shall we raise another kip to : as discussed propose using a attribute bit for delete/compaction flag as well/or instead of null value and updating compaction logic to look at that delelete

[GitHub] kafka pull request #2045: Fix for kafka-4295: kafka-console-consumer.sh does...

GitHub user amethystic reopened a pull request: https://github.com/apache/kafka/pull/2045 Fix for kafka-4295: kafka-console-consumer.sh does not delete the temporary group in zookeeper Since consumer stop logic and zk node removal code are in separate threads, so when two threads e