Build failed in Jenkins: kafka-trunk-jdk8 #1122

2016-12-20 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA-4485; Follower should be in the isr if its FetchRequest has -- [...truncated 17555 lines...] org.apache.kafka.streams.KafkaStreamsTest > shouldNotGetAllTasksWithSto

Question about KIP-73 pr merge

2016-12-20 Thread Json Tu
Hi all, We are now using kafka 0.9.0 in our product enviroment, and we add one broker to the cluster,and execute reassign partitions between all brokers,we find our network card and disk io is very high. and I know KIP-73 has resolved this problem, but I wonder can I merge it to my kafk

Jenkins build is back to normal : kafka-trunk-jdk7 #1775

2016-12-20 Thread Apache Jenkins Server
See

Re: [VOTE] 0.10.1.1 RC1

2016-12-20 Thread Guozhang Wang
+1 from myself as well. This vote passes with 5 +1 votes (3 bindings) and no 0 or -1 votes. +1 votes PMC Members: * Gwen Shapira * Jun Rao * Guozhang Wang Community: * Vahid Hashemian * Swen Moczarski 0 votes * No votes -1 votes * No votes Vote thread: http://markmail.org/message/urbrfmaxofm

[jira] [Commented] (KAFKA-4485) Follower should be in the isr if its FetchRequest has fetched up to the logEndOffset of leader

2016-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15766182#comment-15766182 ] ASF GitHub Bot commented on KAFKA-4485: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-4485) Follower should be in the isr if its FetchRequest has fetched up to the logEndOffset of leader

2016-12-20 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-4485. Resolution: Fixed Issue resolved by pull request 2208 [https://github.com/apache/kafka/pull/2208] > Followe

[GitHub] kafka pull request #2208: KAFKA-4485; Follower should be in the isr if its F...

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2208 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Jenkins build is back to normal : kafka-trunk-jdk8 #1121

2016-12-20 Thread Apache Jenkins Server
See

Build failed in Jenkins: kafka-trunk-jdk7 #1774

2016-12-20 Thread Apache Jenkins Server
See Changes: [jason] MINOR: Replace TopicAndPartition with TopicPartition in `Log` and [wangguoz] MINOR: Clarify log deletion configuration options in server.properties -- [...truncated 7935 l

Re: Kafka controlled shutdown hangs when there are large number of topics in the cluster

2016-12-20 Thread James Cheng
> On Dec 19, 2016, at 10:13 AM, Gwen Shapira wrote: > > Can you try setting auto.leader.rebalance.enable=false in your > configuration (for all brokers) and see if it solves this problem? > We've had some reports regarding this feature interfering with > controlled shutdown. > Gwen, are there

Build failed in Jenkins: kafka-trunk-jdk8 #1120

2016-12-20 Thread Apache Jenkins Server
See Changes: [ismael] MINOR: Support auto-incrementing offsets in MemoryRecordsBuilder [jason] MINOR: Replace TopicAndPartition with TopicPartition in `Log` and -- [...truncated 7944 lines...]

[jira] [Commented] (KAFKA-1972) JMX Tool output for CSV format does not handle attributes with comma in their value

2016-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765908#comment-15765908 ] ASF GitHub Bot commented on KAFKA-1972: --- Github user rekhajoshm closed the pull requ

[GitHub] kafka pull request #45: KAFKA-1972: JMXTool multiple attributes

2016-12-20 Thread rekhajoshm
Github user rekhajoshm closed the pull request at: https://github.com/apache/kafka/pull/45 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] kafka pull request #49: KAFKA-269: run-test.sh async test

2016-12-20 Thread rekhajoshm
Github user rekhajoshm closed the pull request at: https://github.com/apache/kafka/pull/49 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[jira] [Commented] (KAFKA-269) ./system_test/producer_perf/bin/run-test.sh without --async flag does not run

2016-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765906#comment-15765906 ] ASF GitHub Bot commented on KAFKA-269: -- Github user rekhajoshm closed the pull request

Jenkins build is back to normal : kafka-trunk-jdk7 #1773

2016-12-20 Thread Apache Jenkins Server
See

[jira] [Commented] (KAFKA-242) Subsequent calls of ConsumerConnector.createMessageStreams cause Consumer offset to be incorrect

2016-12-20 Thread Jeff Widman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765835#comment-15765835 ] Jeff Widman commented on KAFKA-242: --- Does this bug still exist in 0.10? > Subsequent cal

[jira] [Commented] (KAFKA-1833) OfflinePartitionLeaderSelector may return null leader when ISR and Assgined Broker have no common

2016-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765833#comment-15765833 ] ASF GitHub Bot commented on KAFKA-1833: --- Github user tedxia closed the pull request

[GitHub] kafka pull request #39: KAFKA-1833: OfflinePartitionLeaderSelector may retur...

2016-12-20 Thread tedxia
Github user tedxia closed the pull request at: https://github.com/apache/kafka/pull/39 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[GitHub] kafka pull request #27: Clarify log deletion configuration options in server...

2016-12-20 Thread MarkRose
Github user MarkRose closed the pull request at: https://github.com/apache/kafka/pull/27 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #28: Clarify log deletion configuration options in server...

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/28 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[GitHub] kafka pull request #2268: MINOR: Replace TopicAndPartition with TopicPartiti...

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2268 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #2282: MINOR: Support auto-incrementing offsets in Memory...

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2282 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #2231: NOMERGE: Test new Jenkins PR plugin v4

2016-12-20 Thread ijuma
Github user ijuma closed the pull request at: https://github.com/apache/kafka/pull/2231 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

Re: [DISCUSS] KIP-101: Alter Replication Protocol to use Leader Generation rather than High Watermark for Truncation

2016-12-20 Thread Apurva Mehta
Hi Ben, Thanks for the KIP. It is very well written and explains the problem and solution very nicely. I have one --very minor-- question. In the 'steps' section, you write: > 4.6 The follower starts fetching from the leader from its log end offset. The use of 'its' is a bit ambiguous here. I pr

Re: [VOTE] KIP-90 Remove zkClient dependency from Streams

2016-12-20 Thread Edoardo Comar
+1 (non-binding) thanks! -- Edoardo Comar IBM MessageHub eco...@uk.ibm.com IBM UK Ltd, Hursley Park, SO21 2JN IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Han

Re: [VOTE] KIP-90 Remove zkClient dependency from Streams

2016-12-20 Thread Apurva Mehta
+1 On Tue, Dec 20, 2016 at 1:01 PM, Hojjat Jafarpour wrote: > Hi all, > > Seems that there is no opposition to this KIP. This email is to start the > voting for this KIP. > Once again the KIP is for removing zkClient dependency from Streams. Please > check out the KIP page: > > https://cwiki.apa

Re: [VOTE] KIP-90 Remove zkClient dependency from Streams

2016-12-20 Thread Jay Kreps
+1 On Tue, Dec 20, 2016 at 1:01 PM, Hojjat Jafarpour wrote: > Hi all, > > Seems that there is no opposition to this KIP. This email is to start the > voting for this KIP. > Once again the KIP is for removing zkClient dependency from Streams. Please > check out the KIP page: > > https://cwiki.apa

[jira] [Created] (KAFKA-4564) When the destination brokers are down or misconfigured in config, Streams should fail fast

2016-12-20 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-4564: Summary: When the destination brokers are down or misconfigured in config, Streams should fail fast Key: KAFKA-4564 URL: https://issues.apache.org/jira/browse/KAFKA-4564

Re: [VOTE] KIP-90 Remove zkClient dependency from Streams

2016-12-20 Thread Ismael Juma
Thanks for the KIP, +1 (binding). On Tue, Dec 20, 2016 at 1:01 PM, Hojjat Jafarpour wrote: > Hi all, > > Seems that there is no opposition to this KIP. This email is to start the > voting for this KIP. > Once again the KIP is for removing zkClient dependency from Streams. Please > check out the

[jira] [Commented] (KAFKA-4198) Transient test failure: ConsumerBounceTest.testConsumptionWithBrokerFailures

2016-12-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765468#comment-15765468 ] Guozhang Wang commented on KAFKA-4198: -- Saw another case of this failure, but with di

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2016-12-20 Thread radai
when the leader decides to commit a TX (of X msgs, known at this point), it writes an "intent to append X msgs" msg (control?) followed by the X msgs (at this point it is the leader and therefor point of sync, so this can be done with no "foreign" msgs in between). if there's a crash/change of lead

[GitHub] kafka pull request #2283: HOTFIX: Convert exception to warning since clearly...

2016-12-20 Thread enothereska
GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/2283 HOTFIX: Convert exception to warning since clearly it is happening. You can merge this pull request into a Git repository by running: $ git pull https://github.com/enothereska/kafka hotfix

[jira] [Updated] (KAFKA-4563) State transitions error PARTITIONS_REVOKED to NOT_RUNNING

2016-12-20 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-4563: Description: When starting and stopping streams quickly, the following exception is thrown: java.la

Re: [VOTE] KIP-90 Remove zkClient dependency from Streams

2016-12-20 Thread Damian Guy
+1 On Tue, 20 Dec 2016 at 21:16 Sriram Subramanian wrote: > +1 > > On Tue, Dec 20, 2016 at 1:13 PM, Guozhang Wang wrote: > > > +1. Thanks! > > > > On Tue, Dec 20, 2016 at 1:01 PM, Hojjat Jafarpour > > wrote: > > > > > Hi all, > > > > > > Seems that there is no opposition to this KIP. This emai

Re: [DISCUSS] KIP-99: Add Global Tables to Kafka Streams

2016-12-20 Thread Damian Guy
Hi Guozhang, Thanks for your input. Answers below, but i'm thinking we should remove joins from GlobalKTables for the time being and re-visit if necessary in the future. 1. with a global table the joins are never really materialized (at least how i see it), rather they are just views on the exist

Build failed in Jenkins: kafka-trunk-jdk8 #1119

2016-12-20 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-4540: Suspended tasks that are not assigned to the StreamThread [wangguoz] MINOR: Add more exception information in ProcessorStateManager [jason] KAFKA-4554; Fix ReplicaBuffer.verifyChecksum to use itera

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2016-12-20 Thread Apurva Mehta
@Radai, regarding the replication for inflight transactional messages. I think Jay and Joel have addressed the need for transactional messages to be persisted durably at the moment we enter the pre-commit phase. If we don't have durable persistence of these messages, we can't have idempotent and a

Re: [VOTE] KIP-90 Remove zkClient dependency from Streams

2016-12-20 Thread Sriram Subramanian
+1 On Tue, Dec 20, 2016 at 1:13 PM, Guozhang Wang wrote: > +1. Thanks! > > On Tue, Dec 20, 2016 at 1:01 PM, Hojjat Jafarpour > wrote: > > > Hi all, > > > > Seems that there is no opposition to this KIP. This email is to start the > > voting for this KIP. > > Once again the KIP is for removing z

Re: [VOTE] KIP-90 Remove zkClient dependency from Streams

2016-12-20 Thread Guozhang Wang
+1. Thanks! On Tue, Dec 20, 2016 at 1:01 PM, Hojjat Jafarpour wrote: > Hi all, > > Seems that there is no opposition to this KIP. This email is to start the > voting for this KIP. > Once again the KIP is for removing zkClient dependency from Streams. Please > check out the KIP page: > > https://

[VOTE] KIP-90 Remove zkClient dependency from Streams

2016-12-20 Thread Hojjat Jafarpour
Hi all, Seems that there is no opposition to this KIP. This email is to start the voting for this KIP. Once again the KIP is for removing zkClient dependency from Streams. Please check out the KIP page: https://cwiki.apache.org/confluence/display/KAFKA/KIP-90+-+Remove+zkClient+ dependency+from+St

Build failed in Jenkins: kafka-trunk-jdk8 #1118

2016-12-20 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-4473: RecordCollector should handle retriable exceptions more -- [...truncated 32680 lines...] org.apache.kafka.streams.KafkaStreamsTest > shouldNotGetAllTasksWit

Build failed in Jenkins: kafka-trunk-jdk7 #1772

2016-12-20 Thread Apache Jenkins Server
See Changes: [jason] KAFKA-4554; Fix ReplicaBuffer.verifyChecksum to use iterators instead of -- [...truncated 3870 lines...] kafka.log.LogTest > testReadWithTooSmallMaxLength STARTED kafka.lo

Re: [DISCUSS] KIP-101: Alter Replication Protocol to use Leader Generation rather than High Watermark for Truncation

2016-12-20 Thread Ben Stopford
Hi all So having gone through a few extra failure scenarios it appears it is still possible for logs to diverge if the unclean.leader.election setting is enabled. The protocol could be evolved further to protect against this. The issue is that it adds significant complexity, and potentially impact

[GitHub] kafka pull request #2282: MINOR: Support auto-incrementing offsets in Memory...

2016-12-20 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2282 MINOR: Support auto-incrementing offsets in MemoryRecordsBuilder You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka builder-autoi

Re: [DISCUSS] KIP-99: Add Global Tables to Kafka Streams

2016-12-20 Thread Guozhang Wang
One more thing to add: 6. For KGlobalTable, it is always bootstrapped from the beginning while for other KTables, we are enabling users to override their resetting position as in https://github.com/apache/kafka/pull/2007 Should we consider doing the same for KGlobalTable as well? Guozhang On

Re: [DISCUSS] KIP-99: Add Global Tables to Kafka Streams

2016-12-20 Thread Guozhang Wang
Thanks for the very well written proposal, and sorry for the very-late review. I have a few comments here: 1. We are introducing a "queryableViewName" in the GlobalTable join results, while I'm wondering if we should just add a more general function like "materialize" to KTable and KGlobalTable wi

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2016-12-20 Thread Jay Kreps
I don't think the simple approach of writing to a local store (in memory or on disk) and then copying out to the destination topics would work but there could well be more sophisticated things that would. As you say, it is fine for the data to be un-replicated while you are accumulating the transac

[jira] [Commented] (KAFKA-4554) ReplicaBuffer.verifyChecksum should use use iterators instead of iterables

2016-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765006#comment-15765006 ] ASF GitHub Bot commented on KAFKA-4554: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #2280: KAFKA-4554: Fix ReplicaBuffer.verifyChecksum to us...

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2280 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-4554) ReplicaBuffer.verifyChecksum should use use iterators instead of iterables

2016-12-20 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-4554: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull requ

[jira] [Updated] (KAFKA-4561) Ordering of operations in StreamThread.shutdownTasksAndState may void at-least-once guarantees

2016-12-20 Thread Damian Guy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4561: -- Status: Patch Available (was: In Progress) > Ordering of operations in StreamThread.shutdownTasksAndSta

[jira] [Commented] (KAFKA-4561) Ordering of operations in StreamThread.shutdownTasksAndState may void at-least-once guarantees

2016-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15764971#comment-15764971 ] ASF GitHub Bot commented on KAFKA-4561: --- GitHub user dguy opened a pull request:

[GitHub] kafka pull request #2281: KAFKA-4561: Ordering of operations in StreamThread...

2016-12-20 Thread dguy
GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2281 KAFKA-4561: Ordering of operations in StreamThread.shutdownTasksAndState may void at-least-once guarantees In `shutdownTasksAndState` and `suspendTasksAndState` we commit offsets BEFORE we flush any s

Kafka Summit 2017

2016-12-20 Thread Gwen Shapira
Hi Kafka Fans, Just in case you didn't hear / read: Last year was the first Kafka Summit and it was quite successful. So we are doing two this year: May in NYC and Aug in SF. You can read more details here: https://kafka-summit.org/ and you can use the code "kafkcom17" for a $50 community discou

[jira] [Updated] (KAFKA-4540) Suspended tasks that are not assigned to the StreamThread need to be closed before new active and standby tasks are created

2016-12-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-4540: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[GitHub] kafka pull request #2266: KAFKA-4540: Suspended tasks that are not assigned ...

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2266 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #2276: MINOR: Add more exception information in Processor...

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2276 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2016-12-20 Thread radai
obviously anything committed would need to be replicated to all followers - just like current msgs. what im trying to say is that in-flight data (written as part of an ongoing TX and not committed yet) does not necessarily need to be replicated, or even written out to disk. taken to the extreme it

[jira] [Commented] (KAFKA-4540) Suspended tasks that are not assigned to the StreamThread need to be closed before new active and standby tasks are created

2016-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15764886#comment-15764886 ] ASF GitHub Bot commented on KAFKA-4540: --- Github user asfgit closed the pull request

[jira] [Created] (KAFKA-4563) State transitions error PARTITIONS_REVOKED to NOT_RUNNING

2016-12-20 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-4563: --- Summary: State transitions error PARTITIONS_REVOKED to NOT_RUNNING Key: KAFKA-4563 URL: https://issues.apache.org/jira/browse/KAFKA-4563 Project: Kafka Issue T

[jira] [Commented] (KAFKA-4473) RecordCollector should handle retriable exceptions more strictly

2016-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15764822#comment-15764822 ] ASF GitHub Bot commented on KAFKA-4473: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #2249: KAFKA-4473: RecordCollector should handle retriabl...

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2249 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2016-12-20 Thread Jay Kreps
Cool. It sounds like you guys will sync up and come up with a specific proposal. I think point (3) does require full replication of the pre-commit transaction, but I'm not sure, and I would be very happy to learn otherwise. That was actually the blocker on that alternate proposal. From my point of

[jira] [Updated] (KAFKA-4473) RecordCollector should handle retriable exceptions more strictly

2016-12-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-4473: - Resolution: Fixed Fix Version/s: 0.10.2.0 Status: Resolved (was: Patch Available

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2016-12-20 Thread Joel Koshy
Just got some time to go through most of this thread and KIP - great to see this materialize and discussed!! I will add more comments in the coming days on some of the other "tracks" in this thread; but since Radai brought up the double-journaling approach that we had discussed I thought I would mo

Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

2016-12-20 Thread Manikumar
Hi Devs, If there are no more comments, I will start vote on this KIP later this week. Thanks On Fri, Dec 16, 2016 at 12:28 PM, Manikumar wrote: > Hi, > > >> Can you add a sample Jaas configuration using delegation tokens to the >> KIP? >> > > Will add sample Jaas configuration. > > >> To mak

Re: [VOTE] 0.10.1.1 RC1

2016-12-20 Thread Moczarski, Swen
+1 (non-binding) Thanks for preparing the release. We installed on our test system, at first glance looks good. Am 12/15/16, 10:29 PM schrieb "Guozhang Wang" : Hello Kafka users, developers and client-developers, This is the second, and hopefully the last candidate for the release

Jenkins build is back to normal : kafka-trunk-jdk8 #1117

2016-12-20 Thread Apache Jenkins Server
See

[jira] [Work started] (KAFKA-4561) Ordering of operations in StreamThread.shutdownTasksAndState may void at-least-once guarantees

2016-12-20 Thread Damian Guy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4561 started by Damian Guy. - > Ordering of operations in StreamThread.shutdownTasksAndState may void > at-least-onc

[jira] [Updated] (KAFKA-4554) ReplicaBuffer.verifyChecksum should use use iterators instead of iterables

2016-12-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4554: --- Description: This regressed in https://github.com/apache/kafka/commit/b58b6a1bef0ecdc2107a415e222af09

[jira] [Updated] (KAFKA-4554) ReplicaBuffer.verifyChecksum should use use iterators instead of iterables

2016-12-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4554: --- Summary: ReplicaBuffer.verifyChecksum should use use iterators instead of iterables (was: ReplicaVeri

[jira] [Updated] (KAFKA-4554) ReplicaVerificationToolTest.test_replica_lags system test failure

2016-12-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4554: --- Fix Version/s: 0.10.2.0 > ReplicaVerificationToolTest.test_replica_lags system test failure >

[jira] [Updated] (KAFKA-4554) ReplicaVerificationToolTest.test_replica_lags system test failure

2016-12-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4554: --- Reviewer: Jason Gustafson Status: Patch Available (was: Reopened) > ReplicaVerificationToolTest

Build failed in Jenkins: kafka-trunk-jdk8 #1116

2016-12-20 Thread Apache Jenkins Server
See Changes: [ismael] MINOR: Remove incomplete gradle wrapper infrastructure -- [...truncated 17461 lines...] org.apache.kafka.streams.KafkaStreamsTest > shouldReturnFalseOnCloseWhenThreadsHav

[GitHub] kafka pull request #2222: KAFKA-4500: code improvements

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4500) Kafka Code Improvements

2016-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15764121#comment-15764121 ] ASF GitHub Bot commented on KAFKA-4500: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-4500) Kafka Code Improvements

2016-12-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4500. Resolution: Fixed Fix Version/s: 0.10.2.0 Issue resolved by pull request [https://github

[jira] [Commented] (KAFKA-4554) ReplicaVerificationToolTest.test_replica_lags system test failure

2016-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15764099#comment-15764099 ] ASF GitHub Bot commented on KAFKA-4554: --- GitHub user ijuma opened a pull request:

[GitHub] kafka pull request #2280: KAFKA-4554: Fix `ReplicaBuffer.verifyChecksum` to ...

2016-12-20 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/2280 KAFKA-4554: Fix `ReplicaBuffer.verifyChecksum` to use iterators instead of iterables This was changed in b58b6a1bef0 and caused the `ReplicaVerificationToolTest.test_replica_lags` system test to

[jira] [Resolved] (KAFKA-3808) Transient failure in ReplicaVerificationToolTest

2016-12-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3808. Resolution: Fixed Assignee: (was: Ismael Juma) > Transient failure in ReplicaVerificationT

[jira] [Comment Edited] (KAFKA-3808) Transient failure in ReplicaVerificationToolTest

2016-12-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15763960#comment-15763960 ] Ismael Juma edited comment on KAFKA-3808 at 12/20/16 11:17 AM: -

[jira] [Commented] (KAFKA-3808) Transient failure in ReplicaVerificationToolTest

2016-12-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15763960#comment-15763960 ] Ismael Juma commented on KAFKA-3808: The new failures are being tracked in KAFKA-4285

[jira] [Commented] (KAFKA-4554) ReplicaVerificationToolTest.test_replica_lags system test failure

2016-12-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15763957#comment-15763957 ] Ismael Juma commented on KAFKA-4554: I reopened this because it's a new issue and not

[jira] [Reopened] (KAFKA-4554) ReplicaVerificationToolTest.test_replica_lags system test failure

2016-12-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reopened KAFKA-4554: Assignee: Ismael Juma > ReplicaVerificationToolTest.test_replica_lags system test failure > --

[GitHub] kafka pull request #1722: remove incomplete gradle wrapper infrastructure

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1722 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4562) deadlock heartbeat, metadata-manager, request-handler

2016-12-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15763903#comment-15763903 ] Ismael Juma commented on KAFKA-4562: Thanks for the report. Same as KAFKA-3994/KAFKA-4

[jira] [Created] (KAFKA-4562) deadlock heartbeat, metadata-manager, request-handler

2016-12-20 Thread Hans Kowallik (JIRA)
Hans Kowallik created KAFKA-4562: Summary: deadlock heartbeat, metadata-manager, request-handler Key: KAFKA-4562 URL: https://issues.apache.org/jira/browse/KAFKA-4562 Project: Kafka Issue Typ

[jira] [Created] (KAFKA-4561) Ordering of operations in StreamThread.shutdownTasksAndState may void at-least-once guarantees

2016-12-20 Thread Damian Guy (JIRA)
Damian Guy created KAFKA-4561: - Summary: Ordering of operations in StreamThread.shutdownTasksAndState may void at-least-once guarantees Key: KAFKA-4561 URL: https://issues.apache.org/jira/browse/KAFKA-4561