Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Thanks for the KIP. It looks I am late to the party :) I just went through the KIP and doc for one pass. Some quick feedback: 1. UnrecognizedMessageException is a synchronous exception? It is good for it to catch the attention, but seems not intuitive because the exception is not

Re: Reg: ACLS

The default principal for Plaintext transport is "ANONYMOUS". On Sun, Dec 11, 2016 at 12:58 AM, BigData dev wrote: > Hi, > > bin/kafka-acls.sh --topic kafka-testtopic --add -allow-host 9.30.15.19 > --operation Write --authorizer-properties >

Build failed in Jenkins: kafka-trunk-jdk8 #1096

See Changes: [me] KAFKA-4140: Upgrade to ducktape 0.6.0 and make system tests parallel [ismael] KAFKA-4431; Make consumer heartbeat thread a daemon thread -- [...truncated 14541 lines...]

[jira] [Resolved] (KAFKA-4431) HeartbeatThread should be a daemon thread

[ https://issues.apache.org/jira/browse/KAFKA-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4431. Resolution: Fixed Fix Version/s: 0.10.2.0 Issue resolved by pull request 2234

[GitHub] kafka pull request #2234: KAFKA-4431: Make consumer heartbeat thread a daemo...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2234 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4431) HeartbeatThread should be a daemon thread

[ https://issues.apache.org/jira/browse/KAFKA-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740830#comment-15740830 ] ASF GitHub Bot commented on KAFKA-4431: --- Github user asfgit closed the pull request at:

[jira] [Resolved] (KAFKA-4140) Update system tests to allow running tests in parallel

[ https://issues.apache.org/jira/browse/KAFKA-4140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-4140. -- Resolution: Fixed Fix Version/s: 0.10.2.0 Issue resolved by pull request

[jira] [Commented] (KAFKA-4140) Update system tests to allow running tests in parallel

[ https://issues.apache.org/jira/browse/KAFKA-4140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740786#comment-15740786 ] ASF GitHub Bot commented on KAFKA-4140: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #1834: [WIP] KAFKA-4140: make system tests parallel frien...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1834 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740688#comment-15740688 ] ASF GitHub Bot commented on KAFKA-4497: --- GitHub user becketqin opened a pull request:

[GitHub] kafka pull request #2243: KAFKA-4497: LogCleaner appended the wrong offset t...

GitHub user becketqin opened a pull request: https://github.com/apache/kafka/pull/2243 KAFKA-4497: LogCleaner appended the wrong offset to time index. Backport the fix to 0.10.1. The code in trunk has been changed, so we need to provide a different fix for this problem in 0.10.1.

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740662#comment-15740662 ] ASF GitHub Bot commented on KAFKA-4497: --- GitHub user becketqin opened a pull request:

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Apurva and Jason -- appreciate the detailed explanation, thank you! Jason - > In many applications, there is already a natural identifier which is already used to divide the workload. For example, in Kafka Streams and Kafka Connect, we have a taskId. For applications where there is no natural

[GitHub] kafka pull request #2242: KAFKA-4497: Fix the ByteBufferMessageSet.filterInt...

GitHub user becketqin opened a pull request: https://github.com/apache/kafka/pull/2242 KAFKA-4497: Fix the ByteBufferMessageSet.filterInto There seems a few bugs in `ByteBufferMessageSet.filterInto`, including incorrect offsets, messages read and retained. This patch fixes them and

Re: [DISCUSS] KIP-101: Alter Replication Protocol to use Leader Generation rather than High Watermark for Truncation

Pretty happy to see a KIP tackling this problem! One comment below. The "Extending LeaderEpoch to include Returning Leaders" states: "To protect against this eventuality the controller will maintain a cached mapping of [broker -> Zookeeper CZXID] (CZXID is a unique and monotonic 64-bit number)

Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag

Hi Ewen So this is what got discussed in the discussion thread. We wanted to go for supporting all four combinations Tombstone + non null value Tombstone + null value Non tombstone + non null value Non tombstone + null value The issues were mostly around compatibility with old consumers. In

Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag

Michael, It kind of depends on how you want to interpret the tombstone flag. If it's purely a producer-facing Kafka-level thing that we treat as internal to the broker and log cleaner once the record is sent, then your approach makes sense. You're just moving copying the null-indicates-delete

Re: [DISCUSS] KIP-101: Alter Replication Protocol to use Leader Generation rather than High Watermark for Truncation

Good to see this KIP being proposed. Back when I added the epoch to the replication protocol, we discussed adding it to the log due to the failure scenarios listed in the KIP but I failed to convince people that it was worth the effort needed to upgrade the cluster (especially after we asked

Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag

Hi Jay, Why wouldn't that work, the tombstone value is only looked at by the broker, on a topic configured for compaction as such is benign on non compacted topics. This is as much as sending a null value currently Regards Mike Sent using OWA for iPhone

Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag

Hey Michael, I'm not quite sure that works as that would translate ALL null values to tombstones, even for non-compacted topics that use null as an acceptable value sent by the producer and expected by the consumer. -Jay On Sun, Dec 11, 2016 at 3:26 AM, Michael Pearce

[jira] [Updated] (KAFKA-4522) Using Disruptor instead of Array Blocking queue in Kafka Producer

[ https://issues.apache.org/jira/browse/KAFKA-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratik kumar updated KAFKA-4522: Description: Kafka Producer currently uses Java's Array Blocking Queue to store outbound kafka

[jira] [Updated] (KAFKA-4522) Using Disruptor instead of Array Blocking queue in Kafka Producer

[ https://issues.apache.org/jira/browse/KAFKA-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratik kumar updated KAFKA-4522: Description: Kafka Producer currently uses Java's Array Blocking Queue to store outbound kafka

[jira] [Updated] (KAFKA-4522) Using Disruptor instead of Array Blocking queue in Kafka Producer

[ https://issues.apache.org/jira/browse/KAFKA-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratik kumar updated KAFKA-4522: Description: Kafka Producer currently uses Java's Array Blocking Queue to store outbound kafka

[jira] [Updated] (KAFKA-4522) Using Disruptor instead of Array Blocking queue in Kafka Producer

[ https://issues.apache.org/jira/browse/KAFKA-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratik kumar updated KAFKA-4522: Description: Kafka Producer currently uses Java's Array Blocking Queue to store outbound kafka

[jira] [Created] (KAFKA-4522) Using Disruptor instead of Array Blocking queue in Kafka Producer

Pratik kumar created KAFKA-4522: --- Summary: Using Disruptor instead of Array Blocking queue in Kafka Producer Key: KAFKA-4522 URL: https://issues.apache.org/jira/browse/KAFKA-4522 Project: Kafka

[jira] [Commented] (KAFKA-4514) Add Codec for ZStandard Compression

[ https://issues.apache.org/jira/browse/KAFKA-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15739673#comment-15739673 ] Lee Dongjin commented on KAFKA-4514: [~ijuma][~tgraves] Hello. I am interested in this issue. If it

[DISCUSS] KIP-101: Alter Replication Protocol to use Leader Generation rather than High Watermark for Truncation

Hi All Please find the below KIP which describes a proposed solution to a couple of issues that have been observed with the replication protocol. In short, the proposal replaces the use of the High Watermark, for follower log trunctation, with an alternate Generation Marker. This uniquely

Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag

Hi Ewen, I think the easiest way to show this is with code. As you can see we keep the existing behaviour for code/binaries calling the pre-existing constructors, whereby if the value is null the tombstone is set to true. Regards Mike /** * Creates a record with a specified

[jira] [Updated] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4497: --- Fix Version/s: 0.10.1.1 > log cleaner breaks on timeindex > --- > >

Re: [VOTE] KIP-100 - Relax Type constraints in Kafka Streams API

+1 (binding) My only concern was around compatibility. It seems like the one case it is incompatible would be, at worst, an extremely unusual edge case (and I *think* can be restricted further to "not source compatible for anyone extending the affected classes / interfaces and override affected