[GitHub] kafka pull request: Fixed ConsumerRecord constructor javadoc

[sslavic]

GitHub user sslavic opened a pull request:

https://github.com/apache/kafka/pull/85

Fixed ConsumerRecord constructor javadoc

Refactoring of ConsumerRecord made in 
https://github.com/apache/kafka/commit/0699ff2ce60abb466cab5315977a224f1a70a4da#diff-fafe8d3a3942f3c6394927881a9389b2
 left ConsumerRecord constructor javadoc inconsistent with implementation.

This patch fixes ConsumerRecord constructor javadoc to be inline with 
implementation.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sslavic/kafka patch-3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/85.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #85


commit 4445cbfcff711b23271e8449813fbeb7535967e6
Author: Stevo Slavić 
Date:   2015-07-18T12:01:13Z

Fixed ConsumerRecord constructor javadoc

Refactoring of ConsumerRecord made in 
https://github.com/apache/kafka/commit/0699ff2ce60abb466cab5315977a224f1a70a4da#diff-fafe8d3a3942f3c6394927881a9389b2
 left ConsumerRecord constructor javadoc inconsistent with implementation.

This patch fixes ConsumerRecord constructor javadoc to be inline with 
implementation.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Official Kafka Gitter Room?

[Ismael Juma]

Hi Grant,

I have been considering proposing the same thing myself for similar
reasons, but I also have some doubts. More comments inline.

On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke  wrote:

> I think it could be useful to have a place to chat that is associated with
> the Kafka repo.


Agreed.


> Note that we do currently have an IRC channel, but from my
> experience its a ghost town.
>

I've recently joined #apache-kafka and activity is low, yes, but there is
some[1]. The bigger issue, perhaps, is that contributors are not there. Do
we think that this would be different with Gitter? And, if so, what is the
reason? Is it because no application needs to be installed? In theory,
IRCCloud provides something similar for IRC (although the limitations of
the trial and the upgrade nagging can be annoying) and we could link to the
channel in GitHub's readme.

Also, are there any Apache projects who use Gitter? There was a thread in
the Spark mailing list, but the response wasn't positive[2]. We would need
Apache Infra to set-up Gitter for us as they are the only ones that have
write access to the repository (the set-up is really easy though, which is
definitely a positive).

Gitter may be the right answer, I just want to make sure it actually helps
us have a more active channel instead of splitting the existing activity
between two channels.

Thoughts?

Best,
Ismael

[1] https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
[2]
http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html

Re: [DISCUSS] KIP-27 - Conditional Publish

[Ben Kirwin]

Just wanted to flag a little discussion that happened on the ticket:
https://issues.apache.org/jira/browse/KAFKA-2260?focusedCommentId=14632259&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14632259

In particular, Yasuhiro Matsuda proposed an interesting variant on
this that performs the offset check on the message key (instead of
just the partition), with bounded space requirements, at the cost of
potentially some spurious failures. (ie. the produce request may fail
even if that particular key hasn't been updated recently.) This
addresses a couple of the drawbacks of the per-key approach mentioned
at the bottom of the KIP.

On Fri, Jul 17, 2015 at 6:47 PM, Ben Kirwin  wrote:
> Hi all,
>
> So, perhaps it's worth adding a couple specific examples of where this
> feature is useful, to make this a bit more concrete:
>
> - Suppose I'm using Kafka as a commit log for a partitioned KV store,
> like Samza or Pistachio (?) do. We bootstrap the process state by
> reading from that partition, and log all state updates to that
> partition when we're running. Now imagine that one of my processes
> locks up -- GC or similar -- and the system transitions that partition
> over to another node. When the GC is finished, the old 'owner' of that
> partition might still be trying to write to the commit log at the same
> as the new one is. A process might detect this by noticing that the
> offset of the published message is bigger than it thought the upcoming
> offset was, which implies someone else has been writing to the log...
> but by then it's too late, and the commit log is already corrupt. With
> a 'conditional produce', one of those processes will have it's publish
> request refused -- so we've avoided corrupting the state.
>
> - Envision some copycat-like system, where we have some sharded
> postgres setup and we're tailing each shard into its own partition.
> Normally, it's fairly easy to avoid duplicates here: we can track
> which offset in the WAL corresponds to which offset in Kafka, and we
> know how many messages we've written to Kafka already, so the state is
> very simple. However, it is possible that for a moment -- due to
> rebalancing or operator error or some other thing -- two different
> nodes are tailing the same postgres shard at once! Normally this would
> introduce duplicate messages, but by specifying the expected offset,
> we can avoid this.
>
> So perhaps it's better to say that this is useful when a single
> producer is *expected*, but multiple producers are *possible*? (In the
> same way that the high-level consumer normally has 1 consumer in a
> group reading from a partition, but there are small windows where more
> than one might be reading at the same time.) This is also the spirit
> of the 'runtime cost' comment -- in the common case, where there is
> little to no contention, there's no performance overhead either. I
> mentioned this a little in the Motivation section -- maybe I should
> flesh that out a little bit?
>
> For me, the motivation to work this up was that I kept running into
> cases, like the above, where the existing API was almost-but-not-quite
> enough to give the guarantees I was looking for -- and the extension
> needed to handle those cases too was pretty small and natural-feeling.
>
> On Fri, Jul 17, 2015 at 4:49 PM, Ashish Singh  wrote:
>> Good concept. I have a question though.
>>
>> Say there are two producers A and B. Both producers are producing to same
>> partition.
>> - A sends a message with expected offset, x1
>> - Broker accepts is and sends an Ack
>> - B sends a message with expected offset, x1
>> - Broker rejects it, sends nack
>> - B sends message again with expected offset, x1+1
>> - Broker accepts it and sends Ack
>> I guess this is what this KIP suggests, right? If yes, then how does this
>> ensure that same message will not be written twice when two producers are
>> producing to same partition? Producer on receiving a nack will try again
>> with next offset and will keep doing so till the message is accepted. Am I
>> missing something?
>>
>> Also, you have mentioned on KIP, "it imposes little to no runtime cost in
>> memory or time", I think that is not true for time. With this approach
>> producers' performance will reduce proportionally to number of producers
>> writing to same partition. Please correct me if I am missing out something.
>>
>>
>> On Fri, Jul 17, 2015 at 11:32 AM, Mayuresh Gharat <
>> gharatmayures...@gmail.com> wrote:
>>
>>> If we have 2 producers producing to a partition, they can be out of order,
>>> then how does one producer know what offset to expect as it does not
>>> interact with other producer?
>>>
>>> Can you give an example flow that explains how it works with single
>>> producer and with multiple producers?
>>>
>>>
>>> Thanks,
>>>
>>> Mayuresh
>>>
>>> On Fri, Jul 17, 2015 at 10:28 AM, Flavio Junqueira <
>>> fpjunque...@yahoo.com.invalid> wrote:
>>>
>>> > I like this feature, it reminds me of c

[jira] [Commented] (KAFKA-2260) Allow specifying expected offset on produce

[Ben Kirwin (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632537#comment-14632537
 ] 

Ben Kirwin commented on KAFKA-2260:
---

Ah, clever! Thanks for sharing this -- I've linked to it on the discussion 
thread.

{quote}
When changing the number of sub-partitions, the broker doesn't have to 
recompute sub-partition high water marks. It can initialize all array elements 
with the partition's high water mark.
{quote}

It seems we could also do this initialization every time the log is opened -- 
and avoid any persistent storage for the 'sub-partition' offsets at all. This 
would remove another major drawback of the per-key approach.

> Allow specifying expected offset on produce
> ---
>
> Key: KAFKA-2260
> URL: https://issues.apache.org/jira/browse/KAFKA-2260
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Ben Kirwin
>Assignee: Ewen Cheslack-Postava
>Priority: Minor
> Attachments: expected-offsets.patch
>
>
> I'd like to propose a change that adds a simple CAS-like mechanism to the 
> Kafka producer. This update has a small footprint, but enables a bunch of 
> interesting uses in stream processing or as a commit log for process state.
> h4. Proposed Change
> In short:
> - Allow the user to attach a specific offset to each message produced.
> - The server assigns offsets to messages in the usual way. However, if the 
> expected offset doesn't match the actual offset, the server should fail the 
> produce request instead of completing the write.
> This is a form of optimistic concurrency control, like the ubiquitous 
> check-and-set -- but instead of checking the current value of some state, it 
> checks the current offset of the log.
> h4. Motivation
> Much like check-and-set, this feature is only useful when there's very low 
> contention. Happily, when Kafka is used as a commit log or as a 
> stream-processing transport, it's common to have just one producer (or a 
> small number) for a given partition -- and in many of these cases, predicting 
> offsets turns out to be quite useful.
> - We get the same benefits as the 'idempotent producer' proposal: a producer 
> can retry a write indefinitely and be sure that at most one of those attempts 
> will succeed; and if two producers accidentally write to the end of the 
> partition at once, we can be certain that at least one of them will fail.
> - It's possible to 'bulk load' Kafka this way -- you can write a list of n 
> messages consecutively to a partition, even if the list is much larger than 
> the buffer size or the producer has to be restarted.
> - If a process is using Kafka as a commit log -- reading from a partition to 
> bootstrap, then writing any updates to that same partition -- it can be sure 
> that it's seen all of the messages in that partition at the moment it does 
> its first (successful) write.
> There's a bunch of other similar use-cases here, but they all have roughly 
> the same flavour.
> h4. Implementation
> The major advantage of this proposal over other suggested transaction / 
> idempotency mechanisms is its minimality: it gives the 'obvious' meaning to a 
> currently-unused field, adds no new APIs, and requires very little new code 
> or additional work from the server.
> - Produced messages already carry an offset field, which is currently ignored 
> by the server. This field could be used for the 'expected offset', with a 
> sigil value for the current behaviour. (-1 is a natural choice, since it's 
> already used to mean 'next available offset'.)
> - We'd need a new error and error code for a 'CAS failure'.
> - The server assigns offsets to produced messages in 
> {{ByteBufferMessageSet.validateMessagesAndAssignOffsets}}. After this 
> changed, this method would assign offsets in the same way -- but if they 
> don't match the offset in the message, we'd return an error instead of 
> completing the write.
> - To avoid breaking existing clients, this behaviour would need to live 
> behind some config flag. (Possibly global, but probably more useful 
> per-topic?)
> I understand all this is unsolicited and possibly strange: happy to answer 
> questions, and if this seems interesting, I'd be glad to flesh this out into 
> a full KIP or patch. (And apologies if this is the wrong venue for this sort 
> of thing!)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 36593: Patch for KAFKA-2347

[Jiangjie Qin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36593/#review92178
---

Ship it!


LGTM. Thanks.

- Jiangjie Qin


On July 18, 2015, 5:02 a.m., Ashish Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36593/
> ---
> 
> (Updated July 18, 2015, 5:02 a.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-2347
> https://issues.apache.org/jira/browse/KAFKA-2347
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-2347: Add setConsumerRebalanceListener method to 
> ZookeeperConsuemrConnector java api
> 
> 
> Diffs
> -
> 
>   core/src/main/scala/kafka/javaapi/consumer/ConsumerConnector.java 
> ca74ca8abf03478b6de4ec1f82fbac379e7603f1 
> 
> Diff: https://reviews.apache.org/r/36593/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ashish Singh
> 
>

Re: Review Request 36593: Patch for KAFKA-2347

[Ismael Juma]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36593/#review92179
---


There is a typo in the commit message: "ZookeeperConsuemrConnector". The rest 
looks good.

- Ismael Juma


On July 18, 2015, 5:02 a.m., Ashish Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36593/
> ---
> 
> (Updated July 18, 2015, 5:02 a.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-2347
> https://issues.apache.org/jira/browse/KAFKA-2347
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-2347: Add setConsumerRebalanceListener method to 
> ZookeeperConsuemrConnector java api
> 
> 
> Diffs
> -
> 
>   core/src/main/scala/kafka/javaapi/consumer/ConsumerConnector.java 
> ca74ca8abf03478b6de4ec1f82fbac379e7603f1 
> 
> Diff: https://reviews.apache.org/r/36593/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ashish Singh
> 
>

Re: Review Request 33620: Patch for KAFKA-1690

[Sriharsha Chintalapani]

> On May 15, 2015, 10:54 p.m., Joel Koshy wrote:
> > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java,
> >  line 321
> > 
> >
> > Actually, can you describe how this would be done (say, for dealing 
> > with revoked certificates after an client authenticates)? Per your jira 
> > comment we can use an authorizer to block the client in this case, but if 
> > you have a proposal on handling periodic renegotiation it would be useful 
> > to discuss that. I agree we don't need to implement it now.

I added the SSL renegotiation as part of the latest patch. Please take a look.


- Sriharsha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33620/#review83993
---


On June 23, 2015, 8:18 p.m., Sriharsha Chintalapani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33620/
> ---
> 
> (Updated June 23, 2015, 8:18 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1690
> https://issues.apache.org/jira/browse/KAFKA-1690
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. SSLFactory tests.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Added 
> PrincipalBuilder.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Fixed minor 
> issues with the patch.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Fixed minor 
> issues with the patch.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> Merge remote-tracking branch 'refs/remotes/origin/trunk' into KAFKA-1690-V1
> 
> 
> KAFKA-1690. Broker side ssl changes.
> 
> 
> KAFKA-1684. SSL for socketServer.
> 
> 
> KAFKA-1690. Added SSLProducerSendTest and fixes to get right port for SSL.
> 
> 
> Merge branch 'trunk' into KAFKA-1690-V1
> 
> 
> KAFKA-1690. Post merge fixes.
> 
> 
> KAFKA-1690. Added SSLProducerSendTest.
> 
> 
> KAFKA-1690. Minor fixes based on patch review comments.
> 
> 
> Diffs
> -
> 
>   build.gradle 30d1cf2f1ff9ed3f86a060da8099bb0774b4cf91 
>   checkstyle/import-control.xml f2e6cec267e67ce8e261341e373718e14a8e8e03 
>   clients/src/main/java/org/apache/kafka/clients/ClientUtils.java 
> 0d68bf1e1e90fe9d5d4397ddf817b9a9af8d9f7a 
>   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
> 2c421f42ed3fc5d61cf9c87a7eaa7bb23e26f63b 
>   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
> 48fe7961e2215372d8033ece4af739ea06c6457b 
>   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
> daff34db5bf2144e9dc274b23dc56b88f4efafdc 
>   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
> 951c34c92710fc4b38d656e99d2a41255c60aeb7 
>   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
> 5a37580ec69af08b97cf5b43b241790ba8c129dd 
>   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
> aa264202f2724907924985a5ecbe74afc4c6c04b 
>   clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java 
> bae528d31516679bed88ee61b408f209f185a8cc 
>   clients/src/main/java/org/apache/kafka/common/config/SSLConfigs.java 
> PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/Authenticator.java 
> PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/ByteBufferSend.java 
> df0e6d5105ca97b7e1cb4d334ffb7b443506bd0b 
>   clients/src/main/java/org/apache/kafka/common/network/Channel.java 
> PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/ChannelBuilder.java 
> PRE-CREATION 
>   
> clients/src/main/java/org/apache/kafka/common/network/DefaultAuthenticator.java
>  PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/NetworkReceive.java 
> 3ca0098b8ec8cfdf81158465b2d40afc47eb6f80 
>   
> clients/src/main/java/org/apache/kafka/common/network/PlainTextChannelBuilder.java
>  PRE-CREATION 
>   
> clients/src/main/java/org/apache/kafka/common/network/PlainTextTransportLayer.java
>  PRE-CREATION 
>   
> clients/src/main/java/org/apache/kafka

Re: Review Request 33620: Patch for KAFKA-1690

[Sriharsha Chintalapani]

> On June 30, 2015, 3:03 p.m., Jun Rao wrote:
> > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java,
> >  line 602
> > 
> >
> > Not sure if we should do the flip here. The caller may not be able to 
> > read all bytes in appReadBuffer once. When the caller calls 
> > readFromAppBuffer() again, we shouldn't do the flip again.

As discussed offline. we are doing flip & compact.


- Sriharsha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33620/#review89677
---


On June 23, 2015, 8:18 p.m., Sriharsha Chintalapani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33620/
> ---
> 
> (Updated June 23, 2015, 8:18 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1690
> https://issues.apache.org/jira/browse/KAFKA-1690
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. SSLFactory tests.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Added 
> PrincipalBuilder.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Fixed minor 
> issues with the patch.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Fixed minor 
> issues with the patch.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> Merge remote-tracking branch 'refs/remotes/origin/trunk' into KAFKA-1690-V1
> 
> 
> KAFKA-1690. Broker side ssl changes.
> 
> 
> KAFKA-1684. SSL for socketServer.
> 
> 
> KAFKA-1690. Added SSLProducerSendTest and fixes to get right port for SSL.
> 
> 
> Merge branch 'trunk' into KAFKA-1690-V1
> 
> 
> KAFKA-1690. Post merge fixes.
> 
> 
> KAFKA-1690. Added SSLProducerSendTest.
> 
> 
> KAFKA-1690. Minor fixes based on patch review comments.
> 
> 
> Diffs
> -
> 
>   build.gradle 30d1cf2f1ff9ed3f86a060da8099bb0774b4cf91 
>   checkstyle/import-control.xml f2e6cec267e67ce8e261341e373718e14a8e8e03 
>   clients/src/main/java/org/apache/kafka/clients/ClientUtils.java 
> 0d68bf1e1e90fe9d5d4397ddf817b9a9af8d9f7a 
>   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
> 2c421f42ed3fc5d61cf9c87a7eaa7bb23e26f63b 
>   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
> 48fe7961e2215372d8033ece4af739ea06c6457b 
>   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
> daff34db5bf2144e9dc274b23dc56b88f4efafdc 
>   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
> 951c34c92710fc4b38d656e99d2a41255c60aeb7 
>   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
> 5a37580ec69af08b97cf5b43b241790ba8c129dd 
>   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
> aa264202f2724907924985a5ecbe74afc4c6c04b 
>   clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java 
> bae528d31516679bed88ee61b408f209f185a8cc 
>   clients/src/main/java/org/apache/kafka/common/config/SSLConfigs.java 
> PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/Authenticator.java 
> PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/ByteBufferSend.java 
> df0e6d5105ca97b7e1cb4d334ffb7b443506bd0b 
>   clients/src/main/java/org/apache/kafka/common/network/Channel.java 
> PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/ChannelBuilder.java 
> PRE-CREATION 
>   
> clients/src/main/java/org/apache/kafka/common/network/DefaultAuthenticator.java
>  PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/NetworkReceive.java 
> 3ca0098b8ec8cfdf81158465b2d40afc47eb6f80 
>   
> clients/src/main/java/org/apache/kafka/common/network/PlainTextChannelBuilder.java
>  PRE-CREATION 
>   
> clients/src/main/java/org/apache/kafka/common/network/PlainTextTransportLayer.java
>  PRE-CREATION 
>   
> clients/src/main/java/org/apache/kafka/common/network/SSLChannelBuilder.java 
> PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/SSLFactory.java 
> PRE-CREATION 
>   
> clients/src/main/java/org/apache/kafka/common

Re: Official Kafka Gitter Room?

[Neha Narkhede]

FWIW, if the apache-kafka channel was on Slack, I'd hang out there :-)

On Sat, Jul 18, 2015 at 7:20 AM, Ismael Juma  wrote:

> Hi Grant,
>
> I have been considering proposing the same thing myself for similar
> reasons, but I also have some doubts. More comments inline.
>
> On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke  wrote:
>
> > I think it could be useful to have a place to chat that is associated
> with
> > the Kafka repo.
>
>
> Agreed.
>
>
> > Note that we do currently have an IRC channel, but from my
> > experience its a ghost town.
> >
>
> I've recently joined #apache-kafka and activity is low, yes, but there is
> some[1]. The bigger issue, perhaps, is that contributors are not there. Do
> we think that this would be different with Gitter? And, if so, what is the
> reason? Is it because no application needs to be installed? In theory,
> IRCCloud provides something similar for IRC (although the limitations of
> the trial and the upgrade nagging can be annoying) and we could link to the
> channel in GitHub's readme.
>
> Also, are there any Apache projects who use Gitter? There was a thread in
> the Spark mailing list, but the response wasn't positive[2]. We would need
> Apache Infra to set-up Gitter for us as they are the only ones that have
> write access to the repository (the set-up is really easy though, which is
> definitely a positive).
>
> Gitter may be the right answer, I just want to make sure it actually helps
> us have a more active channel instead of splitting the existing activity
> between two channels.
>
> Thoughts?
>
> Best,
> Ismael
>
> [1] https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
> [2]
>
> http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html
>



-- 
Thanks,
Neha

Re: Merge improvements back into Kafka Metrics?

[Neha Narkhede]

Felix,

That sounds great and thanks for volunteering to submit the patch. We are
happy to take the change above pending review. I also like the idea of
releasing the kafka-metrics as a separate artifact as part of every Kafka
release.

Again, thanks for your help and look forward to the patch.

Thanks,
Neha

On Fri, Jul 17, 2015 at 11:33 AM, Felix GV  wrote:

> Hi,
>
> We've been using the new Kafka Metrics within Voldemort for a little while
> now, and we have made some improvements to the library that you might like
> to copy back into Kafka proper. You can view the changes that went in after
> we forked here:
>
> https://github.com/tehuti-io/tehuti/commits/master
>
> The most critical ones are probably these two:
>
>- A pretty simpe yet nasty bug in Percentile that pretty much made
>Histograms useless otherwise:
>
> https://github.com/tehuti-io/tehuti/commit/913dcc0dcc79e2ce87a4c3e52a1affe2aaae9948
>- A few improvements to SampledStat (unfortunately littered across
>several commits) were made to prevent spurious values from being
> measured
>out of a disproportionately small time window (either initally, or
> because
>all windows expired in the case of an infrequently used stat) :
>
> https://github.com/tehuti-io/tehuti/blob/master/src/main/java/io/tehuti/metrics/stats/SampledStat.java
>
> There were other minor changes here and there, to make the APIs more usable
> (IMHO) though that may be a matter of personal taste more than correctness.
>
> If you're interested in the above changes, I could put together a patch and
> file a JIRA. Or someone else can do it if they prefer.
>
> On an unrelated note, if you do merge the changes back into Kafka, it would
> be nice if you considered releasing kafka-metrics as a standalone artifact.
> Voldemort could depend on kafka-metrics rather than tehuti if it was fixed
> properly, but it would be a stretch for Voldemort to depend on all of Kafka
> (or even Kafka clients...). The fork was just to iterate quicker at the
> time we needed this, but it would be nice to bring it back together.
>
> Let me know if I can help in any way.
>
> --
> *Felix GV*
> Senior Software Engineer
> Data Infrastructure
> LinkedIn
>
> f...@linkedin.com
> linkedin.com/in/felixgv
>



-- 
Thanks,
Neha

Re: Review Request 36593: Patch for KAFKA-2347

[Ashish Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36593/
---

(Updated July 18, 2015, 10:14 p.m.)


Review request for kafka.


Bugs: KAFKA-2347
https://issues.apache.org/jira/browse/KAFKA-2347


Repository: kafka


Description (updated)
---

KAFKA-2347: Add setConsumerRebalanceListener method to 
ZookeeperConsumerConnector java api


Diffs (updated)
-

  core/src/main/scala/kafka/javaapi/consumer/ConsumerConnector.java 
ca74ca8abf03478b6de4ec1f82fbac379e7603f1 

Diff: https://reviews.apache.org/r/36593/diff/


Testing
---


Thanks,

Ashish Singh

[jira] [Commented] (KAFKA-2347) Add setConsumerRebalanceListener method to ZookeeperConsuemrConnector java api.

[Ashish K Singh (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632621#comment-14632621
 ] 

Ashish K Singh commented on KAFKA-2347:
---

Updated reviewboard https://reviews.apache.org/r/36593/
 against branch trunk

> Add setConsumerRebalanceListener method to ZookeeperConsuemrConnector java 
> api.
> ---
>
> Key: KAFKA-2347
> URL: https://issues.apache.org/jira/browse/KAFKA-2347
> Project: Kafka
>  Issue Type: Bug
>Reporter: Jiangjie Qin
>Assignee: Ashish K Singh
> Attachments: KAFKA-2347.patch, KAFKA-2347_2015-07-18_15:14:26.patch
>
>
> The setConsumerRebalanceListener() method is in scala API but not in java 
> api. Needs to add it back.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-2347) Add setConsumerRebalanceListener method to ZookeeperConsuemrConnector java api.

[Ashish K Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-2347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish K Singh updated KAFKA-2347:
--
Attachment: KAFKA-2347_2015-07-18_15:14:26.patch

> Add setConsumerRebalanceListener method to ZookeeperConsuemrConnector java 
> api.
> ---
>
> Key: KAFKA-2347
> URL: https://issues.apache.org/jira/browse/KAFKA-2347
> Project: Kafka
>  Issue Type: Bug
>Reporter: Jiangjie Qin
>Assignee: Ashish K Singh
> Attachments: KAFKA-2347.patch, KAFKA-2347_2015-07-18_15:14:26.patch
>
>
> The setConsumerRebalanceListener() method is in scala API but not in java 
> api. Needs to add it back.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 36593: Patch for KAFKA-2347

[Ismael Juma]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36593/#review92190
---

Ship it!


Ship It!

- Ismael Juma


On July 18, 2015, 10:14 p.m., Ashish Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36593/
> ---
> 
> (Updated July 18, 2015, 10:14 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-2347
> https://issues.apache.org/jira/browse/KAFKA-2347
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-2347: Add setConsumerRebalanceListener method to 
> ZookeeperConsumerConnector java api
> 
> 
> Diffs
> -
> 
>   core/src/main/scala/kafka/javaapi/consumer/ConsumerConnector.java 
> ca74ca8abf03478b6de4ec1f82fbac379e7603f1 
> 
> Diff: https://reviews.apache.org/r/36593/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ashish Singh
> 
>

Re: Official Kafka Gitter Room?

[Andrew Psaltis]

+1 for slack

On Saturday, July 18, 2015, Neha Narkhede  wrote:

> FWIW, if the apache-kafka channel was on Slack, I'd hang out there :-)
>
> On Sat, Jul 18, 2015 at 7:20 AM, Ismael Juma  > wrote:
>
> > Hi Grant,
> >
> > I have been considering proposing the same thing myself for similar
> > reasons, but I also have some doubts. More comments inline.
> >
> > On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke  > wrote:
> >
> > > I think it could be useful to have a place to chat that is associated
> > with
> > > the Kafka repo.
> >
> >
> > Agreed.
> >
> >
> > > Note that we do currently have an IRC channel, but from my
> > > experience its a ghost town.
> > >
> >
> > I've recently joined #apache-kafka and activity is low, yes, but there is
> > some[1]. The bigger issue, perhaps, is that contributors are not there.
> Do
> > we think that this would be different with Gitter? And, if so, what is
> the
> > reason? Is it because no application needs to be installed? In theory,
> > IRCCloud provides something similar for IRC (although the limitations of
> > the trial and the upgrade nagging can be annoying) and we could link to
> the
> > channel in GitHub's readme.
> >
> > Also, are there any Apache projects who use Gitter? There was a thread in
> > the Spark mailing list, but the response wasn't positive[2]. We would
> need
> > Apache Infra to set-up Gitter for us as they are the only ones that have
> > write access to the repository (the set-up is really easy though, which
> is
> > definitely a positive).
> >
> > Gitter may be the right answer, I just want to make sure it actually
> helps
> > us have a more active channel instead of splitting the existing activity
> > between two channels.
> >
> > Thoughts?
> >
> > Best,
> > Ismael
> >
> > [1] https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
> > [2]
> >
> >
> http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html
> >
>
>
>
> --
> Thanks,
> Neha
>

Re: Review Request 36590: Patch for KAFKA-2275

[Jason Gustafson]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36590/#review92194
---



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
(line 1039)


Would it make sense to move the large part of this implementation into 
Fetcher? We've moved a lot of similar logic out of KafkaConsumer to keep its 
implementation simple. This also might make unit testing easier.



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
(line 1054)


Adding the topic to the Metadata object means that from this point forward, 
we will always fetch the associated metadata for whatever topics were used in 
partitionsFor, even if we don't actually care about them anymore. Seems a 
little unfortunate, though I doubt it's much of an issue since users would 
probably only call this method for subscribed topics.



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
(line 1061)


Seems like we force a metadata refresh even if there are not topics whose 
metadata is missing.



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
(line 1069)


It's not clear to me how this is sufficient in order to get all topics. 
Wouldn't this only include topics which have been added to Metadata?


Perhaps a minor concern, but I feel a little wary about mutating the Metadata 
state that is used internally by KafkaConsumer in this approach. Feels like 
there ought to be a way to request the metadata we're interested in directly 
instead. It would involve a change to NetworkClient, but it might be worth 
looking into, at least to see the level of effort.

- Jason Gustafson


On July 18, 2015, 4:39 a.m., Ashish Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36590/
> ---
> 
> (Updated July 18, 2015, 4:39 a.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-2275
> https://issues.apache.org/jira/browse/KAFKA-2275
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> Return metadata for all topics if empty list is passed to partitionsFor
> 
> 
> KAFKA-2275: Add a "Map> partitionsFor(String... 
> topics)" API to the new consumer
> 
> 
> Diffs
> -
> 
>   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
> 252b759c0801f392e3526b0f31503b4b8fbf1c8a 
>   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
> bea3d737c51be77d5b5293cdd944d33b905422ba 
>   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
> c14eed1e95f2e682a235159a366046f00d1d90d6 
>   core/src/test/scala/integration/kafka/api/ConsumerTest.scala 
> 3eb5f95731a3f06f662b334ab2b3d0ad7fa9e1ca 
> 
> Diff: https://reviews.apache.org/r/36590/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ashish Singh
> 
>

Re: Official Kafka Gitter Room?

[Gwen Shapira]

Slack sounds good and I'm not a fan of gitter (we use it for Apache
Sqoop, I dislike the GUI and the guilt-inducing emails about all the
messages I missed...)
Do they have free rooms for open source projects?

On Sat, Jul 18, 2015 at 6:10 PM, Andrew Psaltis
 wrote:
> +1 for slack
>
> On Saturday, July 18, 2015, Neha Narkhede  wrote:
>
>> FWIW, if the apache-kafka channel was on Slack, I'd hang out there :-)
>>
>> On Sat, Jul 18, 2015 at 7:20 AM, Ismael Juma > > wrote:
>>
>> > Hi Grant,
>> >
>> > I have been considering proposing the same thing myself for similar
>> > reasons, but I also have some doubts. More comments inline.
>> >
>> > On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke > > wrote:
>> >
>> > > I think it could be useful to have a place to chat that is associated
>> > with
>> > > the Kafka repo.
>> >
>> >
>> > Agreed.
>> >
>> >
>> > > Note that we do currently have an IRC channel, but from my
>> > > experience its a ghost town.
>> > >
>> >
>> > I've recently joined #apache-kafka and activity is low, yes, but there is
>> > some[1]. The bigger issue, perhaps, is that contributors are not there.
>> Do
>> > we think that this would be different with Gitter? And, if so, what is
>> the
>> > reason? Is it because no application needs to be installed? In theory,
>> > IRCCloud provides something similar for IRC (although the limitations of
>> > the trial and the upgrade nagging can be annoying) and we could link to
>> the
>> > channel in GitHub's readme.
>> >
>> > Also, are there any Apache projects who use Gitter? There was a thread in
>> > the Spark mailing list, but the response wasn't positive[2]. We would
>> need
>> > Apache Infra to set-up Gitter for us as they are the only ones that have
>> > write access to the repository (the set-up is really easy though, which
>> is
>> > definitely a positive).
>> >
>> > Gitter may be the right answer, I just want to make sure it actually
>> helps
>> > us have a more active channel instead of splitting the existing activity
>> > between two channels.
>> >
>> > Thoughts?
>> >
>> > Best,
>> > Ismael
>> >
>> > [1] https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
>> > [2]
>> >
>> >
>> http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html
>> >
>>
>>
>>
>> --
>> Thanks,
>> Neha
>>

Re: Official Kafka Gitter Room?

[Gwen Shapira]

So, as an experiment, I created:
https://apachekafka.slack.com

I figured we'll give it a whirl for a week or two for dev discussions,
see how it goes and if we have activity we can add this to the website
and announce on the lists.

Gwen

On Sat, Jul 18, 2015 at 6:18 PM, Gwen Shapira  wrote:
> Slack sounds good and I'm not a fan of gitter (we use it for Apache
> Sqoop, I dislike the GUI and the guilt-inducing emails about all the
> messages I missed...)
> Do they have free rooms for open source projects?
>
> On Sat, Jul 18, 2015 at 6:10 PM, Andrew Psaltis
>  wrote:
>> +1 for slack
>>
>> On Saturday, July 18, 2015, Neha Narkhede  wrote:
>>
>>> FWIW, if the apache-kafka channel was on Slack, I'd hang out there :-)
>>>
>>> On Sat, Jul 18, 2015 at 7:20 AM, Ismael Juma >> > wrote:
>>>
>>> > Hi Grant,
>>> >
>>> > I have been considering proposing the same thing myself for similar
>>> > reasons, but I also have some doubts. More comments inline.
>>> >
>>> > On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke >> > wrote:
>>> >
>>> > > I think it could be useful to have a place to chat that is associated
>>> > with
>>> > > the Kafka repo.
>>> >
>>> >
>>> > Agreed.
>>> >
>>> >
>>> > > Note that we do currently have an IRC channel, but from my
>>> > > experience its a ghost town.
>>> > >
>>> >
>>> > I've recently joined #apache-kafka and activity is low, yes, but there is
>>> > some[1]. The bigger issue, perhaps, is that contributors are not there.
>>> Do
>>> > we think that this would be different with Gitter? And, if so, what is
>>> the
>>> > reason? Is it because no application needs to be installed? In theory,
>>> > IRCCloud provides something similar for IRC (although the limitations of
>>> > the trial and the upgrade nagging can be annoying) and we could link to
>>> the
>>> > channel in GitHub's readme.
>>> >
>>> > Also, are there any Apache projects who use Gitter? There was a thread in
>>> > the Spark mailing list, but the response wasn't positive[2]. We would
>>> need
>>> > Apache Infra to set-up Gitter for us as they are the only ones that have
>>> > write access to the repository (the set-up is really easy though, which
>>> is
>>> > definitely a positive).
>>> >
>>> > Gitter may be the right answer, I just want to make sure it actually
>>> helps
>>> > us have a more active channel instead of splitting the existing activity
>>> > between two channels.
>>> >
>>> > Thoughts?
>>> >
>>> > Best,
>>> > Ismael
>>> >
>>> > [1] https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
>>> > [2]
>>> >
>>> >
>>> http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html
>>> >
>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Neha
>>>

[GitHub] kafka pull request: Addresses Jira Kafka-2236 to eliminate an arra...

[jhspaybar]

GitHub user jhspaybar opened a pull request:

https://github.com/apache/kafka/pull/86

Addresses Jira Kafka-2236 to eliminate an array index out of bounds e…

…xception when segments are appended between checks

First pull request, so if I did this incorrectly, let me know.  The 
contribution guidelines here http://kafka.apache.org/contributing.html said to 
post patch files directly to Jira, but it looked like Github was very active, 
but I can post over there as well if preferred.  There didn't seem to be 
existing unit tests here, and about the only way I could see to add them to 
catch this bug and prevent regressions would be quite intrusive.  If this is 
okay to merge, great!  If not due to tests, any tips on how I should go about 
testing this would be appreciated(I would think making the function public, 
annotate with @VisibleForTesting maybe, and lots of mocks?)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jhspaybar/kafka kafka-2236

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/86.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #86


commit 1922ae69acba1601d7843ae8d2687ddf70dfc4a6
Author: William Thurston 
Date:   2015-07-19T02:54:05Z

Addresses Jira Kafka-2236 to eliminate an array index out of bounds 
exception when segments are appended between checks




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Official Kafka Gitter Room?

[Gwen Shapira]

FYI:
Joining Slack requires an invitation. We can take invite-requests in
the mailing lists (like we do for KIP hangout).
I wanted to point this out as this may be considered non-Apache
(although I didn't find any specific wording against invite-only
channels in addition to the mailing list).

So, if you want to try it out, yell for an invite :)

On Sat, Jul 18, 2015 at 6:26 PM, Gwen Shapira  wrote:
> So, as an experiment, I created:
> https://apachekafka.slack.com
>
> I figured we'll give it a whirl for a week or two for dev discussions,
> see how it goes and if we have activity we can add this to the website
> and announce on the lists.
>
> Gwen
>
> On Sat, Jul 18, 2015 at 6:18 PM, Gwen Shapira  wrote:
>> Slack sounds good and I'm not a fan of gitter (we use it for Apache
>> Sqoop, I dislike the GUI and the guilt-inducing emails about all the
>> messages I missed...)
>> Do they have free rooms for open source projects?
>>
>> On Sat, Jul 18, 2015 at 6:10 PM, Andrew Psaltis
>>  wrote:
>>> +1 for slack
>>>
>>> On Saturday, July 18, 2015, Neha Narkhede  wrote:
>>>
 FWIW, if the apache-kafka channel was on Slack, I'd hang out there :-)

 On Sat, Jul 18, 2015 at 7:20 AM, Ismael Juma >>> > wrote:

 > Hi Grant,
 >
 > I have been considering proposing the same thing myself for similar
 > reasons, but I also have some doubts. More comments inline.
 >
 > On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke >>> > wrote:
 >
 > > I think it could be useful to have a place to chat that is associated
 > with
 > > the Kafka repo.
 >
 >
 > Agreed.
 >
 >
 > > Note that we do currently have an IRC channel, but from my
 > > experience its a ghost town.
 > >
 >
 > I've recently joined #apache-kafka and activity is low, yes, but there is
 > some[1]. The bigger issue, perhaps, is that contributors are not there.
 Do
 > we think that this would be different with Gitter? And, if so, what is
 the
 > reason? Is it because no application needs to be installed? In theory,
 > IRCCloud provides something similar for IRC (although the limitations of
 > the trial and the upgrade nagging can be annoying) and we could link to
 the
 > channel in GitHub's readme.
 >
 > Also, are there any Apache projects who use Gitter? There was a thread in
 > the Spark mailing list, but the response wasn't positive[2]. We would
 need
 > Apache Infra to set-up Gitter for us as they are the only ones that have
 > write access to the repository (the set-up is really easy though, which
 is
 > definitely a positive).
 >
 > Gitter may be the right answer, I just want to make sure it actually
 helps
 > us have a more active channel instead of splitting the existing activity
 > between two channels.
 >
 > Thoughts?
 >
 > Best,
 > Ismael
 >
 > [1] https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
 > [2]
 >
 >
 http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html
 >



 --
 Thanks,
 Neha

New Producer and "acks" configuration

[Gwen Shapira]

Hi,

I was looking into the different between acks = 0 and acks = 1 in the
new producer, and was a bit surprised at what I found.

Basically, if I understand correctly, the only difference is that with
acks = 0, if the leader fails to append locally, it closes the network
connection silently and with acks = 1, it sends an actual error
message.

Which seems to mean that with acks = 0, any failed produce will lead
to metadata refresh and a retry (because network error), while acks =
1 will lead to either retries or abort, depending on the error.

Not only this doesn't match the documentation, it doesn't even make
much sense...
"acks = 0" was supposed to somehow makes things "less safe but
faster", and it doesn't seem to be doing that any more. I'm not even
sure there's any case where the "acks = 0" behavior is desirable.

Is it my misunderstanding, or did we somehow screw up the logic here?

Gwen