Re: [VOTE] 1.0.0 RC3

Does the voting deadline also need an update? > *** Please download, test and vote by Friday, October 20, 8pm PT On Wed, Oct 25, 2017 at 10:37 AM, Guozhang Wang wrote: > Ted: > > Thanks for the reminder. Yes it is a typo. In fact this is the "forth" > candidate of the release, not the "third" o

Re: [kafka-clients] [VOTE] 1.0.0 RC3

+1. passed kafka-python integration tests, and manually verified producer/consumer on both compressed and non-compressed data. -Dana On Mon, Oct 23, 2017 at 6:00 PM, Guozhang Wang wrote: > Hello Kafka users, developers and client-developers, > > This is the third candidate for release of Apache

Re: [kafka-clients] [VOTE] 1.0.0 RC2

1.0.0-RC2 passed all kafka-python integration tests. Excited for this release -- great work everyone! -Dana On Tue, Oct 17, 2017 at 9:47 AM, Guozhang Wang wrote: > Hello Kafka users, developers and client-developers, > > This is the third candidate for release of Apache Kafka 1.0.0. The main >

[jira] [Created] (KAFKA-5465) FetchResponse v0 does not return any messages when max_bytes smaller than v2 message set

Dana Powers created KAFKA-5465: -- Summary: FetchResponse v0 does not return any messages when max_bytes smaller than v2 message set Key: KAFKA-5465 URL: https://issues.apache.org/jira/browse/KAFKA-5465

Re: [DISCUSS] KIP-144: Exponential backoff for broker reconnect attempts

clumping of reconnects, but that's maybe ok. Another idea would be to make jitter configurable. Full jitter would be 100%. No jitter 0%. Equal Jitter 50%. Etc. On May 8, 2017 5:28 PM, "Dana Powers" wrote: > For some discussion of jitter and exponential back-office, I found thi

Re: [DISCUSS] KIP-144: Exponential backoff for broker reconnect attempts

s more sense to allow "one last doubling" which > > > may > > > bring us slightly over the maximum, but much closer to it. I.e. have > the > > > effective maximum be in [max.backoff - backoff, max.backoff + backoff] > > > range rather than half that.

Re: [VOTE] KIP-144: Exponential backoff for broker reconnect attempts

+1 ! On May 6, 2017 4:49 AM, "Edoardo Comar" wrote: > +1 (non binding) > thanks > -- > Edoardo Comar > IBM MessageHub > eco...@uk.ibm.com > IBM UK Ltd, Hursley Park, SO21 2JN > > IBM United Kingdom Limited Registered in England and Wales with numbe

Re: [VOTE] Add REST Server to Apache Kafka

-1 On Wed, Oct 26, 2016 at 9:23 AM, Shekar Tippur wrote: > +1 > > Thanks

Re: [kafka-clients] [VOTE] 0.10.1.0 RC3

+1 -- passes kafka-python integration tests On Mon, Oct 17, 2016 at 1:28 PM, Jun Rao wrote: > Thanks for preparing the release. Verified quick start on scala 2.11 binary. > +1 > > Jun > > On Fri, Oct 14, 2016 at 4:29 PM, Jason Gustafson wrote: >> >> Hello Kafka users, developers and client-devel

Re: [kafka-clients] [VOTE] 0.10.1.0 RC2

+1; all kafka-python integration tests pass. -Dana On Wed, Oct 12, 2016 at 10:41 AM, Jason Gustafson wrote: > Hello Kafka users, developers and client-developers, > > One more RC for 0.10.1.0. I think we're getting close! > > Release plan: > https://cwiki.apache.org/confluence/display/KAFKA/Rel

Re: [DISCUSS] KIP-82 - Add Record Headers

> I agree with the critique of compaction not having a value. I think we should > consider fixing that directly. Agree that the compaction issue is troubling: compacted "null" deletes are incompatible w/ headers that must be packed into the message value. Are there any alternatives on compaction

Consul / Zookeeper [was Re: any update on this?]

[+ dev list] I have not worked on KAFKA-1793 directly, but I believe most of the work so far has been in removing all zookeeper dependencies from clients. The two main areas for that are (1) consumer rebalancing, and (2) administrative commands. 1) Consumer rebalancing APIs were added to the brok

Re: [VOTE] 0.10.1 Release Plan

+1

Re: Review request for KAFKA-3600

I think one of the hard parts is that while we agree that a call to ApiVersions is required, I don't think there is agreement on how to use the response. In kafka-python, for example, to support backwards compatibility we set a single "api" configuration value during the client constructor. This v

Re: [VOTE] KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change

+1 (still confused about bindings) On Tue, Aug 9, 2016 at 6:24 AM, Ismael Juma wrote: > Thanks for the KIP. +1 (binding) > > Ismael > > On Mon, Aug 8, 2016 at 7:53 PM, Vahid S Hashemian > wrote: > >> I would like to initiate the voting process for KIP-70 ( >> https://cwiki.apache.org/confluence/d

Re: [VOTE] 0.10.0.1 RC2

passed kafka-python integration tests, +1 -Dana On Fri, Aug 5, 2016 at 9:35 AM, Tom Crayford wrote: > Heroku has tested this using the same performance testing setup we used to > evaluate the impact of 0.9 -> 0.10 (see https://engineering. > heroku.com/blogs/2016-05-27-apache-kafka-010-evaluati

Re: [kafka-clients] [VOTE] 0.10.0.1 RC0

+1 tested against kafka-python integration test suite = pass. Aside: as the scope of kafka gets bigger, it may be useful to organize release notes into functional groups like core, brokers, clients, kafka-streams, etc. I've found this useful when organizing kafka-python release notes. -Dana On

Re: [DISCUSS] KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change

This is a nice change. Great KIP write up. -Dana On Fri, Jul 22, 2016 at 10:07 AM, Vahid S Hashemian wrote: > Thanks Ismael. > > What do you think is the best way to check with Storm / Spark users? Their > mailing list? > > Thanks. > > Regards, > --Vahid > > > > > From: Ismael Juma > To:

Re: [DISCUSS] KIP-53 Add custom policies for reconnect attempts to NetworkdClient

Thanks for the confirmation, Ismael. I will write something up for further discussion. -Dana On Jul 5, 2016 4:43 PM, "Ismael Juma" wrote: > Hi Dana, > > Thanks for the PR. Technically, a (simple) KIP is required when proposing > new configs. > > Ismael > > On

Re: [VOTE] KIP-4 Delete Topics Schema

+1

Re: [VOTE] KIP-4 Create Topics Schema

+1 -- thanks for the update On Mon, Jun 20, 2016 at 10:49 AM, Grant Henke wrote: > I have update the patch and wiki based on the feedback in the discussion > thread. The only change is that instead of logging and disconnecting in the > case of invalid messages (duplicate topics or both arguments)

[jira] [Commented] (KAFKA-3878) Support exponential backoff for broker reconnect attempts

[ https://issues.apache.org/jira/browse/KAFKA-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15339941#comment-15339941 ] Dana Powers commented on KAFKA-3878: Proposed implementation at https://github

[jira] [Created] (KAFKA-3878) Support exponential backoff for broker reconnect attempts

Dana Powers created KAFKA-3878: -- Summary: Support exponential backoff for broker reconnect attempts Key: KAFKA-3878 URL: https://issues.apache.org/jira/browse/KAFKA-3878 Project: Kafka Issue

Re: [DISCUSS] KIP-53 Add custom policies for reconnect attempts to NetworkdClient

I took a stab at implementing a simplified exponential + randomized backoff policy here: https://github.com/apache/kafka/pull/1523 Rather than changing public interfaces / using plugins, it just adds a new client configuration "reconnect.backoff.max" that can be used to enable increasing backoff w

Consumer autocommit interference via unintentional/internal poll() calls

I searched through jira and the mailing list for prior discussion of this and could not find any. Forgive me if I missed it, and if so please send a link! It was raised in the kafka-python issue list by an astute reader that the KafkaConsumer autocommit semantics can be accidentally broken by cons

Re: [DISCUSS] KIP-4 Create Topic Schema

last resort. I think there is a good opportunity to avoid it in this case, and I think the api would be better if done that way. -Dana > On Fri, Jun 17, 2016 at 12:04 AM, Dana Powers wrote: > >> Why disconnect the client on a InvalidRequestException? The 2 errors >> you are catching

Re: [DISCUSS] KIP-4 Create Topic Schema

topic_error_codes => topic error_code >> topic => STRING >> error_code => INT16 >> >> CreateTopicsResponse contains a map between topic and topic creation >> result error code (see New Protocol Errors >> <https://cwiki.apache.org/confluence/d

Re: [DISCUSS] KIP-4 Create Topic Schema

On Wed, Jun 15, 2016 at 12:19 PM, Ismael Juma wrote: > Hi Grant, > > Comments below. > > On Wed, Jun 15, 2016 at 6:52 PM, Grant Henke wrote: >> >> The one thing I want to avoid is to many super specific error codes. I am >> not sure how much of a problem it really is but in the case of wire >> pr

Re: [DISCUSS] KIP-4 Create Topic Schema

> Ewen's reply sums up my thoughts on the error handling points. It doesn't > seem ideal to justify the wire protocol behaviour based on the Java > implementation. If we need map-like semantics in the protocol, then maybe > we need a `Map` type to complement `Array`? Otherwise, I still think we > s

Re: [kafka-clients] Adding new client to wiki

I believe wiki requests usually go to the kafka dev mailing list (cc'd) On May 26, 2016 6:51 AM, "Vijay Jadhav" wrote: > > > Hi, > > Can someone point out what is procedure for adding "libasynckafkaclient > " (C++ based single > threaded asynchrono

Re: [VOTE] 0.10.0.0 RC6

+1 -- passed kafka-python integration tests -Dana On Fri, May 20, 2016 at 11:16 AM, Joe Stein wrote: > +1 ran quick start from source and binary release > > On Fri, May 20, 2016 at 1:07 PM, Ewen Cheslack-Postava > wrote: > >> +1 validated connect with a couple of simple connectors and console >

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

16 at 1:13 AM, Jun Rao wrote: > > > Thanks for the response. +1 on the KIP. > > > > Jun > > > > On Thu, Apr 28, 2016 at 9:01 AM, Dana Powers > > wrote: > > > > > Sure thing. Yes, the substantive change is fixing the HC checksum. > > >

Re: KIP-57: Interoperable LZ4 Framing

the classes with Javadoc are > public: https://kafka.apache.org/090/javadoc/index.html) and the > information in the KIP is a bit stale when compared to the PR. > > Ismael > > On Fri, May 6, 2016 at 10:12 PM, Ismael Juma wrote: > >> On Sun, May 1, 2016 at 3:57 AM, Dana P

Re: KIP-57: Interoperable LZ4 Framing

Updated: https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing Should I restart the vote? -Dana On Fri, May 6, 2016 at 2:12 PM, Ismael Juma wrote: > On Sun, May 1, 2016 at 3:57 AM, Dana Powers wrote: >> >> > 2. We're completely disabl

Re: [jira] [Updated] (KAFKA-3160) Kafka LZ4 framing code miscalculates header checksum

You can assign to me also

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

icate varying feature > support if we had to further change that code? What I want to avoid is > clients in some languages having to work with broker version numbers > instead of protocol version numbers due to further incompatibilities we > might find. > > -Ewen > > On Th

Re: [VOTE] 0.10.0.0 RC2

I was unable to use/test the rc2 binary artifact without manually patching kafka-run-class.sh . I'd vote for a new release candidate. -Dana On Mon, May 2, 2016 at 5:44 AM, Ismael Juma wrote: > On second thought, will people be able to test the binary packages if they > can't start the broker? >

Re: KIP-57: Interoperable LZ4 Framing

On Fri, Apr 29, 2016 at 6:29 PM, Ewen Cheslack-Postava wrote: > Two questions: > > 1. My understanding based on KIP-35 is that this won't be a problem for > clients that want to support older broker versions since they will use v0 > produce requests with broken checksum to send to those, and any b

[jira] [Commented] (KAFKA-3615) Exclude test jars in CLASSPATH of kafka-run-class.sh

[ https://issues.apache.org/jira/browse/KAFKA-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265561#comment-15265561 ] Dana Powers commented on KAFKA-3615: This PR has a bug that breaks the class

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

g left unsupported is dependent-block compression, which >our implementation does not currently support. > > > Thanks, > > Jun > > On Mon, Apr 25, 2016 at 2:26 PM, Dana Powers wrote: > >> Hi all, >> >> Initiating a vote thread because the

[VOTE] KIP-57: Interoperable LZ4 Framing

Hi all, Initiating a vote thread because the KIP-57 proposal is specific to the 0.10 release. KIP-57 can be accessed here: . The related JIRA is https://issues.apache.org/jira/browse/KAFKA-3160 and working gith

KIP-57: Interoperable LZ4 Framing

Hi all, I've written up a new KIP based on KAFKA-3160 / fixing LZ4 framing. The write-up is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing Please take a look if you are using LZ4 compression or are interested in framing specs. One question: does anyon

wiki privs

I'd like to create a new KIP page on the wiki, but I can't figure out how to create a new page or edit an existing page. Do I need certain privileges on my user account to be able to do this? Thanks, -Dana

[jira] [Created] (KAFKA-3550) Broker does not honor MetadataRequest api version; always returns v0 MetadataResponse

Dana Powers created KAFKA-3550: -- Summary: Broker does not honor MetadataRequest api version; always returns v0 MetadataResponse Key: KAFKA-3550 URL: https://issues.apache.org/jira/browse/KAFKA-3550

Re: [VOTE] KIP-4 Metadata Schema (Round 2)

+1 On Apr 11, 2016 21:55, "Gwen Shapira" wrote: > +1 > > On Mon, Apr 11, 2016 at 10:42 AM, Grant Henke wrote: > > Based on the discussion in the previous vote thread > > < > http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema > > > > I also would like to include a behav

[jira] [Commented] (KAFKA-3160) Kafka LZ4 framing code miscalculates header checksum

[ https://issues.apache.org/jira/browse/KAFKA-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234221#comment-15234221 ] Dana Powers commented on KAFKA-3160: Magnus: have you made any progress on this?

[jira] [Updated] (KAFKA-3160) Kafka LZ4 framing code miscalculates header checksum

[ https://issues.apache.org/jira/browse/KAFKA-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dana Powers updated KAFKA-3160: --- Description: KAFKA-1493 partially implements the LZ4 framing specification, but it incorrectly

[jira] [Updated] (KAFKA-3160) Kafka LZ4 framing code miscalculates header checksum

[ https://issues.apache.org/jira/browse/KAFKA-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dana Powers updated KAFKA-3160: --- Affects Version/s: 0.9.0.1 Labels: compatibility compression lz4 (was

Re: How will KIP-35 and KIP-43 play together?

I don't have anything specific to say wrt SASL features, but I think this circumstance makes it clear that there are only 2 ways forward: (1) official java client continues releasing w/ broker versioning as an implicit compatibility test ("java client X.Y requires broker X.Y") AND support is added

Re: KIP-4 Wiki Update

Perhaps python has corrupted my brain, but a null arg seems quite clean to me: getTopics() -> returns all getTopics([]) -> returns none getTopics([foo, bar]) -> returns foo and bar only -Dana On Wed, Mar 30, 2016 at 9:10 AM, Jason Gustafson wrote: > > > > Yes, I think empty should be "no topic

Re: KIP-4 Wiki Update

Grant - sorry I was unable to attend. Getting API access to admin functionality has been a big ask for python client users. I like this KIP a lot. I reviewed the details quickly. Here are some comments: MetadataRequest v1: long-term / conceptually, I think a "null" topic list aligns better with f

Re: [VOTE] 0.10.0.0 RC1

+1 -- verified that all kafka-python integration tests now pass On Mon, Mar 28, 2016 at 2:34 PM, Gwen Shapira wrote: > Hello Kafka users, developers and client-developers, > > This is the second candidate for release of Apache Kafka 0.10.0.0. > > This is a major release that includes: > > (1) Ne

Re: [VOTE] KIP-35: Retrieving protocol version

(and we caught this breaks twice in the last two weeks) * Error handling of protocol mismatches Ashish kindly agreed to think about this and improve the KIP. We'll resume the vote as soon as he's back :) Gwen On Wed, Mar 23, 2016 at 5:55 PM, Dana Powers wrote: > speaking of pend

Re: [VOTE] KIP-35: Retrieving protocol version

speaking of pending KIPs, what's the status on this one? On Fri, Mar 18, 2016 at 9:47 PM, Ashish Singh wrote: > Hey Jay, > > Answers inline. > > On Fri, Mar 18, 2016 at 10:45 AM, Jay Kreps wrote: > > Hey Ashish, > > > > Couple quick things: > > > > 1. You list as a rejected alternative "making

[jira] [Commented] (KAFKA-3442) FetchResponse size exceeds max.partition.fetch.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208721#comment-15208721 ] Dana Powers commented on KAFKA-3442: +1. verified test passes on trunk co

[jira] [Commented] (KAFKA-3442) FetchResponse size exceeds max.partition.fetch.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207142#comment-15207142 ] Dana Powers commented on KAFKA-3442: sounds good to me! > FetchResponse size

[jira] [Commented] (KAFKA-3442) FetchResponse size exceeds max.partition.fetch.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207068#comment-15207068 ] Dana Powers commented on KAFKA-3442: I think old clients might work better if

[jira] [Commented] (KAFKA-3442) FetchResponse size exceeds max.partition.fetch.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15206876#comment-15206876 ] Dana Powers commented on KAFKA-3442: [~junrao] Actually I meant adding the error

[jira] [Commented] (KAFKA-3442) FetchResponse size exceeds max.partition.fetch.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15206790#comment-15206790 ] Dana Powers commented on KAFKA-3442: yes, kafka-python uses the same check

[jira] [Commented] (KAFKA-3442) FetchResponse size exceeds max.partition.fetch.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205655#comment-15205655 ] Dana Powers commented on KAFKA-3442: In all prior broker releases clients check

[jira] [Comment Edited] (KAFKA-3442) FetchResponse size exceeds max.partition.fetch.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205411#comment-15205411 ] Dana Powers edited comment on KAFKA-3442 at 3/22/16 12:1

[jira] [Commented] (KAFKA-3442) FetchResponse size exceeds max.partition.fetch.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205411#comment-15205411 ] Dana Powers commented on KAFKA-3442: # clone kafka-python repo git clone h

[jira] [Updated] (KAFKA-3442) FetchResponse size exceeds max.partition.fetch.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dana Powers updated KAFKA-3442: --- Priority: Blocker (was: Major) > FetchResponse size exceeds max.partition.fetch.by

Re: [VOTE] 0.10.0.0 RC0

I filed a bug re max.partition.fetch.bytes here: https://issues.apache.org/jira/browse/KAFKA-3442 Would it be useful to create a 0.10.0.0-rc0 version for JIRA tickets? Or should issues just get filed against 0.10.0.0 ? -Dana On Mon, Mar 21, 2016 at 1:53 PM, Gwen Shapira wrote: > Hello Kafka u

[jira] [Created] (KAFKA-3442) FetchResponse size exceeds max.partition.fetch.bytes

Dana Powers created KAFKA-3442: -- Summary: FetchResponse size exceeds max.partition.fetch.bytes Key: KAFKA-3442 URL: https://issues.apache.org/jira/browse/KAFKA-3442 Project: Kafka Issue Type

Re: [DISCUSS] KIP-35 - Retrieve protocol version

confluent.io > > > > > > > > > >> >> > wrote: > > > > > > > > > >> >> > > > > > > > > > >> >>> Yeah, Gwen's example is a good one. And it doesn't > > even > > > >

Re: [DISCUSS] KIP-35 - Retrieve protocol version

On Thu, Mar 17, 2016 at 1:42 PM, Gwen Shapira wrote: > "I think I would make this approach work by looking at the released server > version documentation for each version that I am trying to support and test > against*, manually identify the expected "protocol vectors" each supports, > store that

Re: [VOTE] KIP-35: Retrieving protocol version

+1 On Thu, Mar 17, 2016 at 4:09 PM, Ashish Singh wrote: > As the KIP has been modified since we started this vote, the vote is > restarted from now. > > The updated KIP is available at > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-35+-+Retrieving+protocol+version > . > > On Mon, Mar

Re: [DISCUSS] KIP-35 - Retrieve protocol version

On Thu, Mar 17, 2016 at 2:07 PM, Jason Gustafson wrote: > It would also be nice to discuss the way the current client > versioning works and why it is inadequate for third-party clients. > My understanding of the java client versioning is that it is not backwards-compatible. Instructions are to

Re: [DISCUSS] KIP-35 - Retrieve protocol version

Is a linear protocol int consistent with the current release model? It seems like that would break down w/ the multiple release branches that are all simultaneously maintained? Or is it implicit that no patch release can ever bump the protocol int? Or maybe the protocol int gets some extra "wiggle"

Re: [DISCUSS] KIP-35 - Retrieve protocol version

There seems to be a lot of tension between support for release-deploys vs. trunk-deploys. Is there a better way to handle this? In my experience the vast majority of third-party client code is written managing compatibility on the first case (release-deploys). I would prefer a simple approach that

Re: [DISCUSS] KIP-35 - Retrieve protocol version

In kafka-python we've been doing something like: if version >= (0, 9): Do cool new stuff elif version >= (0, 8, 2): Do some older stuff else: raise UnsupportedVersionError This will break if / when the new 0.9 apis are completely removed from some future release, but should handle inte

Re: [DISCUSS] KIP-35 - Retrieve protocol version

This is a fantastic and much-needed KIP. All third-party clients have had to deal with this issue. In my experience most clients are either declaring they only support version broker version X, or are spending a lot of time hacking around the issue. I think the community of non-java drivers would s

[jira] [Commented] (KAFKA-3200) MessageSet from broker seems invalid

[ https://issues.apache.org/jira/browse/KAFKA-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131142#comment-15131142 ] Dana Powers commented on KAFKA-3200: the minimum Message payload size is 26 byte

[jira] [Commented] (KAFKA-3177) Kafka consumer can hang when position() is called on a non-existing partition.

[ https://issues.apache.org/jira/browse/KAFKA-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15127701#comment-15127701 ] Dana Powers commented on KAFKA-3177: A similar infinite loop happens when

[jira] [Commented] (KAFKA-1493) Use a well-documented LZ4 compression format and remove redundant LZ4HC option

[ https://issues.apache.org/jira/browse/KAFKA-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15120072#comment-15120072 ] Dana Powers commented on KAFKA-1493: filed KAFKA-3160 > Use a well-documen

[jira] [Created] (KAFKA-3160) Kafka LZ4 framing code miscalculates header checksum

Dana Powers created KAFKA-3160: -- Summary: Kafka LZ4 framing code miscalculates header checksum Key: KAFKA-3160 URL: https://issues.apache.org/jira/browse/KAFKA-3160 Project: Kafka Issue Type

[jira] [Commented] (KAFKA-1493) Use a well-documented LZ4 compression format and remove redundant LZ4HC option

[ https://issues.apache.org/jira/browse/KAFKA-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15119703#comment-15119703 ] Dana Powers commented on KAFKA-1493: Hi all - it appears that the header checksum

[jira] [Commented] (KAFKA-3088) 0.9.0.0 broker crash on receipt of produce request with empty client ID

[ https://issues.apache.org/jira/browse/KAFKA-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111020#comment-15111020 ] Dana Powers commented on KAFKA-3088: But client ids are not globally unique, eve

The JIRA Awakens [KAFKA-1841]

Hi all, I've been helping debug an issue filed against kafka-python related to compatibility w/ Hortonworks 2.3.0.0 kafka release. As I understand it, HDP is currently based on snapshots of apache/kafka trunk, merged with some custom patches from HDP itself. In this case, HDP's 2.3.0.0 kafka rele

Re: group protocol/metadata documentation

od reason not to, all consumer > implementation should use the same consumer protocol format so that tooling > will work correctly. > > Thanks, > Jason > > > > On Tue, Nov 24, 2015 at 4:16 PM, Dana Powers > wrote: > > > Hi all - I've been reading t

group protocol/metadata documentation

Hi all - I've been reading through the wiki docs and mailing list threads for the new JoinGroup/SyncGroup/Heartbeat APIs, hoping to add functionality to the python driver. It appears that there is a shared notion of group "protocols" (client publishes supported protocols, coordinator picks protocol

Re: Writing a client: Connection pooling

cc: kafka-clients mailing list On May 18, 2015 4:24 PM, "Warren Falk" wrote: > Thanks Guozhang, > > Actually your previous email was clear and I understood it. Current broker > design means that parallel requests require parallel connections. > > But I think you misunderstood me. I am not askin

Re: 0.8.3 release plan

Re last beta, I think it was extremely useful. We identified big issues wrt API versioning and third-party client compatibility. Even if the server code doesnt get deployed widely, I think the beta period is still a good signal to third party client devs to rev up tests and make any updates neces

Re: Cannot stop Kafka server if zookeeper is shutdown first

While on the subject of zkclient, also consider KAFKA-1793. A more abstract interface to the distributed coordination service that could be configured to use alternatives like consul or etcd would be very useful imho. Dana FWIW - the ZkClient project team have merged the pull request that I had s

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

> clients don't break on unknown errors maybe true for the official java clients, but I dont think the assumption holds true for community-maintained clients and users of those clients. kafka-python generally follows the fail-fast philosophy and raises an exception on any unrecognized error code

Re: 0.8.2.0 behavior change: ReplicaNotAvailableError

nd I think we should work to tighten that process up. Dana Powers Rdio, Inc. dana.pow...@rd.io rdio.com/people/dpkp/ On Wed, Jan 14, 2015 at 5:43 PM, Jun Rao wrote: > Hi, Dana, > > Thanks for reporting this. I investigated this a bit more. What you > observed is the following: a

[jira] [Commented] (KAFKA-1649) Protocol documentation does not indicate that ReplicaNotAvailable can be ignored

[ https://issues.apache.org/jira/browse/KAFKA-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277807#comment-14277807 ] Dana Powers commented on KAFKA-1649: I am only testing from the wire-protocol l

0.8.2.0 behavior change: ReplicaNotAvailableError

to be flagged as a possible issue for 3rd party clients. Also KAFKA-1649 doesn't look like it was ever actually resolved... The protocol document does not mention anything about clients ignoring this error code. Dana Powers Rdio, Inc. dana.pow...@rd.io rdio.com/people/dpkp/

[jira] [Commented] (KAFKA-1649) Protocol documentation does not indicate that ReplicaNotAvailable can be ignored

[ https://issues.apache.org/jira/browse/KAFKA-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277690#comment-14277690 ] Dana Powers commented on KAFKA-1649: I think this could be a problem -- it is

Re: Compatibility + Unknown APIs

PI in the future because it is not supported, or keep retrying because the network is flaking. Dana Powers Rdio, Inc. dana.pow...@rd.io rdio.com/people/dpkp/ On Mon, Jan 12, 2015 at 3:51 PM, Jay Kreps wrote: > Yeah I totally agree--using the closed socket to indicate "not supported" &g

Compatibility + Unknown APIs

API to get the published version of the server (0.8.2 v. 0.8.1.1, etc). Thoughts? Dana Powers Rdio, Inc. dana.pow...@rd.io rdio.com/people/dpkp/ *stacktrace: ``` [2015-01-12 13:03:55,719] ERROR Closing socket for /127.0.0.1 because of error (kafka.network.Processor) kafka.common.KafkaException: Wro

[jira] [Commented] (KAFKA-1634) Improve semantics of timestamp in OffsetCommitRequests and update documentation

[ https://issues.apache.org/jira/browse/KAFKA-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14265042#comment-14265042 ] Dana Powers commented on KAFKA-1634: possibly related to this JIRA: KAFKA-1841 .

[jira] [Updated] (KAFKA-1841) OffsetCommitRequest API - timestamp field is not versioned

[ https://issues.apache.org/jira/browse/KAFKA-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dana Powers updated KAFKA-1841: --- Description: Timestamp field was added to the OffsetCommitRequest wire protocol api for 0.8.2 by

[jira] [Created] (KAFKA-1841) OffsetCommitRequest API - timestamp field is not versioned

Dana Powers created KAFKA-1841: -- Summary: OffsetCommitRequest API - timestamp field is not versioned Key: KAFKA-1841 URL: https://issues.apache.org/jira/browse/KAFKA-1841 Project: Kafka Issue