[
https://issues.apache.org/jira/browse/KAFKA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke resolved KAFKA-2124.
Resolution: Duplicate
Assignee: Grant Henke
> gradlew is not working on a fresh check
[
https://issues.apache.org/jira/browse/KAFKA-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke resolved KAFKA-4906.
Resolution: Won't Fix
> Support 0.9 brokers with a newer Producer or Consumer vers
[
https://issues.apache.org/jira/browse/KAFKA-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927449#comment-15927449
]
Grant Henke commented on KAFKA-4906:
[~ijuma] Following up with a summary of some of our out of band
[
https://issues.apache.org/jira/browse/KAFKA-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-4906:
---
Fix Version/s: (was: 0.10.2.1)
> Support 0.9 brokers with a newer Producer or Consumer vers
Grant Henke created KAFKA-4906:
--
Summary: Support 0.9 brokers with a newer Producer or Consumer
version
Key: KAFKA-4906
URL: https://issues.apache.org/jira/browse/KAFKA-4906
Project: Kafka
ka.apache.org/msg65697.html
> > >
> > > best,
> > > Colin
> > >
> >
> >
> >
> > --
> > *Gwen Shapira*
> > Product Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > <http://www.confluent.io/blog>
> >
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
KIP-
> > 122%3A+Add+Reset+Consumer+Group+Offsets+tooling
> >
> >
> > Thanks!
> >
> > Jorge.
> >
>
>
>
> --
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter <https://twitter.com/
Feel free to create a KIP based on that discussion and drive the jira. I
don't think a KIP exists yet.
On Thu, Mar 2, 2017 at 1:28 PM, Jeff Widman <j...@netskope.com> wrote:
> Thanks, that's the ticket I was thinking of.
>
> On Thu, Mar 2, 2017 at 11:19 AM, Grant Henke <g
searched, but couldn't find anything in JIRA... am I imagining things?
>
> Now that there's API support for creating topics, the version bump to
> 0.11.0 seems like a good time to re-evaluate whether this default should be
> flipped to false.
>
> I'm happy to create a KIP if need
>
> > The vote will run for a minimum of 72 hours.
> >
> > Thanks,
> > Ismael
> >
>
>
>
> --
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> <http://www.confluent.io/blog>
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
[
https://issues.apache.org/jira/browse/KAFKA-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879124#comment-15879124
]
Grant Henke commented on KAFKA-2729:
I am curious if everyone on this Jira is actually seeing
[
https://issues.apache.org/jira/browse/KAFKA-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872139#comment-15872139
]
Grant Henke commented on KAFKA-4754:
{quote}
This could expose the password to anyone who is able
[
https://issues.apache.org/jira/browse/KAFKA-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872127#comment-15872127
]
Grant Henke commented on KAFKA-4754:
{quote}
Hmm. It is not a good practice to pass passwords through
[
https://issues.apache.org/jira/browse/KAFKA-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-4754:
---
Status: Patch Available (was: Open)
> Correctly parse '=' characters in command line overri
[
https://issues.apache.org/jira/browse/KAFKA-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860597#comment-15860597
]
Grant Henke commented on KAFKA-4754:
Its worth noting, it was also possible to echo out passwords
Grant Henke created KAFKA-4754:
--
Summary: Correctly parse '=' characters in command line overrides
Key: KAFKA-4754
URL: https://issues.apache.org/jira/browse/KAFKA-4754
Project: Kafka
Issue
org/confluence/display/KAFKA/KIP-
> 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
> >
> >
> > The vote will run for a minimum of 72 hours.
> >
> > Thanks,
> > Ismael
> >
> >
> >
> > Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> 3AU
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
[
https://issues.apache.org/jira/browse/KAFKA-4746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858709#comment-15858709
]
Grant Henke commented on KAFKA-4746:
I just mean that often when working with a compacted topic you
Grant Henke created KAFKA-4746:
--
Summary: Offsets can be committed for the offsets topic
Key: KAFKA-4746
URL: https://issues.apache.org/jira/browse/KAFKA-4746
Project: Kafka
Issue Type: Bug
gt;
> > >> Thanks,
> > >> Manikumar
> > >>
> > >>
> > >>
> > >>
> > >> >
> > >> >
> > >> > >
> > >> > > Thanks.
> > >> > > Manikumar
> > >> > >
> > >> > >
> > >> > > >
> > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> > >> manikumar.re...@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi,
> > >> > > > >
> > >> > > > > I would like to initiate the vote on KIP-48:
> > >> > > > >
> > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > >> > > > > Delegation+token+support+for+Kafka
> > >> > > > >
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Manikumar
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> >
> >
> > --
> > Gwen Shapira
> > Product Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter | blog
> >
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>>>> I see. Now I agree one-broker-per-disk should be more efficient in
> >> terms
> >>>> of
> >>>>> GC since each broker probably needs less than 1/10 of the memory
> >>>> available
> >>>>> on a typical machine nowadays. I will remove this from the reason of
> >>>>> rejection.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> Disk failure is the "easy" case. The "hard" case, which is
> >>>>>> unfortunately also the much more common case, is disk misbehavior.
> >>>>>> Towards the end of their lives, disks tend to start slowing down
> >>>>>> unpredictably. Requests that would have completed immediately
> before
> >>>>>> start taking 20, 100 500 milliseconds. Some files may be readable
> and
> >>>>>> other files may not be. System calls hang, sometimes forever, and
> the
> >>>>>> Java process can't abort them, because the hang is in the kernel.
> It
> >>>> is
> >>>>>> not fun when threads are stuck in "D state"
> >>>>>> http://stackoverflow.com/questions/20423521/process-perminan
> >>>>>> tly-stuck-on-d-state
> >>>>>> . Even kill -9 cannot abort the thread then. Fortunately, this is
> >>>>>> rare.
> >>>>>>
> >>>>>
> >>>>> I agree it is a harder problem and it is rare. We probably don't have
> >> to
> >>>>> worry about it in this KIP since this issue is orthogonal to whether
> or
> >>>> not
> >>>>> we use JBOD.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> Another approach we should consider is for Kafka to implement its
> own
> >>>>>> storage layer that would stripe across multiple disks. This
> wouldn't
> >>>>>> have to be done at the block level, but could be done at the file
> >>>> level.
> >>>>>> We could use consistent hashing to determine which disks a file
> should
> >>>>>> end up on, for example.
> >>>>>>
> >>>>>
> >>>>> Are you suggesting that we should distribute log, or log segment,
> >> across
> >>>>> disks of brokers? I am not sure if I fully understand this approach.
> My
> >>>> gut
> >>>>> feel is that this would be a drastic solution that would require
> >>>>> non-trivial design. While this may be useful to Kafka, I would prefer
> >> not
> >>>>> to discuss this in detail in this thread unless you believe it is
> >>>> strictly
> >>>>> superior to the design in this KIP in terms of solving our use-case.
> >>>>>
> >>>>>
> >>>>>> best,
> >>>>>> Colin
> >>>>>>
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Dong
> >>>>>>>
> >>>>>>> On Wed, Jan 25, 2017 at 11:34 AM, Colin McCabe <cmcc...@apache.org
> >
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Dong,
> >>>>>>>>
> >>>>>>>> Thanks for the writeup! It's very interesting.
> >>>>>>>>
> >>>>>>>> I apologize in advance if this has been discussed somewhere else.
> >>>>> But
> >>>>>> I
> >>>>>>>> am curious if you have considered the solution of running multiple
> >>>>>>>> brokers per node. Clearly there is a memory overhead with this
> >>>>>> solution
> >>>>>>>> because of the fixed cost of starting multiple JVMs. However,
> >>>>> running
> >>>>>>>> multiple JVMs would help avoid scalability bottlenecks. You could
> >>>>>>>> probably push more RPCs per second, for example. A garbage
> >>>>> collection
> >>>>>>>> in one broker would not affect the others. It would be
> interesting
> >>>>> to
> >>>>>>>> see this considered in the "alternate designs" design, even if you
> >>>>> end
> >>>>>>>> up deciding it's not the way to go.
> >>>>>>>>
> >>>>>>>> best,
> >>>>>>>> Colin
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Thu, Jan 12, 2017, at 10:46, Dong Lin wrote:
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> We created KIP-112: Handle disk failure for JBOD. Please find the
> >>>>> KIP
> >>>>>>>>> wiki
> >>>>>>>>> in the link https://cwiki.apache.org/confl
> >>>> uence/display/KAFKA/KIP-
> >>>>>>>>> 112%3A+Handle+disk+failure+for+JBOD.
> >>>>>>>>>
> >>>>>>>>> This KIP is related to KIP-113
> >>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>>>>>>> 113%3A+Support+replicas+movement+between+log+directories>:
> >>>>>>>>> Support replicas movement between log directories. They are
> >>>> needed
> >>>>> in
> >>>>>>>>> order
> >>>>>>>>> to support JBOD in Kafka. Please help review the KIP. You
> >>>> feedback
> >>>>> is
> >>>>>>>>> appreciated!
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Dong
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >>
>
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
t; > > admin
> > > client.
> >
> > Thanks for the comments, Becket!
> >
> > I agree that topic configuration change should be in the administrative
> > client. I have not thought about partition movement or preferred leader
> > election. It probab
; > would
> > > >> usually be chosen due to the nature of the data, rather than based
> on
> > > who
> > > >> produced the data, and so it makes sense to me that the durability
> > > should
> > > >> be on the entire topic, not by the producer.
>
; https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 119%3A+Drop+Support+for+Scala+2.10+in+Kafka+0.11
>
> Please take a look. Your feedback is appreciated.
>
> Thanks,
> Ismael
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
d when the
> > next major release will happen, but we know that it won't happen before
> > June 2017.
> >
> > Please take a look at the proposal and share your feedback.
> >
> > Thanks,
> > Ismael
> >
> > [1] http://search-hadoop.com/m/Kafka/uyzND1oIhV61GS5Sf2
> >
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
ng, sometimes forever, and
> the
> > >>>> Java process can't abort them, because the hang is in the kernel.
> It
> > >> is
> > >>>> not fun when threads are stuck in "D state"
> > >>>> http://stackoverflow.com/questions/20423521/process-perminan
> > >>>> tly-stuck-on-d-state
> > >>>> . Even kill -9 cannot abort the thread then. Fortunately, this is
> > >>>> rare.
> > >>>>
> > >>>
> > >>> I agree it is a harder problem and it is rare. We probably don't have
> > to
> > >>> worry about it in this KIP since this issue is orthogonal to whether
> or
> > >> not
> > >>> we use JBOD.
> > >>>
> > >>>
> > >>>>
> > >>>> Another approach we should consider is for Kafka to implement its
> own
> > >>>> storage layer that would stripe across multiple disks. This
> wouldn't
> > >>>> have to be done at the block level, but could be done at the file
> > >> level.
> > >>>> We could use consistent hashing to determine which disks a file
> should
> > >>>> end up on, for example.
> > >>>>
> > >>>
> > >>> Are you suggesting that we should distribute log, or log segment,
> > across
> > >>> disks of brokers? I am not sure if I fully understand this approach.
> My
> > >> gut
> > >>> feel is that this would be a drastic solution that would require
> > >>> non-trivial design. While this may be useful to Kafka, I would prefer
> > not
> > >>> to discuss this in detail in this thread unless you believe it is
> > >> strictly
> > >>> superior to the design in this KIP in terms of solving our use-case.
> > >>>
> > >>>
> > >>>> best,
> > >>>> Colin
> > >>>>
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Dong
> > >>>>>
> > >>>>> On Wed, Jan 25, 2017 at 11:34 AM, Colin McCabe <cmcc...@apache.org
> >
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hi Dong,
> > >>>>>>
> > >>>>>> Thanks for the writeup! It's very interesting.
> > >>>>>>
> > >>>>>> I apologize in advance if this has been discussed somewhere else.
> > >>> But
> > >>>> I
> > >>>>>> am curious if you have considered the solution of running multiple
> > >>>>>> brokers per node. Clearly there is a memory overhead with this
> > >>>> solution
> > >>>>>> because of the fixed cost of starting multiple JVMs. However,
> > >>> running
> > >>>>>> multiple JVMs would help avoid scalability bottlenecks. You could
> > >>>>>> probably push more RPCs per second, for example. A garbage
> > >>> collection
> > >>>>>> in one broker would not affect the others. It would be
> interesting
> > >>> to
> > >>>>>> see this considered in the "alternate designs" design, even if you
> > >>> end
> > >>>>>> up deciding it's not the way to go.
> > >>>>>>
> > >>>>>> best,
> > >>>>>> Colin
> > >>>>>>
> > >>>>>>
> > >>>>>> On Thu, Jan 12, 2017, at 10:46, Dong Lin wrote:
> > >>>>>>> Hi all,
> > >>>>>>>
> > >>>>>>> We created KIP-112: Handle disk failure for JBOD. Please find the
> > >>> KIP
> > >>>>>>> wiki
> > >>>>>>> in the link https://cwiki.apache.org/confl
> > >> uence/display/KAFKA/KIP-
> > >>>>>>> 112%3A+Handle+disk+failure+for+JBOD.
> > >>>>>>>
> > >>>>>>> This KIP is related to KIP-113
> > >>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >>>>>> 113%3A+Support+replicas+movement+between+log+directories>:
> > >>>>>>> Support replicas movement between log directories. They are
> > >> needed
> > >>> in
> > >>>>>>> order
> > >>>>>>> to support JBOD in Kafka. Please help review the KIP. You
> > >> feedback
> > >>> is
> > >>>>>>> appreciated!
> > >>>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> Dong
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
a regex pattern.
> > > Currently
> > > >> no
> > > >>>> topics match. Consumer offset is "latest"
> > > >>>> 3) producer publishes to a topic that matches the string or regex
> > > >> pattern.
> > > >>>> 4) broker immediately creates a topic, writes the message, and
> also
> > > >>>> notifies the consumer group that a rebalance needs to happen to
> > assign
> > > >> the
> > > >>>> topic_partition to one of the consumers..
> > > >>>> 5) rebalance is fairly quick, maybe a second or so
> > > >>>> 6) a consumer is assigned to the newly-created topic_partition
> > > >>>>
> > > >>>> At this point, we've got a consumer steadily polling the recently
> > > >> created
> > > >>>> topic_partition. However, the consumer.poll() never returns any
> > > messages
> > > >>>> published between topic creation and when the consumer was
> assigned
> > to
> > > >> the
> > > >>>> topic_partition. I'm guessing this may be because when the
> consumer
> > is
> > > >>>> assigned to the topic_partition it doesn't find any, so it uses
> the
> > > >> latest
> > > >>>> offset, which happens to be after the messages that were published
> > to
> > > >>>> create the topic.
> > > >>>>
> > > >>>> This is surprising because the consumer technically was subscribed
> > to
> > > >> the
> > > >>>> topic before the messages were produced, so you'd think the
> consumer
> > > >> would
> > > >>>> receive these messages.
> > > >>>>
> > > >>>> Is this known behavior? A bug in Kafka broker? Or a bug in my
> client
> > > >>>> library?
> > > >>
> > > >>
> > >
> > >
> >
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
; >
> > Jun
> >
> > On Wed, Jan 11, 2017 at 2:51 PM, Gwen Shapira <g...@confluent.io> wrote:
> >
> > > The PMC for Apache Kafka has invited Grant Henke to join as a
> > > committer and we are pleased to announce that he has accepted!
> > >
&
retention (dormant)
> >
> >
> > KIP-58 was adopted since then and it probably makes sense to add KIP-10
> to
> > the list.
> >
> > Are people OK with this? Feel free to suggest KIPs that should or should
> > not be in the inactive/dormant list.
> >
> > Ismael
> >
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Grant Henke created KAFKA-4525:
--
Summary: Kafka should not require SSL trust store password
Key: KAFKA-4525
URL: https://issues.apache.org/jira/browse/KAFKA-4525
Project: Kafka
Issue Type: Bug
[
https://issues.apache.org/jira/browse/KAFKA-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke resolved KAFKA-2552.
Resolution: Duplicate
> Certain admin commands such as partition assignment fail on large clust
Grant Henke created KAFKA-4203:
--
Summary: Java producer default max message size does not align
with broker default
Key: KAFKA-4203
URL: https://issues.apache.org/jira/browse/KAFKA-4203
Project: Kafka
Grant Henke created KAFKA-4157:
--
Summary: Transient system test failure in
replica_verification_test.test_replica_lags
Key: KAFKA-4157
URL: https://issues.apache.org/jira/browse/KAFKA-4157
Project
[
https://issues.apache.org/jira/browse/KAFKA-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-4157:
---
Status: Patch Available (was: Open)
> Transient system test fail
ough I created a new vote thread, Gmail placed the messages
> in
> > > the discuss thread, making it not as visible as required. It's
> important
> > to
> > > mention that two +1s were cast by Gwen and Sriram:
> > >
> > > http://mail-archives.apa
gt;
> > >>>>>>>> Cc: "priv...@kafka.apache.org <javascript:;>" <
> > >>>> priv...@kafka.apache.org <javascript:;>>
> > >>>>>>>> Date: 09/06/2016 03:26 PM
> > >>>>>>>> Subject:[ANNOUNCE] New committer: Jason Gustafson
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> The PMC for Apache Kafka has invited Jason Gustafson to join
> > >> as a
> > >>>>>>>> committer and
> > >>>>>>>> we are pleased to announce that he has accepted!
> > >>>>>>>>
> > >>>>>>>> Jason has contributed numerous patches to a wide range of
> > >> areas,
> > >>>>>> notably
> > >>>>>>>> within the new consumer and the Kafka Connect layers. He has
> > >>>>> displayed
> > >>>>>>>> great taste and judgement which has been apparent through his
> > >>>>>> involvement
> > >>>>>>>> across the board from mailing lists, JIRA, code reviews to
> > >>>>> contributing
> > >>>>>>>> features, bug fixes and code and documentation improvements.
> > >>>>>>>>
> > >>>>>>>> Thank you for your contribution and welcome to Apache Kafka,
> > >>> Jason!
> > >>>>>>>> --
> > >>>>>>>> Thanks,
> > >>>>>>>> Neha
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>> --
> > >>> Ashish h
> > >>>
> > >>
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Rajini
> >
> >
>
>
> --
> -Regards,
> Mayuresh R. Gharat
> (862) 250-7125
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
; we
> > > > > > >> > need
> > > > > > >> > > this going forward (the new consumer doesn't even expose
> it
> > at
> > > > the
> > > > > > >> > moment).
> > > > > > >> > >
> > > > > > >> >
> > > > > > >> > The KIP in its current form isn't adding a callback. So
> you'd
> > > > still
> > > > > > >> need to
> > > > > > >> > scan the cache and remove any expired offsets, however you
> > > > wouldn't
> > > > > > send
> > > > > > >> > the tombstone messages.
> > > > > > >> > Having a callback sounds useful, though it isn't clear to me
> > how
> > > > you
> > > > > > >> would
> > > > > > >> > know which offsets to remove from the cache on segment
> > > deletion? I
> > > > > > will
> > > > > > >> > look into it.
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > > 2. This KIP could also be useful for expiration in the
> case
> > > of a
> > > > > > cache
> > > > > > >> > > maintained on the client, but I don't see an obvious way
> > that
> > > > we'd
> > > > > > be
> > > > > > >> > able
> > > > > > >> > > to leverage it since there's no indication to the client
> > when
> > > a
> > > > > > >> segment
> > > > > > >> > has
> > > > > > >> > > been deleted (unless they reload the cache from the
> > beginning
> > > of
> > > > > the
> > > > > > >> > log).
> > > > > > >> > > One approach I can think of would be to write
> corresponding
> > > > > > >> tombstones as
> > > > > > >> > > necessary when a segment is removed, but that seems pretty
> > > > heavy.
> > > > > > Have
> > > > > > >> > you
> > > > > > >> > > considered this problem?
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > We've not considered this and I'm not sure we want to as
> part
> > of
> > > > > this
> > > > > > >> KIP.
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Damian
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > On Mon, Aug 8, 2016 at 12:41 AM, Damian Guy <
> > > > damian@gmail.com
> > > > > >
> > > > > > >> > wrote:
> > > > > > >> > >
> > > > > > >> > > > Hi,
> > > > > > >> > > >
> > > > > > >> > > > We have created KIP 71: Enable log compaction and
> deletion
> > > to
> > > > > > >> co-exist`
> > > > > > >> > > >
> > > > > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > > >> > > > 71%3A+Enable+log+compaction+and+deletion+to+co-exist
> > > > > > >> > > >
> > > > > > >> > > > Please take a look. Feedback is appreciated.
> > > > > > >> > > >
> > > > > > >> > > > Thank you
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
log retention (dormant)
Thank you,
Grant
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
ri, Jul 22, 2016 at 4:13 AM, Jim Jagielski <j...@jagunet.com>
> wrote:
> > > >
> > > >> On Jul 21, 2016, at 10:57 PM, Ismael Juma <ism...@juma.me.uk>
> wrote:
> > > >>
> > > >> Hi Grant,
> > > >>
> >
Gwen Shapira has been active in the Kafka community since she became a
> > Kafka committer
> > about a year ago. I am glad to announce that Gwen is now a member of
> Kafka
> > PMC.
> >
> > Congratulations, Gwen!
> >
> > Jun
> >
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
with various options.Thank you,Jayesh Thakrar
> >
> >
> >
>
>
>
> --
> Gwen Shapira
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter | blog
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
[
https://issues.apache.org/jira/browse/KAFKA-4032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-4032:
---
Status: Patch Available (was: Open)
> Uncaught exceptions when autocreating top
[
https://issues.apache.org/jira/browse/KAFKA-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-4038:
---
Status: Patch Available (was: Open)
> Transient fail
[
https://issues.apache.org/jira/browse/KAFKA-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke reassigned KAFKA-4038:
--
Assignee: Grant Henke
> Transient fail
[
https://issues.apache.org/jira/browse/KAFKA-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419471#comment-15419471
]
Grant Henke commented on KAFKA-3959:
I would like to present an alternative option. This problem
[
https://issues.apache.org/jira/browse/KAFKA-4032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419350#comment-15419350
]
Grant Henke commented on KAFKA-4032:
I will make a patch for this shortly
> Uncaught exceptions w
[
https://issues.apache.org/jira/browse/KAFKA-4032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke reassigned KAFKA-4032:
--
Assignee: Grant Henke
> Uncaught exceptions when autocreating top
ou expect all consumer instances in the
> consumer
> > group to be stopped before copying the offsets? Some applications may not
> > want to do that.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Tue, Aug 9, 2016 at 10:01 AM, Grant Henke
Kafka? Any thoughts on the approach?
Thanks,
Grant
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
[
https://issues.apache.org/jira/browse/KAFKA-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-3934:
---
Summary: Start scripts enable GC by default with no way to disable (was:
kafka-server-start.sh
possible. Any
> ideas?
> >
> > The problem w/ being more descriptive is that its possible that
> > it restricts potential use cases if people think that somehow
> > their use case wouldn't fit.
> >
> >> 4. There is no CreateAcls or DeleteAcls (unlike Crea
gt; >
> >> > * Documentation:
> >> > http://kafka.apache.org/0100/documentation.html
> >> >
> >> > * Protocol:
> >> > http://kafka.apache.org/0100/protocol.html
> >> >
> >> > * Successful Jenkins builds for the 0.10.0 branch:
> >> > Unit/integration tests: *https://builds.apache.org/job
> >> /kafka-0.10.0-jdk7/182/
> >> > <https://builds.apache.org/job/kafka-0.10.0-jdk7/182/>*
> >> > System tests: *https://jenkins.confluent.io/
> >> job/system-test-kafka-0.10.0/138/
> >> > <https://jenkins.confluent.io/job/system-test-kafka-0.10.0/138/>*
> >> >
> >> > Thanks,
> >> > Ismael
> >>
> >>
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
, in addition to the
> > current bound on the number of messages.
> >
> > This comes after several incidents at Linkedin where a sudden "spike" of
> > large message batches caused an out of memory exception.
> >
> > Thank you,
> >
> >Radai
> >
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
t this be addressed in KIP-50 as well, though it
> >> has
> >> > >> some compatibility concerns.
> >> >
> >> > Isn't KIP-50 itself one gigantic compatibility concern? I don't see
> >> > how your suggestions make it any worse...
> >
[
https://issues.apache.org/jira/browse/KAFKA-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-2507:
---
Fix Version/s: 0.11.0.0
> Replace ControlledShutdown{Request,Respo
[
https://issues.apache.org/jira/browse/KAFKA-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-3934:
---
Status: Patch Available (was: Open)
> kafka-server-start.sh enables GC by default with no
Hi Parth,
Are you still working on this? If you need any help please don't hesitate
to ask.
Thanks,
Grant
On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao wrote:
> Parth,
>
> Thanks for the reply.
>
> It makes sense to only allow the renewal by users that authenticated using
>
y issues, especially because the Authorizer
> interface
> exposes no details about propagating the ACLs to other nodes.
> - See Request Forwarding
>
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operati
[
https://issues.apache.org/jira/browse/KAFKA-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-2946:
---
Status: Patch Available (was: In Progress)
> DeleteTopic - protocol and server side implementat
> > > > create the topic via `AdminClient` if that's what they need.
> > > > 2. Like Grant, I'm unsure that will generally be used in a positive
> > way.
> > > > However, if this is what we need to be able to deprecate server
> > > auto-topic
&g
Grant Henke created KAFKA-3934:
--
Summary: kafka-server-start.sh enables GC by default with no way
to disable
Key: KAFKA-3934
URL: https://issues.apache.org/jira/browse/KAFKA-3934
Project: Kafka
If people like this approach, perhaps we could define a topic spec (if
> all
> > fields besides topic name are empty it use "cluster defaults"). Then the
> > AdminClient would have an idempotent create method that takes a spec and
> > verifies that the spec
ind of feel once you start adding AdminClient methods to the producer
> and consumer it's not really clear where to stop--e.g. if I can create I
> should be able to delete, list, etc.
>
> -Jay
>
> On Tue, Jun 28, 2016 at 9:26 AM, Grant Henke <ghe...@cloudera.com>
Grant
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
://github.com/apache/kafka/pull/1489
On Fri, Jun 24, 2016 at 5:43 PM, Gwen Shapira <g...@confluent.io> wrote:
> +1
>
> On Thu, Jun 23, 2016 at 8:32 PM, Grant Henke <ghe...@cloudera.com> wrote:
> > I would like to initiate the voting process for the "KIP-4 Delete Topics
> &g
are whether or not the broker
> is
> >> the controller
> >> b. there is already a separate boolean property,
> KafkaController.isActive
> >> which contains this information.
> >>
> >> Thanks,
> >>
> >> Roger
> >>
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
A/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)>*
A sample patch is on github:
https://github.com/granthenke/kafka/tree/delete-wire-new
Note: This branch and patch is based on the CreateTopic request/response
PR. I will open a PR once that patch is complete.
Here is a link to the past discussion on the mailing list:
*http://search-hadoop.com/m/uyzND1fokOBn6uNd2=+DISCUSS+KIP+4+Delete+Topic+Schema
<http://search-hadoop.com/m/uyzND1fokOBn6uNd2=+DISCUSS+KIP+4+Delete+Topic+Schema>*
Thank you,
Grant
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
..@harsha.io> wrote:
> > > >
> > > >> +1 (binding)
> > > >> -Harsha
> > > >>
> > > >> On Mon, Jun 20, 2016, at 11:33 AM, Ismael Juma wrote:
> > > >> > +1 (binding)
> > > >> >
> &g
ption, because you asked for 0 timeout, you can assume the
> > message was
> > > valid and the topic deletion was triggered.
> >
> > In the timeout <= 0 case, if the client should always ignore and treat
> the timeout
> > error code as "OK", sh
ot "complete" in time. This allows the client
to know which topics actions completed and which timed out. I updated the
wiki to explicitly call this out in the response section.
Thanks,
Grant
On Tue, Jun 21, 2016 at 5:44 AM, Ismael Juma <ism...@juma.me.uk> wrote:
> Thanks Gra
tions-request>
>below
>
> Delete Topics Response
>
>
>
> DeleteTopics Response (Version: 0) => [topic_error_codes]
> topic_error_codes => topic error_code
> topic => STRING
> error_code => INT16
>
> DeleteTopicsResponse is similar
t; >
> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote:
> > > +1.
> > >
> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma <ism...@juma.me.uk>
> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > On Thu,
can potentially just remember the
> > topics that violated those constraints in the request and handle them
> > accordingly with the right error code w/o disconnecting.
> >
> > Thanks,
> >
> > Jun
> >
> > On Fri, Jun 17, 2016 at 8:40 AM, Dana Powers
or the
> offending create_topic_request, not the entire connection? The
> CreateTopicsResponse returns a map of topics to error codes. You could
> just return the topic that caused the error and an
> InvalidRequestException error code.
>
> -Dana
>
> On Thu, Jun 16, 2016 at 8:37 A
nce/display/KAFKA/KIP-62%3A+Allow+consumer+to+send+heartbeats+from+a+background+thread
> > > > > > .
> > > > > >
> > > > > > After some discussion on this list, I think we were in agreement
> > that
> > > > > this
> &g
FKA-2945)
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945)>*
A pull request is available implementing the proposed changes here:
https://github.com/apache/kafka/pull/1489
Here is a link to the past discussion on the mailing list:
*http://search-hadoop.com/m/uyzND1rfG6v1oixmZ=+DISCUSS+KIP+4+Create+Topic+Schema
<http://search-hadoop.com/m/uyzND1rfG6v1oixmZ=+DISCUSS+KIP+4+Create+Topic+Schema>*
Thank you,
Grant
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
t; > number
> > > > of
> > > > > users may still be using Java 7 by the time Kafka 0.10.1.0 is
> > released.
> > > > > More specifically, we care about the subset who would be able to
> > > upgrade
> > > > to
> > > > > Kafka 0.10.1.0, but would not be able to upgrade the Java version.
> It
> > > > would
> > > > > be great if we could quantify this in some way.
> > > > >
> > > > > What do you think?
> > > > >
> > > > > Ismael
> > > > >
> > > > > [1] https://java.com/en/download/faq/java_7.xml
> > > > > [2]
> > https://blogs.oracle.com/thejavatutorials/entry/jdk_8_is_released
> > > > > [3] http://openjdk.java.net/projects/jdk9/
> > > > > [4] https://github.com/apache/cassandra/blob/trunk/README.asc
> > > > > [5]
> > > https://lucene.apache.org/#highlights-of-this-lucene-release-include
> > > > > [6] http://akka.io/news/2015/09/30/akka-2.4.0-released.html
> > > > > [7] https://issues.apache.org/jira/browse/HADOOP-11858
> > > > > [8] https://webtide.com/jetty-9-3-features/
> > > > > [9] http://markmail.org/message/l7s276y3xkga2eqf
> > > > > [10]
> > > > >
> > > > >
> > > >
> > >
> >
> https://intellij-support.jetbrains.com/hc/en-us/articles/206544879-Selecting-the-JDK-version-the-IDE-will-run-under
> > > > > [11] http://markmail.org/message/l7s276y3xkga2eqf
> > > > >
> > > >
> > >
> >
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
nistrativeoperations-request>
>below
>
> Create Topics Response
>
>
>
> CreateTopics Response (Version: 0) => [topic_error_codes]
> topic_error_codes => topic error_code
> topic => STRING
> error_code => INT16
>
> CreateTopicsResp
gt; >
> > Comments below.
> >
> > On Wed, Jun 15, 2016 at 6:52 PM, Grant Henke <ghe...@cloudera.com>
> wrote:
> >>
> >> The one thing I want to avoid is to many super specific error codes. I
> am
> >> not sure how much of a problem it really
[
https://issues.apache.org/jira/browse/KAFKA-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332524#comment-15332524
]
Grant Henke commented on KAFKA-3818:
A few older threads mention that its possible to get clumping
col? In the code, negative timeouts are typically
> disallowed or they mean an infinite timeout (we have moved from the latter
> to the former in some of the Java networking code in recent releases).
> Ismael
>
> On Tue, Jun 14, 2016 at 11:51 PM, Grant Henke <ghe...@cloudera.com&
[
https://issues.apache.org/jira/browse/KAFKA-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-3691:
---
Fix Version/s: 0.10.0.1
> Confusing logging during metadata update time
[
https://issues.apache.org/jira/browse/KAFKA-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-3691:
---
Fix Version/s: 0.10.1.0
> Confusing logging during metadata update time
[
https://issues.apache.org/jira/browse/KAFKA-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-3691:
---
Affects Version/s: 0.10.0.0
> Confusing logging during metadata update time
[
https://issues.apache.org/jira/browse/KAFKA-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-3691:
---
Status: Patch Available (was: Open)
> Confusing logging during metadata update time
quest` then?
>
Sure, I will update that in the patch and wiki.
P.S. I fixed a couple of typos I spotted on the wiki page, I hope that's OK.
>
Absolutely. Feel free to improve the wiki anytime.
Thanks,
Grant
On Tue, Jun 14, 2016 at 3:09 PM, Ismael Juma <ism...@juma.me.uk> wrote
a batch of
> topics, and different error codes are returned.
>
> Guozhang
>
>
>
> On Mon, Jun 13, 2016 at 6:54 PM, Grant Henke <ghe...@cloudera.com> wrote:
>
> > Thanks for the review Jun.
> >
> > You probably want to make it clearer if timeout > 0, wha
lete" means. In the first implementation, it really means
> that the topic metadata is propagated to the controller's metadata cache.
>
> Jun
>
> On Fri, Jun 10, 2016 at 9:21 AM, Grant Henke <ghe...@cloudera.com> wrote:
>
> > Now that Kafka 0.10 has bee
have a way to get
> "ack" from replicas, we will be able to add the new behavior without
> changing the protocol (just the semantics of waiting).
>
> Gwen
>
> On Fri, Jun 10, 2016 at 7:21 PM, Grant Henke <ghe...@cloudera.com> wrote:
> > Now that Kafka 0.10 ha
which would
> maintain its own connection pool etc, would be used internally by the
> producer (and potentially consumer) or if they would just reuse the request
> objects.
>
> Just trying to write this down to sanity check that it will work.
>
> -Jay
>
> On Fri, J
Response (Version: 0) => [topic_error_codes]
> topic_error_codes => topic error_code
> topic => STRING
> error_code => INT16
>
> CreateTopicResponse contains a map between topic and topic creation
> result error code (see New Protocol Errors
> <https://cwi
[
https://issues.apache.org/jira/browse/KAFKA-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-3789:
---
Status: Patch Available (was: Open)
> Upgrade Snappy to fix snappy decompression err
Grant Henke created KAFKA-3789:
--
Summary: Upgrade Snappy to fix snappy decompression errors
Key: KAFKA-3789
URL: https://issues.apache.org/jira/browse/KAFKA-3789
Project: Kafka
Issue Type: Bug
[
https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke reassigned KAFKA-3764:
--
Assignee: Grant Henke
> Error processing append operation on partit
[
https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312916#comment-15312916
]
Grant Henke commented on KAFKA-3764:
It looks like this is likely caused by
https://github.com/xerial
gt; https://cwiki.apache.org/confluence/display/KAFKA/KIP-62%3A+Allow+consumer+to+send+heartbeats+from+a+background+thread
> .
> Have a look and let me know what you think.
>
> Thanks,
> Jason
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
gt;>>>>
> >>>>> Jiangjie (Becket) Qin
> >>>>>
> >>>>>
> >>>>> On Tue, May 24, 2016 at 4:21 PM, Gwen Shapira <g...@confluent.io>
> >>> wrote:
> >>>>>
> >>>>>> +1 (binding)
> >>>>>>
> >>>>>> Thanks for responding to all my original concerns in the discussion
> >>>> thread.
> >>>>>>
> >>>>>> On Tue, May 24, 2016 at 1:37 PM, Eric Wasserman <
> >>>> eric.wasser...@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I would like to begin voting on KIP-58 - Make Log Compaction Point
> >>>>>>> Configurable
> >>>>>>>
> >>>>>>> KIP-58 is here: <
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-58+-+Make+Log+Compaction+Point+Configurable
> >>>>>>>>
> >>>>>>>
> >>>>>>> The Jira ticket KAFKA-1981 Make log compaction point configurable
> >>>>>>> is here: <https://issues.apache.org/jira/browse/KAFKA-1981>
> >>>>>>>
> >>>>>>> The original pull request is here: <
> >>>>>>> https://github.com/apache/kafka/pull/1168>
> >>>>>>> (this includes configurations for size and message count lags that
> >>> will
> >>>>>> be
> >>>>>>> removed per discussion of KIP-58).
> >>>>>>>
> >>>>>>> The vote will run for 72 hours.
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> Thanks,
> >> Ewen
> >>
>
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
tiychuk,
> Dong Lin, Dongjoon Hyun, Drausin Wulsin, Duncan Sands, Dustin Cote,
> Eamon Zhang, edoardo, Edward Ribeiro, Eno Thereska, Ewen
> Cheslack-Postava, Flavio Junqueira, Francois Visconte, Frank Scholten,
> Gabriel Zhang, gaob13, Geoff Anderson, glikson, Grant Henke, Greg
> Fodor, Guoz
[
https://issues.apache.org/jira/browse/KAFKA-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KAFKA-3717:
---
Summary: Support building aggregate javadoc for all project modules (was:
On 0.10.0 branch, building
[
https://issues.apache.org/jira/browse/KAFKA-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285897#comment-15285897
]
Grant Henke commented on KAFKA-3717:
Currently the build places a javadoc directory and jar in each
1 - 100 of 753 matches
Mail list logo