/consumer/ZookeeperConsumerConnector.scala
fbc680fde21b02f11285a4f4b442987356abd17b
Diff: https://reviews.apache.org/r/26291/diff/
Testing
---
Thanks,
Mayuresh Gharat
automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26291/#review55702
-------
On Oct. 9, 2014, 12:29 a.m., Mayuresh Gharat wrote:
>
> ---
> This is an
8ea7368dc394a497164ea093ff8e9f2e6a94b1de
core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala
fbc680fde21b02f11285a4f4b442987356abd17b
Diff: https://reviews.apache.org/r/26291/diff/
Testing
---
Thanks,
Mayuresh Gharat
it:
https://reviews.apache.org/r/26291/#review55456
---
On Oct. 9, 2014, 12:29 a.m., Mayuresh Gharat wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https:/
/PartitionAssignorTest.scala
9ceae222ca5bf63e8131de0db2a94126c8b57b59
Diff: https://reviews.apache.org/r/26291/diff/
Testing
---
Thanks,
Mayuresh Gharat
/unit/kafka/consumer/PartitionAssignorTest.scala
9ceae222ca5bf63e8131de0db2a94126c8b57b59
Diff: https://reviews.apache.org/r/26291/diff/
Testing
---
Thanks,
Mayuresh Gharat
tps://reviews.apache.org/r/27534/#comment101864>
This causes many system tests (approx 98) to fail
- Mayuresh Gharat
On Nov. 3, 2014, 7:46 p.m., Ewen Cheslack-Postava wrote:
>
> ---
> This is an automatically generated e-mail. To reply,
> On Nov. 8, 2014, 3:01 a.m., Mayuresh Gharat wrote:
> > system_test/utils/kafka_system_test_utils.py, line 2403
> > <https://reviews.apache.org/r/27534/diff/1/?file=747724#file747724line2403>
> >
> > This causes many system tests (approx 98) to fail
> On Nov. 8, 2014, 3:01 a.m., Mayuresh Gharat wrote:
> > system_test/utils/kafka_system_test_utils.py, line 2403
> > <https://reviews.apache.org/r/27534/diff/1/?file=747724#file747724line2403>
> >
> > This causes many system tests (approx 98) to fail
Congratulations John!
Thanks,
Mayuresh
On Tue, Nov 12, 2019 at 4:54 PM Vahid Hashemian
wrote:
> Congratulations John!
>
> --Vahid
>
> On Tue, Nov 12, 2019 at 4:38 PM Adam Bellemare
> wrote:
>
> > Congratulations John, and thanks for all your help on KIP-213!
> >
> > > On Nov 12, 2019, at 6:24
Hi Lucas,
Thanks for the KIP.
I am trying to understand why you think "The memory consumption can rise
given the total number of queued requests can go up to 2x" in the impact
section. Normally the requests from controller to a Broker are not high
volume, right ?
Thanks,
Mayuresh
On Tue, Jul 1
adding the extra
> >> config with a default value, say 20, guards us from issues in those
> >> troublesome times, and IMO there isn't much downside of adding the extra
> >> config.
> >>
> >> @Mayuresh
> >> Good catch, this sentence is an obsol
y only concern is that this design is tied to the coincidence
> that
> > > > > > we have two request priorities and there are two ends to a deque.
> > > > > > Hence by using the proposed design, it seems the network layer is
> > > > > > more tightly coupled with upper layer logic, e.g. if we were to
> add
> > > >
Actually nvm, correlationId is reset in case of connection loss, I think.
Thanks,
Mayuresh
On Thu, Jul 19, 2018 at 11:11 AM Mayuresh Gharat
wrote:
> I agree with Dong that out-of-order processing can happen with having 2
> separate queues as well and it can even happen today.
> Can w
ing and never reset,
> hence a broker can leverage that to properly reject obsolete requests.
> Thoughts?
>
> Thanks,
> Lucas
>
> On Thu, Jul 19, 2018 at 12:11 PM, Mayuresh Gharat <
> gharatmayures...@gmail.com> wrote:
>
> > Actually nvm, correlationId is
>> I agree that there is no strong ordering when there are more than one
> >>> socket connections. Currently, we rely on controllerEpoch and
> leaderEpoch
> >>> to ensure that the receiving broker picks up the latest state for each
> >>> partition.
gt; If that's the case, I think this probably deserves a separate doc and
> > > > design independent of this KIP.
> > > >
> > > > Lucas
> > > >
> > > >
> > > >
> > > > On Mon, Jul 23, 2018 at 12:39 PM, Dong Lin
Congrats Dong !!!
Thanks,
Mayuresh
On Mon, Aug 20, 2018 at 1:36 PM Gwen Shapira wrote:
> Congrats Dong Lin! Well deserved!
>
> On Mon, Aug 20, 2018, 3:55 AM Ismael Juma wrote:
>
> > Hi everyone,
> >
> > Dong Lin became a committer in March 2018. Since then, he has remained
> > active in the c
Hi Senthil,
Kafka does allow search message by timestamp after KIP-33 :
https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index#KIP-33-Addatimebasedlogindex-Searchmessagebytimestamp
The new consumer does provide you a way to get offsets by timestamp. You
can use thes
Nice write up Radai.
I think what Jun said is a valid concern.
If I am not wrong as per the proposal, we are depending on the entire
pipeline to flow smoothly from accepting requests to handling it, calling
KafkaApis and handing back the responses.
Thanks,
Mayuresh
On Mon, Aug 8, 2016 at 12:22
Nice write up Ben.
I agree with Joel for keeping this simple by excluding the partitions from
the fetch request/response when the quota is violated at the follower or
leader instead of having a separate set of threads for handling the quota
and non quota cases. Even though its different from the c
y
> >> I'm wondering if it is easier to return empty for those partitions
> entirely
> >> in the fetch response - they will make progress in the subsequent
> fetch. If
> >> they don't make fast enough progress then that would be a case for
> raising
at's a good question. I think if the response size (after leader
> > > throttling) is smaller than min.bytes, we will just delay the sending
> of
> > > the response up to max.wait as we do now. This should prevent frequent
> > > empty responses to the follower.
>
Congrats Gwen :)
Thanks,
Mayuresh
On Thu, Aug 18, 2016 at 10:27 AM, Gwen Shapira wrote:
> Thanks team Kafka :) Very excited and happy to contribute and be part
> of this fantastic community.
>
>
>
> On Thu, Aug 18, 2016 at 9:52 AM, Guozhang Wang wrote:
> > Congrats Gwen!
> >
> > On Thu, Aug 1
Hi,
RequestTimeout is used for 2 cases :
1) Timing out the batches sitting in the accumulator.
2) Requests that are already sent over the wire and you have not yet heard
from the server.
In a case where there is a network partition, the client might not detect
it, till the actual TCP timeout that
Hi Becket,
Thanks for the write up. I had few minor comments :
1) The wiki says ListOffsetRequest v0 still behaves in the old way.
---> It would be good to include what it means although that's not the part
of this KIP.
You can add a note what V0 means like :
(Kafka allows querying offsets of mes
congrats Jason !
Thanks,
Mayuresh
On Wed, Sep 7, 2016 at 5:16 AM, Eno Thereska wrote:
> Congrats Jason!
>
> Eno
> > On 7 Sep 2016, at 10:00, Rajini Sivaram
> wrote:
> >
> > Congrats, Jason!
> >
> > On Wed, Sep 7, 2016 at 8:29 AM, Flavio P JUNQUEIRA
> wrote:
> >
> >> Congrats, Jason. Well don
Great !
Thanks,
Mayuresh
On Tue, Sep 20, 2016 at 2:43 PM, Becket Qin wrote:
> Awesome!
>
> On Mon, Sep 19, 2016 at 11:42 PM, Neha Narkhede wrote:
>
> > Nice!
> > On Mon, Sep 19, 2016 at 11:33 PM Ismael Juma wrote:
> >
> > > Well done everyone. :)
> > >
> > > On 20 Sep 2016 5:23 am, "Jason Gu
Hi Ben,
Just for my understanding :
When you said :
What we really want to do is apply:
LeaderThrottle [104,107,109]
FollowerThrottle [105,113]
---> This means that we want to apply leader throttle to only replica that
is the leader out of 104, 107, 109 right?
Also when you said :
[104,107,10
@Kostya
Regarding "To get around this we have an awful *cough* solution whereby we
have to send our message wrapper with the headers and null content, and
then we have an application that has to consume from all the compacted
topics and when it sees this message it produces back in a null payload
e to the middle) - its still possible
> > though. however, the kafka code base is far from being iovec friendly
> > anyway.
> >
> > 2. yes.
> >
> >
> >
> >
> >
> > On Thu, Oct 6, 2016 at 8:58 AM, K Burstev wrote:
> >
> > > @Mayuresh
Hi David,
Thanks for the KIP.
I had some questions/suggestions :
It would be great if you can explain with an example about how the min
offset for all the consumers will be calculated, in the KIP.
What I meant was, it would be great to understand with a pseudo
code/workflow if possible, how each
ess of the
select timeout). It should be possible to combine these but that will likely
require a callback to handle the KIP-19 timeout and I think that should really
be a separate jira.
- Mayuresh Gharat
On Sept. 19, 2015, 2:27 a.m., Mayuresh Gharat
re/src/test/scala/unit/kafka/utils/TestUtils.scala
09b8444c2add87f0f70dbb182e892977a6b5c243
Diff: https://reviews.apache.org/r/36858/diff/
Testing
---
Thanks,
Mayuresh Gharat
re/src/test/scala/unit/kafka/utils/TestUtils.scala
09b8444c2add87f0f70dbb182e892977a6b5c243
Diff: https://reviews.apache.org/r/36858/diff/
Testing (updated)
---
Unit test pass
Thanks,
Mayuresh Gharat
> On Sept. 28, 2015, 11:04 p.m., Mayuresh Gharat wrote:
> > core/src/main/scala/kafka/controller/ControllerChannelManager.scala, line
> > 109
> > <https://reviews.apache.org/r/36858/diff/13/?file=1077372#file1077372line109>
> >
> > The request time ou
Nice write-up.
Just had a question, instead of returning an empty response back to the
client, would it be better for the broker to return a response that gives
some more info to the client regarding the min version they need to upgrade
to in order to communicate with the broker.
Thanks,
Mayure
, and then we restart again removing the
> inter.broker.protocol.version. With this api the brokers could agree on a
> version to communicate with, and when bounced re-negotiate to the new
> version.
>
>
> On Mon, Sep 28, 2015 at 10:26 PM, Mayuresh Gharat <
> gharatmay
+1 Ewen.
Currently the while creating the RecordBatch we extract the key and value
and add it to the batch. At a very high level, we will have to extract the
key and Value (minimum atleast) from each record in the RecordBatch, create
a wrapper with the exception,the key and the value and add it to
d e-mail. To reply, visit:
https://reviews.apache.org/r/36858/#review101511
-------
On Sept. 28, 2015, 11:16 p.m., Mayuresh Gharat wrote:
>
> ---
> This is an a
This might have been explained before. I had a question :
In the KIP it says :
"One ZooKeeper setting of interest on the server side is
zookeeper.allowSaslFailedClients. If this is false, then clients trying to
authenticate with an incorrect configuration will have their connections
dropped. Otherw
+ dev
Hi
There has been discussion on the ticket :
https://issues.apache.org/jira/browse/KAFKA-2887, that we are going to
deprecate auto-creation of topic or at least turn it off by default once we
have the CreateTopics API. It also says the patch is available for this.
The only other ticket tha
esh,
>
> The ticket that Grant Henke has been working on is here:
> https://issues.apache.org/jira/browse/KAFKA-2945. Is that what you were
> looking for?
>
> -Jason
>
> On Thu, Jan 7, 2016 at 1:50 PM, Mayuresh Gharat <
> gharatmayures...@gmail.com>
> wrote:
>
&
CreateTopic request to create the topic or exit. Is there a
ticket for this?
Thanks,
Mayuresh
On Thu, Jan 7, 2016 at 3:27 PM, Mayuresh Gharat
wrote:
> Hi Jason,
>
> Thanks. I had found this ticket before but the KAFKA-1507 also has some
> context about this and I was confused basic
gt; Hope that helps.
>
> Thanks,
> Grant
>
> On Thu, Jan 7, 2016 at 6:27 PM, Mayuresh Gharat <
> gharatmayures...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I might have missed out the discussions on this :(
> >
> > I had some more questions :
&g
I agree with Nacho.
+1 for the KIP.
Thanks,
Mayuresh
On Fri, Oct 21, 2016 at 11:46 AM, Nacho Solis
wrote:
> I think a separate KIP is a good idea as well. Note however that potential
> decisions in this KIP could affect the other KIP.
>
> Nacho
>
> On Fri, Oct 21, 2016 at 10:23 AM, Jun Rao w
I see the reasoning Magnus described.
If you check the docs https://kafka.apache.org/documentation#messageformat,
it says :
1 byte "magic" identifier to allow format changes, value is 0 or 1
Moreover as per comments in the code :
/**
* The "magic" value
* When magic value is 0, the message
fore require a change in
> > the magic byte field. The attribute bit is sufficient for indicating the
> > new interpretation of attribute.bit5 to indicate a tombstone.
> >
> > Bumping the version number and changing the attribute bit keeps it
> simple.
> >
> >
&g
+1 @Joel.
I think a clear migration plan of upgrading and downgrading of server and
clients along with handling of issues that Joel mentioned, on the KIP would
be really great.
Thanks,
Mayuresh
On Wed, Oct 26, 2016 at 3:31 PM, Joel Koshy wrote:
> I'm not sure why it would be useful, but it sho
> > > the
> > > > > > > config values of *og.retention.min.offset or
> > > > > > **og.retention.min.timestamp*
> > > > > > > that
> > > > > > > they send to the brokers).
> > > &
which I think is a
> feasible requirement.
>
>
> Guozhang
>
>
> On Fri, Oct 28, 2016 at 9:42 AM, Mayuresh Gharat <
> gharatmayures...@gmail.com
> > wrote:
>
> > Hi Guozhang,
> >
> > I agree that pushing out the complexity of coordination to the client
On a side note just a use case that came to my mind, if we ever plan to add
headers to kafka, we can provide this functionality of compaction based on
headers and make the compaction policy pluggable on the broker side. This
might give flexibility for doing compaction in future.
Thanks,
Mayuresh
n is, where did this delete come from? What processing
> > job
> > > > emitted it? What input to the processing job caused this delete
> to
> > be
> > > > produced? For example, if a record in topic A was processed and
> > > caused a
> > &
> > >
> > > > > 1) Tracability. I would like to know who issued
> this
> > delete
> > > > > tombstone. It might include the hostname, IP of the
> producer of
> > the
> > > > delete.
> > > >
port a null value non-tombstone message in a log compacted topic, but I
> am not sure if that has any use case.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
>
> On Wed, Nov 9, 2016 at 9:23 AM, Mayuresh Gharat <
> gharatmayures...@gmail.com>
> wrote:
>
> > I thin
v-store, so a
> map API.
> map.put(key, null) is not the same as map.remove(key), which to me
> means a
> null value should not represent a delete. a delete should be
> explicit
> (meaning flag).
>
>
> On Wed, Nov 9, 2016 at
omebody sends null
> > - broker accepts flags/variable/headers and zero length values as
> > tombstones
> >
> > Conversion from one to the other may be possible but I would rather not
> > have to do it.
> >
> > Nacho
> >
> >
> > On Wed, No
stage approach is because we want to end up just supporting
> tombstone marker (not both)
>
> But we accept we need to allow organisations and systems a period of
> transition (this is what stage 1 provides)
>
>
>
>
>
> Fro
ersion config (i.e. magic value), we know for
> sure when to down convert the messages to adapt to older clients. Otherwise
> we will have to always scan all the messages. It would probably work but
> relies on guess or inference.
>
> Thanks,
>
> Jiangjie (Becket) Qin
&
And the old apis
> > would not be allowed to be called as the broker now expects all clients
> to
> > be using latest api only.
> >
> > At this second phase stage only also we can support a null value in a
> > compacted topic without it being treated as deletion. W
ting magic byte. The interpretation of an old message format should not
> be changed and should always be supported until deprecated.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Mon, Nov 14, 2016 at 11:28 AM, Mayuresh Gharat <
> gharatmayures...@gmail.com> wrote:
>
&g
n, Nov 14, 2016 at 1:43 PM, Michael Pearce >
> > > wrote:
> > >
> > > > I like the idea of up converting and then just having the logic to
> look
> > > > for tombstones. It makes that quite clean in nature.
> > > >
> > > > It
16, 2016 at 5:08 PM, Mayuresh Gharat <
> gharatmayures...@gmail.com
> > wrote:
>
> >- If we don't bump up the magic byte, on the broker side, the broker
> >will always have to look at both tombstone bit and the value when do
> the
> >
missing
something, it sounds like kafka will be stuck with supporting both null
value and tombstone bit for log compaction for life long, which does not
look like a good end state.
Thanks,
Mayuresh
On Wed, Nov 16, 2016 at 9:32 AM, Mayuresh Gharat wrote:
> Hi Ismael,
>
> That'
l always be interpreted to a tombstone.
>
> One option is that we keep the current way, i.e. do not support such
> message. It would be good to know if there is a concrete use case for such
> message. If there is not, we can probably just not support it.
>
> Thanks,
>
&g
n step (3) is this correct?
>
> Cheers
> Mike
>
>
>
> Sent using OWA for iPhone
> ________
> From: Mayuresh Gharat
> Sent: Thursday, November 17, 2016 5:18:41 PM
> To: dev@kafka.apache.org
> Subject: Re: [DISCUSS] KIP-87 - A
Hi Michael,
I have updated the migration section of the KIP. Can you please take a look?
Thanks,
Mayuresh
On Fri, Nov 18, 2016 at 9:07 AM, Mayuresh Gharat wrote:
> Hi Michael,
>
> That whilst sending tombstone and non null value, the consumer can expect
> only to receive the non-
11/2016, 15:52, "Michael Pearce"
> wrote:
> >
> > Hi Mayuresh,
> >
> > LGTM. Ive just made one small adjustment updating the wire
> protocol to
> > show the magic byte bump.
> >
---
Thanks,
Mayuresh Gharat
/KafkaProducer.java
<https://reviews.apache.org/r/32440/#comment125718>
Since its a Producer level config, is this change needed. We can keep it as
an instance variable. Also since the compression type does not change, the
"private final" makes it more clear. What do you think?
- Mayu
> On March 24, 2015, 5 p.m., Mayuresh Gharat wrote:
> > clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java,
> > line 134
> > <https://reviews.apache.org/r/32440/diff/1/?file=904417#file904417line134>
> >
> > Since its a Produ
tps://reviews.apache.org/r/31369/#comment125947>
messageNo++ can be outside the if-else block and used once.
- Mayuresh Gharat
On March 4, 2015, 1:51 a.m., Ashish Singh wrote:
>
> ---
> This is an automatically generated e-mail.
> On March 25, 2015, 4:48 p.m., Mayuresh Gharat wrote:
> > examples/src/main/java/kafka/examples/Producer.java, line 56
> > <https://reviews.apache.org/r/31369/diff/6/?file=884185#file884185line56>
> >
> > messageNo++ can be outside the if-else block and us
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32440/#review77787
---
Ship it!
- Mayuresh Gharat
On March 25, 2015, 6:29 p.m., Grant
You can take a look at this :
https://issues.apache.org/jira/browse/KAFKA-1848
Thanks,
Mayuresh
On Sun, Mar 29, 2015 at 4:45 PM, Jiangjie Qin (JIRA)
wrote:
>
> [
> https://issues.apache.org/jira/browse/KAFKA-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
>
> J
I suppose adding this timeout will help. In cases were the broker is not
completely down but stops responding to the produce request, tools like
Mirror Makers will hang since they are waiting for responses. Adding this
timeout enables it to fail the current request and retry with fresh
metadata.
T
Agreed we also need to change in the code of Sender.java to indicate that
it resembles REPLICATION_TIMEOUT and not the request Timeout.
Thanks,
Mayuresh
On Thu, Apr 16, 2015 at 1:08 PM, Jiangjie Qin
wrote:
> Hi Guozhang,
>
> By implicit timeout for close() and flush(), I meant that currently w
Just a quick question, can we handle REQUEST TIMEOUT as disconnections and
do a fresh MetaDataRequest and retry instead of failing the request?
Thanks,
Mayuresh
On Mon, May 4, 2015 at 10:32 AM, Jiangjie Qin
wrote:
> I incorporated Ewen and Guozhang’s comments in the KIP page. Want to speed
>
+1 Becket. That would give enough time for clients to move. We should make
this change very clear.
Thanks,
Mayuresh
On Tue, May 12, 2015 at 1:45 PM, Jiangjie Qin
wrote:
> Hey Ewen,
>
> Very good summary about the compatibility. What you proposed makes sense.
> So basically we can do the follow
t; 2. If a broker is down, hopefully metadata refresh will find the new
> broker and we will not try to reconnect to the broker anymore.
>
> Comments are welcome!
>
> Thanks.
>
> Jiangjie (Becket) Qin
>
> On 5/12/15, 2:59 PM, "Mayuresh Gharat" wrote:
>
> >
d not see many new connections
> > get created.
> > 2. If a broker is down, hopefully metadata refresh will find the new
> > broker and we will not try to reconnect to the broker anymore.
> >
> > Comments are welcome!
> >
> > Thanks.
> >
> > Jiang
M, Jiangjie Qin
> >>
> >> wrote:
> >>
> >>> I modified the WIKI page to incorporate the feedbacks from mailing list
> >>> and KIP hangout.
> >>>
> >>> - Added the deprecation plan for TIMEOUT_CONFIG
> >>> - Added t
include both the time in
> >the accumulator an the time taken for the first request, etc. In other
> >words this is the upper bound on the time to the Future being satisfied.
> >
> >*replication.timeout* will default to something reasonable but maybe you
> >can override
quest.timeout* is the bound on the time after send() complete until
> you
> > >get an acknowledgement. This covers the connection timeout, and the time
> > >in
> > >the accumulator. So to implement this, the time we set in the request
> sent
> > >via N
Hi,
I had a question about TopicMetadata Request.
Currently the way it works is :
1) Suppose a topic T1 does not exist.
2) Client wants to produce data to T1 using producer P1.
3) Since T1 does not exist, P1 issues a TopicMetadata request to kafka.
This in turn creates the default number of partit
config in the producer to control whether
> TopicCreateRequest should be issued or not on UnknownTopicException. If
> this is disabled and the topic doesn't exist, send will fail and the user
> is expected to create the topic manually.
>
> Thanks,
>
> Jun
>
>
> On
+1.
Thanks,
Mayuresh
On Mon, Jun 1, 2015 at 6:51 PM, Joe Stein wrote:
> We should probably have some release/vXYZ section so that over time we can
> keep track of what KIP where approved for what release, etc/
>
> Anything not in a release folder (we could do now release/v0.8.3.0 for
> everyth
+1 (non-binding). Thanks Jiangjie.
Thanks,
Mayuresh
On Tue, Jun 2, 2015 at 4:49 PM, Ewen Cheslack-Postava
wrote:
> +1 non-binding. Thanks for all the work on this Jiangjie!
>
> Aditya -- see discussion of partitionsFor() API, which needs to also use
> one of those timeouts, and is a fetch rat
Hi Folks,
I was reading docs on Confluent Schema Registry about Compatibility :
https://docs.confluent.io/platform/current/schema-registry/avro.html#compatibility-types
I was confused with "BACKWARDS" vs "BACKWARDS_TRANSITIVE".
If we have 3 schemas X, X-1, X-2 and configure a schema registry wit
Congratulations Luke!
Thanks,
Mayuresh
On Wed, Feb 9, 2022, 5:24 PM Ismael Juma wrote:
> Congratulations Luke!
>
> On Wed, Feb 9, 2022 at 3:22 PM Guozhang Wang wrote:
>
> > The PMC for Apache Kafka has invited Luke Chen (showuon) as a committer
> and
> > we are pleased to announce that he has
Hi Wesley,
Thanks for the KIP and sorry for being late to the party.
I wanted to understand, the scenario you mentioned in Proposed changes :
-
>
> Estimate the earliest message timestamp of an un-compacted log segment. we
> only need to estimate earliest message timestamp for un-compacted log
>
Hi Colin/Zahari,
I have created a ticket for the similar/same feature :
https://issues.apache.org/jira/browse/KAFKA-7548
We (Linkedin) had a use case in Samza at Linkedin when they moved from the
SimpleConsumer to KafkaConsumer and they wanted to do this pause and resume
pattern.
They realized the
f configuration as this
> data should not be thrown away at all. Submitting a PR sounds great,
> although I feel a bit jealous you (LinkedIn) beat me to my first kafka
> commit ;) Not sure how things stand with the voting process ?
>
> Zahari
>
>
>
> On Th
Hi Zahari,
Created the patch here : https://github.com/apache/kafka/pull/5844
Thanks,
Mayuresh
On Thu, Oct 25, 2018 at 4:42 PM Mayuresh Gharat
wrote:
> Hi Zahari,
>
> Oops. We had planned to put this patch upstream but somehow slipped my
> mind. We were recently going over hotf
o reflect the fact
> that
> > as
> > > already discussed - we do not really need any kind of configuration as
> > this
> > > data should not be thrown away at all. Submitting a PR sounds great,
> > > although I feel a bit jealous you (LinkedIn) beat me to my f
Thanks Jun for sharing this. Looks nice !
Do we intend to shed light on how much time is required, on an average, for
new Leader election. Also would it be good to add "if the controller waits
for the LeaderAndIsrResponses before sending shutDown_OK to the shutting
down broker".
Thanks,
Mayuresh
Hi Lincong,
Thanks for the KIP.
As Colin pointed out, would it better to expose certain specific pieces of
information from the request/response like api key, request headers, record
counts, client ID instead of the entire request/response objects ? This
enables us to change the request response
Hi Boyang,
Thanks for the KIP and sorry for being late to the party. This KIP is
really useful for us at Linkedin.
I had a few questions :
The idea of having static member name seems nice, but instead of a config,
would it be possible for it to be passed in to the consumer at runtime?
This is be
s topic. I will do more analysis and make a trade-off
> comparison. Nice catch!
>
>
> I hope the explanations make sense to you. I will keep polishing on the
> edge cases and details.
>
>
> Best,
>
> Boyang
>
>
> From: Mayuresh Gharat
Hi Boyang,
Thanks for updating the KIP. This is a step good direction for stateful
applications and also mirroring applications whose latency is affected due
to the rebalance issues that we have today.
I had a few questions on the current version of the KIP :
For the effectiveness of the KIP, con
101 - 200 of 633 matches
Mail list logo
