Guozhang Wang created KAFKA-8854:
Summary: Optimize bulk loading of RocksDB during state restoration
of Streams
Key: KAFKA-8854
URL: https://issues.apache.org/jira/browse/KAFKA-8854
Project: Kafka
Hello Pere,
Thanks for you interest in contributing to Kafka, I've added you to the
contributors list and you should be able to create wiki pages now.
BTW there's an on-going KIP-500 which aims at removing ZK dependency of
Kafka recently; depending when it would be voted and be adopted, I think
The reduction on network cost is a valid point: letting the consumer to do
the filtering would waste the network bandwidth as many bytes would be
dropped on the floor directly afterwards. But note that letting brokers to
filter on fetch response means that we cannot use nio zero-copy as we need
to
l error is when the producer has been
> fenced by another instance.
>
> Thanks,
> Jason
>
> On Mon, Aug 26, 2019 at 6:05 PM Guozhang Wang wrote:
>
> > Hi Jason,
> >
> > I've made another pass on the wiki page and it reads much better now. One
> > more clarificat
>> Hi Everyone,
> >>>
> >>> Sorry for the long delay on this KIP. I have updated it to include the
> >>> handling of INVALID_PRODUCER_ID_MAPPING as suggested above. If there
are
> >> no
> >>> further comments, I will plan to start a vote
[
https://issues.apache.org/jira/browse/KAFKA-8412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-8412.
--
Resolution: Fixed
> Still a nullpointer exception thrown on shutdown while flushing bef
t; >
> > It's worth noting that such an assignment would still not be considered
> > "balanced", so the ultimately balanced final state of the assignment
> (after
> > task movements) would still have the desired property that each stateful
> &g
ul
> and stateless task is evenly spread across the cluster.
>
> Does that seem reasonable?
>
> Thanks,
> -John
>
> On Wed, Aug 21, 2019 at 11:22 AM Guozhang Wang wrote:
>
> > Hello John,
> >
> > I've made another pass on the wiki page again, overall LG
h some details, I think it causes some problems,
> which
> >> I've written up in the "rejected alternatives" part of the KIP:
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-441%3A+Smooth+Scaling+Out+for+Kafka+Streams#KIP-441:SmoothScalingOutforKafkaStreams-Addi
%3A+Smooth+Scaling+Out+for+Kafka+Streams#KIP-441:SmoothScalingOutforKafkaStreams-Addinganewkindoftask(%22moving%22,%22recovering%22,%22learner%22)fortaskmovements
>
> Please give it a read and let me know what you think.
>
> Thanks,
> -John
>
> On Thu, Aug 8, 2019 at 7:57 PM Gu
Hello folks,
I'd like to start a voting thread the following KIP to improve the Kafka
Streams metrics mechanism to users. This includes 1) renaming changes in
the public StreamsMetrics utils API, and 2) a major cleanup on the Streams'
own built-in metrics hierarchy.
Details can be found here:
rds", and "expired-window-records-drop".
3. renamed the util functions of StreamsMetrics as `addLatencyRateTotal`
and `addRateTotal` sensors.
Since I feel it has incorporated all of your comments I'm going to start
the vote thread for this KIP now.
Guozhang
On Tue, Aug 20, 2019 at 9
nsors to record invocations but none to
> record amounts. Is that intentional? No need to add it to this KIP, I
> am just curious.
>
> Best,
> Bruno
>
> On Tue, Aug 20, 2019 at 1:13 AM Guozhang Wang wrote:
> >
> > Hi Bruno,
> >
> > Just realized that for `
Hi Bruno,
Just realized that for `addRateSensor` and `addLatencyAndRateSensor` we've
actually added the total invocation metric already.
Guozhang
On Mon, Aug 19, 2019 at 4:11 PM Guozhang Wang wrote:
> Hi Bruno,
>
>
> On Tue, Aug 6, 2019 at 1:51 AM Bruno Cadonna wrote:
>
Hi Bruno,
On Tue, Aug 6, 2019 at 1:51 AM Bruno Cadonna wrote:
> Hi Guozhang,
>
> I left my comments inline.
>
> On Thu, Jul 18, 2019 at 8:28 PM Guozhang Wang wrote:
> >
> > Hello Bruno,
> >
> > Thanks for the feedbacks, replied inline.
> >
>
rt!
> > > >
> > > > 2019년 8월 14일 (수) 오후 8:49, Mickael Maison >님이
> > > 작성:
> > > >
> > > > Hi Guozhang,
> > > >
> > > > Thanks for taking a look.
> > > >
> > > > 1. Right, I updated the titles of
lance listener is to consolidate the entry point of consumer internal
> > states. Compared with letting consumer generate a deep-copy of metadata
> > every time we call #sendOffsetsToTransactions, using a callback seems
> > reducing unnecessary updates towards the metadata. WDY
o code the callback by hand, do you think
> > the
> > > > > convenience we sacrifice here worth the simplification benefit?
> > > > >
> > > > >
> > > > > > 3. Can you clarify the behavior of the clients when th
+1 (binding). This is a great KIP, thanks Jason!
Regarding the naming of the zkVersion, I'm actually fine to name it more
generally and leave a note that at the moment its value is defined as the
zk version.
Guozhang
On Mon, Aug 12, 2019 at 2:22 PM Jason Gustafson wrote:
> Hi Viktor,
>
> I
Hi Mickael,
Thanks for the KIP!
Just some minor comments.
1. Java class names are stale, e.g. "CommitOffsetsOptions.java" should be
"AlterOffsetsOptions".
2. I'd suggest we change the future structure of "AlterOffsetsResult" to
*KafkaFuture>>*
This is because we will have a hierarchy of
+1 (binding).
Thanks Jason!
On Wed, Aug 7, 2019 at 11:18 AM Jason Gustafson wrote:
> Hi All,
>
> I'd like to start a vote on KIP-496:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-496%3A+Administrative+API+to+delete+consumer+offsets
> .
> +1
> from me of course.
>
> -Jason
>
--
for example in how
> > long we block on #poll.
> > So in addition to giving them their own name -- let's go with restoring
> > task for now -- they really
> > do seem to deserve being their own distinct task. We can optimize them
> for
> > efficient conversion
> > t
assigned to the same
> instance they had run before the rebalance. As far as I can see this
> special case is not handled in the algorithm.
>
> Best,
> Bruno
>
>
>
> On Thu, Aug 8, 2019 at 8:24 AM Guozhang Wang wrote:
> >
> > 1. Sounds good, just wanted to c
[
https://issues.apache.org/jira/browse/KAFKA-4600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-4600.
--
Resolution: Fixed
> Consumer proceeds on when ConsumerRebalanceListener fa
[
https://issues.apache.org/jira/browse/KAFKA-8493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-8493.
--
Resolution: Fixed
Fix Version/s: 2.4.0
> Add PartitionsLost API in RebalanceListe
and the assignor makes a decision
> about it, the instance should remain in REBALANCING state, right? If so,
> then this should prevent the race condition. If not, then we do indeed need
> to do something about it.
>
> 5. Good idea. I think that today, you can only see the consumer l
Hello Sophie,
Thanks for the proposed KIP. I left some comments on the wiki itself, and I
think I'm still not very clear on a couple or those:
1. With this proposal, does that mean with num.standby.replicas == 0, we
may sometimes still have some standby tasks which may violate the config?
2. I
; of passing in the consumer instance?
>
> Boyang
>
> On Fri, Aug 2, 2019 at 4:36 PM Guozhang Wang wrote:
>
> > Thanks Boyang,
> >
> > I've made another pass on KIP-447 as well as
> > https://github.com/apache/kafka/pull/7078, and have some minor comments
to tack on some rebalance-related metrics as part of this
> > KIP
> > > as well. The details can be found in the sub-task JIRA:
> > > https://issues.apache.org/jira/browse/KAFKA-8609
> > >
> > > On Thu, May 30, 2019 at 5:09 PM Guozhang Wang
> > wrote:
&g
fig in each release to
> add a new accepted value?
>
>
> -Matthias
>
> On 8/2/19 1:07 PM, Guozhang Wang wrote:
> > Hello Matthias,
> >
> > That's a good question. Thinking about a bit more, I'd like to propose
> that
> > we just list all the version numbers sin
nformation on the assignor anyways.
Guozhang
On Tue, Jul 30, 2019 at 1:55 PM Boyang Chen
wrote:
> Thank you Guozhang for the reply. We will consider the interface change
> from 429 as a backup plan for 447.
>
> And bumping this thread for more discussion.
>
> On Mon, Jul 22, 2
Hi Vinoth,
You've been added to the list, cheers!
Guozhang
On Fri, Aug 2, 2019 at 2:50 PM Vinoth Chandar wrote:
> I am interested in picking up KAFKA-7149
> Can I be added to the list? My jira id : vinoth
>
--
-- Guozhang
ht want to rename it to "newest" /
> "current" or something like this?
>
> Another alternative may be to rename it to "since-2.3" (or similar) --
> however, this may require to rename the config if we change metrics in a
> future release (hence, it's not my pr
with the newly added metrics.
Guozhang
On Mon, Jul 29, 2019 at 9:31 AM Guozhang Wang wrote:
> +1 from myself as well (binding).
>
>
> I'm closing this vote thread with the following votes:
>
> binding +1s: 4 (Guozhang, Jun, Jason, Bill).
>
>
> Thanks everyone who've re
Hello,
I've added you as a contributor to Kafka.
Cheers,
Guozhang
On Wed, Jul 31, 2019 at 5:31 PM Tharindu Amila <
tharindu.am...@instaclustr.com> wrote:
> Hi Team,
>
> Request you to add me to the contributor list. Jira username : tharindu
>
> Thanks and Regards,
>
>
> *Tharindu
[
https://issues.apache.org/jira/browse/KAFKA-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-8704.
--
Resolution: Fixed
Fix Version/s: 2.4.0
> Add PartitionAssignor adapter for backwa
ence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
> >
> Please let me know if you have any other questions and/or concerns.
>
> Regards,
> Levani
>
> > On Jul 31, 2019, at 1:24 AM, Guozhang Wang wrote:
> >
> > Hello Levani,
> &
Hello Levani,
Thanks for the KIP! Just got a quick question here about the scope: why do
we only want this for `KStream`, not `KTable#groupBy` for example?
Guozhang
On Tue, Jul 30, 2019 at 1:27 PM Bill Bejeck wrote:
> Thanks for the KIP Levani.
>
> +1 (binding)
>
> -Bill
>
> On Tue, Jul 30,
her comments.
>
> Thanks,
> Jason
>
> On Mon, Jul 29, 2019 at 9:10 AM Guozhang Wang wrote:
>
> > Thanks for the replies Jason!
> >
> > 2. No I do not see any problems, just trying to understand how restrict
> we
> > are applying this rule :) Piggy-backi
+1 (binding)
On Thu, Jul 25, 2019 at 7:39 PM Matthias J. Sax
wrote:
> +1 (binding)
>
> On 7/25/19 1:05 PM, Bill Bejeck wrote:
> > All,
> >
> > After a great discussion on KIP-479 (
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-479%3A+Add+Materialized+to+Join
> )
> > I'd
> > like
vant. This is the
> approach that I modeled. That being said, if it's ok with you, I'd prefer
> to leave this decision to the implementation. I think the main point is
> that once the leader receives the latest update, it can discard any pending
> state.
>
>
> Thanks,
> Ja
Guozhang Wang created KAFKA-8729:
Summary: Augment ProduceResponse error messaging for specific
culprit records
Key: KAFKA-8729
URL: https://issues.apache.org/jira/browse/KAFKA-8729
Project: Kafka
+1 from myself as well (binding).
I'm closing this vote thread with the following votes:
binding +1s: 4 (Guozhang, Jun, Jason, Bill).
Thanks everyone who've reviewed and voted on the KIP!
Guozhang
On Mon, Jul 29, 2019 at 9:30 AM Guozhang Wang wrote:
> Yeah my thinking is that chang
t that need not block the KIP. I'm +1
> overall.
>
> Thanks,
> Jason
>
> On Fri, Jul 26, 2019 at 4:24 PM Guozhang Wang wrote:
>
> > Hi Jason, thanks for the comments.
> >
> > 1. Yes that's a good point. Will move it to `errors`.
> >
> > 2. The p
Hi Jason,
Thanks for the KIP. It looks good to me overall.
1. Just clarifying the "CurrentVersion" field in the newly proposed
protocol. Does it contains the same value as zkVersion that controller get
from ZK?
2. As for the comment in the KIP: "We can either await the update or we can
send a
on perspective, an empty group is just treated as a case where the
> subscription is empty, which makes all offsets subject to expiration.
>
>
> Thanks,
> Jason
>
> On Thu, Jul 25, 2019 at 1:41 PM Guozhang Wang wrote:
>
> > Hi Jason,
> >
> > Thanks for the KIP!
patibility section. Are you suggesting that we would use the new error
> code even for old versions of the produce request?
>
> Thanks,
> Jason
>
>
>
>
> On Tue, Jul 16, 2019 at 3:46 PM Guozhang Wang wrote:
>
> > Hello folks,
> >
> > I'd like to start a
Hi Jason,
Thanks for the KIP! I've made a pass on it and here are few comments:
1. " before the clients which ." --> incomplete sentence?
2. " Any committed offset for a partition which is not currently subscribed
to is subject to expiration." --> this may be an implementation detail, but
are
+1 (binding).
Thanks Daniyar!
On Wed, Jul 24, 2019 at 12:04 PM John Roesler wrote:
> Thanks, Daniyar,
>
> I'm +1 (nonbinding)
>
> -John
>
> On Tue, Jun 11, 2019 at 1:45 PM Development wrote:
>
> > Hello,
> >
> > I want to start a vote for KIP-466 <
> >
>
if
> that's ok with you.
>
> -John
>
> On Tue, Jul 23, 2019 at 5:38 PM Guozhang Wang wrote:
>
> > Hi John,
> >
> > I left another question regarding Transformer in the DISCUSS thread.
> Other
> > than that I think this KIP is ready. Thanks!
> >
&g
s beyond the scope of this KIP, but this KIP is a
> precondition for these further improvements.
>
> I'm copying your comment onto the ticket for posterity.
>
> Thanks!
> -John
>
> On Tue, Jul 23, 2019 at 5:38 PM Guozhang Wang wrote:
>
> > Hi John,
> >
> >
[
https://issues.apache.org/jira/browse/KAFKA-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-8675.
--
Resolution: Not A Problem
> "Main" consumers are not unsubsribed on Kafka
Hi John,
I left another question regarding Transformer in the DISCUSS thread. Other
than that I think this KIP is ready. Thanks!
Guozhang
On Tue, Jul 16, 2019 at 9:01 AM John Roesler wrote:
> Hi Dev,
>
> After a good discussion, I'd like to start the vote for KIP-478
>
> > >> For the end goal, if possible, here's what I propose.
> > > >>
> > > >>1. We keep the scope of the KIP the same, *but we only
> implement* *it in
> > > >>phases*
> > > >>2. Phase one could includ
Hello Brian,
I think your main question is to distinguish 1) broker is alive but there's
no new data coming into the source topics to process, and 2) broker is not
alive and hence nothing is readable, in your monitoring system. I agree
that currently process-rate / last-record-timestamp cannot
Thanks Sönke! I just made a quick pass on those tickets as well I think
your assessments are right.
Guozhang
On Fri, Jul 19, 2019 at 4:09 AM Sönke Liebau
wrote:
> All,
>
> I left a few comments on some old but still open jiras in an attempt to
> clean up a little bit.
>
> Since probably no
Thanks everyone for your inputs, I've updated the wiki page accordingly.
@Bruno: please let me know if you have any further thoughts per my replies
above.
Guozhang
On Mon, Jul 22, 2019 at 6:30 PM Guozhang Wang wrote:
> Thanks Boyang,
>
> I've thought about exposing time vi
ean the partition time.
>
> On Thu, Jul 18, 2019 at 11:29 AM Guozhang Wang wrote:
>
> > Hi Boyang,
> >
> > What do you mean by `per partition latency`?
> >
> > Guozhang
> >
> > On Mon, Jul 1, 2019 at 9:28 AM Boyang Chen
> > wrote:
> >
>
ntical
indications.
> Boyang
>
> On Fri, Jul 19, 2019 at 3:57 PM Guozhang Wang wrote:
>
> > Boyang, thanks for the updated proposal!
> >
> > 3.a. As Jason mentioned, with EOS enabled we still need to augment the
> > offset fetch request with a boolean to indicat
AFKA/KIP-447%3A+Producer+scalability+for+exactly+once+semantics
> > >
> > > and
> > > >> written a design doc
> > > >> <
> > >
> >
> https://docs.google.com/document/d/1LhzHGeX7_Lay4xvrEXxfciuDWATjpUXQhrEIkph9qRE/edit#
> > &g
[
https://issues.apache.org/jira/browse/KAFKA-8392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-8392.
--
Resolution: Fixed
Fix Version/s: 2.4.0
> Kafka broker leaks metric when partit
+1 (binding). Thanks Daniyar!
Guozhang
On Tue, Jul 16, 2019 at 2:13 PM John Roesler wrote:
> Thanks, Daniyar!
>
> I'm +1 (nonbinding)
>
> -John
>
> On Tue, Jul 16, 2019 at 3:14 PM Development wrote:
> >
> > Hi,
> >
> > I’d like to start a vote thread for KIP-466.
> >
> > This addition will
ords that are processed after
> > the retention period of the window. Is this correct?
> >
> > 4) Is there an actual difference between skipped and dropped records?
> > If not, shall we unify the terminology?
> >
> > 5) What happens with removed metrics when the user sets
period and the latter is for records that are processed after
> > > the retention period of the window. Is this correct?
> > >
> > > 4) Is there an actual difference between skipped and dropped records?
> > > If not, shall we unify the terminology?
> > &
them with 2.2- as well; for other ones that are removed
/ replaced like thread-level skipped-records we should still maintain them.
Best,
> Bruno
>
> On Thu, Jun 27, 2019 at 6:11 PM Guozhang Wang wrote:
> >
> > Hello folks,
> >
> > As 2.3 is released now, I'd l
l, which admittedly indeed leaves a hole of not being able to
> > cover
> > > > all
> > > > > internal names here
> > > >
> > > > I think it's important to close this gap. Naming entities seems to a
> > > > binary feature: i
+1 (binging).
This is a great cleanup, thanks John!
Guozhang
On Wed, Jul 17, 2019 at 11:26 AM Ryanne Dolan wrote:
> +1 (non-binding)
>
> Thanks for the interesting discussion.
>
> Ryanne
>
> On Fri, Jul 12, 2019, 2:49 PM Ryanne Dolan wrote:
>
> > John, I'm glad to learn I'm not the only one
[
https://issues.apache.org/jira/browse/KAFKA-8218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-8218.
--
Resolution: Not A Problem
> IllegalStateException while accessing context in Transfor
[
https://issues.apache.org/jira/browse/KAFKA-7176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-7176.
--
Resolution: Fixed
Fix Version/s: 2.3.0
> State store metrics for migrated ta
[
https://issues.apache.org/jira/browse/KAFKA-7850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-7850.
--
Resolution: Fixed
Fix Version/s: 2.3.0
KStreamTestDriver has been removed since 2.3.0
Hello folks,
I'd like to start a voting thread on KIP-467 to improve error communication
and handling for producer response:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-467
%3A+Augment+ProduceResponse+error+messaging+for+specific+culprit+records
Thanks,
-- Guozhang
to start the voting thread soon.
Guozhang
On Thu, Jun 20, 2019 at 4:24 PM Guozhang Wang wrote:
> Hi Jun,
>
> Thanks for your comments.
>
> 1. Yeah I think APIException would not make a distinct call here anymore,
> and what really matters is the RetriableException. Updat
Hello folks,
This is just a kind reminder of the Bay Area Kafka® meetup tomorrow,
Tuesday 6:30pm, at Confluent's Palo Alto HQ.
*RSVP and Register* (if you intend to attend in person):
https://www.meetup.com/KafkaBayArea/events/262715611/
please read instructions within the link to register for
lutes the interface for its _actual_ use
> case of materializing a table view. Of course, to solve the immediate
> problem, all we need is the store name, but we might feel better about
> adding the store name to Joined if we _also_ feel like in the future,
> we would add store/log/cache con
Hello folks,
As 2.3 is released now, I'd like to bump up this KIP discussion again for
your reviews.
Guozhang
On Thu, May 23, 2019 at 4:44 PM Guozhang Wang wrote:
> Hello Patrik,
>
> Since we are rolling out 2.3 and everyone is busy with the release now
> this KIP does n
gt; > > wrote:
> > > > >
> > > > >> >
> > > > >> > I think co-locating does have some merits here, i.e. letting the
> > > > >> > ConsumerCoordinator which has the source-of-truth of assignment
> to
> >
or this KIP, because it is inherently
> impossible to
> achieve co-locating topic partition of transaction log and consumed offset
> topics.
>
>
> > Thanks,
> > Jason
> >
> On Mon, Jun 24, 2019 at 10:07 AM Boyang Chen
> > wrote:
> >
> >
reaking source compatibility.
>
> If the others feel "kind of favorable" with this overall vision, maybe
> we can make one or more Jira tickets to capture it, and then just
> alter _this_ proposal to `processor.api.Processor` (etc).
>
> WDYT?
> -John
>
> On Sun,
bilities we
> don't require, but are still pretty nice:
> * configure the bytes store
> * set a name for the store
> * configure caching
> * configure logging
> * configure segment interval
>
> Not sure where this nets us out, but it's food for thought.
> -John
>
>
Hello Boyang,
Thanks for the KIP, I have some comments below:
1. "Once transactions are complete, the call will return." This seems
different from the existing behavior, in which we would return a retriable
CONCURRENT_TRANSACTIONS and let the client to retry, is this intentional?
2. "an
in
> a
> > stream, consider merging the join into multi-way joins (yes, this is a
> very
> > hand-wavy thought, but the point here is that we do not try to tackle it
> > now but leave it for a better solution :).
>
> I really like this idea! I agree with you in that this a
t; -John
> >>>
> >>> On Thu, Jun 20, 2019 at 4:34 PM Matthias J. Sax
wrote:
> >>>>
> >>>> Just want to second what Sophie said about the stores. The type of a
> >>>> used stores is completely independent of input/output types.
> >>>>
&
Hello Levani,
You should be able to create new wiki page now.
Thanks,
Guozhang
On Sun, Jun 23, 2019 at 3:18 PM Levani Kokhreidze
wrote:
> Hi,
>
> Please give me permission for creating KIP <
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals>.
> You can find link
Hi Colin,
Thanks for the new RC, +1 (binding).
Verified the javadoc, maven repo, and ran unit tests on 2.12 binary.
Guozhang
On Fri, Jun 21, 2019 at 1:23 PM Colin McCabe wrote:
> Hi Ismael,
>
> Good catch. This should be fixed now.
>
> It seems that if the previously staged Sonatype
Guozhang Wang created KAFKA-8584:
Summary: Allow "bytes" type to generated a ByteBuffer rather than
byte arrays
Key: KAFKA-8584
URL: https://issues.apache.org/jira/browse/KAFKA-8584
Proj
Guozhang Wang created KAFKA-8581:
Summary: Augment ProduceResponse error messaging for specific
culprit records
Key: KAFKA-8581
URL: https://issues.apache.org/jira/browse/KAFKA-8581
Project: Kafka
[
https://issues.apache.org/jira/browse/KAFKA-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-8106.
--
Resolution: Fixed
Fix Version/s: 2.4.0
> Reducing the allocation and copy
assertThat(outputTopic2.readKeyValue(), equalTo(new KeyValue<>("Hello",
> 1L)));
> inputTopic2.pipeInput(1L, "Hello");
>
>
> Jukka
>
> to 20. kesäk. 2019 klo 23.52 Guozhang Wang (wangg...@gmail.com) kirjoitti:
>
> > Hello Jukka,
> >
nt records, which error code and error message should the server
> return to the client?
>
> Jun
>
> On Sat, May 11, 2019 at 12:44 PM Guozhang Wang wrote:
>
> > Hello everyone,
> >
> > I'd like to start a discussion thread on this newly created KIP to
> imp
Hello Jukka,
Thanks for writing the KIP, I have a couple of quick questions:
1) Is "TestRecord" an existing class that you propose to piggy-back on?
Right now we have a scala TestRecord case class but I doubt that was your
proposal, or are you proposing to add a new Java class?
2) Would the new
Hi John,
Thanks for KIP! I've a few comments below:
1. So far the "Motivation" section is very general, and the only concrete
example that I have in mind is `TransformValues#punctuate`. Do we have any
other concrete issues that drive this KIP? If not then I feel better to
narrow the scope of
+1 (binding)
Thanks Bruno!
Would also be interested to see how much overhead it may incur by enabling
DEBUG metrics now, if it is huge we may consider doing finer-grained
metrics enabling, but that would be another follow-up task.
Guozhang
On Wed, Jun 19, 2019 at 1:37 PM Patrik Kleindl wrote:
Hello Bill,
Thanks for the KIP. Glad to see that we can likely shooting two birds with
one stone. I have some concerns though about those "two birds" themselves:
1. About not breaking compatibility of stream-stream join materialized
stores: I think this is a valid issue to tackle, but after
urposes
> > > [2]. To get this data and publish it into a metric, one has to parse a
> > > string. Since this data is for debugging purposes, I do not know how
> > > stable the output format is. One thing, we could do, is to dump the
> > > string with the compact
Hello Bruno,
I've read through the aggregation section and I think they look good to me.
There are a few minor comments about the wiki page itself:
1) A state store might consist of multiple state stores -> You mean a
`logical` state store be consistent of multiple `physical` store instances?
er. The main
> disadvantage
> >> would be that it increases the number of metrics by a factor of
> >> num_workers, but I'd say this is an acceptable tradeoff. Another
> advantage
> >> of your suggestion is that the public interfaces for WorkerConnector
> would
> >>
+1 (binding)
Verified release notes, javadoc, and run quick start with scala_2.12 binary.
Guozhang
On Mon, Jun 17, 2019 at 7:47 PM Satish Duggana
wrote:
> +1 (non-binding)
>
> - Ran testAll/releaseTarGzAll successfully with no failures.
> - Ran through quickstart of core/streams on builds
Hello Cyrus,
Thanks for the KIP. I just have one nit question about Connect destroyed
tasks: is it an ever-increasing number? If yes, the corresponding metric
value would be increasing indefinitely as well. Is that intentional?
Otherwise, lgtm.
Guozhang
On Mon, Jun 17, 2019 at 1:14 PM Gwen
Hello Elliott,
Did you mean Kafka or Airflow? Maybe you happen to jump on a different
mailing list :)
Guozhang
On Wed, Jun 12, 2019 at 3:09 PM Elliott Shugerman <
eshuger...@medianewsgroup.com> wrote:
> Hello,
>
> Please grant me permission to create an Airflow Enhancement Proposal.
> My
Hi Richard,
Sorry for getting late on this, I've finally get some time to take a look
at https://github.com/apache/kafka/pull/6594 as well as the KIP itself.
Here are some thoughts:
1. The main motivation of this KIP is to be able to distinguish the case
where
a. "Streams client is in an
901 - 1000 of 6729 matches
Mail list logo
