Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-11 Thread John Roesler
Thanks, all, This proposal sounds good to me. We could try to just disable the current flaky tests as a one-time step and configure GitHub only to merge green builds as a bulwark against the future. By definition, flaky tests may not fail during the buggy PR build itself, but they should make

Re: [ANNOUNCE] New Kafka PMC Member: Satish Duggana

2023-10-29 Thread John Roesler
Congratulations, Satish! -John On Sun, Oct 29, 2023, at 08:09, Randall Hauch wrote: > Congratulations, Satish! > > On Sun, Oct 29, 2023 at 1:47 AM Tom Bentley wrote: > >> Congratulations! >> >> On Sun, 29 Oct 2023 at 5:41 PM, Guozhang Wang >> wrote: >> >> > Congratulations Satish! >> > >> > On

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

2023-09-24 Thread John Roesler
Congratulations, Justine! -John On Sun, Sep 24, 2023, at 05:05, Mickael Maison wrote: > Congratulations Justine! > > On Sun, Sep 24, 2023 at 5:04 AM Sophie Blee-Goldman > wrote: >> >> Congrats Justine! >> >> On Sat, Sep 23, 2023, 4:36 PM Tom Bentley wrote: >> >> > Congratulations! >> > >> > On

Re: [VOTE] KIP-954: expand default DSL store configuration to custom types

2023-07-29 Thread John Roesler
Thanks for the KIP, Almog! I'm +1 (binding) I've reviewed the KIP and skimmed the discussion thread. I think this is going to be a very nice improvement. Thanks, -John On Sat, Jul 29, 2023, at 13:26, Guozhang Wang wrote: > Thanks Almog! I made a pass over the updated wiki and have no more

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

2023-07-26 Thread John Roesler
Hello Shay, Thanks for the KIP! I just took a look in preparation to vote, and there are two small-ish things that I'd like to fix first. Apologies if this stuff has already come up in the discussion thread; I only skimmed it. 1. The KIP only mentions the name of the method instead of

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-21 Thread John Roesler
On Wed, 21 Jun 2023 at 18:25, John Roesler wrote: Thanks Nick, That sounds good to me. I can't let (2) slide, though.. Writing and ingesting SST files is not a RocksDB internal, but rather a supported usage pattern on public APIs. Regardless, I think your overall preference is fine with me,

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-21 Thread John Roesler
Thanks Nick, That sounds good to me. I can't let (2) slide, though.. Writing and ingesting SST files is not a RocksDB internal, but rather a supported usage pattern on public APIs. Regardless, I think your overall preference is fine with me, especially if we can internalize this change

Re: [DISCUSS] KIP-941: Range queries to accept null lower and upper bounds

2023-06-21 Thread John Roesler
Hi all, Thanks for the KIP, Lucia! This is a nice change. To Kirk's question (1), the example is a bit misleading. The typical case that would ease user pain is specifically using "null" to indicate an open-ended range, especially since null is not a valid key. I could additionally see an

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-20 Thread John Roesler
the memorable with Rocks WriteBatches route if the cache is disabled? On Tue, 20 Jun 2023, 22:00 John Roesler, wrote: Touché! Ok, I agree that figuring out the case of a disabled cache would be non-trivial. Ingesting single-record SST files will probably not be performant, but benchmarking may prove

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-20 Thread John Roesler
them on commit, when we're already waiting for acks and a RocksDB commit. In this case, how would uncommitted records be read by joins? On Tue, 20 Jun 2023, 20:51 John Roesler, wrote: Ah, sorry Nick, I just meant the regular heap based cache that we maintain in Streams. I see that it's

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-20 Thread John Roesler
g for acks and a RocksDB commit. Thanks, -John On 6/20/23 14:09, Nick Telford wrote: Hi John, By "RecordCache", do you mean the RocksDB "WriteBatch"? I can't find any class called "RecordCache"... Cheers, Nick On Tue, 20 Jun 2023 at 19:42, John Roesler wrote: Hi

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2023-06-20 Thread John Roesler
What I meant in the KIP is that Position offsets are currently stored in a file, and since we can atomically store metadata along with the record batch we commit to RocksDB, we can move our Position offsets in to this metadata too, and gain the same transactional guarantees that

Re: [VOTE] KIP-872: Add Serializer#serializeToByteBuffer() to reduce memory copying

2023-06-20 Thread John Roesler
Hi Divij and ShunKang, I pulled open this thread to see if you needed my vote, but FYI, Divij is a committer now, so he can re-cast his vote as binding. Thanks, -John On 2023/06/20 13:37:04 ShunKang Lin wrote: > Hi all, > > Bump this thread again and see if we could get a few more votes. >

Re: [ANNOUNCE] New committer: Divij Vaidya

2023-06-13 Thread John Roesler
Congratulations, Divij! -John On Tue, Jun 13, 2023, at 09:44, David Arthur wrote: > Congrats Divij! > > On Tue, Jun 13, 2023 at 12:34 PM Igor Soarez wrote: > >> Congratulations Divij! >> >> -- >> Igor >> > > > -- > -David

Re: [VOTE] 3.5.0 RC1

2023-06-07 Thread John Roesler
Thanks for running this release, Mickael! I've verified: * the signature * that I can compile the project * that I can run the tests. I saw one flaky test failure, but I don't think it should block us. Reported as https://issues.apache.org/jira/browse/KAFKA-13531?focusedCommentId=17730190 * the

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-06-07 Thread John Roesler
2023 at 8:32 AM John Roesler wrote: > >> Hello all, >> >> I put in a ticket with Apache Infra to re-send these invites, and they >> told me I should just remove the usernames in one commit and then re-add >> them in a subsequent commit to trigger the invites again. >

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-06-06 Thread John Roesler
Hello Greg, Andrew, and Victoria, I have just re-added you to the file, so you should receive new invites. Please let us know if you have any trouble! Thanks, -John On Tue, Jun 6, 2023, at 10:31, John Roesler wrote: > Hello all, > > I put in a ticket with Apache Infra to re-send thes

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-06-06 Thread John Roesler
ith the assumption that things are working for me even if I > didn't have to accept an invite. Is that a correct assumption? > > -- > Divij Vaidya > > > > On Tue, Jun 6, 2023 at 5:32 PM John Roesler wrote: > >> Hello all, >> >> I put in a ticket with Apache In

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-06-06 Thread John Roesler
il.com> >> > > > > > > wrote: >> > > > > > > >> > > > > > > > +1 on Divij suggestions >> > > > > > > > >> > > > > > > > >> > > > > > > > On Wed, May 2

Re: [VOTE] KIP-923: Add A Grace Period to Stream Table Join

2023-06-06 Thread John Roesler
Thanks for the KIP, Walker! I’m +1 (binding) Thanks, John On Mon, Jun 5, 2023, at 13:39, Victoria Xia wrote: > Hi Walker, > > Thanks for the KIP! Left a clarification question on the discussion thread > just now but it's about an implementation detail, so I don't think it > changes anything in

Re: [VOTE] KIP-925: rack aware task assignment in Kafka Streams

2023-06-05 Thread John Roesler
Thanks, Hao! I'm +1 (binding) -John On 2023/06/05 18:07:56 Walker Carlson wrote: > +1 (binding) > > On Mon, Jun 5, 2023 at 3:14 AM Bruno Cadonna wrote: > > > Hi Hao, > > > > +1 (binding) > > > > Thanks! > > Bruno > > > > On 30.05.23 21:16, Colt McNealy wrote: > > > +1 (non-binding) > > > > >

Re: [DISCUSS] KIP-925: rack aware task assignment in Kafka Streams

2023-05-22 Thread John Roesler
Hi Hao, Thanks for the KIP! Overall, I think this is a great idea. I always wanted to circle back after the Smooth Scaling KIP to put a proper optimization algorithm into place. I think this has the promise to really improve the quality of the balanced assignments we produce. Thanks for

Re: [VOTE] KIP-927: Improve the kafka-metadata-quorum output

2023-05-20 Thread John Roesler
I’m +1 (binding) Thanks, Federico! It looks like a nice improvement to me. -John On Sat, May 20, 2023, at 09:16, Kamal Chandraprakash wrote: > +1 (non binding) > > On Wed, May 17, 2023 at 3:22 AM Divij Vaidya > wrote: > >> +1 (non binding) >> >> Divij Vaidya >> >> >> >> On Tue, May 16, 2023

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-05-15 Thread John Roesler
, Divij Vaidya, Lucas Brutschy, Yash Mayya, Philip Nee, vamossagar12, Christo Lolov, Federico Valeri, and andymg3 Thanks all, -John On 2023/05/12 15:53:40 John Roesler wrote: > Thanks again for bringing this up, David! > > As an update to the community, the PMC has approved a process to

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-05-12 Thread John Roesler
Thanks again for bringing this up, David! As an update to the community, the PMC has approved a process to make use of this feature. Here are the relevant updates: PR to add the policy: https://github.com/apache/kafka-site/pull/510 PR to update the list:

[jira] [Created] (KAFKA-14995) Automate asf.yaml collaborators refresh

2023-05-12 Thread John Roesler (Jira)
John Roesler created KAFKA-14995: Summary: Automate asf.yaml collaborators refresh Key: KAFKA-14995 URL: https://issues.apache.org/jira/browse/KAFKA-14995 Project: Kafka Issue Type

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-04-28 Thread John Roesler
Hi all, This is a great suggestion! It seems like a really good way to make the Apache Kafka project more efficient in general and also smooth the path to committership. I've brought the topic up with the Apache Kafka PMC to consider adopting a policy around the "collaborator" rule. In the

Re: [VOTE] KIP-906 Tools migration guidelines

2023-03-24 Thread John Roesler
Thanks Federico, I'm +1 (binding) -John On Fri, Mar 24, 2023, at 01:11, Federico Valeri wrote: > Bumping this thread for more votes. > > Thanks. > > On Wed, Mar 15, 2023, 9:57 AM Alexandre Dupriez > wrote: > >> Hi, Frederico, >> >> Thanks for the KIP. >> >> Non-binding +1. >> >> Thanks, >>

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-03-24 Thread John Roesler
Thanks for the KIP, Sagar! At first glance, this seems like a very useful feature. A common pain point in Streams is when upstream producers don't send regular updates and stream time cannot advance. This causes stream-time-driven operations to appear to hang, like time windows not closing,

Re: [DISCUSS] KIP-914 Join Processor Semantics for Versioned Stores

2023-03-09 Thread John Roesler
Thanks for the KIP, Victoria! I had some questions/concerns, but you addressed them in the Rejected Alternatives section. Thanks for the thorough proposal! -John On Thu, Mar 9, 2023, at 18:59, Victoria Xia wrote: > Hi everyone, > > I have a proposal for updating Kafka Streams's stream-table

Re: [ANNOUNCE] New Kafka PMC Member: Chris Egerton

2023-03-09 Thread John Roesler
Congratulations, Chris! -John On Thu, Mar 9, 2023, at 20:02, Luke Chen wrote: > Congratulations, Chris! > > On Fri, Mar 10, 2023 at 9:57 AM Yash Mayya wrote: > >> Congratulations Chris! >> >> On Thu, Mar 9, 2023, 23:42 Jun Rao wrote: >> >> > Hi, Everyone, >> > >> > Chris Egerton has been a

Re: [ANNOUNCE] New Kafka PMC Member: David Arthur

2023-03-09 Thread John Roesler
Congratulations, David! -John On Thu, Mar 9, 2023, at 20:18, ziming deng wrote: > Congrats David! > > Ziming > >> On Mar 10, 2023, at 10:02, Luke Chen wrote: >> >> Congratulations, David! >> >> On Fri, Mar 10, 2023 at 9:56 AM Yash Mayya wrote: >> >>> Congrats David! >>> >>> On Thu, Mar 9,

Re: [VOTE] KIP-898: Modernize Connect plugin discovery

2023-02-28 Thread John Roesler
Thanks for the KIP, Greg! I’m +1 (binding) I really appreciate all the care you took in the migration and test design. Thanks, John On Tue, Feb 28, 2023, at 04:33, Federico Valeri wrote: > +1 (non binding) > > Thanks > Fede > > On Tue, Feb 28, 2023 at 10:10 AM Mickael Maison > wrote: >> >>

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-02-23 Thread John Roesler
Thanks for the KIP, Philip! I think your proposal makes sense. I suspect the reason that we previously did the DNS resolution in the constructor is to fail fast if the name is wrong. On the other hand, it's generally a hassle to do failure-prone or slow operations in a constructor, so I'm in

Re: [ANNOUNCE] New committer: Lucas Bradstreet

2023-02-16 Thread John Roesler
Congratulations, Lucas! -John On Thu, Feb 16, 2023, at 19:52, Luke Chen wrote: > Congratulations, Lucas! > > Luke > > On Fri, Feb 17, 2023 at 8:57 AM Guozhang Wang > wrote: > >> Congratulations, Lucas! >> >> On Thu, Feb 16, 2023 at 3:18 PM Kowshik Prakasam >> wrote: >> > >> > Congratulations,

Re: [ANNOUNCE] New committer: Walker Carlson

2023-01-17 Thread John Roesler
Congratulations, Walker! -John On Tue, Jan 17, 2023, at 18:50, Guozhang Wang wrote: > Congrats, Walker! > > On Tue, Jan 17, 2023 at 2:20 PM Chris Egerton > wrote: > >> Congrats, Walker! >> >> On Tue, Jan 17, 2023, 17:07 Bill Bejeck wrote: >> >> > Congratulations, Walker! >> > >> > -Bill >> > >>

Re: [ANNOUNCE] New committer: Satish Duggana

2023-01-17 Thread John Roesler
Ay, sorry about my autocorrect, Satish. On Tue, Jan 17, 2023, at 19:13, John Roesler wrote: > Congratulations, Salish! I missed the announcement before. > > -John > > On Tue, Jan 17, 2023, at 18:53, Guozhang Wang wrote: >> Congratulations, Satish! >> >> On Tue

Re: [ANNOUNCE] New committer: Satish Duggana

2023-01-17 Thread John Roesler
Congratulations, Salish! I missed the announcement before. -John On Tue, Jan 17, 2023, at 18:53, Guozhang Wang wrote: > Congratulations, Satish! > > On Tue, Jan 10, 2023 at 11:31 AM Rajini Sivaram > wrote: > >> Congratulations, Satish! >> >> Regards, >> >> Rajini >> >> On Tue, Jan 10, 2023 at

Re: [ANNOUNCE] New committer: Stanislav Kozlovski

2023-01-17 Thread John Roesler
Congrats, Stanislav! -John On Tue, Jan 17, 2023, at 18:56, Ismael Juma wrote: > Congratulations Stanislav! > > Ismael > > On Tue, Jan 17, 2023 at 7:51 AM Jun Rao wrote: > >> Hi, Everyone, >> >> The PMC of Apache Kafka is pleased to announce a new Kafka committer >> Stanislav Kozlovski. >> >>

Re: [VOTE] KIP-710: Full support for distributed mode in dedicated MirrorMaker 2.0 clusters

2023-01-09 Thread John Roesler
> >> +1 (non-binding) >> >> Thanks for the KIP, this is an important improvement. >> >> Greg >> >> On Thu, Nov 10, 2022 at 7:21 AM John Roesler wrote: >> >> > Thanks for the KIP, Daniel! >> > >> > I'm no MM expert, but I

Re: [DISCUSS] KIP-894: Use incrementalAlterConfigs API for syncing topic configurations

2023-01-07 Thread John Roesler
Hi Tina, Thanks for the KIP! I hope someone with prior MM or Kafka config experience is able to chime in here; I have neither. I took a look at your KIP, and it makes sense to me. I also think your migration plan is a good one. One suggestion: I'm not sure how concerned you are about

Re: [ANNOUNCE] New committer: Edoardo Comar

2023-01-07 Thread John Roesler
Congrats, Edoardo! -John On Fri, Jan 6, 2023, at 20:47, Matthias J. Sax wrote: > Congrats! > > On 1/6/23 5:15 PM, Luke Chen wrote: >> Congratulations, Edoardo! >> >> Luke >> >> On Sat, Jan 7, 2023 at 7:58 AM Mickael Maison >> wrote: >> >>> Congratulations Edo! >>> >>> >>> On Sat, Jan 7, 2023

Re: [ANNOUNCE] New committer: Justine Olshan

2023-01-03 Thread John Roesler
Congrats, Justine! -John On Tue, Jan 3, 2023, at 13:03, Matthias J. Sax wrote: > Congrats! > > On 12/29/22 6:47 PM, ziming deng wrote: >> Congratulations Justine! >> — >> Best, >> Ziming >> >>> On Dec 30, 2022, at 10:06, Luke Chen wrote: >>> >>> Congratulations, Justine! >>> Well deserved! >>>

Re: [ANNOUNCE] New committer: Viktor Somogyi-Vass

2022-12-20 Thread John Roesler
Congratulations, Viktor! -John On Sat, Dec 17, 2022, at 05:24, Viktor Somogyi-Vass wrote: > Thanks again everyone! > > On Fri, Dec 16, 2022, 18:36 Bill Bejeck wrote: > >> Congratulations, Viktor! >> >> -Bill >> >> On Fri, Dec 16, 2022 at 12:32 PM Matthias J. Sax wrote: >> >> > Congrats! >> >

Re: [ANNOUNCE] New committer: Josep Prat

2022-12-20 Thread John Roesler
Congratulations, Josep! -John On Tue, Dec 20, 2022, at 20:02, Luke Chen wrote: > Congratulations, Josep! > > Luke > > On Wed, Dec 21, 2022 at 6:26 AM Viktor Somogyi-Vass > wrote: > >> Congrats Josep! >> >> On Tue, Dec 20, 2022, 21:56 Matthias J. Sax wrote: >> >> > Congrats! >> > >> > On

Re: [DISCUSS] KIP-886 Add Client Producer and Consumer Builders

2022-12-17 Thread John Roesler
Hello Dan, Thanks for the KIP! I just saw the vote thread, took a look at the KIP, and had some questions. Sorry for the delay in reviewing. It can be a hassle, but the KIP document should specify the exact public interfaces you are proposing. This helps to prevent a gap between what

Re: [VOTE] KIP-889 Versioned State Stores

2022-12-15 Thread John Roesler
Thanks for the thorough KIP, Victoria! I'm +1 (binding) -John On 2022/12/15 19:56:21 Victoria Xia wrote: > Hi all, > > I'd like to start a vote on KIP-889 for introducing versioned key-value > state stores to Kafka Streams: >

Re: [VOTE] KIP-893: The Kafka protocol should support nullable structs

2022-12-05 Thread John Roesler
+1 (binding) Thanks, -John On Mon, Dec 5, 2022, at 16:57, Kirk True wrote: > +1 (non-binding) > > On Mon, Dec 5, 2022, at 10:05 AM, Colin McCabe wrote: >> +1 (binding) >> >> best, >> Colin >> >> On Mon, Dec 5, 2022, at 10:03, David Jacot wrote: >> > Hi all, >> > >> > As this KIP-893 is trivial

Re: Ci stability

2022-11-24 Thread John Roesler
ohn! Is there a jira tag or view or something that > can be used to find all the failing tests and maybe even try to fix them > (even if fix just means extending a timeout)? > > > > On Thu, Nov 24, 2022, 16:03 John Roesler wrote: > >> Hi Dan, >> >> Thanks for pointing thi

Re: Ci stability

2022-11-24 Thread John Roesler
Hi Dan, Thanks for pointing this out. Flaky tests are a perennial problem. We knock them out every now and then, but eventually more spring up. I’ve had some luck in the past filing Jira tickets for the failing tests as they pop up in my PRs. Another thing that seems to motivate people is to

Re: Regarding an issue with message distributions in kafka.

2022-11-23 Thread John Roesler
Hi Puneet, Thanks for the question. It sounds like either the three consumers have different group.id configurations or they are using “assign” instead of “subscribe”. You are correct, if the consumers are all in the same consumer group and subscribed to the same topic, Kafka will only assign

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

2022-11-22 Thread John Roesler
Thanks for publishing this alternative, Nick! The benchmark you mentioned in the KIP-844 discussion seems like a compelling reason to revisit the built-in transactionality mechanism. I also appreciate you analysis, showing that for most use cases, the write batch approach should be just fine.

Re: [VOTE] KIP-884: Add config to configure KafkaClientSupplier in Kafka Streams

2022-11-21 Thread John Roesler
I'm +1 (binding) Thanks for the KIP! -John On 2022/11/17 21:06:29 Hao Li wrote: > Hi all, > > I would like start a vote on KIP-884: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-884%3A+Add+config+to+configure+KafkaClientSupplier+in+Kafka+Streams > > > Thanks, > Hao >

Re: Starting out with Kafka

2022-11-13 Thread John Roesler
lugin on IDE and > then recompiling from terminal the projecting is building fine now. > > Thanks John and Ziming for helping out. > > Thanks, > Vinay > > On Fri, Nov 11, 2022 at 11:01 AM John Roesler wrote: > >> Hmm, >> >> I assume you did, but just to be sure, did you co

Re: Streams: clarification needed, checkpoint vs. position files

2022-11-11 Thread John Roesler
Hi all, Just to clarify: there actually is a position file. It was a small detail of the IQv2 implementation to add it, otherwise a persistent store's position would be lost after a restart. Otherwise, Sophie is right on the money. The checkpoint refers to an offset in the changelog, while

Re: Starting out with Kafka

2022-11-11 Thread John Roesler
log). > I'm attaching the result of the unit test here. > Screen Shot 2022-11-11 at 10.45.48 AM.png > TIA. > > On Fri, Nov 11, 2022 at 5:04 AM John Roesler wrote: >> Hello Vinay, >> >> One thing I’ve noticed recently is that I have to click the “build” button >> in

Re: Starting out with Kafka

2022-11-11 Thread John Roesler
Hello Vinay, One thing I’ve noticed recently is that I have to click the “build” button in intellij before I can use the “run” or “debug” buttons. I’m not sure why. Welcome to the community! -John On Fri, Nov 11, 2022, at 02:47, deng ziming wrote: > Hello, Vinay > Kafka uses gradlew as build

Re: [VOTE] KIP-710: Full support for distributed mode in dedicated MirrorMaker 2.0 clusters

2022-11-10 Thread John Roesler
Thanks for the KIP, Daniel! I'm no MM expert, but I've read over the KIP and discussion, and it seems reasonable to me. I'm +1 (binding). Thanks, -John On 2022/10/22 07:38:38 Urbán Dániel wrote: > Hi everyone, > > I would like to start a vote on KIP-710 which aims to support running a >

[jira] [Created] (KAFKA-14364) Support evolving serde with Foreign Key Join

2022-11-07 Thread John Roesler (Jira)
John Roesler created KAFKA-14364: Summary: Support evolving serde with Foreign Key Join Key: KAFKA-14364 URL: https://issues.apache.org/jira/browse/KAFKA-14364 Project: Kafka Issue Type

Re: [VOTE] KIP-852: Optimize calculation of size for log in remote tier

2022-11-06 Thread John Roesler
Hi Divij, Thanks for the KIP! I’ve read through your write-up, and it sounds reasonable to me. I’m +1 (binding) Thanks, John On Tue, Nov 1, 2022, at 05:03, Divij Vaidya wrote: > Hey folks > > The discuss thread for this KIP has been open for a few months with no > concerns being surfaced. I

Re: [DISCUSS] KIP-872: Add Serializer#serializeToByteBuffer() to reduce memory copying

2022-11-06 Thread John Roesler
yte[] with > ByteBuffer, I will document the intent in your KIP and JavaDocs later. > > I will clarify that if a Serializer implements the new method, then the old > one will never be called. > > Best, > ShunKang > > John Roesler 于2022年11月4日周五 22:42写道: > >> Hi ShunKa

Re: [DISCUSS] KIP-872: Add Serializer#serializeToByteBuffer() to reduce memory copying

2022-11-04 Thread John Roesler
Hi ShunKang, Thanks for the KIP! I’ve been wanting to transition toward byte buffers for a while, so this is a nice start. I thought it was a bit weird to flip the buffer inside the serializer, but I see the existing one already does that. I would have thought it would make more sense for

Re: [VOTE] KIP-869: Improve Streams State Restoration Visibility

2022-10-20 Thread John Roesler
Thanks for the KIP, Guozhang! I'm +1 (binding) -John On Wed, Oct 12, 2022, at 16:36, Nick Telford wrote: > Can't wait! > +1 (non-binding) > > On Wed, 12 Oct 2022, 18:02 Guozhang Wang, > wrote: > >> Hello all, >> >> I'd like to start a vote for the following KIP, aiming to improve Kafka >>

Re: [ANNOUNCE] New committer: Deng Ziming

2022-10-10 Thread John Roesler
Congratulations, Ziming! On Mon, Oct 10, 2022, at 12:05, Justine Olshan wrote: > Congratulations Ziming! I'll always remember the help you provided with > topic IDs. > Very well deserved! > > On Mon, Oct 10, 2022 at 9:53 AM Matthew Benedict de Detrich > wrote: > >> Congratulations! >> >> On Mon,

Re: [VOTE] 3.3.1 RC0

2022-09-30 Thread John Roesler
Hi José, I verified the signatures and ran all the unit tests, as well as the Streams integration tests with: > ./gradlew -version > > > Gradle 7.4.2 > > > Build time:

Re: [VOTE] 3.3.0 RC2

2022-09-26 Thread John Roesler
Thanks for running this, David! I've verified the signatures, looked at the docs, and run the quickstart (ZK and KRaft). I also ran the unit tests, as well as all the tests for Streams locally. The docs look a little malformed (the "collapse/expand" button floats over the text, the collapsed

[jira] [Created] (KAFKA-14254) Format timestamps in assignor logs as dates instead of integers

2022-09-21 Thread John Roesler (Jira)
John Roesler created KAFKA-14254: Summary: Format timestamps in assignor logs as dates instead of integers Key: KAFKA-14254 URL: https://issues.apache.org/jira/browse/KAFKA-14254 Project: Kafka

[jira] [Created] (KAFKA-14253) StreamsPartitionAssignor should print the member count in assignment logs

2022-09-21 Thread John Roesler (Jira)
John Roesler created KAFKA-14253: Summary: StreamsPartitionAssignor should print the member count in assignment logs Key: KAFKA-14253 URL: https://issues.apache.org/jira/browse/KAFKA-14253 Project

Re: [VOTE] KIP-821: Connect Transforms support for nested structures

2022-09-14 Thread John Roesler
00:01, Bill Bejeck wrote: >> >> > Thanks for the KIP, Jorge. >> > >> > This looks like a great addition to Kafka Connect. >> > >> > +1(binding) >> > >> > -Bill >> > >> > On Thu, Apr 21, 2022 at 6:41 PM John Ro

Re: [VOTE] KIP-862: Self-join optimization for stream-stream joins

2022-09-12 Thread John Roesler
Thanks for the updates, Vicky! I've reviewed the KIP and your POC PR, and I'm +1 (binding). Thanks! -John On Mon, Sep 12, 2022, at 09:13, Vasiliki Papavasileiou wrote: > Hey Guozhang, > > Great suggestion, I made the change. > > Best, > Vicky > > On Fri, Sep 9, 2022 at 10:43 PM Guozhang Wang

[jira] [Created] (KAFKA-14202) IQv2: Expose binary store schema to store implementations

2022-09-06 Thread John Roesler (Jira)
John Roesler created KAFKA-14202: Summary: IQv2: Expose binary store schema to store implementations Key: KAFKA-14202 URL: https://issues.apache.org/jira/browse/KAFKA-14202 Project: Kafka

Re: [VOTE] KIP-844: Transactional State Stores

2022-09-01 Thread John Roesler
Thanks for the KIP, Alex! +1 (binding) from me. -John On Thu, Sep 1, 2022, at 09:51, Guozhang Wang wrote: > +1, thanks Alex! > > On Thu, Sep 1, 2022 at 6:33 AM Bruno Cadonna wrote: > >> Thanks for the KIP! >> >> +1 (binding) >> >> Best, >> Bruno >> >> On 01.09.22 15:26, Colt McNealy wrote: >>

Re: Hosting Kafka Videos on ASF YouTube channel

2022-08-25 Thread John Roesler
Thanks all, I’m also +1 on the Kafka Streams videos. Thanks, John On Tue, Aug 9, 2022, at 03:54, Mickael Maison wrote: > Hi, > > I checked the four Streams videos > (https://kafka.apache.org/32/documentation/streams/), they are good > and don't mention any vendors. > +1 (binding) for these

Re: [DISCUSS] KIP-862: Implement self-join optimization

2022-08-12 Thread John Roesler
Thanks for the KIP, Vicky! Re 1/2, I agree with what you both worked out. Re 3: It sounds like you were able to preserve backward compatibility, so I don’t think you need to add any new configs. I think you can just switch it on if people specify “all”. Thanks! -John On Thu, Aug 11, 2022,

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-08-12 Thread John Roesler
Thanks, Sagar! I’m +1 (binding) Can you add a short explanation to each rejected alternative? I was wondering why we wouldn’t provide an overloaded to()/addSink() (the first rejected alternative), and I had to look back at the Streams code to see that they both already accept the partitioner

Re: [DISCUSS] Website changes required for Apache projects

2022-07-28 Thread John Roesler
> >> > > >> >> > > >> On Thu, Jul 21, 2022 at 4:19 AM Divij Vaidya < >> divijvaidy...@gmail.com >> > > >> > > >> wrote: >> > > >> >> > > >>> Thanks for chiming in with your op

Re: [DISCUSS] Website changes required for Apache projects

2022-07-21 Thread John Roesler
e changes i.e. it goes through a community review requiring > votes committers/PMC members. > > Regards, > Divij Vaidya > > > > On Thu, Jul 21, 2022 at 3:57 AM John Roesler wrote: > >> Hi all, >> >> Yes, thanks Divij for driving this! >> >&

Re: [DISCUSS] Website changes required for Apache projects

2022-07-20 Thread John Roesler
Hi all, Yes, thanks Divij for driving this! I tend to agree with Mickael about having vendor branding front-and-center like that. On the other hand, I think the video itself is quite nice, and it's a good thing to put in front of newcomers for a human introduction to the project. I took a look

Re: [VOTE] KIP-851: : Add requireStable flag into ListConsumerGroupOffsetsOptions

2022-06-30 Thread John Roesler
Thanks for the KIP, Guozhang! I’m +1 (binding) -John On Thu, Jun 30, 2022, at 21:17, deng ziming wrote: > Thanks for this KIP, > we have a kafka-consumer-groups.sh shell which is based on the API you > proposed to change, is it worth update it as well? > > -- > Best, > Ziming > >> On Jul 1,

[jira] [Created] (KAFKA-14020) Performance regression in Producer

2022-06-24 Thread John Roesler (Jira)
John Roesler created KAFKA-14020: Summary: Performance regression in Producer Key: KAFKA-14020 URL: https://issues.apache.org/jira/browse/KAFKA-14020 Project: Kafka Issue Type: Bug

Re: [VOTE] KIP-840: Config file option for MessageReader/MessageFormatter in ConsoleProducer/ConsoleConsumer

2022-06-08 Thread John Roesler
Thanks for the KIP, Alexandre! I’m +1 (binding) -John On Wed, Jun 8, 2022, at 20:54, deng ziming wrote: > Thank you for this KIP, > > +1 (non-binding) > > -- > Best, > Ziming > >> On Jun 7, 2022, at 8:53 PM, Alexandre Garnier wrote: >> >> Hi! >> >> A little reminder to vote for this KIP. >>

Re: [VOTE] KIP-827: Expose logdirs total and usable space via Kafka API

2022-05-31 Thread John Roesler
Thanks for the KIP Mickael, I'm +1 (binding) -John On Tue, May 31, 2022, at 18:48, Jun Rao wrote: > Hi, Mickael, > > Thanks for the KIP. +1 > > Jun > > On Wed, May 25, 2022 at 7:54 AM Tom Bentley wrote: > >> Hi Mickael, >> >> Thanks for the KIP! +1 (binding). >> >> Kind regards, >> >> Tom >>

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-31 Thread John Roesler
Generally, I agree with Ismael that having a new, weird name will make it hard to keep them straight. Then again, we need to make them different to prevent confusion about their semantics. To be clear, I'll be a +1 regardless of how we break this dilemma. One suggestion: We currently have

Re: [VOTE] KIP-846: Processor-level Streams metrics for records/bytes Producedd

2022-05-31 Thread John Roesler
+1 (binding) Thanks, John On Mon, May 30, 2022, at 13:00, Bill Bejeck wrote: > +1 (binding) > > -Bill > > On Mon, May 30, 2022 at 4:49 AM Sagar wrote: > >> +1 (non-binding). >> >> Thanks! >> Sagar. >> >> On Mon, May 30, 2022 at 1:11 PM Bruno Cadonna wrote: >> >> > +1 (binding) >> > >> >

Re: [DISCUSS] KIP-846: Task-level Streams metrics for bytes/records Produced

2022-05-28 Thread John Roesler
Thanks for the well motivated and documented KIP, Sophie! I’m in favor of this change. -John On Sat, May 28, 2022, at 06:42, Sophie Blee-Goldman wrote: > Hey all, > > I'd like to propose a very small KIP to add two metrics that will help fill > a gap in the derivable produced and consumed

Re: [DISCUSS] KIP-844: Transactional State Stores

2022-05-24 Thread John Roesler
Thanks for the KIP, Alex! I'm really happy to see your proposal. This improvement fills a long-standing gap. I have a few questions: 1. Configuration The KIP only mentions RocksDB, but of course, Streams also ships with an InMemory store, and users also plug in their own custom state stores.

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-24 Thread John Roesler
in favor of your KIP. Also, sorry for responding to the VOTE thread instead of DISCUSS. I'm +1 (binding) -John On Tue, May 24, 2022, at 09:10, John Roesler wrote: > Hi Sagar, > > Thanks for the KIP! > > I’m not at my computer right now, but I think I confronted a similar > pr

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-24 Thread John Roesler
Hi Sagar, Thanks for the KIP! I’m not at my computer right now, but I think I confronted a similar problem a while back for the Streams metrics. I think that I already made the “addMetric” method to be idempotent, so if it’s already registered, the call just returns the old one instead of

Re: [VOTE] KIP-833: Mark KRaft as Production Ready

2022-05-24 Thread John Roesler
+1 (binding) from me. Thanks, Colin! -John On Tue, May 24, 2022, at 01:58, David Jacot wrote: > +1. Thanks Colin! > > On Tue, May 24, 2022 at 4:50 AM Luke Chen wrote: >> >> +1 from me. >> Kraft is coming!!! >> >> Luke >> >> On Tue, May 24, 2022 at 7:26 AM Israel Ekpo wrote: >> >> > +1

Re: [HEADS-UP] Modification to KIP Template

2022-05-11 Thread John Roesler
+1 from me also. This is a great idea. Thanks, John On Wed, May 11, 2022, at 02:49, Luke Chen wrote: > Hi Mickael, > > +1 to add "test plan" section to KIP template. > Thanks for the improvement! > > Luke > > On Tue, May 10, 2022 at 11:21 PM Mickael Maison > wrote: > >> Hi, >> >> I did not see

Re: [DISCUSS] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-11 Thread John Roesler
>>>> someone using >> > >>>> the pause feature in such a way is because there is something going >> > >>> wrong, >> > >>>> or about >> > >>>> to go wrong, on that particular node. For example as

Re: [DISCUSS] Apache Kafka 3.3.0 Release

2022-05-10 Thread John Roesler
+1 from me! Thanks for volunteering, José. -John On Tue, May 10, 2022, at 17:53, José Armando García Sancio wrote: > Hi all, > > I would like to volunteer for the release of Apache Kafka 3.3.0. If > people agree, I'll start working on the release plan and update this > thread. > > Thanks, >

Re: [VOTE] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-10 Thread John Roesler
Thanks Jim, I’m +1 (binding) -John On Tue, May 10, 2022, at 14:05, Jim Hughes wrote: > Hi all, > > I'm asking for a vote on KIP-834: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211882832 > > Thanks in advance! > > Jim

Re: [VOTE] KIP-832 Allow creating a producer/consumer using a producer/consumer

2022-05-10 Thread John Roesler
+1 (binding) Thanks, François! -John On Tue, May 10, 2022, at 03:09, Bruno Cadonna wrote: > Hi Francois, > > Thanks for the KIP whose link is: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211882578 > > +1 (binding) > > Best, > Bruno > > On 09.05.22 23:23, François Rosière

Re: [DISCUSS] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-09 Thread John Roesler
n it > should be easy for future folks to modify things. > > Overall, I'd frame this KIP as "pause processing resulting in outputs". > > Cheers, > > Jim > > > >> On Mon, May 9, 2022 at 10:33 AM Guozhang Wang wrote: >> >> > I think for named t

Re: [DISCUSS] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-09 Thread John Roesler
ohn, > > Long emails are great; responding inline! > > On Sat, May 7, 2022 at 4:54 PM John Roesler wrote: > >> Thanks for the KIP, Jim! >> >> This conversation seems to highlight that the KIP needs to specify >> some of its behavior as well as its APIs, where the

Re: [DISCUSS] KIP-832 Allow creating a producer/consumer using a producer/consumer config

2022-05-07 Thread John Roesler
he changes to apply: >> >>- Create 2 new constructors in KafkaProducer >>- Create a new constructor in KafkaConsumer and increase de visibility >>of an existing one >>- Create a new constructor in TopologyTestDriver >> >> Kr, >> >>

Re: [DISCUSS] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-07 Thread John Roesler
Thanks for the KIP, Jim! This conversation seems to highlight that the KIP needs to specify some of its behavior as well as its APIs, where the behavior is observable and significant to users. For example: 1. Do you plan to have a guarantee that immediately after calling KafkaStreams.pause(),

Re: [DISCUSS] KIP-832 Allow creating a producer/consumer using a producer/consumer config

2022-05-06 Thread John Roesler
Thanks for the KIP, François! I'm generally in favor of your KIP, since you're proposing to follow the existing pattern of the constructors for both Producer and Consumer, but with the config object instead of Properties or Map configs. Also, because we already have this pattern in Streams, and

Re: [VOTE] KIP-821: Connect Transforms support for nested structures

2022-04-21 Thread John Roesler
Thanks for the KIP, Jorge! I’ve just looked over the KIP, and it looks good to me. I’m +1 (binding) Thanks, John On Thu, Apr 21, 2022, at 09:10, Chris Egerton wrote: > This is a worthwhile addition to the SMTs that ship out of the box with > Kafka Connect. +1 non-binding > > On Thu, Apr 21,

  1   2   3   4   5   6   7   8   9   10   >