Hi Frederik - thanks for the KIP, this will be a fantastic and elegant
addition to Kafka Streams.
I have a higher level question about this, which is that the `poll`
interface returns multiple records and yet the DeserializationException
will be thrown if any record in the batch cannot be
+1 Not binding :)
On Mon, Mar 25, 2024 at 11:11 AM Walker Carlson
wrote:
> Hello everybody,
>
> I think we have had some pretty good discussion on this kip and it seems
> that we are close if not yet settled on the final version.
>
> So I would like to open up the voting for KIP-1024:
>
deferring this to
> a separate KIP.
>
> Best,
> Bruno
>
> On 3/26/24 12:49 AM, Almog Gavra wrote:
> > Hello Folk!
> >
> > Glad to see improvements to the GlobalKTables in discussion! I think they
> > deserve more love :)
> >
> > Scope creep alert
Hello Folk!
Glad to see improvements to the GlobalKTables in discussion! I think they
deserve more love :)
Scope creep alert (which I'm generally against and certainly still support
this KIP without but I want to see if there's an elegant way to address
both problems). The KIP mentions that "Now
Hello Stan,
I wanted to give you a heads up that
https://github.com/apache/kafka/pull/15073 (
https://issues.apache.org/jira/browse/KAFKA-16046) was identified as a
blocker regression and should be merged to trunk by EOD.
Cheers,
Almog
On Tue, Jan 2, 2024 at 4:20 AM Stanislav Kozlovski
wrote:
[
https://issues.apache.org/jira/browse/KAFKA-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Almog Gavra reopened KAFKA-16046:
-
> Stream Stream Joins fail after restoration with deserialization excepti
[
https://issues.apache.org/jira/browse/KAFKA-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Almog Gavra resolved KAFKA-16046.
-
Resolution: Fixed
> Stream Stream Joins fail after restoration with deserialization excepti
Almog Gavra wrote:
> Sorry for the late response to the late reply, hah! I didn't give any
> thought about how we would want to integrate this custom store supplier
> with querying of the stores. My initial intuition suggests that we'd
> probably want a separate API for that, or ju
Almog Gavra created KAFKA-16046:
---
Summary: Stream Stream Joins fail after restoration with
deserialization exceptions
Key: KAFKA-16046
URL: https://issues.apache.org/jira/browse/KAFKA-16046
Project
Almog Gavra created KAFKA-16038:
---
Summary: Periodic Logging fo Current Assignment
Key: KAFKA-16038
URL: https://issues.apache.org/jira/browse/KAFKA-16038
Project: Kafka
Issue Type: Improvement
e
> store instance directly.
>
>
> Guozhang
>
>
> On Sun, Nov 19, 2023 at 5:26 PM Almog Gavra wrote:
> >
> > Hello Guozhang,
> >
> > Thanks for the feedback! For 1 there are tests verifying this and I did
> so
> > manually as well, it does not rev
[
https://issues.apache.org/jira/browse/KAFKA-15774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Almog Gavra resolved KAFKA-15774.
-
Fix Version/s: 3.7.0
Resolution: Fixed
Note that we decided not to have
[
https://issues.apache.org/jira/browse/KAFKA-15215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Almog Gavra resolved KAFKA-15215.
-
Fix Version/s: 3.7.0
Resolution: Fixed
> The default.dsl.store config is not compati
uler which is developed by a different
> group of people rather than the Streams app developers themselves,
> that can only turn on certain features after learning the actual store
> impl suppliers used?
>
> Guozhang
>
> On Sat, Nov 18, 2023 at 2:46 PM Almog Gavra wrote:
>
have any objection to the goal; contrary! It's very annoying
> what we have right now, and if we can improve it, I am totally in favor
> of it.
>
>
> -Matthias
>
> On 11/3/23 8:47 AM, Almog Gavra wrote:
> > Good question :) I have a PR for it already here:
> >
Hello Rohan,
Thanks for the KIP! Thoughts below (seems I have similar comments to
Guozhang, but I had already written this before reading his reply haha!).
They're basically all minor suggestions for improvements, I wouldn't
consider any of them blocking.
1. For the API, thoughts on changing the
> When we call `StreasmBuilder.build()` it will give us a already fully
> wired `Topology`, including all store suppliers needed. I don't see a
> clean way how we could change the store supplier after the fact?
>
>
> -Matthias
>
> On 11/2/23 5:11 PM, Almog Gavra wrote:
> &g
Gavra wrote:
> OK! I think I got everything, but I'll give the KIP another read with
> fresh eyes later. Just a reminder that the voting is open, so go out and
> exercise your civic duty! ;)
>
> - Almog
>
> On Fri, Jul 28, 2023 at 10:38 AM Almog Gavra
> wrote:
>
Almog Gavra created KAFKA-15774:
---
Summary: Respect default.dsl.store Configuration Without Passing
it to StreamsBuilder
Key: KAFKA-15774
URL: https://issues.apache.org/jira/browse/KAFKA-15774
Project
+1 (non-binding) - great improvement, thanks Colt & Eduwer!
On Tue, Oct 17, 2023 at 11:25 AM Guozhang Wang
wrote:
> +1 from me.
>
> On Mon, Oct 16, 2023 at 1:56 AM Lucas Brutschy
> wrote:
> >
> > Hi,
> >
> > thanks again for the KIP!
> >
> > +1 (binding)
> >
> > Cheers,
> > Lucas
> >
> >
> >
>
ss over the updated wiki and have no more
> questions. +1
> >>
> >> Guozhang
> >>
> >> On Wed, Jul 26, 2023 at 8:14 AM Almog Gavra
> wrote:
> >>>
> >>> Hello Everyone,
> >>>
> >>> Opening the voting for K
OK! I think I got everything, but I'll give the KIP another read with fresh
eyes later. Just a reminder that the voting is open, so go out and exercise
your civic duty! ;)
- Almog
On Fri, Jul 28, 2023 at 10:38 AM Almog Gavra wrote:
> Thanks Guozhang & Sophie:
>
> A2. Will clari
in" at first, but I will admit
> it's
> > > > > grown on me. I think it rubbed me the wrong
> > > > > way at first because it's just not part of the standard
> vocabulary in
> > > > > Streams so far. If we envision extending
> > > > > this
Hello Everyone,
Opening the voting for KIP-954. The discussion is converging, but please
feel free to chime in on the last few conversation points if you aren't
happy with where it settled.
zed API" especially in the code section "Stores.java".
>
> A6: Actually I am not sure if I completely follow here. Is this about
> the static methods in class Stores? If yes, I agree with Almog to keep
> this out of the KIP.
>
> Best,
> Bruno
>
>
will open the voting tomorrow! Thanks again everyone for the discussion.
Cheers,
Almog
On Tue, Jul 25, 2023 at 9:20 AM Almog Gavra wrote:
> Glad you like my KIP-secretary skills ;)
>
> A2. I'm definitely happy to take your suggestion here and not do anything
> special w.r.t. Versioned sto
gt; sense in the context of that feature's future plans.
>
> A3: sounds reasonable to me
>
> A5: Also sounds fine to me, though I'll let others chime in with/if they
> have an alternative suggestion/preference. I guess the other contender
> would be something like DSLStoreImpl o
gt;> assume
> >> > > there was no timestamped option for IM stores and that this feature
> >> was
> >> > > incomplete, if they weren't acutely aware of the internal
> >> implementation
> >> > > details (namely that it's only required for
ore implementations) when all
> the
> >> RocksDB stuff is technically internal.
> >>
> >> Basically, I'm suggesting two things: first, call out in some way
> (perhaps
> >> the StoreTypeSpec javadocs) that each StoreTypeSpec is considered a
> public
> >> co
Hi All,
I would like to propose a KIP to expand support for default store types
(KIP-591) to encompass custom store implementations:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-954%3A+expand+default+DSL+store+configuration+to+custom+types
Looking forward to your feedback!
Cheers,
>
> Of course, for this KIP's scope itself, we can just do it for one specific
> rule of "source-changelog". I'm just wondering about the general framework
> for extending the current configuration here.
>
> Guozhang
>
>
> On Wed, Jan 13, 2021 at 10:42 AM
Hello,
I would like to start the discussion thread for KIP-705:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-705%3A+Selectively+Disable+Topology+Optimizations
The KIP is proposing an additional streams configuration that allows to
selectively disable optimizations under the
Almog Gavra created KAFKA-12192:
---
Summary: Add Configuration to Selectively Disable Topology
Optimizations
Key: KAFKA-12192
URL: https://issues.apache.org/jira/browse/KAFKA-12192
Project: Kafka
Another thanks from me! +1 (non-binding)
On Tue, Jan 21, 2020 at 11:04 AM Randall Hauch wrote:
> Thanks again for the KIP and this improvement for Connect.
>
> +1 (binding)
>
> Randall
>
> On Tue, Jan 21, 2020 at 10:45 AM Tom Bentley wrote:
>
> > +1 (non-binding). Thanks for the KIP
t; >> > > > > >
> > >> > > > > > 3. I believe across the whole KIP, when I'm referring to the
> > >> task
> > >> > > > > entity, I
> > >> > > > > > imply the worker task. Not the user code that is running as
> > &g
Hi Konstantine,
Thanks for the KIP! This is going to make automatic integration with
Connect much more powerful.
My thoughts are mostly around freshness of the data and being able to
expose that to users. Riffing on Randall's timestamp question - have we
considered adding some interval at which
Thanks Cyrus! I think this change is a good step in hardening the API. I do
believe that APIs should be defined by functionality and not performance
characteristics, so I'd prefer using List<> over ArrayList<> (the
alternative you mention in rejected). That also gives us leeway in the
future to
; > >> +1 (binding)
> > > >>
> > > >> Randall
> > > >>
> > > >> On Thu, Aug 15, 2019 at 11:59 AM Konstantine Karantasis <
> > > >> konstant...@confluent.io> w
tine Karantasis <
> > >> konstant...@confluent.io> wrote:
> > >>
> > >>> Thanks Almog!
> > >>> Nicely designed and concise KIP.
> > >>>
> > >>> +1 non-binding
> > >>>
> > >>> Konstanti
se the
> JsonConverter for key and/or value converters to use
> `decimal.format=NUMERIC`."
>
> Finally, should we add a short discussion in the Rejected Alternatives
> about the option of leaving JsonConverter untouched and creating a
> different converter implementation?
on.format=NUMERIC`via connector configurations or
> worker configs?
>
> Anyway, great job so far!
>
> Best regards,
>
> Randall
>
> On Thu, Aug 15, 2019 at 12:00 PM Konstantine Karantasis <
> konstant...@confluent.io> wrote:
>
> > Thanks! KIP
oducer~ consumer will be able to read
> binary as today" (again according to my suggestion above, it could be as
> the upgraded source converter ...)
>
> - "consumers cannot consumer NUMERIC data. " -> "consumers cannot read
> NUMERIC data&q
Good catches! Fixed :)
On Thu, Aug 8, 2019 at 10:36 PM Arjun Satish wrote:
> Cool!
>
> Couple of nits:
>
> - In public interfaces, typo: *json.decimal.serialization.fromat*
> - In public interfaces, you use the term "HEX" instead of "BASE64".
>
>
&g
EDIT: everywhere I've been using "HEX" I meant to be using "BASE64". I will
update the KIP to reflect this.
On Wed, Aug 7, 2019 at 9:44 AM Almog Gavra wrote:
> Thanks for the feedback Arjun! I'm happy changing the default config to
> HEX instead of BINARY, no stron
ad of “BINARY”?
>
> Should we call out the fact that JS systems might be susceptible to double
> precision round offs with the new numeric format? here are some people
> discussing a similar problem
> https://github.com/EventStore/EventStore/issues/1541
>
> On Tue, Aug 6,
Hello Everyone,
After discussions on
https://lists.apache.org/thread.html/fa665a6dc59f73ca294a00bcbef2eaa3ad00cc69626e91c516fa4fca@%3Cdev.kafka.apache.org%3E
I've opened this KIP up for formal voting.
KIP:
IP to a
vote.
Almog
On Mon, Jul 29, 2019 at 9:29 AM Almog Gavra wrote:
> I'm mostly happy with your current suggestion (two configs, one for
> serialization and one for deserialization) and your implementation
> suggestion. One thing to note:
>
> > We should _always_ be able to
gree that discussions / plans around
> switching the defaults should not block this KIP.
>
> Andy
>
>
> On Thu, 25 Jul 2019 at 18:26, Almog Gavra wrote:
>
> > Thanks for the replies Andy and Andrew (2x Andy?)!
> >
> > > Is the text decimal a base16 enc
lt of decimal serialization
> to
> > numeric, and text serialization to base 10 in the next major release.
> > (With upgrade notes to match). Though I know this is more contentious, I
> > think it moves us forward in a much more standard way that the current
> > encodin
Hi Everyone!
Kicking off discussion for a new KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-481%3A+SerDe+Improvements+for+Connect+Decimal+type+in+JSON
For those who are interested, I have a prototype implementation that helped
guide my design: https://github.com/agavra/kafka/pull/1
Almog Gavra created KAFKA-8595:
--
Summary: Support SerDe of Decimals in JSON that are not HEX encoded
Key: KAFKA-8595
URL: https://issues.apache.org/jira/browse/KAFKA-8595
Project: Kafka
Issue
Hello everyone - the PR is out and ready to review!
https://github.com/apache/kafka/pull/6728/
On Fri, May 10, 2019 at 11:57 AM Almog Gavra wrote:
> Thanks everyone for the comments and discussion! Closing the voting out
> for this KIP:
>
> * 4 Binding (Randall, Manikumar, Colin,
Ryanne Dolan wrote:
> > > > +1 (non-binding) for the core feature, but I could take or leave the
> > > > builder.
> > > >
> > > > Ryanne
> > > >
> > > > On Fri, May 10, 2019 at 10:43 AM Almog Gavra
> > wrote:
>
ate KIP?
>
> best,
> Colin
>
>
> On Fri, May 10, 2019, at 09:00, Ryanne Dolan wrote:
> > +1 (non-binding) for the core feature, but I could take or leave the
> > builder.
> >
> > Ryanne
> >
> > On Fri, May 10, 2019 at 10:43 AM Almog Gavra wrote:
; check existing builders in `clients` if any exist for what they're doing.
>
> If we didn't add a builder, another option would be a single new
> constructor:
>
> public NewTopic(String name, Optional numPartitions,
> Optional replicationFactor)
>
> Ismael
>
> On Thu, May 9,
vs size?
>
> I am also wondering, what the cut-off line for important configs is,
> that get their own method, vs all other that are set via `config()`.
>
> It seems to be an "random" selection atm.
>
>
> -Matthias
>
> On 5/9/19 6:56 PM, Almog Gavra wrote:
ine, I'd keep it simple and just have the constructor
> > > with just the name parameter.
> > >
> > > Ismael
> > >
> > > On Thu, May 2, 2019 at 1:58 AM Mickael Maison <
> mickael.mai...@gmail.com>
> > > wrote:
> > >
> > >
relying upon the generic
> properties. But I'm fine if others think they should be removed. I'm also
> fine with not deprecating the Connect version of the builder.
>
> On Fri, May 3, 2019 at 11:27 AM Almog Gavra wrote:
>
> > Ack. KIP updated :) Perhaps instead of deprecating the
ss,
> should we ever want to do that (e.g, in Connect). If we make it protected
> or public, then subclassing is easier.
>
> On Thu, May 2, 2019 at 9:44 AM Almog Gavra wrote:
>
> > Thanks for the input Randall. I'm happy adding it natively to NewTopic
> > instead o
c =
>
> NewTopic.build(name).compacted().partitions(1).replicationFactor(3).build();
>
> This is a bit shorter, a bit easier to read (no "new New..."), and more
> discoverable since anyone looking at the NewTopic source or JavaDoc will
> maybe notice it.
>
&
else being set via builder methods.
> > >
> > > This would result in a better long term api as the number of options
> > > increases.
> > >
> > > Sent from my iPhone
> > >
> > > > On 30 Apr 2019, at 16:28, Almog Gavra wrote:
> >
nt from my iPhone
>
> > On 30 Apr 2019, at 16:28, Almog Gavra wrote:
> >
> > Hello Everyone,
> >
> > I'd like to start a discussion on KIP-464:
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-464%3A+Default+Replication+Factor+for+AdminClient%23crea
Hello Everyone!
Kicking off the voting for
https://cwiki.apache.org/confluence/display/KAFKA/KIP-464%3A+Defaults+for+AdminClient%23createTopic
You can see discussion thread here (please respond with suggestions on that
thread):
> them to pick a random number... (not sure there is any configuration that
> can get anyone to think, unfortunately).
>
> On Tue, Apr 30, 2019 at 10:22 AM Almog Gavra wrote:
>
> > I have a preference toward requiring specifying partitions per topic, but
> > I'm happy to
tition count broker default to be used (the one used for auto
> topic creation).
>
> Ismael
>
> On Tue, Apr 30, 2019, 8:39 AM Almog Gavra wrote:
>
> > Hello Everyone,
> >
> > I'd like to start a discussion on KIP-464:
> >
> >
> https://cwi
Hello Everyone,
I'd like to start a discussion on KIP-464:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-464%3A+Default+Replication+Factor+for+AdminClient%23createTopic
It's about allowing users of the AdminClient to supply only a partition
count and to use the default replication factor
Hello Kafka Devs,
I'm following the instructions on
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals,
which first requires me to request edit access on the wiki.
My JIRA Id is: *agavra*
Cheers,
Almog
Almog Gavra created KAFKA-8305:
--
Summary: AdminClient should support creating topics with
`default.replication.factor`
Key: KAFKA-8305
URL: https://issues.apache.org/jira/browse/KAFKA-8305
Project
68 matches
Mail list logo
