le and if they attempted to reuse the Count for
some reason it would throw an error message saying that a store named
“my-store” already exists.
Thanks,
Kyle
From: Damian Guy
Sent: Thursday, June 29, 2017 3:59 AM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] Streams DSL/StateStore Refactoring
Hi
I like more of a builder pattern even though others have voiced against it. The
reason I like it is because it makes it clear to the user that a call to
KGroupedStream#count will return a KTable not some intermediate class that I
need to undetstand.
When trying to program in the fluent API that
especially committers to also take a look at
> > this
> > > proposal and vote.
> > >
> > >
> > > Guozhang
> > >
> > >
> > > On Wed, Jun 7, 2017 at 6:37 PM, Kyle Winkelman <
> winkelman.k...@gmail.com
> > >
> > > wrote:
>
be able to initialize the cogroup
> >
> > On Wed, Jun 14, 2017 at 3:44 PM Kyle Winkelman >
> > wrote:
> >
> > > I will update the kip to have only the aggregator in the first cogroup
> > call
> > > and the initializer and serde in the ag
his. Thus, it might make sense to keep this in mind (or even delay
this
>> > KIP?). It seems a waste of time to discuss all this if we are going to
>> > chance it in 2 month anyway... Just saying.
>> >
>> >
>> > -Matthias
>> >
>> > On 6/13/17
;> > builder-like pattern. This might make the whole discussion void if we
> do
> >> > this. Thus, it might make sense to keep this in mind (or even delay
> this
> >> > KIP?). It seems a waste of time to discuss all this if we are going to
> >> > chance it in 2 month any
. This might make the whole discussion void if we
> do
> > > this. Thus, it might make sense to keep this in mind (or even delay
> this
> > > KIP?). It seems a waste of time to discuss all this if we are going to
> > > chance it in 2 month anyway... Just saying.
> &g
er and serde to
>>>
>>> the
>>>
>>> final aggregate statement, since the serde only applies to the state
>>> store,
>>> and the initializer doesn't bear any relation to the first group in
>>> particular. It would end up looking like
>> .cogroup(grouped3, aggregator3)
> > >> .aggregate(initializer1, aggValueSerde, storeName1);
> > >>
> > >> This doesn't have to be a blocker, but I thought it would make the API
> > >> just
> > >> a tad
r[
> > >> >
> > >> > cart:{Item[no:01]},
> > >> > purchases:{Item[no:07],Item[no:08]},
> > >> >
> > >> > wishList:{}
> > >> > ]
>
gate class is.
>
> Cheers,
> Damian
>
> On Wed, 31 May 2017 at 20:09 Kyle Winkelman
> wrote:
>
> > Hello all,
> >
> > I have spent some more time on this and the best alternative I have come
> up
> > with is:
> > KGroupedStream has a single cogroup
makes more sense to me but doesnt allow the user to
specify different initializers for different tables.
Thanks,
Kyle
On May 24, 2017 7:46 PM, "Kyle Winkelman" wrote:
> Yea I really like that idea I'll see what I can do to update the kip and
> my pr when I have some t
able3 = stream.aggregate(/*per-day window*/)
>
> while underlying we are only using a single store "state-store-name" for
> it.
>
>
> Although this feature is out of the scope of this KIP, I'd like to discuss
> if we can "leave the door open" to make suc
te:
> Thanks for the proposal Kyle, this is a quite common use case to support
> such multi-way table join (i.e. N source tables with N aggregate func)
with
> a single store and N+1 serdes, I have seen lots of people using the
> low-level PAPI to achieve this goal.
>
>
> On Fri
ould also
> be applicable to a co-grouped stream.
>
> Overall I still think this is a really useful addition, but I feel we
> haven't spend much time trying to explore alternative DSLs that could maybe
> generalize better or match our existing syntax more closely.
>
>
Hello all,
I would like to start the vote on KIP-150.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-150+-+Kafka-Streams+Cogroup
Thanks,
Kyle
e output potentially have more partial results?
I.e, for each input on any stream you'd potentially see a record produced?
I think it is worth mentioning this.
Thanks,
Damian
On Tue, 16 May 2017 at 12:26 Kyle Winkelman
wrote:
> No problem, I just wanted to make sure people still had more t
nfluence/display/KAFKA/Release+Plan+0.11.0.0 <
> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+0.11.0.0>)
> so people are busier than usual. Stay tuned.
>
> Eno
> > On 15 May 2017, at 13:25, Kyle Winkelman
> wrote:
> >
> > Damian Guy, could you l
Damian Guy, could you let me know if you plan to review this further? There
is no rush, but if you dont have any additional comments I could start the
voting and finish my WIP PR.
Thanks,
Kyle
On May 9, 2017 11:07 AM, "Kyle Winkelman" wrote:
> Eno, is there anyone else that is an
[
https://issues.apache.org/jira/browse/KAFKA-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003248#comment-16003248
]
Kyle Winkelman commented on KAFKA-5206:
---
I have already made a PR but origin
Kyle Winkelman created KAFKA-5206:
-
Summary: RocksDBSessionStore doesn't use default aggSerde.
Key: KAFKA-5206
URL: https://issues.apache.org/jira/browse/KAFKA-5206
Project: Kafka
Issue
[
https://issues.apache.org/jira/browse/KAFKA-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003245#comment-16003245
]
Kyle Winkelman commented on KAFKA-5205:
---
I have already made a PR but origin
Kyle Winkelman created KAFKA-5205:
-
Summary: CachingSessionStore doesn't use the default keySerde.
Key: KAFKA-5205
URL: https://issues.apache.org/jira/browse/KAFKA-5205
Project: Kafka
ks Kyle, good arguments.
>
> Eno
>
> > On May 7, 2017, at 5:06 PM, Kyle Winkelman
> wrote:
> >
> > *- minor: could you add an exact example (similar to what Jay’s example
> is,
> > or like your Spark/Pig pointers had) to make this super concrete?*
hing) and perhaps we
> > could do the same thing but I'm not sure we know the partitioning so we
> may
> > need an explicit cogroup command that impllies they are already
> > co-partitioned.
> >
> > -Jay
> >
> > On Fri, May 5, 2017 at 5:56 AM
forwards the result.
> I need to look into it and think about it a bit more, but it seems like it
> could be a useful optimization.
>
> On Thu, 4 May 2017 at 23:21 Kyle Winkelman
> wrote:
>
> > I sure can. I have added the following description to my KIP. If this
>
f you propose, and it would help to
> understand it better.
>
> Thanks a lot!
>
>
> -Matthias
>
>
>
> On 5/4/17 11:39 AM, Kyle Winkelman wrote:
> > I have made a pull request. It can be found here.
> >
> > https://github.com/apache/kafka/pull/2975
>
some time to look into it tomorrow. For
> the
> > implementation, can you raise a PR against kafka trunk and mark it as
> WIP?
> > It will be easier to review what you have done.
> >
> > Thanks,
> > Damian
> >
> > On Thu, 4 May 2017 at 11:50 Kyle Wink
I am replying to this in hopes it will draw some attention to my KIP as I
haven't heard from anyone in a couple days. This is my first KIP and my
first large contribution to the project so I'm sure I did something wrong.
;)
On May 1, 2017 4:18 PM, "Kyle Winkelman" wrote:
>
Kyle Winkelman created KAFKA-5172:
-
Summary: CachingSessionStore doesn't fetchPrevious correctly.
Key: KAFKA-5172
URL: https://issues.apache.org/jira/browse/KAFKA-5172
Project: Kafka
Thanks,
Kyle Winkelman
Hello,
I was hoping to get the ability to edit the wiki so I can post a KIP.
Username: winkelman.kyle
Thanks,
Kyle
/cwiki.apache.org/confluence/display/KAFKA/
> Kafka+Improvement+Proposals. Could you see if you can add this there?
>
> Many thanks
> Eno
>
> On Wed, Apr 26, 2017 at 6:00 PM, Kyle Winkelman
> wrote:
>
>> Thank you for your reply.
>>
>> I have attached my f
he sink.
> Meanwhile another record can come in. So multiple records can be in the
> topology at the same time.
>
> Thanks
> Eno
>
>
>
>
>
> On Fri, Apr 14, 2017 at 8:16 PM, Kyle Winkelman
> wrote:
>
>> Eno,
>> Thanks for the response. The figur
Eno,
Thanks for the response. The figure was just a restatement of my questions.
I have made an attempt at a low level processor and it appears to work but
it isn't very pretty and was hoping for something at the streams api level.
I have written some code to show an example of how I see the Cogro
35 matches
Mail list logo
