Re: [DISCUSS] KIP-385: Provide configuration allowing consumer to no throw away prefetched data

2018-10-31 Thread Zahari Dichev
; > > > Hi Colin, Zahari, > > > > Wanted to check if you can review the patch and let me know, if we need > to > > make any changes? > > > > Thanks, > > > > Mayuresh > > > > On Fri, Oct 26, 2018 at 1:41 PM Zahari Dichev

Re: [DISCUSS] KIP-385: Provide configuration allowing consumer to no throw away prefetched data

2018-10-26 Thread Zahari Dichev
k you did an excellent job raising this issue and discussing it. > It's a very good contribution to the project even if you don't end up > writing the patch yourself. I'm going to take a look at the patch today. > If you want to take a look, that would also be good. > > best, > Colin >

Re: [DISCUSS] KIP-385: Provide configuration allowing consumer to no throw away prefetched data

2018-10-25 Thread Zahari Dichev
umed, or we unsubscribe from the > > partition. This would avoid adding a new configuration. > > > > best, > > Colin > > > > > > On Sun, Oct 21, 2018, at 11:54, Zahari Dichev wrote: > > > Hi there, although it has been discussed briefly already in

Requesting wiki permissions

2018-10-24 Thread Zahari Dichev
Hi there Can I get permissions for WIKI id: zaharidichev Zahari

Re: Throwing away prefetched records optimisation.

2018-10-24 Thread Zahari Dichev
: > On Tue, Oct 23, 2018, at 12:38, Zahari Dichev wrote: > > Hi there Matthias, I looked through the code of Kafka Streams. Quite > > impressive work ! If I have to put the logic of buffering within the > > context of what we are doing in Akka though, I might end up with the &g

Re: Throwing away prefetched records optimisation.

2018-10-23 Thread Zahari Dichev
le. > > Ismael > > On 16 Oct 2018 12:30 am, "Zahari Dichev" wrote: > > Hi there Kafka developers, > > I am currently trying to find a solution to an issue that has been > manifesting itself in the Akka streams implementation of the Kafka > connector. When

Re: Throwing away prefetched records optimisation.

2018-10-21 Thread Zahari Dichev
nal buffer of the consumer, before you call `pause()`. > > Curious to to learn why this would not work? How does the pipelining of > fetch requests works in detail? > > > -Matthias > > > On 10/19/18 1:36 PM, Zahari Dichev wrote: > > Hi there Matthias, > &g

[DISCUSS] KIP-385: Provide configuration allowing consumer to no throw away prefetched data

2018-10-21 Thread Zahari Dichev
e than welcome. Zahari Dichev

[jira] [Created] (KAFKA-7526) Allow for not throwing away prefetched data of paused partitions

2018-10-21 Thread Zahari Dichev (JIRA)
Zahari Dichev created KAFKA-7526: Summary: Allow for not throwing away prefetched data of paused partitions Key: KAFKA-7526 URL: https://issues.apache.org/jira/browse/KAFKA-7526 Project: Kafka

Re: Throwing away prefetched records optimisation.

2018-10-19 Thread Zahari Dichev
; > -Matthias > > On 10/18/18 9:27 AM, Jan Filipiak wrote: > > The idea for you would be that Messagechooser could hang on to the > > prefetched messages. > > > > ccing cmcc...@apache.org > > > > @Collin > > just for you to see that MessageChooser is a powerfull a

Re: Throwing away prefetched records optimisation.

2018-10-18 Thread Zahari Dichev
. I suspect > that pause/resume was not intended to be called at high frequency like > that, but I agree with you that the current behavior is needlessly > inefficient. I like your idea of making it configurable. > > Ryanne > > On Thu, Oct 18, 2018, 6:59 AM Zahari Dichev > wrot

Re: Throwing away prefetched records optimisation.

2018-10-18 Thread Zahari Dichev
t sure how to explain the situation in a better way.. Zahari On Thu, Oct 18, 2018 at 9:46 AM Zahari Dichev wrote: > Thanks a lot Jan, > > I will read it. > > Zahari > > On Thu, Oct 18, 2018 at 9:31 AM Jan Filipiak > wrote: > >> especially my suggestions ;) &g

Re: Throwing away prefetched records optimisation.

2018-10-18 Thread Zahari Dichev
; > I think it has suggestions that could be interesting for you > > > > Best Jan > > > > On 16.10.2018 09:29, Zahari Dichev wrote: > >> Hi there Kafka developers, > >> > >> I am currently trying to find a solution to an issue that has been &g

Re: Throwing away prefetched records optimisation.

2018-10-17 Thread Zahari Dichev
I'd suggest they not do > that, and then this problem goes away. > > Ryanne > > On Wed, Oct 17, 2018 at 7:35 AM Zahari Dichev > wrote: > > > Hi there, > > > > Are there any opinions on the matter described in my previous email? I > > think this is quite

Re: Throwing away prefetched records optimisation.

2018-10-17 Thread Zahari Dichev
:29 AM Zahari Dichev wrote: > Hi there Kafka developers, > > I am currently trying to find a solution to an issue that has been > manifesting itself in the Akka streams implementation of the Kafka > connector. When it comes to consuming messages, the implementation relies > h

Throwing away prefetched records optimisation.

2018-10-16 Thread Zahari Dichev
esent) 2. Introduce a facility to pass in a buffer into which the prefetched records are drained when poll is called and paused partitions have some prefetched records. Any opinions on the matter are welcome. Thanks a lot ! Zahari Dichev