Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

Hi, thanks for your mail, felt like this can clarify some things! The thread unfortunately split but as all branches close in on what my suggestion was about Ill pick this to continue Of course only the table the user wants to query would be materialized. (retrieving the queryhandle implies

"End of Batch" event

Hi, I'm pretty new to Kafka Streams. I am using Kafka Streams to ingest large csv file. I need to run some clean-up code after all records in the file are processed. Is there a way to send "End of Batch" event that is guaranteed to be processed after all records? If not is there alternative

Re: possible bug or inconsistency in kafka-clients

that sounds like it. thanks! On Sat, Jan 28, 2017 at 12:49 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Could this be the same issue as the one reported here? > https://issues.apache.org/jira/browse/KAFKA-4547 > > --Vahid > > > > > > From: Koert Kuipers > To:

Re: possible bug or inconsistency in kafka-clients

Could this be the same issue as the one reported here? https://issues.apache.org/jira/browse/KAFKA-4547 --Vahid From: Koert Kuipers To: users@kafka.apache.org Date: 01/27/2017 09:34 PM Subject:possible bug or inconsistency in kafka-clients hello all,

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

Retrieving the Queryable Store from the KTable is a possibility, i.e., you could have something like: QueryableStore store = ktable.queryableStore(); Where QueryableStore (i'm making these names up and not suggesting we use them) is just a proxy to the underlying stores. This would be useful for

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

Hi Gwen, Jan, So in addition to behaving as a hint, providing a state store name (in materialize or overloaded) allows the user to subsequently query by using that name. KTables have internal names, largely opaque to the users. When doing distributed queries with IQ, a state store name is

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

Hi Jan, I understand your concern. One implication of not passing any store name and just getting an IQ handle is that all KTables would need to be materialised. Currently the store name (or proposed .materialize() call) act as hints on whether to materialise the KTable or not. Materialising