For example it could be some "static" information, like a mapping from zip code to city name.
Something that does usually not change over time. -Matthias On 5/25/20 9:55 PM, Pushkar Deole wrote: > Matthias, > > I am wondering what you mean by "Global store hold "axially" data that is > provided from "outside" of the > app" > > will you be able to give some example use case here as to what you mean by > axially data provided from outside app? > > On Sat, May 2, 2020 at 1:58 AM Matthias J. Sax <mj...@apache.org> wrote: > >> Both stores sever a different purpose. >> >> Regular stores allow you to store state the application computes. >> Writing into the changelog is a fault-tolerance mechanism. >> >> Global store hold "axially" data that is provided from "outside" of the >> app. There is no changelog topic, but only the input topic (that is used >> to re-create the global state). >> >> Local stores are sharded and updates are "sync" as they don't need to be >> shared with anybody else. >> >> For global stores, as all instances need to be updated, updates are >> async (we don't know when which instance will update it's own global >> store replica). >> >>>> Say one stream thread updates the topic for global store and starts >>>> processing next event wherein the processor tries to read the global >> store >>>> which may not have been synced with the topic? >> >> Correct. There is no guarantee when the update to the global store will >> be applied. As said, global stores are not designed to hold data the >> application computes. >> >> >> -Matthias >> >> >> On 4/30/20 11:11 PM, Pushkar Deole wrote: >>> thanks... will try with GlobalKTable. >>> As a side question, I didn't really understand the significance of global >>> state store which kind of works in a reverse way to local state store >> i.e. >>> local state store is updated and then saved to changelog topic whereas in >>> case of global state store the topic is updated first and then synced to >>> global state store. Do these two work in sync i.e. the update to topic >> and >>> global state store ? >>> >>> Say one stream thread updates the topic for global store and starts >>> processing next event wherein the processor tries to read the global >> store >>> which may not have been synced with the topic? >>> >>> On Fri, May 1, 2020 at 3:35 AM Matthias J. Sax <mj...@apache.org> wrote: >>> >>>> Yes. >>>> >>>> A `GlobalKTable` uses a global store internally. >>>> >>>> You can also use `StreamsBuilder.addGlobalStore()` or >>>> `Topology.addGlobalStore()` to add a global store "manually". >>>> >>>> >>>> -Matthias >>>> >>>> >>>> On 4/30/20 7:42 AM, Pushkar Deole wrote: >>>>> Thanks Matthias. >>>>> Can you elaborate on the replicated caching layer part? >>>>> When you say global stores, do you mean GlobalKTable created from a >> topic >>>>> e.g. using StreamsBuilder.globalTable(String topic) method ? >>>>> >>>>> On Thu, Apr 30, 2020 at 12:44 PM Matthias J. Sax <mj...@apache.org> >>>> wrote: >>>>> >>>>>> It's not possible to modify state store from "outside". >>>>>> >>>>>> If you want to build a "replicated caching layer", you could use >> global >>>>>> stores and write into the corresponding topics to update all stores. >> Of >>>>>> course, those updates would be async. >>>>>> >>>>>> >>>>>> -Matthias >>>>>> >>>>>> On 4/29/20 10:52 PM, Pushkar Deole wrote: >>>>>>> Hi All, >>>>>>> >>>>>>> I am wondering if this is possible: i have been asked to use state >>>> stores >>>>>>> as a general replicated cache among multiple instances of service >>>>>> instances >>>>>>> however the state store is created through streambuilder but is not >>>>>>> actually modified through stream processor topology however it is to >> be >>>>>>> modified from outside the stream topology. So, essentially, the state >>>>>> store >>>>>>> is just to be created from streambuilder and then to be used as an >>>>>>> application level cache that will get replicated between application >>>>>>> instances. Is this possible using state stores? >>>>>>> >>>>>>> Secondly, if possible, is this a good design approach? >>>>>>> >>>>>>> Appreciate your response since I don't know the internals of state >>>>>> stores. >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >> >> >
signature.asc
Description: OpenPGP digital signature
