Andrew, Thanks for the link. My current use case need to store data in centralized key/value store, that can be updated by one Processor and read by couple of different processors in the cluster. Data speed is low (tracking database schema changes) , so concurrently is not big concern. My use case is explained little bit here: https://github.com/zendesk/maxwell/pull/344 <https://github.com/zendesk/maxwell/pull/344>
+---------+------------+ | Table | Version | +---------+------------+ | Book | v1 | +---------+------------+ | Author | v2 | +---------+------------+ | Sales | v1 | +---------+------------+ | Cart | v1 | +---------+------------+ One processor update schema version number (increment) whenever new schema change event arrived to it. Other processors start writing new DML events to a sub folder, based on schema version number updated by first processes . Thanks Sumanth > On Jul 11, 2016, at 10:33 AM, Andrew Grande <apere...@gmail.com> wrote: > > Sumo, > > Something lightweight I devised here, backed by a simple in mem concurrent > map, for cases when a distributed map is too much > https://github.com/aperepel/nifi-csv-bundle/tree/master/nifi-csv-processors/src/main/java/org/apache/nifi/processors/lookup > > In a cluster, though, it's the true distributed replicated cache that one > must explicitly design for and use. While state framework is ok, the > default implementation backed by a ZK is not meant for high speed > concurrent use. Distributed caches are, however. > > Andrew > > On Sun, Jul 10, 2016, 10:58 PM Sumanth Chinthagunta <xmlk...@gmail.com> > wrote: > >> Thanks Bryan. >> Would be nice if we get support for state sharing across diffrent >> processors in the future. >> -Sumo >> >> Sent from my iPhone >> >>> On Jul 10, 2016, at 7:39 PM, Bryan Bende <bbe...@gmail.com> wrote: >>> >>> Sumo, >>> >>> Two different processor instances (different UUIDs) can not share state >>> that is stored through the state manager. For something like this you >> would >>> likely use the distributed map cache. >>> >>> As Andrew mentioned, the state is accessible across the cluster, so a >>> given processor can access the state from any node because the processor >>> will have the same UUID on each node. >>> >>> -Bryan >>> >>>> On Sunday, July 10, 2016, Andrew Grande <apere...@gmail.com> wrote: >>>> >>>> Sumo, >>>> >>>> IIRC there's a node one selects when setting state. If you invoke with a >>>> cluster mode, the state will be set to a ZK by default. Otherwise just >>>> local to this processor node. >>>> >>>> Andrew >>>> >>>> On Sun, Jul 10, 2016, 10:17 PM Sumanth Chinthagunta <xmlk...@gmail.com >>>> <javascript:;>> >>>> wrote: >>>> >>>>> If I set state from one ExecuteScript processor via stateManager , can >> I >>>>> access same state from other processor ? >>>>> Thanks >>>>> Sumo >>>>> >>>>> Sent from my iPhone >>> >>> >>> -- >>> Sent from Gmail Mobile >>
