I'm afraid that's not what I'm looking for, as I'm just trying to retrieve the local data, from inside my application (but from outside the stream topology), and in some cases it becomes impossible. That is, the stream changes its state from "rebalancing" to "running", but the store is remains inaccessible.
I don't want to have access to state stores located on other instances. In pseudocode: streams.start(); <- here's where the stream starts loopOverLocalData(streams) <- periodical, asynchronous calls to streams.store, which, if the problem occurs, always result in exceptions being thrown. I expected to retrieve the local data this way. On 7 March 2018 at 16:20, Damian Guy <damian....@gmail.com> wrote: > If you have multiple streams instances then the store might only be > available on one of the instances. Using `KafkaStreams.store(..)` will only > locate stores that are currently accessible by that instance. If you need > to be able to locate stores on other instances, then you should probably > have a read of: > https://kafka.apache.org/10/documentation/streams/ > developer-guide/interactive-queries.html#querying-remote- > state-stores-for-the-entire-app > > There is also a convenient 3rd party lib that can help you with this: > https://github.com/lightbend/kafka-streams-query > > On Wed, 7 Mar 2018 at 14:07 detharon <detha...@gmail.com> wrote: > > > Hello, I'm experiencing issues accessing the state stores outside the > Kafka > > stream. My application queries the state store every n seconds using the > > .all() method to retrieve all key value pairs. I know that the state > store > > might not be available, so I guard against the InvalidStateStoreException > > and in case it's caught, I simply invoke the .store method on my stream, > in > > order to get a new store. The problem is, that for some reason the store > > never becomes available. > > > > Some facts: > > - During that time stream processing works correctly, and it successfully > > puts and gets data to and from the store. > > - Stream is in "running" state. I've started logging that because under > > normal circumstances this exception is being thrown when the stream is in > > "rebalancing" phase, but after a while it's gone, as expected, so I can > > distinguish between these two cases. > > - It only happens when I run my app in multiple instances. > > > > I've set the log level to debug, but I can't see anything suspicious in > the > > logs, but maybe there's something particular I should pay attention to? > > I access the store from inside an Akka actor, to which I pass the > reference > > to KafkaStreams object, if that matters. > > > > I ran out of ideas what might have caused that behavior, so any help will > > be greatly appreciated. > > >
