Thanks Andrei. If the external data source client sending batches of 2-3 MB say via TCP socket connection to a bunch of socket streamers (deployed as ignite services deployed to each ignite node) and say of the streamer nodes die, the data source client catching the exception, has to check the cache to see how much of the 2-4MB batch has been flushed to cache and resend the rest? Would setting streamer with overwrite set to true work, if the data source client resend the entire batch? A question regarding streamer with overwrite option set to true. How does the streamer compare the content the data in hand with the data in cache, if each record is being assigned UUID when being inserted to cache?
On Tue, Jan 7, 2020 at 4:40 AM Andrei Aleksandrov <aealexsand...@gmail.com> wrote: > Hi, > > Not flushed data in a data streamer will be lost. Data streamer works > thought some Ignite node and in case if this the node failed it can't > somehow start working with another one. So your application should think > about how to track that all data was loaded (wait for completion of > loading, catch the exceptions, check the cache sizes, etc) and use > another client for data loading in case if previous one was failed. > > BR, > Andrei > > 1/6/2020 2:37 AM, narges saleh пишет: > > Hi All, > > > > Another question regarding ignite's streamer. > > What happens to the data if the streamer node crashes before the > > buffer's content is flushed to the cache? Is the client responsible > > for making sure the data is persisted or ignite redirects the data to > > another node's streamer? > > > > thanks. >
