Thanks Ilya, I'll try mentioned argument. Arseny Kovalchuk
Senior Software Engineer at Synesis skype: arseny.kovalchuk mobile: +375 (29) 666-16-16 LinkedIn Profile <http://www.linkedin.com/in/arsenykovalchuk/en> On 29 January 2018 at 18:58, Ilya Kasnacheev <[email protected]> wrote: > Hello! > > I believe the root cause is this part of log: > > Caused by: java.nio.channels.ClosedByInterruptException > at > java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) > at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:746) > at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727) > at > org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.read(RandomAccessFileIO.java:62) > at > org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:322) > ... 60 more > > > Here, a thread tht was using Persistence file channel got interrupted and > it became dysfunctional after that. Then there's a multitude of problems > caused by this. > > This should not generally happen under normal load, however if you're > seeing such errors, consider setting the following Java argument for Ignite: > -DIGNITE_USE_ASYNC_FILE_IO_FACTORY=true > > Please try to re-run with this setting, see if you still have any issues > > Regards, > > -- > Ilya Kasnacheev > > 2018-01-29 11:06 GMT+03:00 Arseny Kovalchuk <[email protected]>: > >> Hi guys. >> >> We've got 2.3 with persistence enabled. 5 server nodes, 3 client query >> nodes, 1 writer client node (it gets events from Kafka and puts them to >> Ignite with key-val API). All client and server nodes are Pods in >> Kubernetes. >> >> Cluster's worked well for a wile and then has stuck with the error which >> sounded as >> >> Suppressed: org.apache.ignite.IgniteCheckedException: Failed to update >> keys on primary node. >> Caused by: java.lang.ClassCastException: org.apache.ignite.internal.pro >> cessors.cache.persistence.tree.io.DataPageIO cannot be cast to >> org.apache.ignite.internal.processors.cache.persistence.tree.io.BPlusIO >> >> At this stage the cluster doesn't accept data. Please take a look at >> attached logs. >> >> client-log.txt - log of the client, which tries to put data to cluster >> log-instance-N.txt - log of each server node. See ignite-instance-1.txt >> for the error on server node. >> >> Any known issues about that? >> >> Sincerely, >> >> >> Arseny Kovalchuk >> >> Senior Software Engineer at Synesis >> skype: arseny.kovalchuk >> mobile: +375 (29) 666-16-16 <+375%2029%20666-16-16> >> LinkedIn Profile <http://www.linkedin.com/in/arsenykovalchuk/en> >> > >
