Thank you for suggestions. I have 5 node standalone ignite cluster and the main goal is to load data into it and store it for long time for future use. I can't keep Spark workers in memory and assume what my data is ending up in cache distributed inside 5 standalone ignite nodes.
Spark process is starting 5 additional ignite servers ( I can see it from topology snapshot) With standalone=true or standalone=false same issue is happening when I'm running my spark application using YARN. However I found what if Spark is working in local, non-distributed mode (e.g. spark.master=local[2]) records are not getting lost. So it looks like the issue is when spark workers, processing JavaIgniteRDD, are going down. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
