Hi all, We have a web application that connects to a Spark cluster to trigger some calculation there. It also caches big amount of data in the Spark executors' cache.
To meet high availability requirements we need to run 2 instances of our web application on different hosts. Doing this straightforward will mean that the second application fires another set of executors that will initialize their own huge cache totally identical to that for the first application. Ideally we would like to reuse the cache in Spark for the needs of all instances of our applications. I am aware of the possibility to use Tachyon to externalize executors' cache. Currently exploring other options. Is there any way to allow several instances of the same application to connect to the same set of Spark executors? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Several-applications-share-the-same-Spark-executors-or-their-cache-tp21031.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
