Hi there, I would like to use "Ignite for Spark" to save the states of Spark jobs in memory and those states can be used for later jobs. For Shared Deployment, the document only offer two ways to deploy Ignite cluster. First is the standalone deployment, second is MESOS deployment. But Our Spark clusters are running on YARN. My question is: is it possible to run Ignite for Spark on YARN deployment???
I downloaded and installed Ignite on my machine. Next, I referenced the link below for YARN Deployment. http://apacheignite.gridgain.org/docs/yarn-deployment I created the cluster.properties file and ran the application using the command: hadoop jar /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar /u/hongmei/apache-ignite/config/cluster.properties Form the YARN console, The YARN ignite application works ok. It shows running, and 16 containers are allocated for the ignite application. After this step, what should I do in order to run Spark with Ignite on YARN deployment?? Many Thanks!!! Mei -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-for-Spark-on-YARN-Deployment-tp5465.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
