Re: Is it possible to use windows service to start and stop spark standalone cluster
You might also want to see if TaskScheduler helps with that. I have not used it with Windows 2008 R2 but it generally does allow you to schedule a bat file to run on startup On Wed, Mar 11, 2015 at 10:16 AM, Wang, Ningjun (LNG-NPV) ningjun.w...@lexisnexis.com wrote: Thanks for the suggestion. I will try that. Ningjun From: Silvio Fiorito [mailto:silvio.fior...@granturing.com] Sent: Wednesday, March 11, 2015 12:40 AM To: Wang, Ningjun (LNG-NPV); user@spark.apache.org Subject: Re: Is it possible to use windows service to start and stop spark standalone cluster Have you tried Apache Daemon? http://commons.apache.org/proper/commons-daemon/procrun.html From: Wang, Ningjun (LNG-NPV) Date: Tuesday, March 10, 2015 at 11:47 PM To: user@spark.apache.org Subject: Is it possible to use windows service to start and stop spark standalone cluster We are using spark stand alone cluster on Windows 2008 R2. I can start spark clusters by open an command prompt and run the following bin\spark-class.cmd org.apache.spark.deploy.master.Master bin\spark-class.cmd org.apache.spark.deploy.worker.Worker spark:// mywin.mydomain.com:7077 I can stop spark cluster by pressing Ctril-C. The problem is that if the machine is reboot, I have to manually start the spark cluster again as above. Is it possible to use a windows service to start cluster? This way when the machine is reboot, the windows service will automatically restart spark cluster. How to stop spark cluster using windows service is also a challenge. Please advise. Thanks Ningjun
RE: Is it possible to use windows service to start and stop spark standalone cluster
Thanks for the suggestion. I will try that. Ningjun From: Silvio Fiorito [mailto:silvio.fior...@granturing.com] Sent: Wednesday, March 11, 2015 12:40 AM To: Wang, Ningjun (LNG-NPV); user@spark.apache.org Subject: Re: Is it possible to use windows service to start and stop spark standalone cluster Have you tried Apache Daemon? http://commons.apache.org/proper/commons-daemon/procrun.html From: Wang, Ningjun (LNG-NPV) Date: Tuesday, March 10, 2015 at 11:47 PM To: user@spark.apache.orgmailto:user@spark.apache.org Subject: Is it possible to use windows service to start and stop spark standalone cluster We are using spark stand alone cluster on Windows 2008 R2. I can start spark clusters by open an command prompt and run the following bin\spark-class.cmd org.apache.spark.deploy.master.Master bin\spark-class.cmd org.apache.spark.deploy.worker.Worker spark://mywin.mydomain.com:7077 I can stop spark cluster by pressing Ctril-C. The problem is that if the machine is reboot, I have to manually start the spark cluster again as above. Is it possible to use a windows service to start cluster? This way when the machine is reboot, the windows service will automatically restart spark cluster. How to stop spark cluster using windows service is also a challenge. Please advise. Thanks Ningjun
Re: Is it possible to use windows service to start and stop spark standalone cluster
Have you tried Apache Daemon? http://commons.apache.org/proper/commons-daemon/procrun.html From: Wang, Ningjun (LNG-NPV) Date: Tuesday, March 10, 2015 at 11:47 PM To: user@spark.apache.orgmailto:user@spark.apache.org Subject: Is it possible to use windows service to start and stop spark standalone cluster We are using spark stand alone cluster on Windows 2008 R2. I can start spark clusters by open an command prompt and run the following bin\spark-class.cmd org.apache.spark.deploy.master.Master bin\spark-class.cmd org.apache.spark.deploy.worker.Worker spark://mywin.mydomain.com:7077 I can stop spark cluster by pressing Ctril-C. The problem is that if the machine is reboot, I have to manually start the spark cluster again as above. Is it possible to use a windows service to start cluster? This way when the machine is reboot, the windows service will automatically restart spark cluster. How to stop spark cluster using windows service is also a challenge. Please advise. Thanks Ningjun
