Well thats unfortunate, just means I have to scrape the webui for that information. As to why, I have a cluster that is being increased in size to accommodate the processing requirements of a large set of jobs. Its useful to know when the new workers have joined the spark cluster. In my specific case, I may be growing the cluster size by a hundred nodes and if I fail to wait for that initialization to complete the job will not have enough memory to run my jobs.
Aaron On Thu, Mar 24, 2016 at 3:07 AM, Takeshi Yamamuro <linguin....@gmail.com> wrote: > Hi, > > There is no way to get such information from your app. > Why do you need that? > > thanks, > maropu > > On Thu, Mar 24, 2016 at 8:23 AM, Ajaxx <ajack...@pobox.com> wrote: > >> I'm building some elasticity into my model and I'd like to know when my >> workers have come online. It appears at present that the API only >> supports >> getting information about applications. Is there a good way to determine >> how many workers are available? >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Best-way-to-determine-of-workers-tp26586.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> > > > -- > --- > Takeshi Yamamuro >
