For the record, this is what I came up with (ignoring the configurable port
for now):
spark/sbin/start-master.sh
master_ui_response_code=0
while [ $master_ui_response_code -ne 200 ]; do
sleep 1
master_ui_response_code=$(
curl --head --silent --output /dev/null \
Oh, good point. So I guess I should be able to query the master via code
like this before any slaves are started.
On Sat, Apr 11, 2015 at 7:52 PM Ted Yu yuzhih...@gmail.com wrote:
From SparkUI.scala :
def getUIPort(conf: SparkConf): Int = {
conf.getInt(spark.ui.port,
Yeah from what I remember it was set defensively. I don't know of a good
way to check if the master is up though. I guess we could poll the Master
Web UI and see if we get a 200/ok response
Shivaram
On Fri, Apr 10, 2015 at 8:24 PM, Nicholas Chammas
nicholas.cham...@gmail.com wrote:
Check this
So basically, to tell if the master is ready to accept slaves, just poll
http://master-node:4040 for an HTTP 200 response?
On Sat, Apr 11, 2015 at 2:42 PM Shivaram Venkataraman
shiva...@eecs.berkeley.edu wrote:
Yeah from what I remember it was set defensively. I don't know of a good
way to
From SparkUI.scala :
def getUIPort(conf: SparkConf): Int = {
conf.getInt(spark.ui.port, SparkUI.DEFAULT_PORT)
}
Better retrieve effective UI port before probing.
Cheers
On Sat, Apr 11, 2015 at 2:38 PM, Nicholas Chammas
nicholas.cham...@gmail.com wrote:
So basically, to tell if the
