Hi, guys
I have another problem about spark yarn.
Today, i was running start-all.sh when i found only two worker in the
Web Ui, and in fact, I have four nodes.
The only display of the two nodes, one is master, one is slave2.
the '/etc/hosts' file is show below:
/127.0.0.1 localhost//
//169.254.9.148 master//
//169.254.142.119 s1//
//169.254.180.3 s2//
//169.254.250.67 s3/
So, I'm going to see the spark log on slave1 and slave3.
The slave2 log is show below:
16/06/13 03:38:00 INFO util.Utils: Successfully started service
'sparkWorker' on port 39887.
And the slave1 or slave3 log is same:
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind
on port 0. Attempting port 1.
So, I think the problem lies in this util.Utils.
I search through Google, but there is no answer to this problem.
Now how to bind the sparkWorker port?
Thanks
Jay
