hero, Did you check whether there is any exception after retry? If the port is 0, the spark worker should bind to a random port. BTW, what's the spark version?
Regards, - Terry On Mon, Jun 13, 2016 at 4:24 PM, hero <super_big_h...@sina.com> wrote: > Hi, guys > > I have another problem about spark yarn. > Today, i was running start-all.sh when i found only two worker in the Web > Ui, and in fact, I have four nodes. > The only display of the two nodes, one is master, one is slave2. > the '/etc/hosts' file is show below: > > *127.0.0.1 localhost* > *169.254.9.148 master* > *169.254.142.119 s1* > *169.254.180.3 s2* > *169.254.250.67 s3* > > > So, I'm going to see the spark log on slave1 and slave3. > The slave2 log is show below: > > 16/06/13 03:38:00 INFO util.Utils: Successfully started service > 'sparkWorker' on port 39887. > > And the slave1 or slave3 log is same: > > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > 16/06/13 03:38:00 WARN util.Utils: Service 'sparkWorker' could not bind on > port 0. Attempting port 1. > > So, I think the problem lies in this util.Utils. > I search through Google, but there is no answer to this problem. > Now how to bind the sparkWorker port? > > Thanks > Jay >
