Jacek, could you create a JIRA for it? I just reproduced it. It's a bug in how Master handles the Worker disconnection.
Best Regards, Shixiong Zhu 2015-12-10 2:45 GMT-08:00 Jacek Laskowski <ja...@japila.pl>: > Hi, > > I'm on yesterday's master HEAD. > > Pozdrawiam, > Jacek > > -- > Jacek Laskowski | https://medium.com/@jaceklaskowski/ | > http://blog.jaceklaskowski.pl > Mastering Spark https://jaceklaskowski.gitbooks.io/mastering-apache-spark/ > Follow me at https://twitter.com/jaceklaskowski > Upvote at http://stackoverflow.com/users/1305344/jacek-laskowski > > > On Thu, Dec 10, 2015 at 9:50 AM, Sasaki Kai <sas...@treasure-data.com> > wrote: > > Hi, Jacek > > > > What version of Spark do you use? > > I started sbin/start-master.sh script as you did against master HEAD. > But there is no warning log such you pasted. > > While you can specify hostname with -h option, you can also omit it. The > master name can be set automatically with > > the name `hostname` command. You can also try it. > > > > Kai Sasaki > > > >> On Dec 10, 2015, at 5:22 PM, Jacek Laskowski <ja...@japila.pl> wrote: > >> > >> Hi, > >> > >> While toying with Spark Standalone I've noticed the following messages > >> in the logs of the master: > >> > >> INFO Master: Registering worker 192.168.1.6:59919 with 2 cores, 2.0 GB > RAM > >> INFO Master: localhost:59920 got disassociated, removing it. > >> ... > >> WARN Master: Removing worker-20151210090708-192.168.1.6-59919 because > >> we got no heartbeat in 60 seconds > >> INFO Master: Removing worker worker-20151210090708-192.168.1.6-59919 > >> on 192.168.1.6:59919 > >> > >> Why does the message "WARN Master: Removing > >> worker-20151210090708-192.168.1.6-59919 because we got no heartbeat in > >> 60 seconds" appear when the worker should've been gone already (as > >> pointed out in "INFO Master: localhost:59920 got disassociated, > >> removing it.")? > >> > >> Could it be that the ids are different - 192.168.1.6:59919 vs > localhost:59920? > >> > >> I started master using "./sbin/start-master.sh -h localhost" and the > >> workers "./sbin/start-slave.sh spark://localhost:7077". > >> > >> p.s. Are such questions appropriate for this mailing list? > >> > >> Pozdrawiam, > >> Jacek > >> > >> -- > >> Jacek Laskowski | https://medium.com/@jaceklaskowski/ | > >> http://blog.jaceklaskowski.pl > >> Mastering Spark > https://jaceklaskowski.gitbooks.io/mastering-apache-spark/ > >> Follow me at https://twitter.com/jaceklaskowski > >> Upvote at http://stackoverflow.com/users/1305344/jacek-laskowski > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > >> For additional commands, e-mail: dev-h...@spark.apache.org > >> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > >
