Re: Spark: why need a masterLock when sending heartbeat to master
Thanks, I got it ! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-why-need-a-masterLock-when-sending-heartbeat-to-master-tp12256p12297.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Spark: why need a masterLock when sending heartbeat to master
I don't understand why worker need a master lock when sending heartbeat.
Caused by master HA ? Who can explain this in detail? Thanks~
Please refer:
http://stackoverflow.com/questions/25173219/why-does-the-spark-worker-actor-use-a-masterlock
case SendHeartbeat =
masterLock.synchronized {
if (connected) { master ! Heartbeat(workerId) }
}
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-why-need-a-masterLock-when-sending-heartbeat-to-master-tp12256.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
Re: Spark: why need a masterLock when sending heartbeat to master
Yes, good point, I believe the masterLock is now unnecessary altogether.
The reason for its initial existence was that changeMaster() originally
could be called out-of-band of the actor, and so we needed to make sure the
master reference did not change out from under us. Now it appears that all
master mutation and accesses are within the actor (we only call
tryRegisterAllMasters() from a different thread, which does not use the
master field).
If you like, feel free to submit a PR to remove the masterLock.
On Sun, Aug 17, 2014 at 8:58 AM, Victor Sheng victorsheng...@gmail.com
wrote:
I don't understand why worker need a master lock when sending heartbeat.
Caused by master HA ? Who can explain this in detail? Thanks~
Please refer:
http://stackoverflow.com/questions/25173219/why-does-the-spark-worker-actor-use-a-masterlock
case SendHeartbeat =
masterLock.synchronized {
if (connected) { master ! Heartbeat(workerId) }
}
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-why-need-a-masterLock-when-sending-heartbeat-to-master-tp12256.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
