hi, Passion I don't know an exact solution. But yes, the port each executor chosen to communicate with driver is random. I am wondering if it's possible that you can have a node has two ethernet card, configure one card for intranet for Spark and configure one card for WAN. Then connect the rests nodes using the intranet.
And also, I think you might not use WAN for Spark data transfer since the amount of data during shuffle is huge. You got to have a high-speed switch for your cluster. Hopes this answer can help you! Wei -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
