Hi Akshay Thanks for the response please find below the answers to your questions.
1. We are running Spark in cluster mode the cluster manager being Spark's standalone cluster manager. 2. All the ports are open and we preconfigure on what ports the communication should happen and modify firewall rules to allow traffic on these ports. (The functionality is fine till Spark master goes down after 60 mins) 3. Memory consumptions of all the components: Spark Master: S0 S1 E O M CCS YGC YGCT FGC FGCT GCT 0.00 0.00 12.91 35.11 97.08 95.80 5 0.239 2 0.197 0.436 Spark Worker: S0 S1 E O M CCS YGC YGCT FGC FGCT GCT 51.64 0.00 46.66 27.44 97.57 95.85 10 0.381 2 0.233 0.613 Spark Submit Process (Driver): S0 S1 E O M CCS YGC YGCT FGC FGCT GCT 0.00 63.57 93.82 26.29 98.24 97.53 4663 124.648 109 20.910 145.558 Spark executor (Coarse grained): S0 S1 E O M CCS YGC YGCT FGC FGCT GCT 0.00 69.77 17.74 31.13 95.67 90.44 7353 556.888 5 1.572 558.460 -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
