If you set up Spark's metrics reporting to write to the Ganglia backend that will give you a good idea of how much network/disk/CPU is being used and on what machines.
https://spark.apache.org/docs/0.9.0/monitoring.html On Tue, Apr 8, 2014 at 12:57 PM, yxzhao <yxz...@ualr.edu> wrote: > Hi All, > > I want to measure the total network traffic for a Spark Job. But I > did > not see related information from the log. Does anybody know how to measure > it? Thanks very much in advance. > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Measuring-Network-Traffic-for-Spark-Job-tp3912.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
