Hi everyone, I'm working with Spark Streaming, and I need to perform some offline performance measures.
What I'd like to have is a CSV file that reports something like this: *Batch number/timestamp Input Size Total Delay* which is in fact similar to what the UI outputs. I tried to get some metrics (metrics.properties), but I'm having hard time getting precise information on every single batch, since they only have entries concerning the /last/ (completed/received) batch, and values are often different to those appearing in the UI. Can anybody give me some advice on how to get metrics that are close to those of the UI? Thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Total-delay-per-batch-in-a-CSV-file-tp24129.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
