Depends how big the lines are, on a typical HDD you can write at max 10-15MB/s, and on SSDs it can be upto 30-40MB/s.
Thanks Best Regards On Mon, Sep 28, 2015 at 3:57 PM, vaibhavrtk <vaibhav...@gmail.com> wrote: > Hello > > We need a lot of logging for our application about 1000 lines needed to be > logged per message we process and we process 1000 msgs/sec. So total lines > needed to be logged is /1000*1000/sec/. As it is going to be written in a > file. Will writing so much logs will impact the processing power of spark > by > a lot? > If yes, What can be the alternative? > > Note: This much logging is required for the appropriate monitoring of the > application. > Let me know if more information is needed. > > Thanks > Vaibhav > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/log4j-Spark-worker-performance-problem-tp24842.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
