Re: Time is ugly in Spark Streaming....

2015-06-27 Thread Tathagata Das
Could you print the time on the driver (that is, in foreachRDD but before RDD.foreachPartition) and see if it is behaving weird? TD On Fri, Jun 26, 2015 at 3:57 PM, Emrehan Tüzün emrehan.tu...@gmail.com wrote: On Fri, Jun 26, 2015 at 12:30 PM, Sea 261810...@qq.com wrote: Hi, all I find

?????? Time is ugly in Spark Streaming....

2015-06-27 Thread Sea
Yes , things go well now. It is a problem of SimpleDateFormat. Thank you all. -- -- ??: Dumas Hwang;dumas.hw...@gmail.com; : 2015??6??27??(??) 8:16 ??: Tathagata Dast...@databricks.com; : Emrehan

Re: Time is ugly in Spark Streaming....

2015-06-27 Thread Dumas Hwang
Java's SimpleDateFormat is not thread safe. You can consider using DateTimeFormatter if you are using Java 8 or Joda-time On Sat, Jun 27, 2015 at 3:32 AM, Tathagata Das t...@databricks.com wrote: Could you print the time on the driver (that is, in foreachRDD but before RDD.foreachPartition)