On 20 Jul 2015, at 23:20, Harish Butani <rhbutani.sp...@gmail.com> wrote:
> Can you post details on how to reproduce the NPE Essentially it is like this: I have a scala case class that contains a Joda DateTime attribute and instances of this class are updated using updateStateByKey. When a certain condition is reached the instances are turned to Json (using spray.json) and are stored in ElasticSearch. just before creating the JSON I call getYear on the date attribute and that fails with NPE. When insert a getMillis or toString right before the getYear these work just fine. Jan > > On Mon, Jul 20, 2015 at 1:19 PM, algermissen1971 <algermissen1...@icloud.com> > wrote: > Hi Harish, > > On 20 Jul 2015, at 20:37, Harish Butani <rhbutani.sp...@gmail.com> wrote: > > > Hey Jan, > > > > Can you provide more details on the serialization and cache issues. > > My symptom is that I have a Joda DateTime on which I can call toString and > getMillis without problems, but when I call getYear I get a NPE out of the > internal AbstractDateTime. Totally strange but seems to align with issues > others have. > > I am now changing the app to work with millis internally, as that seems to be > a performance improvement regarding serialization anyhow. > > Thanks, > > Jan > > > > > > If you are looking for datetime functionality with spark-sql please > > consider: https://github.com/SparklineData/spark-datetime It provides a > > simple way to combine joda datetime expressions with spark sql. > > > > regards, > > Harish. > > > > On Mon, Jul 20, 2015 at 7:37 AM, algermissen1971 > > <algermissen1...@icloud.com> wrote: > > Hi, > > > > I am having trouble with Joda Time in a Spark application and saw by now > > that I am not the only one (generally seems to have to do with > > serialization and internal caches of the Joda Time objects). > > > > Is there a known best practice to work around these issues? > > > > Jan > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > > For additional commands, e-mail: user-h...@spark.apache.org > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
