Hi again,
On Fri, Jan 16, 2015 at 4:25 PM, Tobias Pfeiffer <t...@preferred.jp> wrote:
> Now I'm wondering where this comes from (I haven't touched this component
> in a while, nor upgraded Spark etc.) [...]
>
So the reason that the error is showing up now is that suddenly data from a
different dataset is showing up in my test dataset... don't ask me...
anyway, this different dataset contains data like
{"Click":"nonclicked", "Impression":1,
"DisplayURL":4401798909506983219, "AdId":21215341, ...}
{"Click":"nonclicked", "Impression":1,
"DisplayURL":14452800566866169008, "AdId":10587781, ...}
and the DisplayURL seems to be too long for Long, while it is still
inferred as a Long column.
So, what to do about this? Is jsonRDD inherently incapable of handling
those long numbers or is it just an issue in the schema inference and I
should file a JIRA issue?
Thanks
Tobias
