Local mode does serDe, so it should expose serialization problems.

On Wed, May 14, 2014 at 10:53 AM, Philip Ogren <philip.og...@oracle.com>wrote:

> Have you actually found this to be true?  I have found Spark local mode to
> be quite good about blowing up if there is something non-serializable and
> so my unit tests have been great for detecting this.  I have never seen
> something that worked in local mode that didn't work on the cluster because
> of different serialization requirements between the two.  Perhaps it is
> different when using Kryo....
>
>
>
> On 05/14/2014 04:34 AM, Andras Nemeth wrote:
>
>> E.g. if I accidentally use a closure which has something non-serializable
>> in it, then my test will happily succeed in local mode but go down in
>> flames on a real cluster.
>>
>
>

Reply via email to