That should work. Gonna give it a try. Thanks !
On Tue, Aug 12, 2014 at 11:01 AM, Marcelo Vanzin <van...@cloudera.com>
wrote:
> You could create a copy of the variable inside your "Parse" class;
> that way it would be serialized with the instance you create when
> calling map() below.
>
> On Tue, Aug 12, 2014 at 10:56 AM, Sunny Khatri <sunny.k...@gmail.com>
> wrote:
> > Are there any other workarounds that could be used to pass in the values
> > from someVariable to the transformation function ?
> >
> >
> > On Tue, Aug 12, 2014 at 10:48 AM, Sean Owen <so...@cloudera.com> wrote:
> >>
> >> I don't think static members are going to be serialized in the
> >> closure? the instance of Parse will be looking at its local
> >> SampleOuterClass, which is maybe not initialized on the remote JVM.
> >>
> >> On Tue, Aug 12, 2014 at 6:02 PM, Sunny Khatri <sunny.k...@gmail.com>
> >> wrote:
> >> > I have a class defining an inner static class (map function). The
> inner
> >> > class tries to refer the variable instantiated in the outside class,
> >> > which
> >> > results in a NullPointerException. Sample Code as follows:
> >> >
> >> > class SampleOuterClass {
> >> >
> >> > private static ArrayList<String> someVariable;
> >> >
> >> > SampleOuterClass() {
> >> > // initialize someVariable
> >> > }
> >> >
> >> > public static class Parse implements Function<...> {
> >> > public TypeReturn call (...) {
> >> > // Try using someVariable: Raises
> NullPointerException
> >> > }
> >> > }
> >> >
> >> > public void run() {
> >> > RDD<> rdd = data.map(new Parse()).rdd()
> >> > }
> >> > }
> >> >
> >> > Am I missing something with how Closures work with Spark or something
> >> > else
> >> > is wrong ?
> >> >
> >> > Thanks
> >> > Sunny
> >> >
> >> >
> >
> >
>
>
>
> --
> Marcelo
>
