define some key of yours, say "my.app.array",
serialize the object to a string, say myObjectString
Then conf.set("my.app.array", myObjectString)
Then in your Mapper.configure() method
String myObjectString = conf.get("my.app.array")
deserialize your object.
A google search on java object serialization to string, will provide you
with many examples, such as
http://www.velocityreviews.com/forums/showpost.php?p=3185744&postcount=10.
On Sat, May 2, 2009 at 11:56 PM, Amandeep Khurana <ama...@gmail.com> wrote:
> Thanks Jason.
>
> My object is relatively small. But how do I pass it via the JobConf object?
> Can you elaborate a bit...
>
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Sat, May 2, 2009 at 11:53 PM, jason hadoop <jason.had...@gmail.com
> >wrote:
>
> > If it is relatively small you can pass it via the JobConf object, storing
> a
> > serialized version of your dataset.
> > If it is larger you can pass a serialized version via the distributed
> > cache.
> > Your map task will need to deserialize the object in the configure
> method.
> >
> > None of the above methods give you an object that is write shared between
> > map tasks.
> >
> > Please remember that the map tasks execute in separate JVM's on distinct
> > machines in the normal MapReduce environment.
> >
> >
> >
> > On Sat, May 2, 2009 at 10:59 PM, Amandeep Khurana <ama...@gmail.com>
> > wrote:
> >
> > > How can I create a global variable for each node running my map task.
> For
> > > example, a common ArrayList that my map function can access for every
> k,v
> > > pair it works on. It doesnt really need to create the ArrayList
> > everytime.
> > >
> > > If I create it in the main function of the job, the map function gets a
> > > null
> > > pointer exception. Where else can this be created?
> > >
> > > Amandeep
> > >
> > >
> > > Amandeep Khurana
> > > Computer Science Graduate Student
> > > University of California, Santa Cruz
> > >
> >
> >
> >
> > --
> > Alpha Chapters of my book on Hadoop are available
> > http://www.apress.com/book/view/9781430219422
> >
>
--
Alpha Chapters of my book on Hadoop are available
http://www.apress.com/book/view/9781430219422
