That sounds just right.
On 5/26/08 7:05 AM, "Deyaa Adranale" <[EMAIL PROTECTED]> wrote: > since the information needed in the MapReduce is just small data as > strings or integers, i did not prefer to write it to the file system. > otherwise, i used the methods conf.setInt(name,intvalue) and > conf.set(name,stringvalue) to give the data to the JobConf object. Then > inside mapper.configure(JobConf) or reducer.configure(JobConf), i used: > job.getInt(name,defaultvalue) and job.get(name) to get the values of the > data assigned previously. > > i think if i had large amounts of data to be shared I could store it in > the DFS and add it to the configeration by calling: > conf.addResource(...) and read it in a similar way inside the > mapper/reduce.configure method. > > is what i said right? any corrections? > > thanks for the support > > Deyaa > > > > > Deyaa Adranale schrieb: >> thanks for the answer. ur suggestion is interesting to try. >> >> i will try it out and write the result here. >> >> >> >> Ted Dunning schrieb: >>> The mapper and reducers are not executing on the same machine as your >>> main >>> program. >>> >>> That means that they cannot access the data you are talking about. The >>> requirement that they be static classes is a reflection of this. >>> >>> If you need to share data, then you need to write it to the file >>> system and >>> read it in the configure method on the mapper or reducer. >>> >>> >>> On 5/21/08 3:17 AM, "Deyaa Adranale" <[EMAIL PROTECTED]> >>> wrote: >>> >>> >>>> hello. >>>> why do we have to set the map and reduce classes as static? >>>> i need inside them to access some data which is not static. what i >>>> should do? >>>> >>>> non static map or reduce classes generates the following exception: >>>> >>>> java.lang.RuntimeException: java.lang.NoSuchMethodException: >>>> hadoop.examples.Simple$MyMapper.() >>>> at >>>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:45) >>>> >>>> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:32) >>>> at >>>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:53) >>>> >>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:209) >>>> at >>>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1210) >>>> Caused by: java.lang.NoSuchMethodException: >>>> hadoop.examples.Simple$MyMapper.() >>>> at java.lang.Class.getConstructor0(Class.java:2705) >>>> at java.lang.Class.getDeclaredConstructor(Class.java:1984) >>>> at >>>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:41) >>>> >>>> ... 4 more >>>> >>>> >>>> >>>> >>>> any ideas? >>>> >>>> thanks in advance, >>>> >>>> Deyaa >>>> >>> >>> >>> >> >> >
