But there is another problem. Is it possible to set the config variable from
Reduce?? I mean once I get the average value in the 1st M-R, I need to
initialise the Config variable to it. Can this be done in Reduce/Map
methods?

 As far as i know, we can SET the config variable only while configuring the
job. And we can only RETRIEVE these variables at the start of the map/reduce
methods. So, while setting up the second job, how do I get the value of the
Reduce's output ?

I apologize if this happens to be a trivial Question...but am pretty much
confused at this point and unable to proceed.

Thanks.

On Sun, Nov 16, 2008 at 1:49 AM, some speed <[EMAIL PROTECTED]> wrote:

> Thank you all!!
> What Milind has said will do the trick for me as a I need accurate values
> for the deviation.
> Passing variables between jobs by means of the Configure method and
> getInt/setInt will make things a lot easier!
>
>
> On Wed, Nov 12, 2008 at 7:07 PM, Milind Bhandarkar <[EMAIL PROTECTED]>wrote:
>
>> Since you need to pass only one number (average) to all mappers, you can
>> pass it through jobconf with a config variable defined by you, say
>> "my.average"..
>>
>> - milind
>>
>>
>> On 11/11/08 8:25 PM, "some speed" <[EMAIL PROTECTED]> wrote:
>>
>> > Thanks for the response. What I am trying is to do is finding the
>> average
>> > and then the standard deviation for a very large set (say a million) of
>> > numbers. The result would be used in further calculations.
>> > I have got the average from the first map-reduce chain. now i need to
>> read
>> > this average as well as the set of numbers to calculate the standard
>> > deviation.  so one file would have the input set and the other
>> "resultant"
>> > file would have just the average.
>> > Please do tell me in case there is a better way of doing things than
>> what i
>> > am doing. Any input/suggestion is appreciated.:)
>> >
>> >
>> >
>> > On Mon, Nov 10, 2008 at 4:22 AM, Amar Kamat <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> Amar Kamat wrote:
>> >>
>> >>> some speed wrote:
>> >>>
>> >>>> I was wondering if it was possible to read the input for a map
>> function
>> >>>> from
>> >>>> 2 different files:
>> >>>>  1st file ---> user-input file from a particular location(path)
>> >>>>
>> >>> Is the input/user file sorted? If yes then you can use "map-side join"
>> for
>> >> performance reasons. See org.apache.hadoop.mapred.join for more
>> details.
>> >>
>> >>> 2nd file=---> A resultant file (has just one <key,value> pair) from a
>> >>>> previous MapReduce job. (I am implementing a chain MapReduce
>> function)
>> >>>>
>> >>> Can you explain in more detail the contents of 2nd file?
>> >>
>> >>>
>> >>>> Now, for every <key,value> pair in the user-input file, I would like
>> to
>> >>>> use
>> >>>> the same <key,value> pair from the 2nd file for some calculations.
>> >>>>
>> >>> Can you explain this in more detail? Can you give some abstracted
>> example
>> >> of how file1 and file2 look like and what operation/processing you want
>> to
>> >> do?
>> >>
>> >>
>> >>>>
>> >>> I guess you might need to do some kind of join on the 2 files. Look at
>> >>> contrib/data_join for more details.
>> >>> Amar
>> >>>
>> >>>> Is it possible for me to do so? Can someone guide me in the right
>> >>>> direction
>> >>>> please?
>> >>>>
>> >>>>
>> >>>> Thanks!
>> >>>>
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>
>>
>>
>> --
>> Milind Bhandarkar
>> Y!IM: GridSolutions
>> 408-349-2136
>> ([EMAIL PROTECTED])
>>
>>
>

Reply via email to