Re: Multithreaded Mapper and Map runner

If only thread is created to run mapper/reducer, how would mapred.child.java.opts be effective ? Please refer to src/mapred/org/apache/hadoop/mapred/TaskRunner.java which is not very long. On Wed, Jun 16, 2010 at 9:10 PM, Jyothish Soman wrote: > > I have another doubt, for cross checking. The nu

Re: Changing default separator for streaming application

Final output is written by OutputFormat. By default, TextOutputFormat will write \t as the key-value separator. You can specify a different key-value separator for TextOutputFormat by specifying the value for configuration property "mapred.textoutputformat.separator". Try setting ' ' for the co

Re: Multithreaded Mapper and Map runner

I have another doubt, for cross checking. The number set in mapred.tasktracker.map/reduce.tasks.maximum creates that many JVM instances, or does it just create that many threads. Though I could not see any explicit statement about it, it was pointed everywhere as if it is a JVM instance. Please do

Re: Problem with Reducer emitting a different Key than Mapper

Yes - that was the problem - thanks On Wed, Jun 16, 2010 at 9:26 AM, Alex Kozlov wrote: > Hi Steve, did you do > > job.setOutputKeyClass(Text.class); > job.setOutputValueClass(Text.class); > > ? > > Alex K > > > On Wed, Jun 16, 2010 at 9:15 AM, Steve Lewis wrote: > >> Problem with Reducer emitti

Re: Problem with Reducer emitting a different Key than Mapper

Hi Steve, did you do job.setOutputKeyClass(Text.class); job.setOutputValueClass(Text.class); ? Alex K On Wed, Jun 16, 2010 at 9:15 AM, Steve Lewis wrote: > Problem with Reducer emitting a different Key than Mapper > IO have the FOllowing code where the Mapper emits a custom Key and the > redu

Problem with Reducer emitting a different Key than Mapper

Problem with Reducer emitting a different Key than Mapper IO have the FOllowing code where the Mapper emits a custom Key and the reducer isa expected to emit text Using Hadoop 0.2 on a local instance I asj the reducer to write Text,Text - thiew is even what the IDE says I should do and what I get

Changing default separator for streaming application

I am trying to develop a streaming MR application by implementing korn-shell based mapper and reducer. I want to use 'space - x20' as the separator between key and value throughout the application. When invoking the application I specified -D stream.map.output.field.separator=" " -D stream.reduce