How do I unsubscribe from this list?
Vanessa van Gelder Analyst Programmer Tel: +27 21 680 1387 | Fax: | Cell: vvangel...@homechoice.co.za HomeChoice http://www.homechoice.co.za/ -----Original Message----- From: anilgupt...@gmail.com [mailto:anilgupt...@gmail.com] On Behalf Of anil gupta Sent: Friday, March 16, 2012 3:39 AM To: common-user@hadoop.apache.org Subject: Re: Suggestion for InputSplit and InputFormat - Split every line. Have a look at NLineInputFormat class in Hadoop. It is build to split the input on the basis of number of lines. On Thu, Mar 15, 2012 at 6:13 PM, Deepak Nettem <deepaknet...@gmail.com>wrote: > Hi, > > I have this use case - I need to spawn as many mappers as the number > of lines in a file in HDFS. This file isn't big (only 10-50 lines). > Actually each line represents the path of another data source that the > Mappers will work on. So each mapper will read 1 line, (the map() > method will need to be called only once), and work on the data source. > > What's the best way to construct InputSplit, InputFormat and > RecordReader to achieve this? I would appreciate any example code :) > > Best, > Deepak > -- Thanks & Regards, Anil Gupta
