RE: hwo to read a text file in Map function until reaching specific line

2009-06-26 Thread Ramakishore Yelamanchilli
I think map function gets the line number as key. You can ignore te other lines after the key value 500. Thanks -Original Message- From: Leiz [mailto:lzhan...@gmail.com] Sent: Friday, June 26, 2009 8:57 AM To: core-user@hadoop.apache.org Subject: hwo to read a text file in Map function

Re: hwo to read a text file in Map function until reaching specific line

2009-06-26 Thread Tarandeep Singh
The TextInputFormat gives byte offset in the file as key and the entire line as value. so it won't work for you. You can modify NLineInputFormat to achieve what you want. NLineInputformat gives each mapper N Lines (in your case N=500) Since you are interested in only first 500 lines of each