Do you mean, without scanning all the files line by line?
I know little about implementation of hadoop, but as a programmer, I can
presume that it's not possible without a complete scan.
But I can suggest a work-around:
- compute number of records manually before putting a file to HDFS.
- Append the computed number to the filename.
- modify InputReader, so that reader appends that number to the key of every
map.
Hope this helps,
Rasit
2009/1/27 Andy Liu andyliu1...@gmail.com
Is there a way to programatically get the number of records in a MapFile
without doing a complete scan?
--
M. Raşit ÖZDAŞ
