The only way to do something like this is get them mapers to use something
like /dev/shm as there storage folder that's 100% memory
outside of that everything is flushed because the mapper exits when its done
the tasktracker is the one delivering the output to the reduce task.
Billy
"paula_ta" <paula...@yahoo.com> wrote in
message news:23617347.p...@talk.nabble.com...
Is it possible that some intermediate data produced by mappers and written
to
the local file system resides in memory in the file system cache and is
never flushed to disk ? Eventually reducers will retrieve this data via
HTTP - possibly without the data ever being written to disk ?
thanks
Paula
--
View this message in context:
http://www.nabble.com/Is-intermediate-data-produced-by-mappers-always-flushed-to-disk---tp23617347p23617347.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.
