You can make use of this configuration to do the same. <property> <description>List of directories to store localized files in. An application's localized file directory will be found in: ${yarn.nodemanager.local-dirs}/usercache/${user}/appcache/application_${appid}. Individual containers' work directories, called container_${contid}, will be subdirectories of this. </description> <name>yarn.nodemanager.local-dirs</name> <value>${hadoop.tmp.dir}/nm-local-dir</value> </property>
Thanks Devaraj k From: John Lilley [mailto:john.lil...@redpoint.net] Sent: 02 July 2013 02:08 To: user@hadoop.apache.org Subject: temporary folders for YARN tasks When a YARN app and its tasks wants to write temporary files, how does it know where to write the files? I am assuming that each task has some temporary space available, and I hope it is available across multiple disk volumes for parallel performance. Are those files cleaned up automatically after task exit? If I want to give lifetime control of the files to an auxiliary service (along the lines of MR shuffle passing files to the aux service), how would I do that, and would that entail different file locations? Thanks John