It turned out our fs.defaultFS was in hdfs-site.xml and not core-site.xml This makes sense since the application manager was dumping the job resources onto its local FS, also hdfs dfs -ls / was returning the local filesystem on each node.
Moving this property to core-site.xml and restarting dfs allowed nodemanagers to localize resources from HDFS. Rookie mistake -- hopefully this thread saves someone else a couple of hours someday -- Iain Wright This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you are not the intended recipient, do not disclose or disseminate the message to anyone except the intended recipient. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender by return email, and delete all copies of this message. On Mon, Sep 29, 2014 at 11:38 AM, iain wright <[email protected]> wrote: > Anyone have any advice on this? I'm going to cross post to the hadoop > users group since this seems to be a YARN related issue > > Best, > >
