Hi, Though I am not an expert in Hadoop, I think Hadoop's one of the main motives is to have data locality (i.e business logic(job) should run locally with the data.. which is further helped by having data replicas on various data nodes so that the task which is loaded locally to the data block would read it) This saves network bandwidth. So May be you can have HDFS and YARN service running on different cluster but I think that's not what Hadoop aims for.. This is just my point of view..
On Fri, Jan 10, 2014 at 3:45 AM, Arpit Agarwal <aagar...@hortonworks.com>wrote: > Hi Kevin, > > *> 2. According to > http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html > <http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html>, > Hadoop 2.2.0 supports Microsoft Windows. How do/Can you configure YARN for > secure container isolation in Windows? It seems that the ContainerExecutor > and DefaultContainerExecutor can detect and run on Windows, but the secure > LinuxContainerExecutor are for *nix systems, so is there anything in place > for maximum security like LCE is?* > > Cluster security is not supported on Windows at the moment. > > > > *> 1. Does YARN need to run on the same machines that are hosting the HDFS > services or can HDFS be remote of a YARN cluster? If this done by placing > the remote HDFS cluster's configuration files (core-site.xml and > hdfs-site.xml) on the YARN cluster's machines?* > > *> 3. If 1 is yes, then is it possible to have a cluster mixed with both > Linux and Windows machines running YARN and working together?* > It should work in theory if you get the configuration right - I have not > tried it out so I am not sure. YARN containers and HDFS datanodes should be > collocated for good performance. The MapReduce compute model especially > depends on access to fast local storage. > > > > On Thu, Jan 9, 2014 at 1:19 PM, Kevin <kevin.macksa...@gmail.com> wrote: > >> Hi, >> >> Three questions about the new Hadoop release regarding YARN: >> >> 1. Does YARN need to run on the same machines that are hosting the HDFS >> services or can HDFS be remote of a YARN cluster? If this done by placing >> the remote HDFS cluster's configuration files (core-site.xml and >> hdfs-site.xml) on the YARN cluster's machines? >> >> 2. According to >> http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html, >> Hadoop 2.2.0 supports Microsoft Windows. How do/Can you configure YARN for >> secure container isolation in Windows? It seems that the ContainerExecutor >> and DefaultContainerExecutor can detect and run on Windows, but the secure >> LinuxContainerExecutor are for *nix systems, so is there anything in place >> for maximum security like LCE is? >> >> 3. If 1 is yes, then is it possible to have a cluster mixed with both >> Linux and Windows machines running YARN and working together? >> >> Thanks, >> Kevin >> > > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
