Re: Setup Remote HDFS for Spark

2014-11-24 Thread Tobias Pfeiffer
Hi, On Sat, Nov 22, 2014 at 12:13 AM, EH eas...@gmail.com wrote: Unfortunately whether it is possible to have both Spark and HDFS running on the same machine is not under our control. :( Right now we have Spark and HDFS running in different machines. In this case, is it still possible to

Re: Setup Remote HDFS for Spark

2014-11-22 Thread Akhil Das
Yes, it is possible to have both spark and hdfs running on the same cluster. We have a lot of clusters running without any issues. And yes, it is possible to hook spark up with remote hdfs. You might feel a bit lag if they are on different networks. Thanks Best Regards On Fri, Nov 21, 2014 at

Re: Setup Remote HDFS for Spark

2014-11-21 Thread EH
Unfortunately whether it is possible to have both Spark and HDFS running on the same machine is not under our control. :( Right now we have Spark and HDFS running in different machines. In this case, is it still possible to hook up a remote HDFS with Spark so that we can use Spark Streaming