Having them on the same network will give you better performance. What problems are you facing? Can you elaborate more about the versions? (Spark and hadoop), Only problem that could happen is the OS's OOM Killer might be killing your HDFS processes due to low memory and such. But you can check the logs to get more clear picture about whats happening.
Thanks Best Regards On Fri, Nov 21, 2014 at 7:58 PM, EH <eas...@gmail.com> wrote: > Hi, > > Are there any way that I can setup a remote HDFS for Spark (more specific, > for Spark Streaming checkpoints)? The reason I'm asking is that our Spark > and HDFS do not run on the same machines. I've been looked around but > still > no clue so far. > > Thanks, > EH > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Setup-Remote-HDFS-for-Spark-tp19481.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
