Hi, Matei Thanks for your response.
I tried to copy the file (1G) from NFS and took 10 seconds. The NFS mount is a LAN environment and the NFS server is running on the same server that Spark is running on. So basically I mount the NFS on the same bare metal machine. Larry On Wed, Dec 17, 2014 at 11:42 AM, Matei Zaharia <matei.zaha...@gmail.com> wrote: > > The problem is very likely NFS, not Spark. What kind of network is it > mounted over? You can also test the performance of your NFS by copying a > file from it to a local disk or to /dev/null and seeing how many bytes per > second it can copy. > > Matei > > > On Dec 17, 2014, at 9:38 AM, Larryliu <larryli...@gmail.com> wrote: > > > > A wordcounting job for about 1G text file takes 1 hour while input from > a NFS > > mount. The same job took 30 seconds while input from local file system. > > > > Is there any tuning required for a NFS mount input? > > > > Thanks > > > > Larry > > > > > > > > -- > > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/wordcount-job-slow-while-input-from-NFS-mount-tp20747.html > > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > > For additional commands, e-mail: user-h...@spark.apache.org > > > >
