Finally, someone reply. thank you, sir! But I am planning to deploy stand alone mode of Spark. I thought there is no need to use hdfs? And my spark is not being built with hadoop/yarn config.
regards, Amin On Tue, Aug 5, 2014 at 10:39 PM, Mayur Rustagi <mayur.rust...@gmail.com> wrote: > Spark is not able to communicate with your hadoop hdfs. Is your hdfs > running, if so can you try to explicitly connect to it with hadoop command > line tools giving full hostname & port. > Or test port using > > telnet localhost 9000 > In all likelyhood either your hdfs is down, bound to wrong port/ip that > spark cannot access > > Mayur Rustagi > Ph: +1 (760) 203 3257 > http://www.sigmoidanalytics.com > @mayur_rustagi <https://twitter.com/mayur_rustagi> > > > > On Tue, Aug 5, 2014 at 11:33 PM, alamin.ishak <alamin.is...@gmail.com> > wrote: > >> Hi, >> Anyone? Any input would be much appreciated >> >> Thanks, >> Amin >> On 5 Aug 2014 00:31, "Al Amin" <[hidden email] >> <http://user/SendEmail.jtp?type=node&node=11477&i=0>> wrote: >> >>> Hi all, >>> >>> Any help would be much appreciated. >>> >>> Thanks, >>> Al >>> >>> >>> On Mon, Aug 4, 2014 at 7:09 PM, Al Amin <[hidden email] >>> <http://user/SendEmail.jtp?type=node&node=11477&i=1>> wrote: >>> >>>> Hi all, >>>> >>>> I have setup 2 nodes (master and slave1) on stand alone mode. Tried >>>> running SparkPi example and its working fine. However when I move on to >>>> wordcount its giving me below error: >>>> >>>> 14/08/04 21:40:33 INFO storage.MemoryStore: ensureFreeSpace(32856) >>>> called with curMem=0, maxMem=311387750 >>>> 14/08/04 21:40:33 INFO storage.MemoryStore: Block broadcast_0 stored as >>>> values in memory (estimated size 32.1 KB, free 296.9 MB) >>>> 14/08/04 21:40:34 INFO ipc.Client: Retrying connect to server: master/ >>>> 10.0.1.27:9000. Already tried 0 time(s). >>>> 14/08/04 21:40:35 INFO ipc.Client: Retrying connect to server: master/ >>>> 10.0.1.27:9000. Already tried 1 time(s). >>>> 14/08/04 21:40:36 INFO ipc.Client: Retrying connect to server: master/ >>>> 10.0.1.27:9000. Already tried 2 time(s). >>>> 14/08/04 21:40:37 INFO ipc.Client: Retrying connect to server: master/ >>>> 10.0.1.27:9000. Already tried 3 time(s). >>>> 14/08/04 21:40:38 INFO ipc.Client: Retrying connect to server: master/ >>>> 10.0.1.27:9000. Already tried 4 time(s). >>>> 14/08/04 21:40:39 INFO ipc.Client: Retrying connect to server: master/ >>>> 10.0.1.27:9000. Already tried 5 time(s). >>>> 14/08/04 21:40:40 INFO ipc.Client: Retrying connect to server: master/ >>>> 10.0.1.27:9000. Already tried 6 time(s). >>>> 14/08/04 21:40:41 INFO ipc.Client: Retrying connect to server: master/ >>>> 10.0.1.27:9000. Already tried 7 time(s). >>>> 14/08/04 21:40:42 INFO ipc.Client: Retrying connect to server: master/ >>>> 10.0.1.27:9000. Already tried 8 time(s). >>>> 14/08/04 21:40:43 INFO ipc.Client: Retrying connect to server: master/ >>>> 10.0.1.27:9000. Already tried 9 time(s). >>>> Exception in thread "main" java.lang.RuntimeException: >>>> java.net.ConnectException: Call to master/10.0.1.27:9000 failed on >>>> connection exception: java.net.ConnectException: Connection refused >>>> >>>> >>>> 1) how to fix this issue? I have configure hostname --fqdn accordingly. >>>> >>>> 2) I could see that in my logs that my master/worker deploy >>>> configuration is -Xms512m -Xmx512m. Is there any way that I can increase >>>> it? Or 512mb is just fine? AFAIK, spark require huge memory. >>>> >>>> 3) I have a hadoop cluster and its working. Could anyone point me how >>>> to integrate Yarn with Spark? Any good tutorial would be very useful >>>> >>>> >>>> Thanks, >>>> Al >>>> >>> >>> >> ------------------------------ >> View this message in context: Re: Configuration setup and Connection >> refused >> <http://apache-spark-user-list.1001560.n3.nabble.com/Re-Configuration-setup-and-Connection-refused-tp11477.html> >> Sent from the Apache Spark User List mailing list archive >> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >> > >
