Spark Streaming socketTextStream

[fredwolfinger]

Good morning,

I have taken the socketTextStream example and instead of running on a local
Spark instance, I have pushed it to my Spark cluster in AWS (1 master with 5
slave nodes). I am getting the following error that appears to indicate that
all the slaves are trying to read from localhost: when all I really want
is the single master node to read from it's localhost: and batch up what
it receives. Can anyone help me with what I might be missing with the way I
am submitting the job?

14/06/10 13:12:49 INFO scheduler.ReceiverTracker: Registered receiver for
stream 0 from akka.tcp://spark@SLAVE-INTERNAL-IP:39710
14/06/10 13:12:49 ERROR scheduler.ReceiverTracker: Deregistered receiver for
stream 0: Restarting receiver with delay 2000ms: Error connecting to
localhost: - java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.init(Socket.java:425)
at java.net.Socket.init(Socket.java:208)
at
org.apache.spark.streaming.dstream.SocketReceiver.receive(SocketInputDStream.scala:71)
at
org.apache.spark.streaming.dstream.SocketReceiver$$anon$2.run(SocketInputDStream.scala:57)



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-socketTextStream-tp7326.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Spark Streaming socketTextStream

[fredwolfinger]

Worked! Thanks so much!

Fred

Fred Wolfinger
Research Staff Member, CyberPoint Labs
direct +1 410 779 6741
mobile +1 443 655 3322

CyberPoint International

621 East Pratt Street, Suite 300

Baltimore MD 21202-3140

phone +1 410 779 6700

www.cyberpointllc.com http://www.cyberpointllc.com/





If you believe you received this e-mail in error, please notify the sender
immediately, delete the e-mail from your computer and do not copy or
disclose it to anyone else.



The information in this email constitutes the proprietary information of
Cyber Point International, LLC (DBA CyberPoint), and should be accessed only
by the individual to whom it is addressed. The information in this email and
any attachments may not be used, copied or disclosed without the consent of
CyberPoint. CyberPoint is not responsible for any damages caused by your
unauthorized use of the materials in this email.


From:  Akhil Das-2 [via Apache Spark User List]
ml-node+s1001560n7328...@n3.nabble.com
Date:  Tuesday, June 10, 2014 10:16 AM
To:  Fred Wolfinger fwolfin...@cyberpointllc.com
Subject:  Re: Spark Streaming socketTextStream

You can use the master's IP address (Or whichever machine you chose to run
the nc command) instead of localhost.



If you reply to this email, your message will be added to the discussion
below:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-socketTe
xtStream-tp7326p7328.html
To unsubscribe from Spark Streaming socketTextStream, click here
http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jt
p?macro=unsubscribe_by_codenode=7326code=ZndvbGZpbmdlckBjeWJlcnBvaW50bGxjL
mNvbXw3MzI2fC0xMDE2NjE2NjAy .
NAML 
http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jt
p?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.
namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.vi
ew.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemai
l.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aem
ail.naml 




smime.p7s (6K) 
http://apache-spark-user-list.1001560.n3.nabble.com/attachment/7330/0/smime.p7s




--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-socketTextStream-tp7326p7330.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.