HI All,

Right now I'm trying to execute a script using this command:

nohup $SPARK_HOME/bin/spark-shell < $HOME/my-script.scala > $HOME/my-script.log 
2>&1 &

my-script.scala just have 1 line of code:
println("hallo world")

But after waiting for a minute, I still don't receive the result from 
spark-shell.
And the output log only contains:

Spark assembly has been built with Hive, including Datanucleus jars on classpath
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/data1/hadoop/spark-1.0.0-bin-hadoop2/lib/spark-assembly-1.0.0-hadoop2.2.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/data1/hadoop/hbase-0.96.0-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/08/28 09:38:27 INFO spark.SecurityManager: Changing view acls to: hadoop
14/08/28 09:38:27 INFO spark.SecurityManager: SecurityManager: authentication 
disabled; ui acls disabled; users with view permissions: Set(hadoop)
14/08/28 09:38:27 INFO spark.HttpServer: Starting HTTP Server
14/08/28 09:38:27 INFO server.Server: jetty-8.y.z-SNAPSHOT
14/08/28 09:38:27 INFO server.AbstractConnector: Started 
SocketConnector@0.0.0.0:49382<mailto:SocketConnector@0.0.0.0:49382>

My question is:
What is the right way to execute spark script?
I really want to use spark-shell as a way to run cron job in the future, just 
like when I run R script.

Best regards,
Henry

________________________________
The privileged confidential information contained in this email is intended for 
use only by the addressees as indicated by the original sender of this email. 
If you are not the addressee indicated in this email or are not responsible for 
delivery of the email to such a person, please kindly reply to the sender 
indicating this fact and delete all copies of it from your computer and network 
server immediately. Your cooperation is highly appreciated. It is advised that 
any unauthorized use of confidential information of Winbond is strictly 
prohibited; and any information in this email irrelevant to the official 
business of Winbond shall be deemed as neither given nor endorsed by Winbond.

Reply via email to