[
https://issues.apache.org/jira/browse/SYSTEMML-745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312694#comment-15312694
]
Glenn Weidner commented on SYSTEMML-745:
----------------------------------------
Note the DML script runs successfully (e.g., 'Hello Apache SystemML' visible in
log for simple hello world example). Since creating spark context is an
expensive operation, it is done only when required. However, the Spark
Application Master apparently checks if it's been created and if not, fails the
job:
16/06/02 07:48:48 INFO DMLScript: SystemML Statistics:
Total execution time: 0.528 sec.
Number of executed MR Jobs: 0.
16/06/02 07:48:48 INFO DMLScript: END DML run 06/02/2016 07:48:48
16/06/02 07:48:48 INFO ApplicationMaster: Final app status: SUCCEEDED,
exitCode: 0
16/06/02 07:48:57 ERROR ApplicationMaster: SparkContext did not initialize
after waiting for 100000 ms. Please check earlier log output for errors.
Failing the application.
> Spark context not initialized error under yarn-cluster for small data
> ---------------------------------------------------------------------
>
> Key: SYSTEMML-745
> URL: https://issues.apache.org/jira/browse/SYSTEMML-745
> Project: SystemML
> Issue Type: Bug
> Components: Runtime
> Reporter: Glenn Weidner
>
> Sample command to reproduce issue:
> ./bin/spark-submit --master yarn-cluster --class
> org.apache.sysml.api.DMLScript ../systemml/lib/systemml-0.10.0-incubating.jar
> -s "print('Hello Apache SystemML')
> Work-around is to add '-exec spark':
> ./bin/spark-submit --master yarn-cluster --class
> org.apache.sysml.api.DMLScript ../systemml/lib/systemml-0.10.0-incubating.jar
> -s "print('Hello Apache SystemML') -exec spark
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
