[
https://issues.apache.org/jira/browse/SPARK-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050659#comment-14050659
]
Hari Shreedharan commented on SPARK-2345:
-----------------------------------------
Currently, the job (like saveAsTextFile or saveAsHadoopFile) on the DStream
will cause the rdd.save calls to be executed on sparkContext.runJob, which in
turn will call the foreachfunc which is passed to the ForEachDStream. So a case
where this DStream is saved off works fine.
But if you simply do a register and have the foreachfunc do some processing and
custom writes may cause the application to be run locally.
> ForEachDStream should have an option of running the foreachfunc on Spark
> ------------------------------------------------------------------------
>
> Key: SPARK-2345
> URL: https://issues.apache.org/jira/browse/SPARK-2345
> Project: Spark
> Issue Type: Bug
> Reporter: Hari Shreedharan
>
> Today the Job generated simply calls the foreachfunc, but does not run it on
> spark itself using the sparkContext.runJob method.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
