[
https://issues.apache.org/jira/browse/SPARK-1853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14051998#comment-14051998
]
Tathagata Das commented on SPARK-1853:
--------------------------------------
If you look at what is shown in any spark program's stage description, it shows
the lines of "user code" that created that stage. For streaming, it shows the
lines of internal code (spark streaming code) instead of the user code that
created it.
So in this case of the screenshot, it should show
4520 - take at Tutorial.scala:34
4521 - map at Tutorial.scala:XXX
...
4513 - reduceByKey at Tutorial.scala:YYY
> Show Streaming application code context (file, line number) in Spark Stages UI
> ------------------------------------------------------------------------------
>
> Key: SPARK-1853
> URL: https://issues.apache.org/jira/browse/SPARK-1853
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.0.0
> Reporter: Tathagata Das
> Assignee: Mubarak Seyed
> Fix For: 1.1.0
>
> Attachments: Screen Shot 2014-07-03 at 2.54.05 PM.png
>
>
> Right now, the code context (file, and line number) shown for streaming jobs
> in stages UI is meaningless as it refers to internal DStream:<random line>
> rather than user application file.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
