Re: Debug spark core and streaming programs in scala

>From https://spark.apache.org/docs/latest/monitoring.html#metrics : - JmxSink: Registers metrics for viewing in a JMX console. FYI On Sun, May 15, 2016 at 11:54 PM, Mich Talebzadeh wrote: > Have you tried Spark GUI on 4040. This will show jobs being executed by

Re: Debug spark core and streaming programs in scala

Have you tried Spark GUI on 4040. This will show jobs being executed by executors is each stage and the line of code as well. [image: Inline images 1] Also command line tools like jps and jmonitor HTH Dr Mich Talebzadeh LinkedIn *

Debug spark core and streaming programs in scala

Hi I have scala program consisting of spark core and spark streaming APIs Is there any open source tool that i can use to debug the program for performance reasons? My primary interest is to find the block of codes that would be exeuted on driver and what would go to the executors. Is there JMX