[ https://issues.apache.org/jira/browse/SPARK-12803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097933#comment-15097933 ]
Sean Owen commented on SPARK-12803: ----------------------------------- You wouldn't know which task executes on which executor anyway ahead of time, so being able to pick one executor doesn't do a lot. You would connect a profile to any executor and watch it execute tasks, and presumably one is about as good as the next for that purpose. > Consider adding ability to profile specific instances of executors in spark > --------------------------------------------------------------------------- > > Key: SPARK-12803 > URL: https://issues.apache.org/jira/browse/SPARK-12803 > Project: Spark > Issue Type: Improvement > Components: Java API > Reporter: Rajesh Balamohan > > It would be useful to profile specific instances of executors as opposed to > adding profiler details to all executors via > "spark.executor.extraJavaOptions". > Setting the number of executors to just 1 and profiling wouldn't be much > useful (in some cases, most of the time with single executor mode would be > spent in terms of reading data from remote node). At the same time, setting > profiling option to all executors could just create too many number of > snapshots; making it harder to analyze. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org