[ https://issues.apache.org/jira/browse/SPARK-32186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-32186: --------------------------------- Description: 1. Python Profiler: https://spark.apache.org/docs/2.3.0/api/python/_modules/pyspark/profiler.html, {code} >>> sc._conf.set("spark.python.profile", "true") >>> rdd = sc.parallelize(range(100)).map(str) >>> rdd.count() 100 >>> sc.show_profiles() {code} 2. Python Debugger, e.g.) https://github.com/holdenk/remote-python-debugging-4-spark (?) 3. Monitoring Python Workers: {{top}}, `{{ls}}`, etc. (?) 4. ... was: 1. Python Profiler: https://spark.apache.org/docs/2.3.0/api/python/_modules/pyspark/profiler.html, {code} >>> sc._conf.set("spark.python.profile", "true") >>> rdd = sc.parallelize(range(100)).map(str) >>> rdd.count() 100 >>> sc.show_profiles() {code} 2. Python Debugger, e.g.) https://github.com/holdenk/remote-python-debugging-4-spark 3. Monitoring Python Workers: {{top}}, `{{ls}}`, etc. 4. ... > User Guide - Debugging > ---------------------- > > Key: SPARK-32186 > URL: https://issues.apache.org/jira/browse/SPARK-32186 > Project: Spark > Issue Type: Sub-task > Components: Documentation, PySpark > Affects Versions: 3.1.0 > Reporter: Hyukjin Kwon > Priority: Major > > 1. Python Profiler: > https://spark.apache.org/docs/2.3.0/api/python/_modules/pyspark/profiler.html, > > {code} > >>> sc._conf.set("spark.python.profile", "true") > >>> rdd = sc.parallelize(range(100)).map(str) > >>> rdd.count() > 100 > >>> sc.show_profiles() > {code} > 2. Python Debugger, e.g.) > https://github.com/holdenk/remote-python-debugging-4-spark (?) > 3. Monitoring Python Workers: {{top}}, `{{ls}}`, etc. (?) > 4. ... -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org