[ https://issues.apache.org/jira/browse/SPARK-32186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-32186. ---------------------------------- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29639 [https://github.com/apache/spark/pull/29639] > Development - Debugging > ----------------------- > > Key: SPARK-32186 > URL: https://issues.apache.org/jira/browse/SPARK-32186 > Project: Spark > Issue Type: Sub-task > Components: Documentation, PySpark > Affects Versions: 3.1.0 > Reporter: Hyukjin Kwon > Assignee: Hyukjin Kwon > Priority: Major > Fix For: 3.1.0 > > > 1. Python Profiler: > https://spark.apache.org/docs/2.3.0/api/python/_modules/pyspark/profiler.html, > > {code} > >>> sc._conf.set("spark.python.profile", "true") > >>> rdd = sc.parallelize(range(100)).map(str) > >>> rdd.count() > 100 > >>> sc.show_profiles() > {code} > 2. Python Debugger, e.g.) > https://www.jetbrains.com/help/pycharm/remote-debugging-with-product.html#remote-interpreter > \(?\) > 3. Monitoring Python Workers: {{top}}, `{{ls}}`, etc. \(?\) > 4. PyCharm setup \(?\) (at https://issues.apache.org/jira/browse/SPARK-32189) -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org