Re: GC issue - Ext Root Scanning

2021-11-16 Thread Kohki Nishio
Since it's a stop the world activity, it's supposed to finish as soon as possible. it's parallel marking phase, so 23 threads (which is picked by JVM) should be fine. I don't have any other apps running on this machine. Spark is working perfectly fine except for one job, The eden size (8G) is also

Apache Spark 3.2.0 | Pyspark | Pycharm Setup

2021-11-16 Thread Anil Kulkarni
Hi Spark community, I am having a hard time setting up my Pycharm to work with pyspark. Can any of you point me to documentation available? Things I have tried till now : 1. Download and Install Apache spark 2. Add pyspark package in pycharm. 3. Add SPARK_HOME. PYTHONPATH, HADOOP_HOME e