Thomas Graves created SPARK-33406: ------------------------------------- Summary: k8s pyspark shell doesn't honor pyspark memory setting Key: SPARK-33406 URL: https://issues.apache.org/jira/browse/SPARK-33406 Project: Spark Issue Type: Bug Components: Kubernetes Affects Versions: 3.1.0 Reporter: Thomas Graves
When running the interactive pyspark shell in client mode on k8s, it doesn't honor thespark.executor.pyspark.memory memory setting when requesting containers from k8s. If you run it in cluster mode and specify a python script then it does work. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org