Re: Push-based shuffle SPIP

Hi, I have updated the results of running a large-shuffle application and TPCH queries in this document . The new test results which are linked from the document are here

Manual allocation of Spark executors during Spark application runtime

Hi all, I am wondering if there is a method to manually tune the number of Spark executors when dynamic allocation is enabled on a Spark application. Say for example I have I PySpark shell application running on Kubernetes: ``` $ python > from pyspark.sql import SparkSession > spark = (