Try "spark.shuffle.io.numConnectionsPerPeer=10"
On Fri, Aug 30, 2019 at 10:22 AM Daniel Zhang wrote:
> Hi, All:
> We are testing the EMR and compare with our on-premise HDP solution. We
> use one application as the test:
> EMR (5.21.1) with Hadoop 2.8.5 + Spark 2.4.3 vs HDP (2.6.3) with Hadoop
Hi, All:
We are testing the EMR and compare with our on-premise HDP solution. We use one
application as the test:
EMR (5.21.1) with Hadoop 2.8.5 + Spark 2.4.3 vs HDP (2.6.3) with Hadoop 2.7.3 +
Spark 2.2.0
The application is very simple, just read Parquet raw file, then do a
