[ https://issues.apache.org/jira/browse/SPARK-16357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zeweichen updated SPARK-16357: ------------------------------ Environment: Apache Spark 1.6.1 BigBench 1.0.1 jdk 1.8.0_73 was: Apache Spark 1.6.1 BigBench 1.0.1 > After enabling Spark shuffle RPC encryption using 3DES, Sparksql query has > poor performance when running in parallel. > --------------------------------------------------------------------------------------------------------------------- > > Key: SPARK-16357 > URL: https://issues.apache.org/jira/browse/SPARK-16357 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.6.1 > Environment: Apache Spark 1.6.1 > BigBench 1.0.1 > jdk 1.8.0_73 > Reporter: zeweichen > Priority: Minor > > During our performance test of Spark shuffle RPC encryption using 3DES on > Sparksql, we found that throughput test of 2 stream lasts 2.68X time than > power test, which is much larger than it should be. Q30 cost 0.5 hour in > stream1, but q30 last 2.5 hours in stream 0 which is much longer than it > should be. This caused that queries in stream1 go ahead much faster than > stream0. So it works like single stream and can not fully use CPU resource. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org