[ https://issues.apache.org/jira/browse/SPARK-19693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuming Wang updated SPARK-19693: -------------------------------- Comment: was deleted (was: I'm working on.) > SET mapreduce.job.reduces automatically converted to > spark.sql.shuffle.partitions > --------------------------------------------------------------------------------- > > Key: SPARK-19693 > URL: https://issues.apache.org/jira/browse/SPARK-19693 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.1.0, 2.2.0 > Reporter: Yuming Wang > > How to reproduce: > {code} > spark-sql> set spark.sql.shuffle.partitions; > spark.sql.shuffle.partitions 200 > Time taken: 7.118 seconds, Fetched 1 row(s) > spark-sql> set mapred.reduce.tasks=10; > spark.sql.shuffle.partitions 10 > Time taken: 5.968 seconds, Fetched 1 row(s) > spark-sql> set mapreduce.job.reduces=20; > set mapreduce.job.reduces 20 > Time taken: 0.233 seconds, Fetched 1 row(s) > spark-sql> set spark.sql.shuffle.partitions; > spark.sql.shuffle.partitions 10 > Time taken: 0.099 seconds, Fetched 1 row(s) > {code} > The {{SET mapreduce.job.reduces}} should automatically converted to > {{spark.sql.shuffle.partitions}}, it's similar to {{set mapred.reduce.tasks}}. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org