[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679104#comment-17679104 ] Apache Spark commented on SPARK-40303: -- User 'wangyum' has created a pull request for this issue: https://github.com/apache/spark/pull/39671 > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > Attachments: TestApiBenchmark.scala, TestApis.java, > TestParameters.java > > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.run() > } > {code} > {noformat} > Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 > Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > Benchmark count distinct: Best Time(ms) Avg Time(ms) > Stdev(ms)Rate(M/s) Per Row(ns) Relative > > 60 count distinct with codegen 628146 628146 >0 0.0 314072.8 1.0X > 60 count distinct without codegen147635 147635 >0 0.0 73817.5 4.3X > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679103#comment-17679103 ] Apache Spark commented on SPARK-40303: -- User 'wangyum' has created a pull request for this issue: https://github.com/apache/spark/pull/39671 > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > Attachments: TestApiBenchmark.scala, TestApis.java, > TestParameters.java > > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.run() > } > {code} > {noformat} > Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 > Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > Benchmark count distinct: Best Time(ms) Avg Time(ms) > Stdev(ms)Rate(M/s) Per Row(ns) Relative > > 60 count distinct with codegen 628146 628146 >0 0.0 314072.8 1.0X > 60 count distinct without codegen147635 147635 >0 0.0 73817.5 4.3X > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619978#comment-17619978 ] Yuming Wang commented on SPARK-40303: - How to run benchmark code: # Download latest spark: https://spark.apache.org/downloads.html # start spark-shell {code:sh} tar -zxf spark-3.3.1-bin-hadoop3.tgz cd spark-3.3.1-bin-hadoop3 bin/spark-shell --master "local[2]" {code} # Run benchmark code: {code:scala} val dir = "/tmp/spark/benchmark" val N = 200 val columns = Range(0, 100).map(i => s"id % $i AS id$i") spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) Seq(40, 60).foreach { cnt => val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => s"count(distinct $c)") val start = System.currentTimeMillis() spark.read.parquet(dir).selectExpr(selectExps: _*).collect() println(cnt + "|" + (System.currentTimeMillis() - start)) } {code} # Output: {noformat} Before: 40|280273 60|581743 After backport JDK-8159720 to JDK 8: 40|20582 60|49688 {noformat} > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > Attachments: TestApiBenchmark.scala, TestApis.java, > TestParameters.java > > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.run() > } > {code} > {noformat} > Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 > Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > Benchmark count distinct: Best Time(ms) Avg Time(ms) > Stdev(ms)Rate(M/s) Per Row(ns) Relative > > 60 count distinct with codegen 628146 628146 >0 0.0 314072.8 1.0X > 60 count distinct without codegen147635 147635 >0 0.0 73817.5 4.3X > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603883#comment-17603883 ] Yang Jie commented on SPARK-40303: -- I did a simple experiment to compare the following scenarios: # A method with 127 input parameters # Encapsulate the input parameters of the above method as a specific type, including 127 fields, and create a new parameter object before each call # Encapsulate the input parameters of the above method as a specific type, including 127 fields, reuse one parameter object and reset + refill parameter data before each call I confirmed that the JIT compilation failure mentioned above will occur in the 1&2 test scenarios, the test result as follows: Java 8 {code:java} OpenJDK 64-Bit Server VM 1.8.0_345-b01 on Linux 5.15.0-1019-azure Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz Test sum: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative Use multiple parameters method 35772 36161 550 0.3 3577.2 1.0X Use TestParameters create new 38701 38783 115 0.3 3870.1 0.9X Use TestParameters reuse 17986 18125 196 0.6 1798.6 2.0X {code} Java 11 {code:java} OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz Test sum: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative Use multiple parameters method 12253 12286 46 0.8 1225.3 1.0X Use TestParameters create new 13644 13665 30 0.7 1364.4 0.9X Use TestParameters reuse 13188 13219 44 0.8 1318.8 0.9X {code} Java 17 {code:java} OpenJDK 64-Bit Server VM 17.0.4+8-LTS on Linux 5.15.0-1019-azure Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz Test sum: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative Use multiple parameters method 14044 14128 119 0.7 1404.4 1.0X Use TestParameters create new 16174 16289 162 0.6 1617.4 0.9X Use TestParameters reuse 15633 15638 8 0.6 1563.3 0.9X {code} >From the test results, encapsulating and reusing specific parameter types will >only alleviate the problem when using Java 8, but running programs using Java >11 or Java 17 seems to be a simpler and more effective way. So for the current issue, I suggest upgrading the Java runtime environment to solve the problem. Uploaded the test program to the attachment > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > Attachments: TestApiBenchmark.scala, TestApis.java, > TestParameters.java > > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mod
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599616#comment-17599616 ] Kris Mok commented on SPARK-40303: -- Nice findings [~LuciferYang]! {quote} After some experiments, I found when the number of parameters exceeds 50, the performance of the case in the Jira description will significant deterioration. {quote} Sounds reasonable. Note that in a STD compilation, the only things that need to be live at the method entry are the method parameters (both implicit ones like {{this}}, and explicit ones); however, for an OSR compilation, it would be all of the parameters/local variables that are live at the loop entry point, so in this case both the {{doConsume}} parameters and the local variables contribute to the problem. Just FYI I have an old write on PrintCompilation and OSR here: https://gist.github.com/rednaxelafx/1165804#file-notes-md (Gee, just realized that was from 11 years ago...) {quote} maybe try to make the input parameters of the `doConsume` method fixed length will help, such as using a List or Array {quote} Welp, hoisting the parameters into an Arguments object is rather common in "code splitting" in code generators. Since we're already doing codegen, it's possible to generate tailor-made Arguments classes to retain the type information. Using List/Array would require extra boxing for primitive types and it's less ideal. (An array-based box is already used in Spark SQL's codegen in the form of the {{references}} array. Indeed the type info is lost on the interface level and you'd have to do a cast when you get data out of it. It's still usable though.) > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.run() > } > {code} > {noformat} > Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 > Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > Benchmark count distinct: Best Time(ms) Avg Time(ms) > Stdev(ms)Rate(M/s) Per Row(ns) Relative > > 60 count distinct with codegen 628146 628146 >0 0.0 314072.8 1.0X > 60 count distinct without codegen147635 147635 >0 0.0 73817.5 4.3X > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599332#comment-17599332 ] Yang Jie commented on SPARK-40303: -- Thank you for your explain, very clear [~rednaxelafx] . I guess this may related to the number of `doConsume` method parameters. After some experiments, I found when the number of parameters exceeds 50, the performance of the case in the Jira description will significant deterioration. If need change the code, maybe try to make the input parameters of the `doConsume` method fixed length will help, such as using a List or Array, but this will lose the parameter type, and I am not sure whether it is feasible in practice > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.run() > } > {code} > {noformat} > Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 > Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > Benchmark count distinct: Best Time(ms) Avg Time(ms) > Stdev(ms)Rate(M/s) Per Row(ns) Relative > > 60 count distinct with codegen 628146 628146 >0 0.0 314072.8 1.0X > 60 count distinct without codegen147635 147635 >0 0.0 73817.5 4.3X > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599314#comment-17599314 ] Yang Jie commented on SPARK-40303: -- do some investigate on linux with spark `local[2]` module, maybe help: # Upgrading to Java 17 will significantly improve performance (about 10times over Java 8: From 20 minutes to 1.9 minutes) # When using Java 8, adding `spark.sql.CodeGenerator.validParamLength = 48` is helpful for performance improvement (From 20 minutes to 3.1 minutes) > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.run() > } > {code} > {noformat} > Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 > Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > Benchmark count distinct: Best Time(ms) Avg Time(ms) > Stdev(ms)Rate(M/s) Per Row(ns) Relative > > 60 count distinct with codegen 628146 628146 >0 0.0 314072.8 1.0X > 60 count distinct without codegen147635 147635 >0 0.0 73817.5 4.3X > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599302#comment-17599302 ] Kris Mok commented on SPARK-40303: -- Interesting, thanks for posting your findings, [~LuciferYang]! In the JDK8 case, the message: {code} COMPILE SKIPPED: unsupported calling sequence (not retryable) {code} Indicates that the compiler deemed this method should not be attempted to compile again on any tier of compilation, and because this is an OSR compilation (i.e. loop compilation), this will mark the method as "never try to perform OSR compilation again on all tiers". On JDK17 this is relaxed a bit and it'll try tier 1. Note that OSR compilation and STD compilation (normal compilation) are separate things. Marking one as not compilation doesn't affect the other. I haven't checked the IR yet but if I had to guess, the reason why this method is recorded as not OSR compilable is because there are too many live local variables at the OSR (loop) entry point, beyond what the HotSpot JVM could support. So only OSR is affected, STD should still be fine. In general, the tiered compilation system in the HotSpot JVM works as: - tier 0: interpreter - tier 1: C1 no profiling (best code quality for C1, same as HotSpot Client VM's C1) - tier 2: C1 basic profiling (lower performance than tier 1, only used when the target level is tier 3 but the C1 queue is too long) - tier 3: C1 full profiling (lower performance than tier 2, for collecting profile to perform tier 4 profile-guided optimization) - tier 4: C2 As such, tiers 2 and 3 are only useful if tier 4 is available. So if a method is recorded as "not compilable on tier 4", the only realistic option left is to try tier 1. > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.run() > } > {code} > {noformat} > Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 > Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > Benchmark count distinct: Best Time(ms) Avg Time(ms) > Stdev(ms)Rate(M/s) Per Row(ns) Relative > > 60 count distinct with codegen 628146 628146 >0 0.0 314072.8 1.0X > 60 count distinct without codegen147635 147635 >0 0.0 73817.5 4.3X > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599263#comment-17599263 ] Yang Jie commented on SPARK-40303: -- If you run with Java 17, the performance gap will be smaller. The compilation logs of `hashAgg_doConsume_0` and `hashAgg_doAggregateWithKeys_0` as follows: {code:java} 102158 22568 3 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$ (2051 bytes) 102180 22606 4 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$ (2051 bytes) 102228 22606 4 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$ (2051 bytes) COMPILE SKIPPED: unsupported incoming calling sequence (retry at different tier) 102228 22619 1 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$ (2051 bytes) 102240 22568 3 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$ (2051 bytes) made not entrant 218296 24067 3 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2::hashAgg_doConsume_0$ (2052 bytes) 218463 22568 3 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$ (2051 bytes) made zombie {code} {code:java} 105832 22708 % 3 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ @ 38 (2712 bytes) 105955 22709 3 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ (2712 bytes) 108247 22741 % 4 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ @ 38 (2712 bytes) 108484 22741 % 4 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ @ 38 (2712 bytes) COMPILE SKIPPED: unsupported calling sequence (retry at different tier) 108727 22708 % 3 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ @ 38 (2712 bytes) made not entrant 108727 22743 % 1 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ @ 38 (2712 bytes) 218463 22708 % 3 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ @ 38 (2712 bytes) made zombie {code} > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.run() > } > {code} > {noformat} > Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 > Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > Benchmark count distinct: Best Time(ms) Avg Time(ms) > Stdev(ms)Rate(M/s) Per Row(ns) Relative > > 60 count distinct with codegen 628146 628146 >0 0.0 314072.8
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599262#comment-17599262 ] Yang Jie commented on SPARK-40303: -- {code:java} 64265 21820 3 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$ (2051 bytes) 64308 21862 4 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$ (2051 bytes) 64350 21862 4 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$ (2051 bytes) COMPILE SKIPPED: unsupported incoming calling sequence (not retryable) {code} {code:java} 103941 22065 % 3 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ @ 38 (2712 bytes) 104602 22067 3 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ (2712 bytes) 135979 22141 % 4 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ @ 38 (2712 bytes) 137245 22141 % 4 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ @ 38 (2712 bytes) COMPILE SKIPPED: unsupported calling sequence (not retryable) {code} The compilation relevant logs are as above when Java 8 is used > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.run() > } > {code} > {noformat} > Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 > Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > Benchmark count distinct: Best Time(ms) Avg Time(ms) > Stdev(ms)Rate(M/s) Per Row(ns) Relative > > 60 count distinct with codegen 628146 628146 >0 0.0 314072.8 1.0X > 60 count distinct without codegen147635 147635 >0 0.0 73817.5 4.3X > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599254#comment-17599254 ] Yang Jie commented on SPARK-40303: -- Run use Java 8 with `-XX:+PrintCompilation`, I found the following logs: {code:java} 64350 21862 4 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$ (2051 bytes) COMPILE SKIPPED: unsupported incoming calling sequence (not retryable) 137245 22141 % 4 org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$ @ 38 (2712 bytes) COMPILE SKIPPED: unsupported calling sequence (not retryable) {code} > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.run() > } > {code} > {noformat} > Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 > Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > Benchmark count distinct: Best Time(ms) Avg Time(ms) > Stdev(ms)Rate(M/s) Per Row(ns) Relative > > 60 count distinct with codegen 628146 628146 >0 0.0 314072.8 1.0X > 60 count distinct without codegen147635 147635 >0 0.0 73817.5 4.3X > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598831#comment-17598831 ] Yuming Wang commented on SPARK-40303: - cc [~cloud_fan] [~joshrosen] [~rednaxelafx] > The performance will be worse after codegen > --- > > Key: SPARK-40303 > URL: https://issues.apache.org/jira/browse/SPARK-40303 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.4.0 >Reporter: Yuming Wang >Priority: Major > > {code:scala} > import org.apache.spark.benchmark.Benchmark > val dir = "/tmp/spark/benchmark" > val N = 200 > val columns = Range(0, 100).map(i => s"id % $i AS id$i") > spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir) > // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100) > Seq(60).foreach{ cnt => > val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => > s"count(distinct $c)") > val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters = > 1) > benchmark.addCase(s"$cnt count distinct with codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "true", > "spark.sql.codegen.factoryMode" -> "FALLBACK") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.addCase(s"$cnt count distinct without codegen") { _ => > withSQLConf( > "spark.sql.codegen.wholeStage" -> "false", > "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") { > spark.read.parquet(dir).selectExpr(selectExps: > _*).write.format("noop").mode("Overwrite").save() > } > } > benchmark.run() > } > {code} > {noformat} > Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 > Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > Benchmark count distinct: Best Time(ms) Avg Time(ms) > Stdev(ms)Rate(M/s) Per Row(ns) Relative > > 60 count distinct with codegen 628146 628146 >0 0.0 314072.8 1.0X > 60 count distinct without codegen147635 147635 >0 0.0 73817.5 4.3X > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org