[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17679104#comment-17679104
]
Apache Spark commented on SPARK-40303:
--
User 'wangyum' has created a pull request for this issue:
https://github.com/apache/spark/pull/39671
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
> Attachments: TestApiBenchmark.scala, TestApis.java,
> TestParameters.java
>
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.run()
> }
> {code}
> {noformat}
> Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark count distinct: Best Time(ms) Avg Time(ms)
> Stdev(ms)Rate(M/s) Per Row(ns) Relative
>
> 60 count distinct with codegen 628146 628146
>0 0.0 314072.8 1.0X
> 60 count distinct without codegen147635 147635
>0 0.0 73817.5 4.3X
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17679103#comment-17679103
]
Apache Spark commented on SPARK-40303:
--
User 'wangyum' has created a pull request for this issue:
https://github.com/apache/spark/pull/39671
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
> Attachments: TestApiBenchmark.scala, TestApis.java,
> TestParameters.java
>
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.run()
> }
> {code}
> {noformat}
> Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark count distinct: Best Time(ms) Avg Time(ms)
> Stdev(ms)Rate(M/s) Per Row(ns) Relative
>
> 60 count distinct with codegen 628146 628146
>0 0.0 314072.8 1.0X
> 60 count distinct without codegen147635 147635
>0 0.0 73817.5 4.3X
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619978#comment-17619978
]
Yuming Wang commented on SPARK-40303:
-
How to run benchmark code:
# Download latest spark: https://spark.apache.org/downloads.html
# start spark-shell
{code:sh}
tar -zxf spark-3.3.1-bin-hadoop3.tgz
cd spark-3.3.1-bin-hadoop3
bin/spark-shell --master "local[2]"
{code}
# Run benchmark code:
{code:scala}
val dir = "/tmp/spark/benchmark"
val N = 200
val columns = Range(0, 100).map(i => s"id % $i AS id$i")
spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
Seq(40, 60).foreach { cnt =>
val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
s"count(distinct $c)")
val start = System.currentTimeMillis()
spark.read.parquet(dir).selectExpr(selectExps: _*).collect()
println(cnt + "|" + (System.currentTimeMillis() - start))
}
{code}
# Output:
{noformat}
Before:
40|280273
60|581743
After backport JDK-8159720 to JDK 8:
40|20582
60|49688
{noformat}
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
> Attachments: TestApiBenchmark.scala, TestApis.java,
> TestParameters.java
>
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.run()
> }
> {code}
> {noformat}
> Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark count distinct: Best Time(ms) Avg Time(ms)
> Stdev(ms)Rate(M/s) Per Row(ns) Relative
>
> 60 count distinct with codegen 628146 628146
>0 0.0 314072.8 1.0X
> 60 count distinct without codegen147635 147635
>0 0.0 73817.5 4.3X
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603883#comment-17603883
]
Yang Jie commented on SPARK-40303:
--
I did a simple experiment to compare the following scenarios：
# A method with 127 input parameters
# Encapsulate the input parameters of the above method as a specific type,
including 127 fields, and create a new parameter object before each call
# Encapsulate the input parameters of the above method as a specific type,
including 127 fields, reuse one parameter object and reset + refill parameter
data before each call
I confirmed that the JIT compilation failure mentioned above will occur in the
1&2 test scenarios， the test result as follows:
Java 8
{code:java}
OpenJDK 64-Bit Server VM 1.8.0_345-b01 on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Test sum: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
Use multiple parameters method 35772 36161
550 0.3 3577.2 1.0X
Use TestParameters create new 38701 38783
115 0.3 3870.1 0.9X
Use TestParameters reuse 17986 18125
196 0.6 1798.6 2.0X {code}
Java 11
{code:java}
OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test sum: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
Use multiple parameters method 12253 12286
46 0.8 1225.3 1.0X
Use TestParameters create new 13644 13665
30 0.7 1364.4 0.9X
Use TestParameters reuse 13188 13219
44 0.8 1318.8 0.9X {code}
Java 17
{code:java}
OpenJDK 64-Bit Server VM 17.0.4+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
Test sum: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
Use multiple parameters method 14044 14128
119 0.7 1404.4 1.0X
Use TestParameters create new 16174 16289
162 0.6 1617.4 0.9X
Use TestParameters reuse 15633 15638
8 0.6 1563.3 0.9X {code}
>From the test results, encapsulating and reusing specific parameter types will
>only alleviate the problem when using Java 8, but running programs using Java
>11 or Java 17 seems to be a simpler and more effective way.
So for the current issue, I suggest upgrading the Java runtime environment to
solve the problem.
Uploaded the test program to the attachment
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
> Attachments: TestApiBenchmark.scala, TestApis.java,
> TestParameters.java
>
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
>
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599616#comment-17599616
]
Kris Mok commented on SPARK-40303:
--
Nice findings [~LuciferYang]!
{quote}
After some experiments, I found when the number of parameters exceeds 50, the
performance of the case in the Jira description will significant deterioration.
{quote}
Sounds reasonable. Note that in a STD compilation, the only things that need to
be live at the method entry are the method parameters (both implicit ones like
{{this}}, and explicit ones); however, for an OSR compilation, it would be all
of the parameters/local variables that are live at the loop entry point, so in
this case both the {{doConsume}} parameters and the local variables contribute
to the problem.
Just FYI I have an old write on PrintCompilation and OSR here:
https://gist.github.com/rednaxelafx/1165804#file-notes-md
(Gee, just realized that was from 11 years ago...)
{quote}
maybe try to make the input parameters of the `doConsume` method fixed length
will help, such as using a List or Array
{quote}
Welp, hoisting the parameters into an Arguments object is rather common in
"code splitting" in code generators. Since we're already doing codegen, it's
possible to generate tailor-made Arguments classes to retain the type
information. Using List/Array would require extra boxing for primitive types
and it's less ideal.
(An array-based box is already used in Spark SQL's codegen in the form of the
{{references}} array. Indeed the type info is lost on the interface level and
you'd have to do a cast when you get data out of it. It's still usable though.)
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.run()
> }
> {code}
> {noformat}
> Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark count distinct: Best Time(ms) Avg Time(ms)
> Stdev(ms)Rate(M/s) Per Row(ns) Relative
>
> 60 count distinct with codegen 628146 628146
>0 0.0 314072.8 1.0X
> 60 count distinct without codegen147635 147635
>0 0.0 73817.5 4.3X
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599332#comment-17599332
]
Yang Jie commented on SPARK-40303:
--
Thank you for your explain, very clear [~rednaxelafx] .
I guess this may related to the number of `doConsume` method parameters. After
some experiments, I found when the number of parameters exceeds 50, the
performance of the case in the Jira description will significant deterioration.
If need change the code, maybe try to make the input parameters of the
`doConsume` method fixed length will help, such as using a List or Array, but
this will lose the parameter type, and I am not sure whether it is feasible in
practice
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.run()
> }
> {code}
> {noformat}
> Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark count distinct: Best Time(ms) Avg Time(ms)
> Stdev(ms)Rate(M/s) Per Row(ns) Relative
>
> 60 count distinct with codegen 628146 628146
>0 0.0 314072.8 1.0X
> 60 count distinct without codegen147635 147635
>0 0.0 73817.5 4.3X
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599314#comment-17599314
]
Yang Jie commented on SPARK-40303:
--
do some investigate on linux with spark `local[2]` module, maybe help:
# Upgrading to Java 17 will significantly improve performance (about 10times
over Java 8: From 20 minutes to 1.9 minutes)
# When using Java 8, adding `spark.sql.CodeGenerator.validParamLength = 48` is
helpful for performance improvement (From 20 minutes to 3.1 minutes)
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.run()
> }
> {code}
> {noformat}
> Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark count distinct: Best Time(ms) Avg Time(ms)
> Stdev(ms)Rate(M/s) Per Row(ns) Relative
>
> 60 count distinct with codegen 628146 628146
>0 0.0 314072.8 1.0X
> 60 count distinct without codegen147635 147635
>0 0.0 73817.5 4.3X
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599302#comment-17599302
]
Kris Mok commented on SPARK-40303:
--
Interesting, thanks for posting your findings, [~LuciferYang]!
In the JDK8 case, the message:
{code}
COMPILE SKIPPED: unsupported calling sequence (not retryable)
{code}
Indicates that the compiler deemed this method should not be attempted to
compile again on any tier of compilation, and because this is an OSR
compilation (i.e. loop compilation), this will mark the method as "never try to
perform OSR compilation again on all tiers". On JDK17 this is relaxed a bit and
it'll try tier 1.
Note that OSR compilation and STD compilation (normal compilation) are separate
things. Marking one as not compilation doesn't affect the other. I haven't
checked the IR yet but if I had to guess, the reason why this method is
recorded as not OSR compilable is because there are too many live local
variables at the OSR (loop) entry point, beyond what the HotSpot JVM could
support.
So only OSR is affected, STD should still be fine.
In general, the tiered compilation system in the HotSpot JVM works as:
- tier 0: interpreter
- tier 1: C1 no profiling (best code quality for C1, same as HotSpot Client
VM's C1)
- tier 2: C1 basic profiling (lower performance than tier 1, only used when the
target level is tier 3 but the C1 queue is too long)
- tier 3: C1 full profiling (lower performance than tier 2, for collecting
profile to perform tier 4 profile-guided optimization)
- tier 4: C2
As such, tiers 2 and 3 are only useful if tier 4 is available. So if a method
is recorded as "not compilable on tier 4", the only realistic option left is to
try tier 1.
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.run()
> }
> {code}
> {noformat}
> Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark count distinct: Best Time(ms) Avg Time(ms)
> Stdev(ms)Rate(M/s) Per Row(ns) Relative
>
> 60 count distinct with codegen 628146 628146
>0 0.0 314072.8 1.0X
> 60 count distinct without codegen147635 147635
>0 0.0 73817.5 4.3X
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599263#comment-17599263
]
Yang Jie commented on SPARK-40303:
--
If you run with Java 17, the performance gap will be smaller. The compilation
logs of `hashAgg_doConsume_0` and `hashAgg_doAggregateWithKeys_0` as follows:
{code:java}
102158 22568 3
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$
(2051 bytes)
102180 22606 4
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$
(2051 bytes)
102228 22606 4
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$
(2051 bytes) COMPILE SKIPPED: unsupported incoming calling sequence (retry
at different tier)
102228 22619 1
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$
(2051 bytes)
102240 22568 3
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$
(2051 bytes) made not entrant
218296 24067 3
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2::hashAgg_doConsume_0$
(2052 bytes)
218463 22568 3
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$
(2051 bytes) made zombie {code}
{code:java}
105832 22708 % 3
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
@ 38 (2712 bytes)
105955 22709 3
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
(2712 bytes)
108247 22741 % 4
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
@ 38 (2712 bytes)
108484 22741 % 4
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
@ 38 (2712 bytes) COMPILE SKIPPED: unsupported calling sequence (retry at
different tier)
108727 22708 % 3
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
@ 38 (2712 bytes) made not entrant
108727 22743 % 1
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
@ 38 (2712 bytes)
218463 22708 % 3
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
@ 38 (2712 bytes) made zombie {code}
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.run()
> }
> {code}
> {noformat}
> Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark count distinct: Best Time(ms) Avg Time(ms)
> Stdev(ms)Rate(M/s) Per Row(ns) Relative
>
> 60 count distinct with codegen 628146 628146
>0 0.0 314072.8 1.0X
> 60
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599262#comment-17599262
]
Yang Jie commented on SPARK-40303:
--
{code:java}
64265 21820 3
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$
(2051 bytes)
64308 21862 4
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$
(2051 bytes)
64350 21862 4
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$
(2051 bytes) COMPILE SKIPPED: unsupported incoming calling sequence (not
retryable) {code}
{code:java}
103941 22065 % 3
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
@ 38 (2712 bytes)
104602 22067 3
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
(2712 bytes)
135979 22141 % 4
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
@ 38 (2712 bytes)
137245 22141 % 4
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
@ 38 (2712 bytes) COMPILE SKIPPED: unsupported calling sequence (not
retryable) {code}
The compilation relevant logs are as above when Java 8 is used
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.run()
> }
> {code}
> {noformat}
> Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark count distinct: Best Time(ms) Avg Time(ms)
> Stdev(ms)Rate(M/s) Per Row(ns) Relative
>
> 60 count distinct with codegen 628146 628146
>0 0.0 314072.8 1.0X
> 60 count distinct without codegen147635 147635
>0 0.0 73817.5 4.3X
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599254#comment-17599254
]
Yang Jie commented on SPARK-40303:
--
Run use Java 8 with `-XX:+PrintCompilation`, I found the following logs:
{code:java}
64350 21862 4
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doConsume_0$
(2051 bytes) COMPILE SKIPPED: unsupported incoming calling sequence (not
retryable)
137245 22141 % 4
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1::hashAgg_doAggregateWithKeys_0$
@ 38 (2712 bytes) COMPILE SKIPPED: unsupported calling sequence (not
retryable) {code}
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.run()
> }
> {code}
> {noformat}
> Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark count distinct: Best Time(ms) Avg Time(ms)
> Stdev(ms)Rate(M/s) Per Row(ns) Relative
>
> 60 count distinct with codegen 628146 628146
>0 0.0 314072.8 1.0X
> 60 count distinct without codegen147635 147635
>0 0.0 73817.5 4.3X
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40303) The performance will be worse after codegen
[
https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17598831#comment-17598831
]
Yuming Wang commented on SPARK-40303:
-
cc [~cloud_fan] [~joshrosen] [~rednaxelafx]
> The performance will be worse after codegen
> ---
>
> Key: SPARK-40303
> URL: https://issues.apache.org/jira/browse/SPARK-40303
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Yuming Wang
>Priority: Major
>
> {code:scala}
> import org.apache.spark.benchmark.Benchmark
> val dir = "/tmp/spark/benchmark"
> val N = 200
> val columns = Range(0, 100).map(i => s"id % $i AS id$i")
> spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)
> // Seq(1, 2, 5, 10, 15, 25, 40, 60, 100)
> Seq(60).foreach{ cnt =>
> val selectExps = columns.take(cnt).map(_.split(" ").last).map(c =>
> s"count(distinct $c)")
> val benchmark = new Benchmark("Benchmark count distinct", N, minNumIters =
> 1)
> benchmark.addCase(s"$cnt count distinct with codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "true",
> "spark.sql.codegen.factoryMode" -> "FALLBACK") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.addCase(s"$cnt count distinct without codegen") { _ =>
> withSQLConf(
> "spark.sql.codegen.wholeStage" -> "false",
> "spark.sql.codegen.factoryMode" -> "NO_CODEGEN") {
> spark.read.parquet(dir).selectExpr(selectExps:
> _*).write.format("noop").mode("Overwrite").save()
> }
> }
> benchmark.run()
> }
> {code}
> {noformat}
> Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark count distinct: Best Time(ms) Avg Time(ms)
> Stdev(ms)Rate(M/s) Per Row(ns) Relative
>
> 60 count distinct with codegen 628146 628146
>0 0.0 314072.8 1.0X
> 60 count distinct without codegen147635 147635
>0 0.0 73817.5 4.3X
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
