Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22103#discussion_r209936266
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
---
@@ -1385,9 +1386,15 @@ object CodeGenerator extends Logging {
try {
val cf = new ClassFile(new ByteArrayInputStream(classBytes))
val stats = cf.methodInfos.asScala.flatMap { method =>
- method.getAttributes().filter(_.getClass.getName ==
codeAttr.getName).map { a =>
+ method.getAttributes().filter(_.getClass eq codeAttr).map { a =>
val byteCodeSize =
codeAttrField.get(a).asInstanceOf[Array[Byte]].length
CodegenMetrics.METRIC_GENERATED_METHOD_BYTECODE_SIZE.update(byteCodeSize)
+
+ if (byteCodeSize > DEFAULT_JVM_HUGE_METHOD_LIMIT) {
+ logInfo("Generated method too long to be JIT compiled: " +
--- End diff --
when we hit this, the JIT will very likely not work and performance may
drop a lot. This even worth a warning...
Since it's just an estimation and Spark SQL can still work, I think info is
fine here.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
