Wan Kun created SPARK-52103:
-------------------------------
Summary: Fallback complex expression whole-stage codegen
Key: SPARK-52103
URL: https://issues.apache.org/jira/browse/SPARK-52103
Project: Spark
Issue Type: Task
Components: SQL
Affects Versions: 4.0.0
Reporter: Wan Kun
The whole-stage code of a complex expression may too long to be JIT compiled.
Add a config to determines whether fallback the complex expressions codegen.
Test query
{code:sql}
SELECT vv
FROM
(
SELECT vv, case vv
when '1' then '1'
when '2' then '2'
when '3' then '3'
when '4' then '4'
when '5' then '5'
when '6' then '6'
when '7' then '7'
when '8' then '8'
when '9' then '9'
when '10' then '10'
when '11' then '11'
when '12' then '12'
when '13' then '13'
when '14' then '14'
when '15' then '15'
when '16' then '16'
when '17' then '17'
when '18' then '18'
when '19' then '19'
when '20' then '20'
when '21' then '21'
when '22' then '22'
when '23' then '23'
when '24' then '24'
when '25' then '25'
when '26' then '26'
when '27' then '27'
when '28' then '28'
when '29' then '29'
when '30' then '30'
when '31' then '31'
when '32' then '32'
else ''
end as cv
FROM (
SELECT regexp_replace(trim(lower(
get_json_object(concat(v,'}'),'$$.s'))),'\\n','') AS vv
FROM values('a') as t(v)
) tmp
) t2
WHERE length(cv) > 0
AND cv not LIKE '%xxx%'
{code}
JIT error message:
25/05/13 13:19:35 INFO CodeGenerator: Generated method too long to be JIT
compiled:
MessageWithContext(org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.generate_doConsume_0$
,{})MessageWithContext(is 21798 bytes,{})
25/05/13 13:19:35 INFO CodeGenerator: Code generated in 1658.222814 ms
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
