[
https://issues.apache.org/jira/browse/SPARK-25084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25084:
--------------------------
Description:
Test Query:
{code:java}
select * from store_sales distribute by (ss_sold_time_sk, ss_item_sk,
ss_customer_sk, ss_cdemo_sk, ss_addr_sk, ss_promo_sk) limit 1;{code}
Exception:
{code:java}
Caused by: org.codehaus.commons.compiler.CompileException: File
'generated.java', Line 131, Column 67: failed to compile:
org.codehaus.commons.compiler.CompileException: File 'generated.java', Line
131, Column 67: One of ', )' expected instead of '['
at
org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.org$apache$spark$sql$catalyst$expressions$codegen$CodeGenerator$$doCompile(CodeGenerator.scala:1435)
at
org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1497)
at
org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1494)
at
org.spark_project.guava.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)
at
org.spark_project.guava.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)
at
org.spark_project.guava.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342){code}
Wrong Codegen:
{code:java}
/* 131 */ private int computeHashForStruct_1(InternalRow mutableStateArray[0],
int value1) {
/* 132 */
/* 133 */
/* 134 */ if (!mutableStateArray[0].isNullAt(5)) {
/* 135 */
/* 136 */ final int element5 = mutableStateArray[0].getInt(5);
/* 137 */ value1 =
org.apache.spark.unsafe.hash.Murmur3_x86_32.hashInt(element5, value1);
/* 138 */
/* 139 */ }{code}
was:
Test Query:
{code:java}
select * from store_sales distribute by (ss_sold_time_sk, ss_item_sk,
ss_customer_sk, ss_cdemo_sk, ss_addr_sk, ss_promo_sk, ss_ext_list_price,
ss_net_profit) limit 1000;{code}
Wrong Codegen:
{code:java}
/* 146 */ private int computeHashForStruct_0(InternalRow mutableStateArray[0],
int value1) {
/* 147 */
/* 148 */
/* 149 */ if (!mutableStateArray[0].isNullAt(0)) {
/* 150 */
/* 151 */ final int element = mutableStateArray[0].getInt(0);
/* 152 */ value1 = org.apache.spark.unsafe.hash.Murmur3_x86_32.hashInt(element,
value1);
/* 153 */
/* 154 */ }{code}
> "distribute by" on multiple columns may lead to codegen issue
> -------------------------------------------------------------
>
> Key: SPARK-25084
> URL: https://issues.apache.org/jira/browse/SPARK-25084
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.1
> Reporter: yucai
> Priority: Blocker
>
> Test Query:
> {code:java}
> select * from store_sales distribute by (ss_sold_time_sk, ss_item_sk,
> ss_customer_sk, ss_cdemo_sk, ss_addr_sk, ss_promo_sk) limit 1;{code}
> Exception:
> {code:java}
> Caused by: org.codehaus.commons.compiler.CompileException: File
> 'generated.java', Line 131, Column 67: failed to compile:
> org.codehaus.commons.compiler.CompileException: File 'generated.java', Line
> 131, Column 67: One of ', )' expected instead of '['
> at
> org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.org$apache$spark$sql$catalyst$expressions$codegen$CodeGenerator$$doCompile(CodeGenerator.scala:1435)
> at
> org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1497)
> at
> org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1494)
> at
> org.spark_project.guava.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)
> at
> org.spark_project.guava.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)
> at
> org.spark_project.guava.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342){code}
> Wrong Codegen:
> {code:java}
> /* 131 */ private int computeHashForStruct_1(InternalRow
> mutableStateArray[0], int value1) {
> /* 132 */
> /* 133 */
> /* 134 */ if (!mutableStateArray[0].isNullAt(5)) {
> /* 135 */
> /* 136 */ final int element5 = mutableStateArray[0].getInt(5);
> /* 137 */ value1 =
> org.apache.spark.unsafe.hash.Murmur3_x86_32.hashInt(element5, value1);
> /* 138 */
> /* 139 */ }{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
