[jira] [Commented] (SPARK-19512) codegen for compare structs fails

2018-05-08 Thread Takeshi Yamamuro (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468417#comment-16468417
 ] 

Takeshi Yamamuro commented on SPARK-19512:
--

can you put a simple query to reproduce here?

> codegen for compare structs fails
> -
>
> Key: SPARK-19512
> URL: https://issues.apache.org/jira/browse/SPARK-19512
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Bogdan Raducanu
>Assignee: Bogdan Raducanu
>Priority: Major
> Fix For: 2.1.1, 2.2.0
>
>
> This (1 struct field)
> {code:java|title=1 struct field}
> spark.range(10)
>   .selectExpr("named_struct('a', id) as col1", "named_struct('a', id+2) 
> as col2")
>   .filter("col1 = col2").count
> {code}
> fails with
> {code}
> [info]   Cause: java.util.concurrent.ExecutionException: java.lang.Exception: 
> failed to compile: org.codehaus.commons.compiler.CompileException: File 
> 'generated.java', Line 144, Column 32: Expression "range_value" is not an 
> rvalue
> {code}
> This (2 struct fields)
> {code:java|title=2 struct fields}
> spark.range(10)
> .selectExpr("named_struct('a', id, 'b', id) as col1", 
> "named_struct('a',id+2, 'b',id+2) as col2")
> .filter($"col1" === $"col2").count
> {code}
> fails with 
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: 1
>   at 
> scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
>   at scala.collection.immutable.List.apply(List.scala:84)
>   at 
> org.apache.spark.sql.catalyst.expressions.BoundReference.doGenCode(BoundAttribute.scala:64)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-19512) codegen for compare structs fails

2018-05-08 Thread howie yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467403#comment-16467403
 ] 

howie yu commented on SPARK-19512:
--

Hi


I think this  is similar issue, may not the same.

I have create a  project can re-produce this error

[https://github.com/howie/spark-codegen-error]

 

 

> codegen for compare structs fails
> -
>
> Key: SPARK-19512
> URL: https://issues.apache.org/jira/browse/SPARK-19512
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Bogdan Raducanu
>Assignee: Bogdan Raducanu
>Priority: Major
> Fix For: 2.1.1, 2.2.0
>
>
> This (1 struct field)
> {code:java|title=1 struct field}
> spark.range(10)
>   .selectExpr("named_struct('a', id) as col1", "named_struct('a', id+2) 
> as col2")
>   .filter("col1 = col2").count
> {code}
> fails with
> {code}
> [info]   Cause: java.util.concurrent.ExecutionException: java.lang.Exception: 
> failed to compile: org.codehaus.commons.compiler.CompileException: File 
> 'generated.java', Line 144, Column 32: Expression "range_value" is not an 
> rvalue
> {code}
> This (2 struct fields)
> {code:java|title=2 struct fields}
> spark.range(10)
> .selectExpr("named_struct('a', id, 'b', id) as col1", 
> "named_struct('a',id+2, 'b',id+2) as col2")
> .filter($"col1" === $"col2").count
> {code}
> fails with 
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: 1
>   at 
> scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
>   at scala.collection.immutable.List.apply(List.scala:84)
>   at 
> org.apache.spark.sql.catalyst.expressions.BoundReference.doGenCode(BoundAttribute.scala:64)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-19512) codegen for compare structs fails

2018-05-08 Thread Takeshi Yamamuro (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467043#comment-16467043
 ] 

Takeshi Yamamuro commented on SPARK-19512:
--

I checked in the released v2.3.0 and the master though, I didn't hit this issue.

> codegen for compare structs fails
> -
>
> Key: SPARK-19512
> URL: https://issues.apache.org/jira/browse/SPARK-19512
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Bogdan Raducanu
>Assignee: Bogdan Raducanu
>Priority: Major
> Fix For: 2.1.1, 2.2.0
>
>
> This (1 struct field)
> {code:java|title=1 struct field}
> spark.range(10)
>   .selectExpr("named_struct('a', id) as col1", "named_struct('a', id+2) 
> as col2")
>   .filter("col1 = col2").count
> {code}
> fails with
> {code}
> [info]   Cause: java.util.concurrent.ExecutionException: java.lang.Exception: 
> failed to compile: org.codehaus.commons.compiler.CompileException: File 
> 'generated.java', Line 144, Column 32: Expression "range_value" is not an 
> rvalue
> {code}
> This (2 struct fields)
> {code:java|title=2 struct fields}
> spark.range(10)
> .selectExpr("named_struct('a', id, 'b', id) as col1", 
> "named_struct('a',id+2, 'b',id+2) as col2")
> .filter($"col1" === $"col2").count
> {code}
> fails with 
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: 1
>   at 
> scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
>   at scala.collection.immutable.List.apply(List.scala:84)
>   at 
> org.apache.spark.sql.catalyst.expressions.BoundReference.doGenCode(BoundAttribute.scala:64)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-19512) codegen for compare structs fails

2018-05-06 Thread howie yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465386#comment-16465386
 ] 

howie yu commented on SPARK-19512:
--

Hi 
I still have this issue in 2.3.0

[https://stackoverflow.com/questions/50185228/spark-left-outer-join-cause-codegenerator-error]

 

> codegen for compare structs fails
> -
>
> Key: SPARK-19512
> URL: https://issues.apache.org/jira/browse/SPARK-19512
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Bogdan Raducanu
>Assignee: Bogdan Raducanu
>Priority: Major
> Fix For: 2.1.1, 2.2.0
>
>
> This (1 struct field)
> {code:java|title=1 struct field}
> spark.range(10)
>   .selectExpr("named_struct('a', id) as col1", "named_struct('a', id+2) 
> as col2")
>   .filter("col1 = col2").count
> {code}
> fails with
> {code}
> [info]   Cause: java.util.concurrent.ExecutionException: java.lang.Exception: 
> failed to compile: org.codehaus.commons.compiler.CompileException: File 
> 'generated.java', Line 144, Column 32: Expression "range_value" is not an 
> rvalue
> {code}
> This (2 struct fields)
> {code:java|title=2 struct fields}
> spark.range(10)
> .selectExpr("named_struct('a', id, 'b', id) as col1", 
> "named_struct('a',id+2, 'b',id+2) as col2")
> .filter($"col1" === $"col2").count
> {code}
> fails with 
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: 1
>   at 
> scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
>   at scala.collection.immutable.List.apply(List.scala:84)
>   at 
> org.apache.spark.sql.catalyst.expressions.BoundReference.doGenCode(BoundAttribute.scala:64)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-19512) codegen for compare structs fails

2017-02-09 Thread Apache Spark (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860053#comment-15860053
 ] 

Apache Spark commented on SPARK-19512:
--

User 'bogdanrdc' has created a pull request for this issue:
https://github.com/apache/spark/pull/16875

> codegen for compare structs fails
> -
>
> Key: SPARK-19512
> URL: https://issues.apache.org/jira/browse/SPARK-19512
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Bogdan Raducanu
>Assignee: Bogdan Raducanu
> Fix For: 2.2.0
>
>
> This (1 struct field)
> {code:java|title=1 struct field}
> spark.range(10)
>   .selectExpr("named_struct('a', id) as col1", "named_struct('a', id+2) 
> as col2")
>   .filter("col1 = col2").count
> {code}
> fails with
> {code}
> [info]   Cause: java.util.concurrent.ExecutionException: java.lang.Exception: 
> failed to compile: org.codehaus.commons.compiler.CompileException: File 
> 'generated.java', Line 144, Column 32: Expression "range_value" is not an 
> rvalue
> {code}
> This (2 struct fields)
> {code:java|title=2 struct fields}
> spark.range(10)
> .selectExpr("named_struct('a', id, 'b', id) as col1", 
> "named_struct('a',id+2, 'b',id+2) as col2")
> .filter($"col1" === $"col2").count
> {code}
> fails with 
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: 1
>   at 
> scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
>   at scala.collection.immutable.List.apply(List.scala:84)
>   at 
> org.apache.spark.sql.catalyst.expressions.BoundReference.doGenCode(BoundAttribute.scala:64)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-19512) codegen for compare structs fails

2017-02-08 Thread Apache Spark (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15857887#comment-15857887
 ] 

Apache Spark commented on SPARK-19512:
--

User 'bogdanrdc' has created a pull request for this issue:
https://github.com/apache/spark/pull/16852

> codegen for compare structs fails
> -
>
> Key: SPARK-19512
> URL: https://issues.apache.org/jira/browse/SPARK-19512
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Bogdan Raducanu
>
> This (1 struct field)
> {code:java|title=1 struct field}
> spark.range(10)
>   .selectExpr("named_struct('a', id) as col1", "named_struct('a', id+2) 
> as col2")
>   .filter("col1 = col2").count
> {code}
> fails with
> {code}
> [info]   Cause: java.util.concurrent.ExecutionException: java.lang.Exception: 
> failed to compile: org.codehaus.commons.compiler.CompileException: File 
> 'generated.java', Line 144, Column 32: Expression "range_value" is not an 
> rvalue
> {code}
> This (2 struct fields)
> {code:java|title=2 struct fields}
> spark.range(10)
> .selectExpr("named_struct('a', id, 'b', id) as col1", 
> "named_struct('a',id+2, 'b',id+2) as col2")
> .filter($"col1" === $"col2").count
> {code}
> fails with 
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: 1
>   at 
> scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
>   at scala.collection.immutable.List.apply(List.scala:84)
>   at 
> org.apache.spark.sql.catalyst.expressions.BoundReference.doGenCode(BoundAttribute.scala:64)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org