[jira] [Commented] (SPARK-45183) ExecutorPodsLifecycleManager delete a pod multi times.

2023-09-19 Thread hgs (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-45183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766802#comment-17766802
 ] 

hgs commented on SPARK-45183:
-

I have compared  Spark 3.2.0 with Spark 3.5.0. The deletion of pod is nothing 
different in `ExecutorPodsLifecycleManager`.So I suspect the version 3.5.0 may 
have the same issure.[~dongjoon] 

> ExecutorPodsLifecycleManager delete a pod multi times.
> --
>
> Key: SPARK-45183
> URL: https://issues.apache.org/jira/browse/SPARK-45183
> Project: Spark
>  Issue Type: Bug
>  Components: Kubernetes
>Affects Versions: 3.2.0
> Environment: Spark 3.2.0
>Reporter: hgs
>Priority: Minor
>
>  Because `ExecutorPodsLifecycleManager`.`removedExecutorsCache` is not thread 
> safe, will cause a pod deleted many times when  
> `ExecutorPodsLifecycleManager`.`onNewSnapshots` called by multi threads.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-45183) ExecutorPodsLifecycleManager delete a pod multi times.

2023-09-15 Thread hgs (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-45183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765886#comment-17765886
 ] 

hgs commented on SPARK-45183:
-

[~dongjoon] , will you have a look at this.

> ExecutorPodsLifecycleManager delete a pod multi times.
> --
>
> Key: SPARK-45183
> URL: https://issues.apache.org/jira/browse/SPARK-45183
> Project: Spark
>  Issue Type: Bug
>  Components: Kubernetes
>Affects Versions: 3.2.0
> Environment: Spark 3.2.0
>Reporter: hgs
>Priority: Minor
>
>  Because `ExecutorPodsLifecycleManager`.`removedExecutorsCache` is not thread 
> safe, will cause a pod deleted many times when  
> `ExecutorPodsLifecycleManager`.`onNewSnapshots` called by multi threads.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-45183) ExecutorPodsLifecycleManager delete a pod multi times.

2023-09-15 Thread hgs (Jira) 
hgs created SPARK-45183:
---

 Summary: ExecutorPodsLifecycleManager delete a pod multi times.
 Key: SPARK-45183
 URL: https://issues.apache.org/jira/browse/SPARK-45183
 Project: Spark
  Issue Type: Bug
  Components: Kubernetes
Affects Versions: 3.2.0
 Environment: Spark 3.2.0
Reporter: hgs


 Because `ExecutorPodsLifecycleManager`.`removedExecutorsCache` is not thread 
safe, will cause a pod deleted many times when  
`ExecutorPodsLifecycleManager`.`onNewSnapshots` called by multi threads.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-44881) Executor stucked on retrying to fetch shuffle data when `java.lang.OutOfMemoryError. unable to create native thread` exception occurred.

2023-08-19 Thread hgs (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-44881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hgs updated SPARK-44881:

Summary: Executor stucked on retrying to fetch shuffle data when 
`java.lang.OutOfMemoryError. unable to create native thread` exception 
occurred.  (was: Executor stucked on retry to fetch shuffle data when 
`java.lang.OutOfMemoryError. unable to create native thread` exception 
occurred.)

> Executor stucked on retrying to fetch shuffle data when 
> `java.lang.OutOfMemoryError. unable to create native thread` exception 
> occurred.
> 
>
> Key: SPARK-44881
> URL: https://issues.apache.org/jira/browse/SPARK-44881
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Core
>Affects Versions: 3.2.0
>Reporter: hgs
>Priority: Minor
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-44881) Executor stucked on retry to fetch shuffle data when `java.lang.OutOfMemoryError. unable to create native thread` exception occurred.

2023-08-19 Thread hgs (Jira) 
hgs created SPARK-44881:
---

 Summary: Executor stucked on retry to fetch shuffle data when 
`java.lang.OutOfMemoryError. unable to create native thread` exception occurred.
 Key: SPARK-44881
 URL: https://issues.apache.org/jira/browse/SPARK-44881
 Project: Spark
  Issue Type: Bug
  Components: Spark Core
Affects Versions: 3.2.0
Reporter: hgs






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-40288) After `RemoveRedundantAggregates`, `PullOutGroupingExpressions` should applied to avoid attribute missing when use complex expression.

2022-09-03 Thread hgs (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-40288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hgs updated SPARK-40288:

Description: 
{{--table}}
{{create}}  {{table}} {{miss_expr(id }}{{{}int{}}}{{{},{}}}{{{}name{}}} 
{{string,age }}{{{}double{}}}{{{}) stored {}}}{{as}} {{textfile}}
{{--data}}
{{insert}} {{overwrite }}{{table}} {{miss_expr 
}}{{{}values{}}}{{{}(1,{}}}{{{}'ox'{}}}{{{},1.0),(1,{}}}{{{}'oox'{}}}{{{},2.0),(2,{}}}{{{}'ox'{}}}{{{},3.0),(2,{}}}{{{}'xxo'{}}}{{{},4.0){}}}
{{--failure sql}}

{{select}} {{{}id,{}}}{{{}name{}}}{{{},nage {}}}{{as}} {{n 
}}{{{}from{}}}{{{}({}}}
{{select}} {{{}id,{}}}{{{}name{}}}{{{},if(age>3,100,200) {}}}{{as}} {{nage 
}}{{from}} {{miss_expr }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},age{}}}
{{) }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},nage{}}}

--error stack
{{Caused by: java.lang.IllegalStateException: Couldn't find age#4 in 
[id#2,name#3,if ((age#4 > 3.0)) 100 else 200#12|#2,name#3,if ((age#4 > 3.0)) 
100 else 200#12]}}
{{at 
org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:80)}}
{{at 
org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:73)}}
{{at 
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:481)}}

  was:
{{--table}}
{{create}}  {{table}} {{miss_expr(id }}{{{}int{}}}{{{},{}}}{{{}name{}}} 
{{string,age }}{{{}double{}}}{{{}) stored {}}}{{as}} {{textfile}}
{{--data}}
{{insert}} {{overwrite }}{{table}} {{miss_expr 
}}{{{}values{}}}{{{}(1,{}}}{{{}'ox'{}}}{{{},1.0),(1,{}}}{{{}'oox'{}}}{{{},2.0),(2,{}}}{{{}'ox'{}}}{{{},3.0),(2,{}}}{{{}'xxo'{}}}{{{},4.0){}}}
{{--failure sql}}
{{insert}} {{overwrite }}{{table}} {{miss_expr}}
{{select}} {{{}id,{}}}{{{}name{}}}{{{},nage {}}}{{as}} {{n 
}}{{{}from{}}}{{{}({}}}
{{select}} {{{}id,{}}}{{{}name{}}}{{{},if(age>3,100,200) {}}}{{as}} {{nage 
}}{{from}} {{miss_expr }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},age{}}}
{{) }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},nage{}}}


--error stack
{{Caused by: java.lang.IllegalStateException: Couldn't find age#4 in 
[id#2,name#3,if ((age#4 > 3.0)) 100 else 200#12|#2,name#3,if ((age#4 > 3.0)) 
100 else 200#12]}}
{{at 
org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:80)}}
{{at 
org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:73)}}
{{at 
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:481)}}


> After `RemoveRedundantAggregates`, `PullOutGroupingExpressions` should 
> applied to avoid attribute missing when use complex expression.
> --
>
> Key: SPARK-40288
> URL: https://issues.apache.org/jira/browse/SPARK-40288
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 3.2.0, 3.3.0
> Environment: spark 3.2.0 spark 3.2.2 spark 3.3.0
>Reporter: hgs
>Priority: Minor
>
> {{--table}}
> {{create}}  {{table}} {{miss_expr(id }}{{{}int{}}}{{{},{}}}{{{}name{}}} 
> {{string,age }}{{{}double{}}}{{{}) stored {}}}{{as}} {{textfile}}
> {{--data}}
> {{insert}} {{overwrite }}{{table}} {{miss_expr 
> }}{{{}values{}}}{{{}(1,{}}}{{{}'ox'{}}}{{{},1.0),(1,{}}}{{{}'oox'{}}}{{{},2.0),(2,{}}}{{{}'ox'{}}}{{{},3.0),(2,{}}}{{{}'xxo'{}}}{{{},4.0){}}}
> {{--failure sql}}
> {{select}} {{{}id,{}}}{{{}name{}}}{{{},nage {}}}{{as}} {{n 
> }}{{{}from{}}}{{{}({}}}
> {{select}} {{{}id,{}}}{{{}name{}}}{{{},if(age>3,100,200) {}}}{{as}} {{nage 
> }}{{from}} {{miss_expr }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},age{}}}
> {{) }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},nage{}}}
> --error stack
> {{Caused by: java.lang.IllegalStateException: Couldn't find age#4 in 
> [id#2,name#3,if ((age#4 > 3.0)) 100 else 200#12|#2,name#3,if ((age#4 > 3.0)) 
> 100 else 200#12]}}
> {{at 
> org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:80)}}
> {{at 
> org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:73)}}
> {{at 
> org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:481)}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-40288) After `RemoveRedundantAggregates`, `PullOutGroupingExpressions` should applied to avoid attribute missing when use complex expression.

2022-08-31 Thread hgs (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-40288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hgs updated SPARK-40288:

Affects Version/s: (was: 3.0.0)

> After `RemoveRedundantAggregates`, `PullOutGroupingExpressions` should 
> applied to avoid attribute missing when use complex expression.
> --
>
> Key: SPARK-40288
> URL: https://issues.apache.org/jira/browse/SPARK-40288
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 3.2.0, 3.3.0
> Environment: spark 3.2.0 spark 3.2.2 spark 3.3.0
>Reporter: hgs
>Priority: Minor
>
> {{--table}}
> {{create}}  {{table}} {{miss_expr(id }}{{{}int{}}}{{{},{}}}{{{}name{}}} 
> {{string,age }}{{{}double{}}}{{{}) stored {}}}{{as}} {{textfile}}
> {{--data}}
> {{insert}} {{overwrite }}{{table}} {{miss_expr 
> }}{{{}values{}}}{{{}(1,{}}}{{{}'ox'{}}}{{{},1.0),(1,{}}}{{{}'oox'{}}}{{{},2.0),(2,{}}}{{{}'ox'{}}}{{{},3.0),(2,{}}}{{{}'xxo'{}}}{{{},4.0){}}}
> {{--failure sql}}
> {{insert}} {{overwrite }}{{table}} {{miss_expr}}
> {{select}} {{{}id,{}}}{{{}name{}}}{{{},nage {}}}{{as}} {{n 
> }}{{{}from{}}}{{{}({}}}
> {{select}} {{{}id,{}}}{{{}name{}}}{{{},if(age>3,100,200) {}}}{{as}} {{nage 
> }}{{from}} {{miss_expr }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},age{}}}
> {{) }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},nage{}}}
> --error stack
> {{Caused by: java.lang.IllegalStateException: Couldn't find age#4 in 
> [id#2,name#3,if ((age#4 > 3.0)) 100 else 200#12|#2,name#3,if ((age#4 > 3.0)) 
> 100 else 200#12]}}
> {{at 
> org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:80)}}
> {{at 
> org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:73)}}
> {{at 
> org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:481)}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-40288) After `RemoveRedundantAggregates`, `PullOutGroupingExpressions` should applied to avoid attribute missing when use complex expression.

2022-08-31 Thread hgs (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-40288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hgs updated SPARK-40288:

Description: 
{{--table}}
{{create}}  {{table}} {{miss_expr(id }}{{{}int{}}}{{{},{}}}{{{}name{}}} 
{{string,age }}{{{}double{}}}{{{}) stored {}}}{{as}} {{textfile}}
{{--data}}
{{insert}} {{overwrite }}{{table}} {{miss_expr 
}}{{{}values{}}}{{{}(1,{}}}{{{}'ox'{}}}{{{},1.0),(1,{}}}{{{}'oox'{}}}{{{},2.0),(2,{}}}{{{}'ox'{}}}{{{},3.0),(2,{}}}{{{}'xxo'{}}}{{{},4.0){}}}
{{--failure sql}}
{{insert}} {{overwrite }}{{table}} {{miss_expr}}
{{select}} {{{}id,{}}}{{{}name{}}}{{{},nage {}}}{{as}} {{n 
}}{{{}from{}}}{{{}({}}}
{{select}} {{{}id,{}}}{{{}name{}}}{{{},if(age>3,100,200) {}}}{{as}} {{nage 
}}{{from}} {{miss_expr }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},age{}}}
{{) }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},nage{}}}


--error stack
{{Caused by: java.lang.IllegalStateException: Couldn't find age#4 in 
[id#2,name#3,if ((age#4 > 3.0)) 100 else 200#12|#2,name#3,if ((age#4 > 3.0)) 
100 else 200#12]}}
{{at 
org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:80)}}
{{at 
org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:73)}}
{{at 
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:481)}}

  was:
{{--table}}
{{create}}  {{table}} {{miss_expr(id }}{{{}int{}}}{{{},{}}}{{{}name{}}} 
{{string,age }}{{{}double{}}}{{{}) stored {}}}{{as}} {{textfile}}
{{--data}}
{{insert}} {{overwrite }}{{table}} {{miss_expr 
}}{{{}values{}}}{{{}(1,{}}}{{{}'ox'{}}}{{{},1.0),(1,{}}}{{{}'oox'{}}}{{{},2.0),(2,{}}}{{{}'ox'{}}}{{{},3.0),(2,{}}}{{{}'xxo'{}}}{{{},4.0){}}}
{{--failure sql}}
{{insert}} {{overwrite }}{{table}} {{miss_expr}}
{{select}} {{{}id,{}}}{{{}name{}}}{{{},nage {}}}{{as}} {{n 
}}{{{}from{}}}{{{}({}}}
{{select}} {{{}id,{}}}{{{}name{}}}{{{},if(age>3,100,200) {}}}{{as}} {{nage 
}}{{from}} {{miss_expr }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},age{}}}
{{) }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},nage{}}}
{{}}
--error stack
{{Caused by: java.lang.IllegalStateException: Couldn't find age#4 in 
[id#2,name#3,if ((age#4 > 3.0)) 100 else 200#12]}}
{{at 
org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:80)}}
{{at 
org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:73)}}
{{at 
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:481)}}


> After `RemoveRedundantAggregates`, `PullOutGroupingExpressions` should 
> applied to avoid attribute missing when use complex expression.
> --
>
> Key: SPARK-40288
> URL: https://issues.apache.org/jira/browse/SPARK-40288
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 3.0.0, 3.2.0, 3.3.0
> Environment: spark 3.2.0 spark 3.2.2 spark 3.3.0
>Reporter: hgs
>Priority: Minor
>
> {{--table}}
> {{create}}  {{table}} {{miss_expr(id }}{{{}int{}}}{{{},{}}}{{{}name{}}} 
> {{string,age }}{{{}double{}}}{{{}) stored {}}}{{as}} {{textfile}}
> {{--data}}
> {{insert}} {{overwrite }}{{table}} {{miss_expr 
> }}{{{}values{}}}{{{}(1,{}}}{{{}'ox'{}}}{{{},1.0),(1,{}}}{{{}'oox'{}}}{{{},2.0),(2,{}}}{{{}'ox'{}}}{{{},3.0),(2,{}}}{{{}'xxo'{}}}{{{},4.0){}}}
> {{--failure sql}}
> {{insert}} {{overwrite }}{{table}} {{miss_expr}}
> {{select}} {{{}id,{}}}{{{}name{}}}{{{},nage {}}}{{as}} {{n 
> }}{{{}from{}}}{{{}({}}}
> {{select}} {{{}id,{}}}{{{}name{}}}{{{},if(age>3,100,200) {}}}{{as}} {{nage 
> }}{{from}} {{miss_expr }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},age{}}}
> {{) }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},nage{}}}
> --error stack
> {{Caused by: java.lang.IllegalStateException: Couldn't find age#4 in 
> [id#2,name#3,if ((age#4 > 3.0)) 100 else 200#12|#2,name#3,if ((age#4 > 3.0)) 
> 100 else 200#12]}}
> {{at 
> org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:80)}}
> {{at 
> org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:73)}}
> {{at 
> org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:481)}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-40288) After `RemoveRedundantAggregates`, `PullOutGroupingExpressions` should applied to avoid attribute missing when use complex expression.

2022-08-31 Thread hgs (Jira) 
hgs created SPARK-40288:
---

 Summary: After `RemoveRedundantAggregates`, 
`PullOutGroupingExpressions` should applied to avoid attribute missing when use 
complex expression.
 Key: SPARK-40288
 URL: https://issues.apache.org/jira/browse/SPARK-40288
 Project: Spark
  Issue Type: Bug
  Components: SQL
Affects Versions: 3.3.0, 3.2.0, 3.0.0
 Environment: spark 3.2.0 spark 3.2.2 spark 3.3.0
Reporter: hgs


{{--table}}
{{create}}  {{table}} {{miss_expr(id }}{{{}int{}}}{{{},{}}}{{{}name{}}} 
{{string,age }}{{{}double{}}}{{{}) stored {}}}{{as}} {{textfile}}
{{--data}}
{{insert}} {{overwrite }}{{table}} {{miss_expr 
}}{{{}values{}}}{{{}(1,{}}}{{{}'ox'{}}}{{{},1.0),(1,{}}}{{{}'oox'{}}}{{{},2.0),(2,{}}}{{{}'ox'{}}}{{{},3.0),(2,{}}}{{{}'xxo'{}}}{{{},4.0){}}}
{{--failure sql}}
{{insert}} {{overwrite }}{{table}} {{miss_expr}}
{{select}} {{{}id,{}}}{{{}name{}}}{{{},nage {}}}{{as}} {{n 
}}{{{}from{}}}{{{}({}}}
{{select}} {{{}id,{}}}{{{}name{}}}{{{},if(age>3,100,200) {}}}{{as}} {{nage 
}}{{from}} {{miss_expr }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},age{}}}
{{) }}{{group}} {{by}} {{{}id,{}}}{{{}name{}}}{{{},nage{}}}
{{}}
--error stack
{{Caused by: java.lang.IllegalStateException: Couldn't find age#4 in 
[id#2,name#3,if ((age#4 > 3.0)) 100 else 200#12]}}
{{at 
org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:80)}}
{{at 
org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:73)}}
{{at 
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:481)}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-37216) Add the Hive macro functionality to SparkSQL

2021-11-05 Thread hgs (Jira) 
hgs created SPARK-37216:
---

 Summary: Add the Hive macro functionality to SparkSQL
 Key: SPARK-37216
 URL: https://issues.apache.org/jira/browse/SPARK-37216
 Project: Spark
  Issue Type: New Feature
  Components: SQL
Affects Versions: 3.3.0
Reporter: hgs


Should add macro to SparkSQL? We have did this in Spark that we use.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org