[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023717#comment-17023717
 ] 

Maciej Szymkiewicz commented on SPARK-30629:


Makes sense. I guess if we have to choose between these two, recursive calls 
will be less common. I think that for now we should disable the test case (I've 
opened a PR for that), as it is clearly no longer applicable.

> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
>  Issue Type: Bug
>  Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests, 
> which cover that problem, but it seems like they have been dead for some 
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
>   f(x)
> }
> SparkR:::cleanClosure(f)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Hossein Falaki (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023715#comment-17023715
 ] 

Hossein Falaki commented on SPARK-30629:


Yes, this is a good example. There must be a solution to avoid the old bug and 
not fail with such a case. I could not find one.

> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
>  Issue Type: Bug
>  Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests, 
> which cover that problem, but it seems like they have been dead for some 
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
>   f(x)
> }
> SparkR:::cleanClosure(f)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023714#comment-17023714
 ] 

Maciej Szymkiewicz commented on SPARK-30629:


Although I still have some doubts... This should terminate (at least for some 
values of {{x}})

{code:r}
  f <- function(x) {
if(x > 0) {
  f(x - 1)
} else {
  x
}
  }
{code}

but it still fails with the same exception (build from 
d5b92b24c41b047c64a4d89cc4061ebf534f0995)

{code}
> SparkR:::cleanClosure(f)
Error: node stack overflow
{code}


> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
>  Issue Type: Bug
>  Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests, 
> which cover that problem, but it seems like they have been dead for some 
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
>   f(x)
> }
> SparkR:::cleanClosure(f)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023713#comment-17023713
 ] 

Maciej Szymkiewicz commented on SPARK-30629:


Thanks for clarification, I'll handle this in moment.

> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
>  Issue Type: Bug
>  Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests, 
> which cover that problem, but it seems like they have been dead for some 
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
>   f(x)
> }
> SparkR:::cleanClosure(f)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Hossein Falaki (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023712#comment-17023712
 ] 

Hossein Falaki commented on SPARK-30629:


Oh yes. I think we can disable that specific test. If it ever ran and passed it 
must have been because of the other bug. Thanks for catching it.

> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
>  Issue Type: Bug
>  Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests, 
> which cover that problem, but it seems like they have been dead for some 
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
>   f(x)
> }
> SparkR:::cleanClosure(f)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023711#comment-17023711
 ] 

Maciej Szymkiewicz commented on SPARK-30629:


[~falaki] Fair enough, but the problem is that we actually test for that 
specific scenario 
https://github.com/apache/spark/blob/d5b92b24c41b047c64a4d89cc4061ebf534f0995/R/pkg/tests/fulltests/test_utils.R#L92,
 which cause test failures.

> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
>  Issue Type: Bug
>  Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests, 
> which cover that problem, but it seems like they have been dead for some 
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
>   f(x)
> }
> SparkR:::cleanClosure(f)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Hossein Falaki (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023710#comment-17023710
 ] 

Hossein Falaki commented on SPARK-30629:


[~zero323] that indefinite recursive call will never execute. So it is better 
to fail during {{cleanClosure()}} rather than execution on workers.

> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
>  Issue Type: Bug
>  Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests, 
> which cover that problem, but it seems like they have been dead for some 
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
>   f(x)
> }
> SparkR:::cleanClosure(f)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023708#comment-17023708
 ] 

Maciej Szymkiewicz commented on SPARK-30629:


CC [~mengxr] [~falaki]

> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
>  Issue Type: Bug
>  Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests, 
> which cover that problem, but it seems like they have been dead for some 
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
>   f(x)
> }
> newF <- cleanClosure(f)
> {code}
> Just looking at the {{cleanClosure}} /  {{processClosure}} pair, that 
> function that is being processed is not added to {{checkedFuncs}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023706#comment-17023706
 ] 

Maciej Szymkiewicz commented on SPARK-30629:


OK, so I checked and it looks like the problem has been introduced by 
SPARK-29777.

> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
>  Issue Type: Bug
>  Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests, 
> which cover that problem, but it seems like they have been dead for some 
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
>   f(x)
> }
> newF <- cleanClosure(f)
> {code}
> Just looking at the {{cleanClosure}} /  {{processClosure}} pair, that 
> function that is being processed is not added to {{checkedFuncs}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org