[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow
[
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023717#comment-17023717
]
Maciej Szymkiewicz commented on SPARK-30629:
Makes sense. I guess if we have to choose between these two, recursive calls
will be less common. I think that for now we should disable the test case (I've
opened a PR for that), as it is clearly no longer applicable.
> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests,
> which cover that problem, but it seems like they have been dead for some
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
> f(x)
> }
> SparkR:::cleanClosure(f)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow
[
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023715#comment-17023715
]
Hossein Falaki commented on SPARK-30629:
Yes, this is a good example. There must be a solution to avoid the old bug and
not fail with such a case. I could not find one.
> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests,
> which cover that problem, but it seems like they have been dead for some
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
> f(x)
> }
> SparkR:::cleanClosure(f)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow
[
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023714#comment-17023714
]
Maciej Szymkiewicz commented on SPARK-30629:
Although I still have some doubts... This should terminate (at least for some
values of {{x}})
{code:r}
f <- function(x) {
if(x > 0) {
f(x - 1)
} else {
x
}
}
{code}
but it still fails with the same exception (build from
d5b92b24c41b047c64a4d89cc4061ebf534f0995)
{code}
> SparkR:::cleanClosure(f)
Error: node stack overflow
{code}
> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests,
> which cover that problem, but it seems like they have been dead for some
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
> f(x)
> }
> SparkR:::cleanClosure(f)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow
[
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023713#comment-17023713
]
Maciej Szymkiewicz commented on SPARK-30629:
Thanks for clarification, I'll handle this in moment.
> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests,
> which cover that problem, but it seems like they have been dead for some
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
> f(x)
> }
> SparkR:::cleanClosure(f)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow
[
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023712#comment-17023712
]
Hossein Falaki commented on SPARK-30629:
Oh yes. I think we can disable that specific test. If it ever ran and passed it
must have been because of the other bug. Thanks for catching it.
> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests,
> which cover that problem, but it seems like they have been dead for some
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
> f(x)
> }
> SparkR:::cleanClosure(f)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow
[
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023711#comment-17023711
]
Maciej Szymkiewicz commented on SPARK-30629:
[~falaki] Fair enough, but the problem is that we actually test for that
specific scenario
https://github.com/apache/spark/blob/d5b92b24c41b047c64a4d89cc4061ebf534f0995/R/pkg/tests/fulltests/test_utils.R#L92,
which cause test failures.
> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests,
> which cover that problem, but it seems like they have been dead for some
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
> f(x)
> }
> SparkR:::cleanClosure(f)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow
[
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023710#comment-17023710
]
Hossein Falaki commented on SPARK-30629:
[~zero323] that indefinite recursive call will never execute. So it is better
to fail during {{cleanClosure()}} rather than execution on workers.
> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests,
> which cover that problem, but it seems like they have been dead for some
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
> f(x)
> }
> SparkR:::cleanClosure(f)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow
[
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023708#comment-17023708
]
Maciej Szymkiewicz commented on SPARK-30629:
CC [~mengxr] [~falaki]
> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests,
> which cover that problem, but it seems like they have been dead for some
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
> f(x)
> }
> newF <- cleanClosure(f)
> {code}
> Just looking at the {{cleanClosure}} / {{processClosure}} pair, that
> function that is being processed is not added to {{checkedFuncs}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow
[
https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023706#comment-17023706
]
Maciej Szymkiewicz commented on SPARK-30629:
OK, so I checked and it looks like the problem has been introduced by
SPARK-29777.
> cleanClosure on recursive call leads to node stack overflow
> ---
>
> Key: SPARK-30629
> URL: https://issues.apache.org/jira/browse/SPARK-30629
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 3.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Major
>
> This problem surfaced while handling SPARK-22817. In theory there are tests,
> which cover that problem, but it seems like they have been dead for some
> reason.
> Reproducible example
> {code:r}
> f <- function(x) {
> f(x)
> }
> newF <- cleanClosure(f)
> {code}
> Just looking at the {{cleanClosure}} / {{processClosure}} pair, that
> function that is being processed is not added to {{checkedFuncs}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
