[jira] [Commented] (SPARK-17728) UDFs are run too many times

2022-06-22 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557756#comment-17557756 ] Josh Rosen commented on SPARK-17728: As of SPARK-36718 in Spark 3.3 I think the

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-10-06 Thread Jacob Eisinger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552708#comment-15552708 ] Jacob Eisinger commented on SPARK-17728: [~hvanhovell], thanks for looking into this! Unless you

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-10-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549514#comment-15549514 ] Herman van Hovell commented on SPARK-17728: --- I think we eventually should add it, however this

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-10-05 Thread Jacob Eisinger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549478#comment-15549478 ] Jacob Eisinger commented on SPARK-17728: Thanks for the great explanation! Are there plans for

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-10-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547358#comment-15547358 ] Herman van Hovell commented on SPARK-17728: --- The second one does not trigger the behavior

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-10-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547352#comment-15547352 ] Herman van Hovell commented on SPARK-17728: --- There are different evaluation paths in Spark SQL:

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-10-03 Thread Jacob Eisinger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15544309#comment-15544309 ] Jacob Eisinger commented on SPARK-17728: Also, it is interesting for me to note that this occurs

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-10-03 Thread Jacob Eisinger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15544303#comment-15544303 ] Jacob Eisinger commented on SPARK-17728: What do you mean by _completely interpreted path_? >

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-10-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543969#comment-15543969 ] Herman van Hovell commented on SPARK-17728: --- First of all, we implement subexpression

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-10-03 Thread Jacob Eisinger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542689#comment-15542689 ] Jacob Eisinger commented on SPARK-17728: Thanks for the explanation and the tricky code snippet! 

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-09-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15535023#comment-15535023 ] Herman van Hovell commented on SPARK-17728: --- I think calling {{explain(true)}} on your plans

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-09-29 Thread Jacob Eisinger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534923#comment-15534923 ] Jacob Eisinger commented on SPARK-17728: Thanks for the explanation, but I still think this is an

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-09-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534792#comment-15534792 ] Herman van Hovell commented on SPARK-17728: --- I am going to close this as not a problem, but

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-09-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534790#comment-15534790 ] Herman van Hovell commented on SPARK-17728: --- Spark assumes UDF's are pure function; we do not

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-09-29 Thread Jacob Eisinger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534583#comment-15534583 ] Jacob Eisinger commented on SPARK-17728: I am a little confused. # Could you explain how a

[jira] [Commented] (SPARK-17728) UDFs are run too many times

2016-09-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534059#comment-15534059 ] Herman van Hovell commented on SPARK-17728: --- You really should not try to use any external