[jira] [Commented] (SPARK-30063) Failure when returning a value from multiple Pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-30063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986817#comment-16986817 ] Ruben Berenguel commented on SPARK-30063: - Wow, this looks bad for now (since grouped_aggs are

[jira] [Commented] (SPARK-30063) Failure when returning a value from multiple Pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-30063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985192#comment-16985192 ] Ruben Berenguel commented on SPARK-30063: - Hi [~tkellogg] I’d like to have a look, do you have

[jira] [Commented] (SPARK-25994) SPIP: Property Graphs, Cypher Queries, and Algorithms

[ https://issues.apache.org/jira/browse/SPARK-25994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920213#comment-16920213 ] Ruben Berenguel commented on SPARK-25994: - Hi [~mju], I’ve had a series of unforeseen increases

[jira] [Commented] (SPARK-25994) SPIP: Property Graphs, Cypher Queries, and Algorithms

[ https://issues.apache.org/jira/browse/SPARK-25994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871677#comment-16871677 ] Ruben Berenguel commented on SPARK-25994: - Hi [~mju] sounds good to me (sorry for the delay,

[jira] [Commented] (SPARK-25994) SPIP: Property Graphs, Cypher Queries, and Algorithms

[ https://issues.apache.org/jira/browse/SPARK-25994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856827#comment-16856827 ] Ruben Berenguel commented on SPARK-25994: - Hi [~mju] I'd like to lend a hand if you feel like it

[jira] [Commented] (SPARK-20787) PySpark can't handle datetimes before 1900

[ https://issues.apache.org/jira/browse/SPARK-20787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811044#comment-16811044 ] Ruben Berenguel commented on SPARK-20787: - Hi [~AdiC], indeed, I have not added additional work.

[jira] [Commented] (SPARK-23904) Big execution plan cause OOM

[ https://issues.apache.org/jira/browse/SPARK-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646726#comment-16646726 ] Ruben Berenguel commented on SPARK-23904: - [~staslos] Interesting, thanks. I guess this still

[jira] [Commented] (SPARK-24347) df.alias() in python API should not clear metadata by default

[ https://issues.apache.org/jira/browse/SPARK-24347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622714#comment-16622714 ] Ruben Berenguel commented on SPARK-24347: - Hi [~axelmagn] I know what the issue is and why it is

[jira] [Commented] (SPARK-24458) Invalid PythonUDF check_1(), requires attributes from more than one child

[ https://issues.apache.org/jira/browse/SPARK-24458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525556#comment-16525556 ] Ruben Berenguel commented on SPARK-24458: - Can't reproduce with Spark 2.2 either, local mode. >

[jira] [Commented] (SPARK-24347) df.alias() in python API should not clear metadata by default

[ https://issues.apache.org/jira/browse/SPARK-24347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523476#comment-16523476 ] Ruben Berenguel commented on SPARK-24347: - Pinging [~hyukjin.kwon], too :) > df.alias() in

[jira] [Commented] (SPARK-24458) Invalid PythonUDF check_1(), requires attributes from more than one child

[ https://issues.apache.org/jira/browse/SPARK-24458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523475#comment-16523475 ] Ruben Berenguel commented on SPARK-24458: - Oh, big facepalm, thanks [~hyukjin.kwon]. My

[jira] [Commented] (SPARK-24458) Invalid PythonUDF check_1(), requires attributes from more than one child

[ https://issues.apache.org/jira/browse/SPARK-24458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520165#comment-16520165 ] Ruben Berenguel commented on SPARK-24458: - [~hyukjin.kwon] I just built 2.3.0 from the tagged

[jira] [Commented] (SPARK-24347) df.alias() in python API should not clear metadata by default

[ https://issues.apache.org/jira/browse/SPARK-24347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519191#comment-16519191 ] Ruben Berenguel commented on SPARK-24347: - [~holdenkarau] [~ueshin] (I ping you since you have

[jira] [Commented] (SPARK-24347) df.alias() in python API should not clear metadata by default

[ https://issues.apache.org/jira/browse/SPARK-24347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519074#comment-16519074 ] Ruben Berenguel commented on SPARK-24347: - Looking into this > df.alias() in python API should

[jira] [Commented] (SPARK-24458) Invalid PythonUDF check_1(), requires attributes from more than one child

[ https://issues.apache.org/jira/browse/SPARK-24458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516908#comment-16516908 ] Ruben Berenguel commented on SPARK-24458: - That's what I thought [~AbdealiJK] (I had `1, 2, 3`)

[jira] [Commented] (SPARK-24458) Invalid PythonUDF check_1(), requires attributes from more than one child

[ https://issues.apache.org/jira/browse/SPARK-24458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516890#comment-16516890 ] Ruben Berenguel commented on SPARK-24458: - [~AbdealiJK] what does your `a.csv` file contain in

[jira] [Commented] (SPARK-23904) Big execution plan cause OOM

[ https://issues.apache.org/jira/browse/SPARK-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499476#comment-16499476 ] Ruben Berenguel commented on SPARK-23904: - Yes [~igreenfi] I'm using that setting for

[jira] [Commented] (SPARK-23904) Big execution plan cause OOM

[ https://issues.apache.org/jira/browse/SPARK-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16496290#comment-16496290 ] Ruben Berenguel commented on SPARK-23904: - Thanks [~igreenfi], still at it then :) > Big

[jira] [Commented] (SPARK-23904) Big execution plan cause OOM

[ https://issues.apache.org/jira/browse/SPARK-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494832#comment-16494832 ] Ruben Berenguel commented on SPARK-23904: - [~igreenfi] that's what I mean, removing the code

[jira] [Commented] (SPARK-23904) Big execution plan cause OOM

[ https://issues.apache.org/jira/browse/SPARK-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494513#comment-16494513 ] Ruben Berenguel commented on SPARK-23904: - [~igreenfi] after a few more tries at reproducing,

[jira] [Commented] (SPARK-23904) Big execution plan cause OOM

[ https://issues.apache.org/jira/browse/SPARK-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494462#comment-16494462 ] Ruben Berenguel commented on SPARK-23904: - Finally, managed to reproduce (takes a long while,

[jira] [Commented] (SPARK-23904) Big execution plan cause OOM

[ https://issues.apache.org/jira/browse/SPARK-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493327#comment-16493327 ] Ruben Berenguel commented on SPARK-23904: - I could not (but had no time to dive too deep), but

[jira] [Commented] (SPARK-23904) Big execution plan cause OOM

[ https://issues.apache.org/jira/browse/SPARK-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473863#comment-16473863 ] Ruben Berenguel commented on SPARK-23904: - [~igreenfi] I didn’t manage to reproduce. I will give

[jira] [Commented] (SPARK-23904) Big execution plan cause OOM

[ https://issues.apache.org/jira/browse/SPARK-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439815#comment-16439815 ] Ruben Berenguel commented on SPARK-23904: - I'll give it a look, maybe there is a way to avoid it

[jira] [Commented] (SPARK-19732) DataFrame.fillna() does not work for bools in PySpark

[ https://issues.apache.org/jira/browse/SPARK-19732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16030102#comment-16030102 ] Ruben Berenguel commented on SPARK-19732: - I'll give this a go! > DataFrame.fillna() does not

[jira] [Commented] (SPARK-19044) PySpark dropna() can fail with AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-19044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16030021#comment-16030021 ] Ruben Berenguel commented on SPARK-19044: - Oh, there's a typo in the "equivalent Scala code" in

[jira] [Commented] (SPARK-19044) PySpark dropna() can fail with AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-19044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028620#comment-16028620 ] Ruben Berenguel commented on SPARK-19044: - Seems vaguely related (at least in the code involved)

[jira] [Comment Edited] (SPARK-19044) PySpark dropna() can fail with AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-19044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028620#comment-16028620 ] Ruben Berenguel edited comment on SPARK-19044 at 5/29/17 8:46 PM: -- Seems

[jira] [Commented] (SPARK-20787) PySpark can't handle datetimes before 1900

[ https://issues.apache.org/jira/browse/SPARK-20787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028256#comment-16028256 ] Ruben Berenguel commented on SPARK-20787: - [~facai] thanks, will give it a shot then (not sure if

[jira] [Commented] (SPARK-20787) PySpark can't handle datetimes before 1900

[ https://issues.apache.org/jira/browse/SPARK-20787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028187#comment-16028187 ] Ruben Berenguel commented on SPARK-20787: - Hi [~facai] are you working on this? I'd be interested

[jira] [Commented] (SPARK-13947) PySpark DataFrames: The error message from using an invalid table reference is not clear

[ https://issues.apache.org/jira/browse/SPARK-13947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883688#comment-15883688 ] Ruben Berenguel commented on SPARK-13947: - I'll give a shot to this one as a first dive into the