[jira] [Created] (SPARK-30817) SparkR ML algorithms parity

2020-02-13 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30817: -- Summary: SparkR ML algorithms parity Key: SPARK-30817 URL: https://issues.apache.org/jira/browse/SPARK-30817 Project: Spark Issue Type:

[jira] [Commented] (SPARK-30747) Update roxygen2 to 7.0.1

2020-02-06 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031471#comment-17031471 ] Maciej Szymkiewicz commented on SPARK-30747: CC [~felixcheung] [~hyukjin.kwon] [~shaneknapp]

[jira] [Updated] (SPARK-30747) Update roxygen2 to 7.0.1

2020-02-06 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30747: --- Description: Currently Spark uses {{roxygen2}} 5.0.1. It is already pretty old

[jira] [Created] (SPARK-30747) Update roxygen2 to 7.0.1

2020-02-06 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30747: -- Summary: Update roxygen2 to 7.0.1 Key: SPARK-30747 URL: https://issues.apache.org/jira/browse/SPARK-30747 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-30681) Add higher order functions API to PySpark

2020-01-30 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30681: --- Description: As of 3.0.0 higher order functions are available in SQL and Scala, but

[jira] [Created] (SPARK-30682) Add higher order functions API to SparkR

2020-01-30 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30682: -- Summary: Add higher order functions API to SparkR Key: SPARK-30682 URL: https://issues.apache.org/jira/browse/SPARK-30682 Project: Spark Issue

[jira] [Created] (SPARK-30681) Add higher order functions API to PySpark

2020-01-30 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30681: -- Summary: Add higher order functions API to PySpark Key: SPARK-30681 URL: https://issues.apache.org/jira/browse/SPARK-30681 Project: Spark Issue

[jira] [Updated] (SPARK-30663) Remove 1.x testthat switch once Jenkins version is updated to 2.x

2020-01-28 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30663: --- Issue Type: Planned Work (was: Bug) > Remove 1.x testthat switch once Jenkins

[jira] [Updated] (SPARK-30663) Remove 1.x testthat switch once Jenkins version is updated to 2.x

2020-01-28 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30663: --- Description: As part of SPARK-23435 proposal we include {{testthat}} 1.x

[jira] [Created] (SPARK-30663) Remove 1.x testthat switch once Jenkins version is updated to 2.x

2020-01-28 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30663: -- Summary: Remove 1.x testthat switch once Jenkins version is updated to 2.x Key: SPARK-30663 URL: https://issues.apache.org/jira/browse/SPARK-30663

[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023717#comment-17023717 ] Maciej Szymkiewicz commented on SPARK-30629: Makes sense. I guess if we have to choose

[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023714#comment-17023714 ] Maciej Szymkiewicz commented on SPARK-30629: Although I still have some doubts... This

[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023713#comment-17023713 ] Maciej Szymkiewicz commented on SPARK-30629: Thanks for clarification, I'll handle this in

[jira] [Comment Edited] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023711#comment-17023711 ] Maciej Szymkiewicz edited comment on SPARK-30629 at 1/26/20 2:46 AM: -

[jira] [Comment Edited] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023711#comment-17023711 ] Maciej Szymkiewicz edited comment on SPARK-30629 at 1/26/20 2:42 AM: -

[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023711#comment-17023711 ] Maciej Szymkiewicz commented on SPARK-30629: [~falaki] Fair enough, but the problem is that

[jira] [Updated] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30629: --- Description: This problem surfaced while handling SPARK-22817. In theory there are

[jira] [Updated] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30629: --- Description: This problem surfaced while handling SPARK-22817. In theory there are

[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023708#comment-17023708 ] Maciej Szymkiewicz commented on SPARK-30629: CC [~mengxr] [~falaki] > cleanClosure on

[jira] [Commented] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-25 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023706#comment-17023706 ] Maciej Szymkiewicz commented on SPARK-30629: OK, so I checked and it looks like the problem

[jira] [Created] (SPARK-30645) collect() support Unicode charactes tests fails on Windows

2020-01-25 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30645: -- Summary: collect() support Unicode charactes tests fails on Windows Key: SPARK-30645 URL: https://issues.apache.org/jira/browse/SPARK-30645 Project: Spark

[jira] [Created] (SPARK-30629) cleanClosure on recursive call leads to node stack overflow

2020-01-23 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30629: -- Summary: cleanClosure on recursive call leads to node stack overflow Key: SPARK-30629 URL: https://issues.apache.org/jira/browse/SPARK-30629 Project:

[jira] [Created] (SPARK-30611) Update testthat dependency

2020-01-22 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30611: -- Summary: Update testthat dependency Key: SPARK-30611 URL: https://issues.apache.org/jira/browse/SPARK-30611 Project: Spark Issue Type:

[jira] [Created] (SPARK-30607) overlay wrappers for SparkR and PySpark

2020-01-22 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30607: -- Summary: overlay wrappers for SparkR and PySpark Key: SPARK-30607 URL: https://issues.apache.org/jira/browse/SPARK-30607 Project: Spark Issue

[jira] [Created] (SPARK-30569) Add DSL functions invoking percentile_approx

2020-01-19 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30569: -- Summary: Add DSL functions invoking percentile_approx Key: SPARK-30569 URL: https://issues.apache.org/jira/browse/SPARK-30569 Project: Spark

[jira] [Updated] (SPARK-30533) Add classes to represent Java Regressors and RegressionModels

2020-01-16 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30533: --- Parent: SPARK-28958 Issue Type: Sub-task (was: Bug) > Add classes to

[jira] [Created] (SPARK-30533) Add classes to represent Java Regressors and RegressionModels

2020-01-16 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30533: -- Summary: Add classes to represent Java Regressors and RegressionModels Key: SPARK-30533 URL: https://issues.apache.org/jira/browse/SPARK-30533 Project:

[jira] [Updated] (SPARK-30504) OneVsRest and OneVsRestModel _from_java and _to_java should handle weightCol

2020-01-13 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30504: --- Description: Current behaviour {code:python} from pyspark.ml.classification import

[jira] [Created] (SPARK-30504) OneVsRest and OneVsRestModel _from_java and _to_java should handle weightCol

2020-01-13 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30504: -- Summary: OneVsRest and OneVsRestModel _from_java and _to_java should handle weightCol Key: SPARK-30504 URL: https://issues.apache.org/jira/browse/SPARK-30504

[jira] [Updated] (SPARK-30493) pyspark.ml.classification.OneVsRestModel shouldn't have setClassifier, setLabelCol and setWeightCol methods

2020-01-12 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30493: --- Summary: pyspark.ml.classification.OneVsRestModel shouldn't have setClassifier,

[jira] [Updated] (SPARK-30493) pyspark.ml.classification.OneVsRestModel shouldn't have setClassifier, setLabelCol, setWeightCol and methods

2020-01-12 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30493: --- Description: These methods don't makes sense in a model, and not present in the

[jira] [Updated] (SPARK-30493) pyspark.ml.classification.OneVsRestModel shouldn't have setClassifier, setLabelCol, setWeightCol and methods

2020-01-12 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30493: --- Summary: pyspark.ml.classification.OneVsRestModel shouldn't have setClassifier,

[jira] [Updated] (SPARK-30493) pyspark.ml.classification.OneVsRestModel shouldn't have setClassifier method

2020-01-12 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-30493: --- Description: Problem introduced with SPARK-29093. >

[jira] [Created] (SPARK-30493) pyspark.ml.classification.OneVsRestModel shouldn't have setClassifier method

2020-01-12 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30493: -- Summary: pyspark.ml.classification.OneVsRestModel shouldn't have setClassifier method Key: SPARK-30493 URL: https://issues.apache.org/jira/browse/SPARK-30493

[jira] [Updated] (SPARK-29748) Remove sorting of fields in PySpark SQL Row creation

2020-01-10 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-29748: --- Labels: release-notes (was: ) > Remove sorting of fields in PySpark SQL Row

[jira] [Commented] (SPARK-27692) Optimize evaluation of udf that is deterministic and has literal inputs

2019-12-31 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006127#comment-17006127 ] Maciej Szymkiewicz commented on SPARK-27692: Could you explain what is the value of this

[jira] [Commented] (SPARK-28264) Revisiting Python / pandas UDF

2019-12-30 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005500#comment-17005500 ] Maciej Szymkiewicz commented on SPARK-28264: Thanks [~hyukjin.kwon]. In general I think

[jira] [Commented] (SPARK-29748) Remove sorting of fields in PySpark SQL Row creation

2019-11-15 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975054#comment-16975054 ] Maciej Szymkiewicz commented on SPARK-29748: [~jhereth] {quote}With simply removing sorting

[jira] [Comment Edited] (SPARK-29748) Remove sorting of fields in PySpark SQL Row creation

2019-11-13 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973695#comment-16973695 ] Maciej Szymkiewicz edited comment on SPARK-29748 at 11/13/19 9:13 PM:

[jira] [Comment Edited] (SPARK-29748) Remove sorting of fields in PySpark SQL Row creation

2019-11-13 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973695#comment-16973695 ] Maciej Szymkiewicz edited comment on SPARK-29748 at 11/13/19 9:11 PM:

[jira] [Commented] (SPARK-29748) Remove sorting of fields in PySpark SQL Row creation

2019-11-13 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973695#comment-16973695 ] Maciej Szymkiewicz commented on SPARK-29748: While this is a step in the right direction I

[jira] [Created] (SPARK-29363) o.a.s.ml.regression.Regressor should be public

2019-10-05 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-29363: -- Summary: o.a.s.ml.regression.Regressor should be public Key: SPARK-29363 URL: https://issues.apache.org/jira/browse/SPARK-29363 Project: Spark

[jira] [Updated] (SPARK-29212) Add common classes without using JVM backend

2019-10-05 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-29212: --- Description: Copied from [https://github.com/apache/spark/pull/25776].   Maciej's

[jira] [Commented] (SPARK-29212) Add common classes without using JVM backend

2019-10-05 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945025#comment-16945025 ] Maciej Szymkiewicz commented on SPARK-29212: [~podongfeng] I've formalized the suggestions,

[jira] [Updated] (SPARK-29212) Add common classes without using JVM backend

2019-10-05 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-29212: --- Description: Copied from [https://github.com/apache/spark/pull/25776].   Maciej's

[jira] [Updated] (SPARK-29212) Add common classes without using JVM backend

2019-10-05 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-29212: --- Description: Copied from [https://github.com/apache/spark/pull/25776].   Maciej's

[jira] [Updated] (SPARK-29212) Add common classes without using JVM backend

2019-10-05 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-29212: --- Description: Copied from [https://github.com/apache/spark/pull/25776].   Maciej's

[jira] [Updated] (SPARK-29212) Add common classes without using JVM backend

2019-10-05 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-29212: --- Description: Copied from [https://github.com/apache/spark/pull/25776].   Maciej's

[jira] [Updated] (SPARK-29212) Add common classes without using JVM backend

2019-10-05 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-29212: --- Description: Copied from [https://github.com/apache/spark/pull/25776].   Maciej's

[jira] [Updated] (SPARK-29212) Add common classes without using JVM backend

2019-10-05 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-29212: --- Description: Copied from [https://github.com/apache/spark/pull/25776].   Maciej's

[jira] [Commented] (SPARK-13802) Fields order in Row(**kwargs) is not consistent with Schema.toInternal method

2019-10-02 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942912#comment-16942912 ] Maciej Szymkiewicz commented on SPARK-13802: [~metasim] namedtuples are the simplest and the

[jira] [Comment Edited] (SPARK-29212) Add common classes without using JVM backend

2019-10-02 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942809#comment-16942809 ] Maciej Szymkiewicz edited comment on SPARK-29212 at 10/2/19 1:41 PM: -

[jira] [Commented] (SPARK-29212) Add common classes without using JVM backend

2019-10-02 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942809#comment-16942809 ] Maciej Szymkiewicz commented on SPARK-29212: [~podongfeng] It sounds about right. I will

[jira] [Commented] (SPARK-27884) Deprecate Python 2 support in Spark 3.0

2019-09-29 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940365#comment-16940365 ] Maciej Szymkiewicz commented on SPARK-27884: According to [related discussion on

[jira] [Commented] (SPARK-29212) Add common classes without using JVM backend

2019-09-26 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938465#comment-16938465 ] Maciej Szymkiewicz commented on SPARK-29212: [~podongfeng] First of all thank your for

[jira] [Updated] (SPARK-28439) pyspark.sql.functions.array_repeat should support Column as count argument

2019-07-18 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-28439: --- Description: In Scala, Spark supports

[jira] [Updated] (SPARK-28439) pyspark.sql.functions.array_repeat should support Column as count argument

2019-07-18 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-28439: --- Description: In Scala Spark supports   {code:java} (Column, Column) => Column

[jira] [Updated] (SPARK-28439) pyspark.sql.functions.array_repeat should support Column as count argument

2019-07-18 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-28439: --- Description: In Scala Spark supports   {code:java} (Column, Column) => Column

[jira] [Updated] (SPARK-28439) pyspark.sql.functions.array_repeat should support Column as count argument

2019-07-18 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-28439: --- Description: In Scala Spark supports   {code:java} (Column, Column) => Column

[jira] [Created] (SPARK-28439) pyspark.sql.functions.array_repeat should support Column as count argument

2019-07-18 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-28439: -- Summary: pyspark.sql.functions.array_repeat should support Column as count argument Key: SPARK-28439 URL: https://issues.apache.org/jira/browse/SPARK-28439

[jira] [Comment Edited] (SPARK-28264) Revisiting Python / pandas UDF

2019-07-11 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882774#comment-16882774 ] Maciej Szymkiewicz edited comment on SPARK-28264 at 7/11/19 9:23 AM: -

[jira] [Comment Edited] (SPARK-28264) Revisiting Python / pandas UDF

2019-07-11 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882774#comment-16882774 ] Maciej Szymkiewicz edited comment on SPARK-28264 at 7/11/19 9:18 AM: -

[jira] [Commented] (SPARK-28264) Revisiting Python / pandas UDF

2019-07-11 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882774#comment-16882774 ] Maciej Szymkiewicz commented on SPARK-28264: Personally I fail to see why some UDF types are

[jira] [Comment Edited] (SPARK-17333) Make pyspark interface friendly with static analysis

2019-01-24 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751040#comment-16751040 ] Maciej Szymkiewicz edited comment on SPARK-17333 at 1/24/19 11:58 AM:

[jira] [Commented] (SPARK-17333) Make pyspark interface friendly with static analysis

2019-01-24 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751040#comment-16751040 ] Maciej Szymkiewicz commented on SPARK-17333: [~Alexander_Gorokhov] Personally I maintain

[jira] [Comment Edited] (SPARK-17333) Make pyspark interface friendly with static analysis

2019-01-24 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751040#comment-16751040 ] Maciej Szymkiewicz edited comment on SPARK-17333 at 1/24/19 11:54 AM:

[jira] [Comment Edited] (SPARK-17333) Make pyspark interface friendly with static analysis

2019-01-24 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751040#comment-16751040 ] Maciej Szymkiewicz edited comment on SPARK-17333 at 1/24/19 11:53 AM:

[jira] [Comment Edited] (SPARK-17333) Make pyspark interface friendly with static analysis

2019-01-24 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751040#comment-16751040 ] Maciej Szymkiewicz edited comment on SPARK-17333 at 1/24/19 11:53 AM:

[jira] [Commented] (SPARK-24359) SPIP: ML Pipelines in R

2018-05-23 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16487209#comment-16487209 ] Maciej Szymkiewicz commented on SPARK-24359: Just my two cents: * As proposed right now,

[jira] [Updated] (SPARK-2620) case class cannot be used as key for reduce

2017-10-22 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-2620: -- Affects Version/s: 2.2.0 > case class cannot be used as key for reduce >

[jira] [Commented] (SPARK-14155) Hide UserDefinedType in Spark 2.0

2017-09-06 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155811#comment-16155811 ] Maciej Szymkiewicz commented on SPARK-14155: [~barrybecker4] Nope: SPARK-7768 > Hide

[jira] [Commented] (SPARK-12157) Support numpy types as return values of Python UDFs

2017-08-23 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138299#comment-16138299 ] Maciej Szymkiewicz commented on SPARK-12157: [~felixcheung] IMHO it is not worth fixing. It

[jira] [Commented] (SPARK-18825) Eliminate duplicate links in SparkR API doc index

2017-05-21 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16018948#comment-16018948 ] Maciej Szymkiewicz commented on SPARK-18825: By all means. I created a PR with one possible

[jira] [Created] (SPARK-20830) PySpark wrappers for explode_outer and posexplode_outer

2017-05-21 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20830: -- Summary: PySpark wrappers for explode_outer and posexplode_outer Key: SPARK-20830 URL: https://issues.apache.org/jira/browse/SPARK-20830 Project: Spark

[jira] [Commented] (SPARK-18825) Eliminate duplicate links in SparkR API doc index

2017-05-20 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16018349#comment-16018349 ] Maciej Szymkiewicz commented on SPARK-18825: I'll try to work on this in the upcoming days

[jira] [Comment Edited] (SPARK-18825) Eliminate duplicate links in SparkR API doc index

2017-05-19 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017817#comment-16017817 ] Maciej Szymkiewicz edited comment on SPARK-18825 at 5/19/17 6:42 PM: -

[jira] [Commented] (SPARK-18825) Eliminate duplicate links in SparkR API doc index

2017-05-19 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017817#comment-16017817 ] Maciej Szymkiewicz commented on SPARK-18825: Originally I thought about patching it for our

[jira] [Commented] (SPARK-18825) Eliminate duplicate links in SparkR API doc index

2017-05-15 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011504#comment-16011504 ] Maciej Szymkiewicz commented on SPARK-18825: It is a bit hack but I made some experiments and

[jira] [Closed] (SPARK-8832) insertInto() throws error in sparkR

2017-05-15 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz closed SPARK-8832. - Resolution: Not A Problem We don't support inserts into RDD-based tables so exception is

[jira] [Created] (SPARK-20729) Reduce boilerplate in Spark ML models

2017-05-12 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20729: -- Summary: Reduce boilerplate in Spark ML models Key: SPARK-20729 URL: https://issues.apache.org/jira/browse/SPARK-20729 Project: Spark Issue

[jira] [Created] (SPARK-20726) R wrapper for SQL broadcast

2017-05-12 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20726: -- Summary: R wrapper for SQL broadcast Key: SPARK-20726 URL: https://issues.apache.org/jira/browse/SPARK-20726 Project: Spark Issue Type:

[jira] [Created] (SPARK-20694) Document DataFrameWriter partitionBy, bucketBy and sortBy in SQL guide

2017-05-10 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20694: -- Summary: Document DataFrameWriter partitionBy, bucketBy and sortBy in SQL guide Key: SPARK-20694 URL: https://issues.apache.org/jira/browse/SPARK-20694

[jira] [Commented] (SPARK-11834) Ignore thresholds in LogisticRegression and update documentation

2017-05-07 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1625#comment-1625 ] Maciej Szymkiewicz commented on SPARK-11834: Sorry, for that. Wrong ticket in PR. > Ignore

[jira] [Updated] (SPARK-20631) LogisticRegression._checkThresholdConsistency should use values not Params

2017-05-07 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-20631: --- Description: {{_checkThresholdConsistency}} incorrectly uses {{getParam}} in attempt

[jira] [Created] (SPARK-20631) LogisticRegression._checkThresholdConsistency should use values not Params

2017-05-07 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20631: -- Summary: LogisticRegression._checkThresholdConsistency should use values not Params Key: SPARK-20631 URL: https://issues.apache.org/jira/browse/SPARK-20631

[jira] [Commented] (SPARK-12467) Get rid of sorting in Row's constructor in pyspark

2017-05-04 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996901#comment-15996901 ] Maciej Szymkiewicz commented on SPARK-12467: [~hyukjin.kwon] Personally I like {{namedtuple}}

[jira] [Comment Edited] (SPARK-12467) Get rid of sorting in Row's constructor in pyspark

2017-05-04 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996702#comment-15996702 ] Maciej Szymkiewicz edited comment on SPARK-12467 at 5/4/17 1:13 PM:

[jira] [Commented] (SPARK-12467) Get rid of sorting in Row's constructor in pyspark

2017-05-04 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996702#comment-15996702 ] Maciej Szymkiewicz commented on SPARK-12467: ??Row has named fields, so it shouldn't depend

[jira] [Comment Edited] (SPARK-12467) Get rid of sorting in Row's constructor in pyspark

2017-05-03 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994748#comment-15994748 ] Maciej Szymkiewicz edited comment on SPARK-12467 at 5/3/17 12:55 PM: -

[jira] [Commented] (SPARK-12467) Get rid of sorting in Row's constructor in pyspark

2017-05-03 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994748#comment-15994748 ] Maciej Szymkiewicz commented on SPARK-12467: Python before 3.6 does not preserve the order of

[jira] [Created] (SPARK-20550) R wrappers for Dataset.alias

2017-05-01 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20550: -- Summary: R wrappers for Dataset.alias Key: SPARK-20550 URL: https://issues.apache.org/jira/browse/SPARK-20550 Project: Spark Issue Type:

[jira] [Created] (SPARK-20544) R wrapper for input_file_name

2017-05-01 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20544: -- Summary: R wrapper for input_file_name Key: SPARK-20544 URL: https://issues.apache.org/jira/browse/SPARK-20544 Project: Spark Issue Type:

[jira] [Created] (SPARK-20535) R wrappers for explode_outer and posexplode_outer

2017-04-29 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20535: -- Summary: R wrappers for explode_outer and posexplode_outer Key: SPARK-20535 URL: https://issues.apache.org/jira/browse/SPARK-20535 Project: Spark

[jira] [Created] (SPARK-20534) Outer generators skip missing records if used alone

2017-04-29 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20534: -- Summary: Outer generators skip missing records if used alone Key: SPARK-20534 URL: https://issues.apache.org/jira/browse/SPARK-20534 Project: Spark

[jira] [Created] (SPARK-20532) SparkR should provide grouping and grouping_id

2017-04-28 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20532: -- Summary: SparkR should provide grouping and grouping_id Key: SPARK-20532 URL: https://issues.apache.org/jira/browse/SPARK-20532 Project: Spark

[jira] [Created] (SPARK-20490) Add eqNullSafe, not and ! to SparkR

2017-04-27 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20490: -- Summary: Add eqNullSafe, not and ! to SparkR Key: SPARK-20490 URL: https://issues.apache.org/jira/browse/SPARK-20490 Project: Spark Issue Type:

[jira] [Created] (SPARK-20438) R wrappers for split and repeat

2017-04-22 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20438: -- Summary: R wrappers for split and repeat Key: SPARK-20438 URL: https://issues.apache.org/jira/browse/SPARK-20438 Project: Spark Issue Type:

[jira] [Commented] (SPARK-20208) Document R fpGrowth support in vignettes, programming guide and code example

2017-04-22 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15980081#comment-15980081 ] Maciej Szymkiewicz commented on SPARK-20208: [~felixcheung] I believe this can be marked as

[jira] [Created] (SPARK-20437) R wrappers for rollup and cube

2017-04-22 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20437: -- Summary: R wrappers for rollup and cube Key: SPARK-20437 URL: https://issues.apache.org/jira/browse/SPARK-20437 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-20208) Document R fpGrowth support in vignettes, programming guide and code example

2017-04-19 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974374#comment-15974374 ] Maciej Szymkiewicz edited comment on SPARK-20208 at 4/19/17 9:27 AM: -

<    1   2   3   4   5   6   7   >