[jira] [Commented] (SPARK-37044) Add Row to __all__ in pyspark.sql.types

2021-10-18 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430059#comment-17430059 ] Maciej Szymkiewicz commented on SPARK-37044: cc [~hyukjin.kwon] > Add Row to __all__ in

[jira] [Created] (SPARK-37044) Add Row to __all__ in pyspark.sql.types

2021-10-18 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-37044: -- Summary: Add Row to __all__ in pyspark.sql.types Key: SPARK-37044 URL: https://issues.apache.org/jira/browse/SPARK-37044 Project: Spark Issue

[jira] [Updated] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-16 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37022: --- Description: [{{black}}|https://github.com/psf/black] is a popular Python code

[jira] [Commented] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-16 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429532#comment-17429532 ] Maciej Szymkiewicz commented on SPARK-37022: Looking at the diffs, the majority of changes

[jira] [Commented] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-15 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429482#comment-17429482 ] Maciej Szymkiewicz commented on SPARK-37022: cc [~hyukjin.kwon] > Use black as a formatter

[jira] [Comment Edited] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-15 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429477#comment-17429477 ] Maciej Szymkiewicz edited comment on SPARK-37022 at 10/15/21, 8:45 PM:

[jira] [Updated] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-15 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37022: --- Attachment: black-diff-stats.txt > Use black as a formatter for the whole PySpark

[jira] [Updated] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-15 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37022: --- Attachment: (was: black-diff-stats.txt) > Use black as a formatter for the

[jira] [Commented] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-15 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429477#comment-17429477 ] Maciej Szymkiewicz commented on SPARK-37022: The attached files show git diff stats for a

[jira] [Updated] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-15 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37022: --- Attachment: black-diff-stats.txt > Use black as a formatter for the whole PySpark

[jira] [Updated] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-15 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37022: --- Attachment: pyproject.toml > Use black as a formatter for the whole PySpark

[jira] [Created] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-15 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-37022: -- Summary: Use black as a formatter for the whole PySpark codebase. Key: SPARK-37022 URL: https://issues.apache.org/jira/browse/SPARK-37022 Project: Spark

[jira] [Created] (SPARK-36997) Test type hints against examples

2021-10-13 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-36997: -- Summary: Test type hints against examples Key: SPARK-36997 URL: https://issues.apache.org/jira/browse/SPARK-36997 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36989) Migrate type hint data tests

2021-10-13 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-36989: --- Component/s: Tests > Migrate type hint data tests > >

[jira] [Updated] (SPARK-36989) Migrate type hint data tests

2021-10-13 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-36989: --- Issue Type: Improvement (was: Bug) > Migrate type hint data tests >

[jira] [Updated] (SPARK-36989) Migrate type hint data tests

2021-10-12 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-36989: --- Description: Before the migration, {{pyspark-stubs}} contained a set of [data

[jira] [Comment Edited] (SPARK-36989) Migrate type hint data tests

2021-10-12 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427889#comment-17427889 ] Maciej Szymkiewicz edited comment on SPARK-36989 at 10/12/21, 7:23 PM:

[jira] [Commented] (SPARK-36989) Migrate type hint data tests

2021-10-12 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427889#comment-17427889 ] Maciej Szymkiewicz commented on SPARK-36989: FYI [~hyukjin.kwon] [~XinrongM] [~ueshin] >

[jira] [Commented] (SPARK-36989) Migrate type hint data tests

2021-10-12 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427888#comment-17427888 ] Maciej Szymkiewicz commented on SPARK-36989: Currently I am working on [some

[jira] [Created] (SPARK-36989) Migrate type hint data tests

2021-10-12 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-36989: -- Summary: Migrate type hint data tests Key: SPARK-36989 URL: https://issues.apache.org/jira/browse/SPARK-36989 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-36985) Future typing errors in pyspark.pandas

2021-10-12 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427600#comment-17427600 ] Maciej Szymkiewicz commented on SPARK-36985: cc [~hyukjin.kwon] [~XinrongM] [~ueshin] >

[jira] [Created] (SPARK-36985) Future typing errors in pyspark.pandas

2021-10-12 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-36985: -- Summary: Future typing errors in pyspark.pandas Key: SPARK-36985 URL: https://issues.apache.org/jira/browse/SPARK-36985 Project: Spark Issue

[jira] [Created] (SPARK-36977) Update docs to reflect that Python 3.6 is no longer supported

2021-10-11 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-36977: -- Summary: Update docs to reflect that Python 3.6 is no longer supported Key: SPARK-36977 URL: https://issues.apache.org/jira/browse/SPARK-36977 Project:

[jira] [Assigned] (SPARK-36944) Remove unused python/pyspark/sql/__init__.pyi

2021-10-08 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-36944: -- Assignee: dch nguyen > Remove unused python/pyspark/sql/__init__.pyi >

[jira] [Resolved] (SPARK-36944) Remove unused python/pyspark/sql/__init__.pyi

2021-10-08 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-36944. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Created] (SPARK-36909) pyspark.sql.catalog stubs are missing annotation for createExternalTable

2021-10-01 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-36909: -- Summary: pyspark.sql.catalog stubs are missing annotation for createExternalTable Key: SPARK-36909 URL: https://issues.apache.org/jira/browse/SPARK-36909

[jira] [Commented] (SPARK-36845) Inline type hint files

2021-09-30 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422989#comment-17422989 ] Maciej Szymkiewicz commented on SPARK-36845: It is an optional library and won't be present

[jira] [Commented] (SPARK-36845) Inline type hint files

2021-09-30 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422576#comment-17422576 ] Maciej Szymkiewicz commented on SPARK-36845: {quote} For the {{_typing.pyi}}, it uses

[jira] [Updated] (SPARK-36894) RDD.toDF should be synchronized with dispatched variants of SparkSession.createDataFrame

2021-09-29 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-36894: --- Description: There are some variants that are supported: * Providing a schema as a

[jira] [Updated] (SPARK-36894) RDD.toDF should be synchronized with dispatched variants of SparkSession.createDataFrame

2021-09-29 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-36894: --- Summary: RDD.toDF should be synchronized with dispatched variants of

[jira] [Created] (SPARK-36894) RDD.toDF should support string schema

2021-09-29 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-36894: -- Summary: RDD.toDF should support string schema Key: SPARK-36894 URL: https://issues.apache.org/jira/browse/SPARK-36894 Project: Spark Issue

[jira] [Commented] (SPARK-36845) Inline type hint files

2021-09-29 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422379#comment-17422379 ] Maciej Szymkiewicz commented on SPARK-36845: Additionally, we should probably rethink

[jira] [Comment Edited] (SPARK-36845) Inline type hint files

2021-09-29 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422371#comment-17422371 ] Maciej Szymkiewicz edited comment on SPARK-36845 at 9/29/21, 8:14 PM:

[jira] [Comment Edited] (SPARK-36845) Inline type hint files

2021-09-29 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422371#comment-17422371 ] Maciej Szymkiewicz edited comment on SPARK-36845 at 9/29/21, 8:13 PM:

[jira] [Commented] (SPARK-36845) Inline type hint files

2021-09-29 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422371#comment-17422371 ] Maciej Szymkiewicz commented on SPARK-36845: I'd recommend adding SPARK-36145 as blocker for

[jira] [Created] (SPARK-36654) Drop type ignores from numpy imports

2021-09-02 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-36654: -- Summary: Drop type ignores from numpy imports Key: SPARK-36654 URL: https://issues.apache.org/jira/browse/SPARK-36654 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-36617) Inconsistencies in approxQuantile annotations

2021-09-02 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-36617: -- Assignee: Cary Lee > Inconsistencies in approxQuantile annotations >

[jira] [Resolved] (SPARK-36617) Inconsistencies in approxQuantile annotations

2021-09-02 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-36617. Fix Version/s: 3.1.3 3.2.0 Resolution: Resolved Issue

[jira] [Commented] (SPARK-36617) Inconsistencies in approxQuantile annotations

2021-08-30 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406956#comment-17406956 ] Maciej Szymkiewicz commented on SPARK-36617: FYI [~hyukjin.kwon] > Inconsistencies in

[jira] [Created] (SPARK-36617) Inconsistencies in approxQuantile annotations

2021-08-30 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-36617: -- Summary: Inconsistencies in approxQuantile annotations Key: SPARK-36617 URL: https://issues.apache.org/jira/browse/SPARK-36617 Project: Spark

[jira] [Resolved] (SPARK-36211) type check fails for `F.udf(...).asNonDeterministic()

2021-07-27 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-36211. Assignee: Luran He Resolution: Fixed > type check fails for

[jira] [Updated] (SPARK-36211) type check fails for `F.udf(...).asNonDeterministic()

2021-07-27 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-36211: --- Fix Version/s: 3.3.0 3.1.3 3.2.0 > type check

[jira] [Commented] (SPARK-36211) type check fails for `F.udf(...).asNonDeterministic()

2021-07-27 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387834#comment-17387834 ] Maciej Szymkiewicz commented on SPARK-36211: Issue resolved by pull request 33399

[jira] [Resolved] (SPARK-34630) Add type hints of pyspark.__version__ and pyspark.sql.Column.contains

2021-03-24 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-34630. Fix Version/s: 3.1.2 3.2.0 Assignee: Danny Meijer

[jira] [Commented] (SPARK-34629) Python type hints improvement

2021-03-13 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300901#comment-17300901 ] Maciej Szymkiewicz commented on SPARK-34629: {quote} Would love to help out here if I may.

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

2021-03-08 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17297752#comment-17297752 ] Maciej Szymkiewicz commented on SPARK-34544: {quote} so that would require that we build our

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

2021-03-01 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292996#comment-17292996 ] Maciej Szymkiewicz commented on SPARK-34544: [~ravwojdyla] > But until pyspark release, how

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

2021-03-01 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292989#comment-17292989 ] Maciej Szymkiewicz commented on SPARK-34544: Overall: * {{DataFrameLike}} should be

[jira] [Comment Edited] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

2021-03-01 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292977#comment-17292977 ] Maciej Szymkiewicz edited comment on SPARK-34544 at 3/1/21, 3:49 PM: -

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

2021-03-01 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292977#comment-17292977 ] Maciej Szymkiewicz commented on SPARK-34544: The problem we're dealing with here is, that

[jira] [Comment Edited] (SPARK-32195) Standardize warning types and messages

2021-02-24 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289833#comment-17289833 ] Maciej Szymkiewicz edited comment on SPARK-32195 at 2/24/21, 10:26 AM:

[jira] [Comment Edited] (SPARK-32195) Standardize warning types and messages

2021-02-24 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289833#comment-17289833 ] Maciej Szymkiewicz edited comment on SPARK-32195 at 2/24/21, 10:26 AM:

[jira] [Commented] (SPARK-32195) Standardize warning types and messages

2021-02-24 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289833#comment-17289833 ] Maciej Szymkiewicz commented on SPARK-32195: [~RISHI_23]  SPARK-33730 is already resolved

[jira] [Created] (SPARK-34191) udf type hint should allow dectorator with named returnType

2021-01-21 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-34191: -- Summary: udf type hint should allow dectorator with named returnType Key: SPARK-34191 URL: https://issues.apache.org/jira/browse/SPARK-34191 Project:

[jira] [Created] (SPARK-34136) Support complex types in pyspark.sql.functions.lit

2021-01-16 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-34136: -- Summary: Support complex types in pyspark.sql.functions.lit Key: SPARK-34136 URL: https://issues.apache.org/jira/browse/SPARK-34136 Project: Spark

[jira] [Created] (SPARK-34132) Update Roxygen version references to 7.1.1

2021-01-15 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-34132: -- Summary: Update Roxygen version references to 7.1.1 Key: SPARK-34132 URL: https://issues.apache.org/jira/browse/SPARK-34132 Project: Spark Issue

[jira] [Commented] (SPARK-30747) Update roxygen2 to 7.0.1

2021-01-14 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265566#comment-17265566 ] Maciej Szymkiewicz commented on SPARK-30747: Thanks for working on this [~shaneknapp]! >

[jira] [Comment Edited] (SPARK-33730) Standardize warning types

2020-12-11 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248051#comment-17248051 ] Maciej Szymkiewicz edited comment on SPARK-33730 at 12/11/20, 5:30 PM:

[jira] [Comment Edited] (SPARK-33730) Standardize warning types

2020-12-11 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248051#comment-17248051 ] Maciej Szymkiewicz edited comment on SPARK-33730 at 12/11/20, 5:29 PM:

[jira] [Commented] (SPARK-33730) Standardize warning types

2020-12-11 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248051#comment-17248051 ] Maciej Szymkiewicz commented on SPARK-33730: {quote}Eh, I think Maciej Szymkiewicz is

[jira] [Commented] (SPARK-33730) Standardize warning types

2020-12-10 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247565#comment-17247565 ] Maciej Szymkiewicz commented on SPARK-33730: [~hyukjin.kwon] We're going to target this for

[jira] [Commented] (SPARK-32681) PySpark type hints support

2020-12-04 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244012#comment-17244012 ] Maciej Szymkiewicz commented on SPARK-32681: Thanks [~hyukjin.kwon]. As far as I am aware

[jira] [Updated] (SPARK-33622) Add array_to_vector function to SparkR

2020-12-01 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-33622: --- Component/s: ML > Add array_to_vector function to SparkR >

[jira] [Created] (SPARK-33622) Add array_to_vector function to SparkR

2020-12-01 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33622: -- Summary: Add array_to_vector function to SparkR Key: SPARK-33622 URL: https://issues.apache.org/jira/browse/SPARK-33622 Project: Spark Issue

[jira] [Created] (SPARK-33563) Expose inverse hyperbolic trig functions in PySpark and SparkR

2020-11-25 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33563: -- Summary: Expose inverse hyperbolic trig functions in PySpark and SparkR Key: SPARK-33563 URL: https://issues.apache.org/jira/browse/SPARK-33563 Project:

[jira] [Created] (SPARK-33457) Adjust mypy configuration

2020-11-15 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33457: -- Summary: Adjust mypy configuration Key: SPARK-33457 URL: https://issues.apache.org/jira/browse/SPARK-33457 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-33252) Migration to NumPy documentation style in MLlib (pyspark.mllib.*)

2020-11-14 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232059#comment-17232059 ] Maciej Szymkiewicz commented on SPARK-33252: I am starting to work on this one. > Migration

[jira] [Created] (SPARK-33415) Column.__repr__ shouldn't encode JVM response

2020-11-10 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33415: -- Summary: Column.__repr__ shouldn't encode JVM response Key: SPARK-33415 URL: https://issues.apache.org/jira/browse/SPARK-33415 Project: Spark

[jira] [Commented] (SPARK-33254) Migration to NumPy documentation style in Core (pyspark.*, pyspark.resource.*, etc.)

2020-11-09 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228867#comment-17228867 ] Maciej Szymkiewicz commented on SPARK-33254: I am working on this one. > Migration to NumPy

[jira] [Created] (SPARK-33313) R/run-tests.sh is not compatible with testthat >= 3.0

2020-11-01 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33313: -- Summary: R/run-tests.sh is not compatible with testthat >= 3.0 Key: SPARK-33313 URL: https://issues.apache.org/jira/browse/SPARK-33313 Project: Spark

[jira] [Created] (SPARK-33304) Add from_avro and to_avro functions to SparkR

2020-10-30 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33304: -- Summary: Add from_avro and to_avro functions to SparkR Key: SPARK-33304 URL: https://issues.apache.org/jira/browse/SPARK-33304 Project: Spark

[jira] [Commented] (SPARK-33251) Migration to NumPy documentation style in ML (pyspark.ml.*)

2020-10-29 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222748#comment-17222748 ] Maciej Szymkiewicz commented on SPARK-33251: Let me work on this one. > Migration to NumPy

[jira] [Created] (SPARK-33258) Add asc_nulls_* and desc_nulls_* methods to SparkR

2020-10-27 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33258: -- Summary: Add asc_nulls_* and desc_nulls_* methods to SparkR Key: SPARK-33258 URL: https://issues.apache.org/jira/browse/SPARK-33258 Project: Spark

[jira] [Created] (SPARK-33257) Support Column inputs in PySpark ordering functions (asc*, desc*)

2020-10-27 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33257: -- Summary: Support Column inputs in PySpark ordering functions (asc*, desc*) Key: SPARK-33257 URL: https://issues.apache.org/jira/browse/SPARK-33257

[jira] [Updated] (SPARK-32085) Migrate to NumPy documentation style

2020-10-22 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-32085: --- Description: https://github.com/numpy/numpydoc For example, Before:

[jira] [Commented] (SPARK-32084) Replace dictionary-based function definitions to proper functions in functions.py

2020-10-20 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217914#comment-17217914 ] Maciej Szymkiewicz commented on SPARK-32084: I got that part and I fully agree. At the end

[jira] [Commented] (SPARK-32084) Replace dictionary-based function definitions to proper functions in functions.py

2020-10-20 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217339#comment-17217339 ] Maciej Szymkiewicz commented on SPARK-32084: [~hyukjin.kwon] I can work on that, though I

[jira] [Updated] (SPARK-33105) Broken installation of source packages on AppVeyor

2020-10-09 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-33105: --- Description: It looks like AppVeyor configuration is broken, which leads to failure

[jira] [Created] (SPARK-33105) Broken installation of source packages on AppVeyor

2020-10-09 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33105: -- Summary: Broken installation of source packages on AppVeyor Key: SPARK-33105 URL: https://issues.apache.org/jira/browse/SPARK-33105 Project: Spark

[jira] [Created] (SPARK-33086) Provide static annotatiions for pyspark.resource.* modules

2020-10-07 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33086: -- Summary: Provide static annotatiions for pyspark.resource.* modules Key: SPARK-33086 URL: https://issues.apache.org/jira/browse/SPARK-33086 Project: Spark

[jira] [Commented] (SPARK-33003) Add type hints guideliness to the documentation

2020-10-07 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209419#comment-17209419 ] Maciej Szymkiewicz commented on SPARK-33003: [~hyukjin.kwon] Do you think we should have

[jira] [Created] (SPARK-33056) Add mapping between external Python types and CalendarIntervalType

2020-10-02 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33056: -- Summary: Add mapping between external Python types and CalendarIntervalType Key: SPARK-33056 URL: https://issues.apache.org/jira/browse/SPARK-33056

[jira] [Created] (SPARK-33055) Add support for CalendarIntervalType in PySpark schema

2020-10-02 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33055: -- Summary: Add support for CalendarIntervalType in PySpark schema Key: SPARK-33055 URL: https://issues.apache.org/jira/browse/SPARK-33055 Project: Spark

[jira] [Created] (SPARK-33054) Support interval type in PySpark

2020-10-02 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33054: -- Summary: Support interval type in PySpark Key: SPARK-33054 URL: https://issues.apache.org/jira/browse/SPARK-33054 Project: Spark Issue Type:

[jira] [Created] (SPARK-33040) Add SparkR API for invoking vector_to_array

2020-09-30 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33040: -- Summary: Add SparkR API for invoking vector_to_array Key: SPARK-33040 URL: https://issues.apache.org/jira/browse/SPARK-33040 Project: Spark

[jira] [Created] (SPARK-33030) Add nth_value in SparkR API

2020-09-29 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33030: -- Summary: Add nth_value in SparkR API Key: SPARK-33030 URL: https://issues.apache.org/jira/browse/SPARK-33030 Project: Spark Issue Type:

[jira] [Created] (SPARK-33003) Add type hints guideliness to the documentation

2020-09-26 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33003: -- Summary: Add type hints guideliness to the documentation Key: SPARK-33003 URL: https://issues.apache.org/jira/browse/SPARK-33003 Project: Spark

[jira] [Created] (SPARK-33002) Post-port removal of non-API stubs

2020-09-26 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33002: -- Summary: Post-port removal of non-API stubs Key: SPARK-33002 URL: https://issues.apache.org/jira/browse/SPARK-33002 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32933) Use keyword-only syntax for keyword_only methods

2020-09-23 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17201017#comment-17201017 ] Maciej Szymkiewicz commented on SPARK-32933: Not a problem [~hyukjin.kwon]. My only concern

[jira] [Created] (SPARK-32949) Add timestamp_seconds to SparkR

2020-09-21 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-32949: -- Summary: Add timestamp_seconds to SparkR Key: SPARK-32949 URL: https://issues.apache.org/jira/browse/SPARK-32949 Project: Spark Issue Type:

[jira] [Created] (SPARK-32946) Add withField function to SparkR

2020-09-19 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-32946: -- Summary: Add withField function to SparkR Key: SPARK-32946 URL: https://issues.apache.org/jira/browse/SPARK-32946 Project: Spark Issue Type:

[jira] [Created] (SPARK-32933) Use keyword-only syntax for keyword_only methods

2020-09-18 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-32933: -- Summary: Use keyword-only syntax for keyword_only methods Key: SPARK-32933 URL: https://issues.apache.org/jira/browse/SPARK-32933 Project: Spark

[jira] [Issue Comment Deleted] (SPARK-32814) Metaclasses are broken for a few classes in Python 3

2020-09-08 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-32814: --- Comment: was deleted (was: https://github.com/apache/spark/pull/29591) >

[jira] [Commented] (SPARK-32814) Metaclasses are broken for a few classes in Python 3

2020-09-08 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191969#comment-17191969 ] Maciej Szymkiewicz commented on SPARK-32814: https://github.com/apache/spark/pull/29664 >

[jira] [Commented] (SPARK-32814) Metaclasses are broken for a few classes in Python 3

2020-09-08 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191967#comment-17191967 ] Maciej Szymkiewicz commented on SPARK-32814: https://github.com/apache/spark/pull/29591 >

[jira] [Created] (SPARK-32814) Metaclasses are broken for a few classes in Python 3

2020-09-07 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-32814: -- Summary: Metaclasses are broken for a few classes in Python 3 Key: SPARK-32814 URL: https://issues.apache.org/jira/browse/SPARK-32814 Project: Spark

[jira] [Commented] (SPARK-30817) SparkR ML algorithms parity

2020-07-28 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1713#comment-1713 ] Maciej Szymkiewicz commented on SPARK-30817: [~dan_z] I think we're in-sync ATM and this

[jira] [Created] (SPARK-30820) Add FMClassifier to SparkR

2020-02-13 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30820: -- Summary: Add FMClassifier to SparkR Key: SPARK-30820 URL: https://issues.apache.org/jira/browse/SPARK-30820 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-30819) Add FMRegressor wrapper to SparkR

2020-02-13 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30819: -- Summary: Add FMRegressor wrapper to SparkR Key: SPARK-30819 URL: https://issues.apache.org/jira/browse/SPARK-30819 Project: Spark Issue Type:

[jira] [Created] (SPARK-30818) Add LinearRegression wrapper to SparkR

2020-02-13 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-30818: -- Summary: Add LinearRegression wrapper to SparkR Key: SPARK-30818 URL: https://issues.apache.org/jira/browse/SPARK-30818 Project: Spark Issue

<    1   2   3   4   5   6   7   >