[jira] [Assigned] (SPARK-37417) Inline type hints for python/pyspark/ml/linalg/__init__.py

[ https://issues.apache.org/jira/browse/SPARK-37417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37417: -- Assignee: Maciej Szymkiewicz > Inline type hints for python/pyspark/ml/linalg

[jira] [Assigned] (SPARK-36885) Inline type hints for python/pyspark/sql/dataframe.py

[ https://issues.apache.org/jira/browse/SPARK-36885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-36885: -- Assignee: Apache Spark > Inline type hints for python/pyspark/sql/dataframe.p

[jira] [Assigned] (SPARK-36885) Inline type hints for python/pyspark/sql/dataframe.py

[ https://issues.apache.org/jira/browse/SPARK-36885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-36885: -- Assignee: (was: Apache Spark) > Inline type hints for python/pyspark/sql/

[jira] [Assigned] (SPARK-37408) Inline type hints for python/pyspark/ml/image.py

[ https://issues.apache.org/jira/browse/SPARK-37408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37408: -- Assignee: Maciej Szymkiewicz (was: Apache Spark) > Inline type hints for pyt

[jira] [Updated] (SPARK-38082) Update minimum numpy version

[ https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-38082: --- Description: Currently, we use set numpy version in {{extras_require}} to be {{>=1.7

[jira] [Updated] (SPARK-38082) Update minimum numpy version

[ https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-38082: --- Description: Currently, we use set numpy version in {{extras_require}} to be {{>=1.7

[jira] [Updated] (SPARK-38082) Update minimum numpy version

[ https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-38082: --- Description: Currently, we use set numpy version in {{extras_require}} to be {{>=1.7

[jira] [Updated] (SPARK-38082) Update minimum numpy version

[ https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-38082: --- Description: Currently, we use set numpy version in {{extras_require}} to be {{>=1.7

[jira] [Commented] (SPARK-38082) Update minimum numpy version

[ https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485327#comment-17485327 ] Maciej Szymkiewicz commented on SPARK-38082: cc [~hyukjin.kwon] [~WeichenXu1

[jira] [Updated] (SPARK-38082) Update minimum numpy version

[ https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-38082: --- Description: Currently, we use set numpy version in {{extras_require}} to be {{>=1.7

[jira] [Updated] (SPARK-38082) Update minimum numpy version

[ https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-38082: --- Description: Currently, we use set numpy version in {{extras_require}} to be {{>=1.7

[jira] [Created] (SPARK-38082) Update minimum numpy version

Maciej Szymkiewicz created SPARK-38082: -- Summary: Update minimum numpy version Key: SPARK-38082 URL: https://issues.apache.org/jira/browse/SPARK-38082 Project: Spark Issue Type: Improvem

[jira] [Assigned] (SPARK-38067) Inconsistent missing values handling in Pandas on Spark to_json

[ https://issues.apache.org/jira/browse/SPARK-38067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-38067: -- Assignee: Bjørn Jørgensen > Inconsistent missing values handling in Pandas on

[jira] [Resolved] (SPARK-38067) Inconsistent missing values handling in Pandas on Spark to_json

[ https://issues.apache.org/jira/browse/SPARK-38067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-38067. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35296

[jira] [Updated] (SPARK-38067) Inconsistent missing values handling in Pandas on Spark to_json

[ https://issues.apache.org/jira/browse/SPARK-38067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-38067: --- Description: If {{ps.DataFrame.to_json}} is called without {{path}} argument, missin

[jira] [Updated] (SPARK-38067) Inconsistent missing values handling in Pandas on Spark to_json

[ https://issues.apache.org/jira/browse/SPARK-38067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-38067: --- Summary: Inconsistent missing values handling in Pandas on Spark to_json (was: Pand

[jira] [Commented] (SPARK-38073) NameError: name 'sc' is not defined when running driver with IPyhon and Pyhon > 3.7

[ https://issues.apache.org/jira/browse/SPARK-38073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484378#comment-17484378 ] Maciej Szymkiewicz commented on SPARK-38073: [~holden] FYI, since you questi

[jira] [Created] (SPARK-38073) NameError: name 'sc' is not defined when running driver with IPyhon and Pyhon > 3.7

Maciej Szymkiewicz created SPARK-38073: -- Summary: NameError: name 'sc' is not defined when running driver with IPyhon and Pyhon > 3.7 Key: SPARK-38073 URL: https://issues.apache.org/jira/browse/SPARK-38073

[jira] [Resolved] (SPARK-37288) Backport update pyspark.since annotation to 3.1 and 3.2

[ https://issues.apache.org/jira/browse/SPARK-37288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37288. Assignee: Maciej Szymkiewicz Resolution: Fixed Issue resolved by pull reques

[jira] [Assigned] (SPARK-37153) Inline type hints for python/pyspark/profiler.py

[ https://issues.apache.org/jira/browse/SPARK-37153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37153: -- Assignee: dch nguyen > Inline type hints for python/pyspark/profiler.py > ---

[jira] [Resolved] (SPARK-37153) Inline type hints for python/pyspark/profiler.py

[ https://issues.apache.org/jira/browse/SPARK-37153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37153. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34731

[jira] [Created] (SPARK-37992) Restore mypy version check in dev/lint-python

Maciej Szymkiewicz created SPARK-37992: -- Summary: Restore mypy version check in dev/lint-python Key: SPARK-37992 URL: https://issues.apache.org/jira/browse/SPARK-37992 Project: Spark Iss

[jira] [Resolved] (SPARK-37981) Deletes columns with all Null as default.

[ https://issues.apache.org/jira/browse/SPARK-37981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37981. Resolution: Duplicate > Deletes columns with all Null as default. > --

[jira] [Comment Edited] (SPARK-37981) Deletes columns with all Null as default.

[ https://issues.apache.org/jira/browse/SPARK-37981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480291#comment-17480291 ] Maciej Szymkiewicz edited comment on SPARK-37981 at 1/21/22, 11:37 PM: ---

[jira] [Commented] (SPARK-37981) Deletes columns with all Null as default.

[ https://issues.apache.org/jira/browse/SPARK-37981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480291#comment-17480291 ] Maciej Szymkiewicz commented on SPARK-37981: This doesn't seem valid. {{dro

[jira] [Resolved] (SPARK-37972) Typing incompatibilities with numpy==1.22.x

[ https://issues.apache.org/jira/browse/SPARK-37972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37972. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35261

[jira] [Assigned] (SPARK-37972) Typing incompatibilities with numpy==1.22.x

[ https://issues.apache.org/jira/browse/SPARK-37972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37972: -- Assignee: Maciej Szymkiewicz > Typing incompatibilities with numpy==1.22.x >

[jira] [Resolved] (SPARK-37083) Inline type hints for python/pyspark/accumulators.py

[ https://issues.apache.org/jira/browse/SPARK-37083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37083. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34363

[jira] [Assigned] (SPARK-37083) Inline type hints for python/pyspark/accumulators.py

[ https://issues.apache.org/jira/browse/SPARK-37083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37083: -- Assignee: dch nguyen > Inline type hints for python/pyspark/accumulators.py >

[jira] [Updated] (SPARK-37972) Typing incompatibilities with numpy==1.22.x

[ https://issues.apache.org/jira/browse/SPARK-37972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37972: --- Description: When type checked against {{numpy==1.22}} mypy detects following issues

[jira] [Updated] (SPARK-37972) Typing incompatibilities with numpy==1.22.x

[ https://issues.apache.org/jira/browse/SPARK-37972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37972: --- Summary: Typing incompatibilities with numpy==1.22.x (was: Typing incompatibilities

[jira] [Created] (SPARK-37972) Typing incompatibilities with numpy~=1.22

Maciej Szymkiewicz created SPARK-37972: -- Summary: Typing incompatibilities with numpy~=1.22 Key: SPARK-37972 URL: https://issues.apache.org/jira/browse/SPARK-37972 Project: Spark Issue T

[jira] [Commented] (SPARK-37498) test_reuse_worker_of_parallelize_range is flaky

[ https://issues.apache.org/jira/browse/SPARK-37498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17476385#comment-17476385 ] Maciej Szymkiewicz commented on SPARK-37498: ??maybe we should add eventuall

[jira] [Assigned] (SPARK-37902) Update annotations to resolve issues detected with mypy==0.931

[ https://issues.apache.org/jira/browse/SPARK-37902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37902: -- Assignee: Maciej Szymkiewicz > Update annotations to resolve issues detected

[jira] [Resolved] (SPARK-37902) Update annotations to resolve issues detected with mypy==0.931

[ https://issues.apache.org/jira/browse/SPARK-37902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37902. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35199

[jira] [Resolved] (SPARK-37429) Inline type hints for python/pyspark/mllib/linalg/__init__.py

[ https://issues.apache.org/jira/browse/SPARK-37429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37429. Fix Version/s: 3.3.0 Assignee: Maciej Szymkiewicz Resolution: Fixed

[jira] [Created] (SPARK-37909) Restore F401 checks

Maciej Szymkiewicz created SPARK-37909: -- Summary: Restore F401 checks Key: SPARK-37909 URL: https://issues.apache.org/jira/browse/SPARK-37909 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-37909) Restore F403 checks

[ https://issues.apache.org/jira/browse/SPARK-37909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37909: --- Summary: Restore F403 checks (was: Restore F401 checks) > Restore F403 checks > ---

[jira] [Created] (SPARK-37902) Update annotations to resolve issues detected with mypy==0.931

Maciej Szymkiewicz created SPARK-37902: -- Summary: Update annotations to resolve issues detected with mypy==0.931 Key: SPARK-37902 URL: https://issues.apache.org/jira/browse/SPARK-37902 Project: S

[jira] [Assigned] (SPARK-37686) Migrate remaining pyspark.sql.functions to _invoke_* style

[ https://issues.apache.org/jira/browse/SPARK-37686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37686: -- Assignee: Maciej Szymkiewicz > Migrate remaining pyspark.sql.functions to _in

[jira] [Resolved] (SPARK-37686) Migrate remaining pyspark.sql.functions to _invoke_* style

[ https://issues.apache.org/jira/browse/SPARK-37686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37686. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34951

[jira] [Resolved] (SPARK-37418) Inline type hints for python/pyspark/ml/param/__init__.py

[ https://issues.apache.org/jira/browse/SPARK-37418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37418. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35136

[jira] [Assigned] (SPARK-37418) Inline type hints for python/pyspark/ml/param/__init__.py

[ https://issues.apache.org/jira/browse/SPARK-37418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37418: -- Assignee: Maciej Szymkiewicz > Inline type hints for python/pyspark/ml/param/

[jira] [Created] (SPARK-37869) Update pytest-mypy-plugins to 1.9.3

Maciej Szymkiewicz created SPARK-37869: -- Summary: Update pytest-mypy-plugins to 1.9.3 Key: SPARK-37869 URL: https://issues.apache.org/jira/browse/SPARK-37869 Project: Spark Issue Type: I

[jira] [Resolved] (SPARK-37852) Enable flake's E741 rule in PySpark

[ https://issues.apache.org/jira/browse/SPARK-37852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37852. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35152

[jira] [Assigned] (SPARK-37852) Enable flake's E741 rule in PySpark

[ https://issues.apache.org/jira/browse/SPARK-37852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37852: -- Assignee: Hyukjin Kwon > Enable flake's E741 rule in PySpark > --

[jira] [Assigned] (SPARK-37085) Missing overloads in functions accepting both varargs and single arg collection

[ https://issues.apache.org/jira/browse/SPARK-37085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37085: -- Assignee: Maciej Szymkiewicz > Missing overloads in functions accepting both

[jira] [Resolved] (SPARK-37085) Missing overloads in functions accepting both varargs and single arg collection

[ https://issues.apache.org/jira/browse/SPARK-37085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37085. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34354

[jira] [Commented] (SPARK-37417) Inline type hints for python/pyspark/ml/linalg/__init__.py

[ https://issues.apache.org/jira/browse/SPARK-37417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17470783#comment-17470783 ] Maciej Szymkiewicz commented on SPARK-37417: {{ml}} and {{mllib}} linalg typ

[jira] [Commented] (SPARK-37415) Inline type hints for python/pyspark/ml/util.py

[ https://issues.apache.org/jira/browse/SPARK-37415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17470768#comment-17470768 ] Maciej Szymkiewicz commented on SPARK-37415: I am going to handle this one,

[jira] [Commented] (SPARK-37416) Inline type hints for python/pyspark/ml/wrapper.py

[ https://issues.apache.org/jira/browse/SPARK-37416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17470767#comment-17470767 ] Maciej Szymkiewicz commented on SPARK-37416: I am going to handle this one,

[jira] [Resolved] (SPARK-37419) Inline type hints for python/pyspark/ml/param/shared.py

[ https://issues.apache.org/jira/browse/SPARK-37419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37419. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34674

[jira] [Assigned] (SPARK-37419) Inline type hints for python/pyspark/ml/param/shared.py

[ https://issues.apache.org/jira/browse/SPARK-37419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37419: -- Assignee: Maciej Szymkiewicz > Inline type hints for python/pyspark/ml/param/

[jira] [Commented] (SPARK-37397) Inline type hints for python/pyspark/ml/base.py

[ https://issues.apache.org/jira/browse/SPARK-37397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17470654#comment-17470654 ] Maciej Szymkiewicz commented on SPARK-37397: I'll handle this once SPARK-374

[jira] [Assigned] (SPARK-37397) Inline type hints for python/pyspark/ml/base.py

[ https://issues.apache.org/jira/browse/SPARK-37397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37397: -- Assignee: (was: Maciej Szymkiewicz) > Inline type hints for python/pyspar

[jira] [Assigned] (SPARK-37397) Inline type hints for python/pyspark/ml/base.py

[ https://issues.apache.org/jira/browse/SPARK-37397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37397: -- Assignee: Maciej Szymkiewicz > Inline type hints for python/pyspark/ml/base.p

[jira] [Commented] (SPARK-37601) Could/should sql.DataFrame.transform accept function parameters?

[ https://issues.apache.org/jira/browse/SPARK-37601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17470304#comment-17470304 ] Maciej Szymkiewicz commented on SPARK-37601: Thanks for pointing this out [~

[jira] [Resolved] (SPARK-37788) ColumnOrName vs Column in PySpark Functions module

[ https://issues.apache.org/jira/browse/SPARK-37788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37788. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35071

[jira] [Assigned] (SPARK-37788) ColumnOrName vs Column in PySpark Functions module

[ https://issues.apache.org/jira/browse/SPARK-37788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37788: -- Assignee: Daniel Davies > ColumnOrName vs Column in PySpark Functions module

[jira] [Commented] (SPARK-37425) Inline type hints for python/pyspark/mllib/recommendation.py

[ https://issues.apache.org/jira/browse/SPARK-37425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468609#comment-17468609 ] Maciej Szymkiewicz commented on SPARK-37425: Hi [~amirkdv]. For the whole {{

[jira] [Resolved] (SPARK-37601) Could/should sql.DataFrame.transform accept function parameters?

[ https://issues.apache.org/jira/browse/SPARK-37601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37601. Fix Version/s: 3.3.0 Resolution: Fixed > Could/should sql.DataFrame.transfo

[jira] [Assigned] (SPARK-37738) PySpark date_add only accepts an integer as it's second parameter

[ https://issues.apache.org/jira/browse/SPARK-37738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37738: -- Assignee: Daniel Davies > PySpark date_add only accepts an integer as it's se

[jira] [Assigned] (SPARK-37738) PySpark date_add only accepts an integer as it's second parameter

[ https://issues.apache.org/jira/browse/SPARK-37738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37738: -- Assignee: (was: Maciej Szymkiewicz) > PySpark date_add only accepts an in

[jira] [Assigned] (SPARK-37738) PySpark date_add only accepts an integer as it's second parameter

[ https://issues.apache.org/jira/browse/SPARK-37738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37738: -- Assignee: Maciej Szymkiewicz > PySpark date_add only accepts an integer as it

[jira] [Resolved] (SPARK-37738) PySpark date_add only accepts an integer as it's second parameter

[ https://issues.apache.org/jira/browse/SPARK-37738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37738. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35032

[jira] [Assigned] (SPARK-37754) Fix black version in dev/reformat-python

[ https://issues.apache.org/jira/browse/SPARK-37754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37754: -- Assignee: Chia-Ping Tsai > Fix black version in dev/reformat-python > ---

[jira] [Resolved] (SPARK-37754) Fix black version in dev/reformat-python

[ https://issues.apache.org/jira/browse/SPARK-37754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37754. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35033

[jira] [Created] (SPARK-37737) Update Black to 21.12.b0

Maciej Szymkiewicz created SPARK-37737: -- Summary: Update Black to 21.12.b0 Key: SPARK-37737 URL: https://issues.apache.org/jira/browse/SPARK-37737 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-37407) Inline type hints for python/pyspark/ml/functions.py

[ https://issues.apache.org/jira/browse/SPARK-37407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37407. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34678

[jira] [Assigned] (SPARK-37407) Inline type hints for python/pyspark/ml/functions.py

[ https://issues.apache.org/jira/browse/SPARK-37407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37407: -- Assignee: dch nguyen > Inline type hints for python/pyspark/ml/functions.py >

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464189#comment-17464189 ] Maciej Szymkiewicz commented on SPARK-34544: Thanks [~hyukjin.kwon]! > pysp

[jira] [Resolved] (SPARK-37704) Update mypy in tests to 0.920

[ https://issues.apache.org/jira/browse/SPARK-37704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37704. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34972

[jira] [Assigned] (SPARK-37704) Update mypy in tests to 0.920

[ https://issues.apache.org/jira/browse/SPARK-37704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37704: -- Assignee: Maciej Szymkiewicz > Update mypy in tests to 0.920 > --

[jira] [Updated] (SPARK-37704) Update mypy in tests to 0.920

[ https://issues.apache.org/jira/browse/SPARK-37704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37704: --- Component/s: Build (was: Project Infra) > Update mypy in tests

[jira] [Updated] (SPARK-37704) Update mypy in tests to 0.920

[ https://issues.apache.org/jira/browse/SPARK-37704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37704: --- Component/s: (was: Build) > Update mypy in tests to 0.920 >

[jira] [Created] (SPARK-37704) Update mypy in tests to 0.920

Maciej Szymkiewicz created SPARK-37704: -- Summary: Update mypy in tests to 0.920 Key: SPARK-37704 URL: https://issues.apache.org/jira/browse/SPARK-37704 Project: Spark Issue Type: Improve

[jira] [Created] (SPARK-37686) Migrare remaining pyspark.sql.functions to _invoke_* style

Maciej Szymkiewicz created SPARK-37686: -- Summary: Migrare remaining pyspark.sql.functions to _invoke_* style Key: SPARK-37686 URL: https://issues.apache.org/jira/browse/SPARK-37686 Project: Spark

[jira] [Updated] (SPARK-37686) Migrate remaining pyspark.sql.functions to _invoke_* style

[ https://issues.apache.org/jira/browse/SPARK-37686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37686: --- Summary: Migrate remaining pyspark.sql.functions to _invoke_* style (was: Migrare r

[jira] [Commented] (SPARK-37683) Nondeterministic type checker error in data tests

[ https://issues.apache.org/jira/browse/SPARK-37683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17461881#comment-17461881 ] Maciej Szymkiewicz commented on SPARK-37683: cc [~hyukjin.kwon] FYI > Nonde

[jira] [Created] (SPARK-37683) Nondeterministic type checker error in data tests

Maciej Szymkiewicz created SPARK-37683: -- Summary: Nondeterministic type checker error in data tests Key: SPARK-37683 URL: https://issues.apache.org/jira/browse/SPARK-37683 Project: Spark

[jira] [Commented] (SPARK-37678) Incorrect annotations in SeriesGroupBy._cleanup_and_return

[ https://issues.apache.org/jira/browse/SPARK-37678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17461679#comment-17461679 ] Maciej Szymkiewicz commented on SPARK-37678: ??It's an actual implementation

[jira] [Resolved] (SPARK-37620) Use more precise types for SparkContext Optional fields (i.e. _gateway, _jvm)

[ https://issues.apache.org/jira/browse/SPARK-37620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37620. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34892

[jira] [Assigned] (SPARK-37620) Use more precise types for SparkContext Optional fields (i.e. _gateway, _jvm)

[ https://issues.apache.org/jira/browse/SPARK-37620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37620: -- Assignee: Byron Hsu > Use more precise types for SparkContext Optional fields

[jira] [Commented] (SPARK-37678) Incorrect annotations in SeriesGroupBy._cleanup_and_return

[ https://issues.apache.org/jira/browse/SPARK-37678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17461609#comment-17461609 ] Maciej Szymkiewicz commented on SPARK-37678: cc [~ueshin] Do you have any th

[jira] [Created] (SPARK-37678) Incorrect annotations in SeriesGroupBy._cleanup_and_return

Maciej Szymkiewicz created SPARK-37678: -- Summary: Incorrect annotations in SeriesGroupBy._cleanup_and_return Key: SPARK-37678 URL: https://issues.apache.org/jira/browse/SPARK-37678 Project: Spar

[jira] [Updated] (SPARK-37668) 'Index' object has no attribute 'levels' in pyspark.pandas.frame.DataFrame.insert

[ https://issues.apache.org/jira/browse/SPARK-37668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37668: --- Summary: 'Index' object has no attribute 'levels' in pyspark.pandas.frame.DataFrame

[jira] [Commented] (SPARK-37668) 'Index' object has no attribute 'levels'

[ https://issues.apache.org/jira/browse/SPARK-37668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17461039#comment-17461039 ] Maciej Szymkiewicz commented on SPARK-37668: cc [~hyukjin.kwon], [~itholic],

[jira] [Created] (SPARK-37668) 'Index' object has no attribute 'levels'

Maciej Szymkiewicz created SPARK-37668: -- Summary: 'Index' object has no attribute 'levels' Key: SPARK-37668 URL: https://issues.apache.org/jira/browse/SPARK-37668 Project: Spark Issue Ty

[jira] [Updated] (SPARK-37620) Use more precise types for SparkContext Optional fields (i.e. _gateway, _jvm)

[ https://issues.apache.org/jira/browse/SPARK-37620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37620: --- Parent: SPARK-37094 Issue Type: Sub-task (was: Improvement) > Use more prec

[jira] [Resolved] (SPARK-37146) Inline type hints for python/pyspark/__init__.py

[ https://issues.apache.org/jira/browse/SPARK-37146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37146. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34433

[jira] [Assigned] (SPARK-37146) Inline type hints for python/pyspark/__init__.py

[ https://issues.apache.org/jira/browse/SPARK-37146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37146: -- Assignee: dch nguyen > Inline type hints for python/pyspark/__init__.py > ---

[jira] [Updated] (SPARK-37632) Drop code targetting Python < 3.7

[ https://issues.apache.org/jira/browse/SPARK-37632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37632: --- Description: We should drop parts of code that target Python 3.6 (or less if these a

[jira] [Created] (SPARK-37632) Drop code targetting Python < 3.7

Maciej Szymkiewicz created SPARK-37632: -- Summary: Drop code targetting Python < 3.7 Key: SPARK-37632 URL: https://issues.apache.org/jira/browse/SPARK-37632 Project: Spark Issue Type: Sub

[jira] [Resolved] (SPARK-36969) Inline type hints for SparkContext

[ https://issues.apache.org/jira/browse/SPARK-36969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-36969. Fix Version/s: 3.3.0 Resolution: Duplicate > Inline type hints for SparkCon

[jira] [Commented] (SPARK-37620) Use more precise types for SparkContext Optional fields (i.e. _gateway, _jvm)

[ https://issues.apache.org/jira/browse/SPARK-37620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17458050#comment-17458050 ] Maciej Szymkiewicz commented on SPARK-37620: FYI [~ueshin] [~itholic] [~Xinr

[jira] [Created] (SPARK-37620) Use more precise types for SparkContext Optional fields (i.e. _gateway, _jvm)

Maciej Szymkiewicz created SPARK-37620: -- Summary: Use more precise types for SparkContext Optional fields (i.e. _gateway, _jvm) Key: SPARK-37620 URL: https://issues.apache.org/jira/browse/SPARK-37620

[jira] [Resolved] (SPARK-37152) Inline type hints for python/pyspark/context.py

[ https://issues.apache.org/jira/browse/SPARK-37152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37152. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34466

[jira] [Assigned] (SPARK-37152) Inline type hints for python/pyspark/context.py

[ https://issues.apache.org/jira/browse/SPARK-37152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37152: -- Assignee: Byron Hsu > Inline type hints for python/pyspark/context.py > -

[jira] [Commented] (SPARK-37551) Argument 1 to "rename" of "DataFrame" has incompatible type in pandas.LocIndexerLike.__getitem__

[ https://issues.apache.org/jira/browse/SPARK-37551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17453626#comment-17453626 ] Maciej Szymkiewicz commented on SPARK-37551: cc [~itholic] [~ueshin], since

[jira] [Created] (SPARK-37551) Argument 1 to "rename" of "DataFrame" has incompatible type in pandas.LocIndexerLike.__getitem__

Maciej Szymkiewicz created SPARK-37551: -- Summary: Argument 1 to "rename" of "DataFrame" has incompatible type in pandas.LocIndexerLike.__getitem__ Key: SPARK-37551 URL: https://issues.apache.org/jira/browse/

<    1   2   3   4   5   6   7   >