[jira] [Updated] (SPARK-30491) Enable dependency audit files to tell dependency classifier

2020-01-11 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-30491: - Shepherd: (was: Xinrong Meng) > Enable dependency audit files to tell dependency classifier >

[jira] [Updated] (SPARK-30491) Enable dependency audit files to tell dependency classifier

2020-01-11 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-30491: - Shepherd: Xinrong Meng > Enable dependency audit files to tell dependency classifier >

[jira] [Created] (SPARK-30491) Enable dependency audit files to tell dependency classifier

2020-01-11 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-30491: Summary: Enable dependency audit files to tell dependency classifier Key: SPARK-30491 URL: https://issues.apache.org/jira/browse/SPARK-30491 Project: Spark

[jira] [Updated] (SPARK-30491) Enable dependency audit files to tell dependency classifier

2020-01-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-30491: - Description: Current dependency audit files under `dev/deps` only show jar names. And there

[jira] [Commented] (SPARK-35344) Make conversion from/to literals data-type-based

2021-06-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363906#comment-17363906 ] Xinrong Meng commented on SPARK-35344: -- I'd like to work on this ticket. > Make conversion from/to

[jira] [Updated] (SPARK-35344) Make conversion from/to literals data-type-based

2021-06-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35344: - Description: Aim to achieve {{F}}{{*.*}}{{lit(np}}{{*.*}}{{int64(1)).}} We can define {{def

[jira] [Updated] (SPARK-35614) Make the conversion to pandas data-type-based for ExtensionDtypes

2021-06-14 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35614: - Summary: Make the conversion to pandas data-type-based for ExtensionDtypes (was: Make the

[jira] [Updated] (SPARK-35681) Support arithmetic operators (+, *) among bool Series/Index

2021-06-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35681: - Description: We ought to support arithmetic operators (+, *) between bool Series/Indexes,

[jira] [Updated] (SPARK-35681) Support arithmetic operators (+, *) among bool Series/Index

2021-06-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35681: - Description: We ought to support arithmetic operators (+, *) between bool Series/Indexes,

[jira] [Created] (SPARK-35681) Support arithmetic operators (+, *) among bool Series/Index

2021-06-08 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35681: Summary: Support arithmetic operators (+, *) among bool Series/Index Key: SPARK-35681 URL: https://issues.apache.org/jira/browse/SPARK-35681 Project: Spark

[jira] [Resolved] (SPARK-35681) Support arithmetic operators (+, *) among bool Series/Index

2021-06-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-35681. -- Resolution: Duplicate > Support arithmetic operators (+, *) among bool Series/Index >

[jira] [Updated] (SPARK-35601) Complete arithmetic operators involving bool literals, Series, and Index

2021-06-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35601: - Description: Completing arithmetic operators involving bool literals, Series, and Index

[jira] [Updated] (SPARK-35601) Complete arithmetic operators involving bool literals, Series, and Index

2021-06-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35601: - Summary: Complete arithmetic operators involving bool literals, Series, and Index (was:

[jira] [Updated] (SPARK-35616) Make astype data-type-based

2021-06-10 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35616: - Description: There are many type checks in `astype` methods. Since `DataTypeOps` is introduced,

[jira] [Updated] (SPARK-35344) Support creating a Column of numpy literal value Support creating a Column of numpy literal value in pandas-on-Spark

2021-06-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35344: - Summary: Support creating a Column of numpy literal value Support creating a Column of numpy

[jira] [Updated] (SPARK-35344) Support creating a Column of numpy literal value Support creating a Column of numpy literal value in pandas-on-Spark

2021-06-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35344: - Description: Spark (`lit` function defined in `pyspark.sql.functions`) doesn't support

[jira] [Updated] (SPARK-35344) Support creating a Column of numpy literal value in pandas-on-Spark

2021-06-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35344: - Summary: Support creating a Column of numpy literal value in pandas-on-Spark (was: Support

[jira] [Commented] (SPARK-35342) Introduce DecimalOps

2021-06-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359316#comment-17359316 ] Xinrong Meng commented on SPARK-35342: -- Certainly, thank you! Yikun Jiang (Jira) 于2021年6月8日

[jira] [Created] (SPARK-35705) Adjust pandas-on-spark `test_groupby_multiindex_columns` test for different pandas versions

2021-06-09 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35705: Summary: Adjust pandas-on-spark `test_groupby_multiindex_columns` test for different pandas versions Key: SPARK-35705 URL: https://issues.apache.org/jira/browse/SPARK-35705

[jira] [Created] (SPARK-35340) Standardize TypeError messages for unsupported arithmetic operations

2021-05-07 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35340: Summary: Standardize TypeError messages for unsupported arithmetic operations Key: SPARK-35340 URL: https://issues.apache.org/jira/browse/SPARK-35340 Project: Spark

[jira] [Created] (SPARK-35337) pandas APIs on Spark: Separate basic operations into data type based structures

2021-05-07 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35337: Summary: pandas APIs on Spark: Separate basic operations into data type based structures Key: SPARK-35337 URL: https://issues.apache.org/jira/browse/SPARK-35337

[jira] [Updated] (SPARK-35342) Introduce DecimalOps

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35342: - Description: In PySpark, {{DoubleType}} and {{FloatType}} could have NaN, whereas

[jira] [Updated] (SPARK-35341) Introduce ExtentionDtypeOps

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35341: - Description: extension_dtypes process __and__, __or__, __rand__, and __ror__ differently with

[jira] [Updated] (SPARK-35338) Separate arithmetic operations into data type based structures

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35338: - Description: For arithmetic operations: {code:java} __add__, __sub__, __mul__, __truediv__, 

[jira] [Created] (SPARK-35343) Make conversion from/to pandas data data-type-based

2021-05-07 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35343: Summary: Make conversion from/to pandas data data-type-based Key: SPARK-35343 URL: https://issues.apache.org/jira/browse/SPARK-35343 Project: Spark Issue

[jira] [Created] (SPARK-35344) Make conversion from/to literals data-type-based

2021-05-07 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35344: Summary: Make conversion from/to literals data-type-based Key: SPARK-35344 URL: https://issues.apache.org/jira/browse/SPARK-35344 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35343) Make conversion from/to pandas data-type-based

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35343: - Summary: Make conversion from/to pandas data-type-based (was: Make conversion from/to pandas

[jira] [Updated] (SPARK-35337) pandas APIs on Spark: Separate basic operations into data type based structures

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35337: - Description: Currently, the same basic operation of all data types is defined in one function,

[jira] [Created] (SPARK-35342) Introduce DecimalOps

2021-05-07 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35342: Summary: Introduce DecimalOps Key: SPARK-35342 URL: https://issues.apache.org/jira/browse/SPARK-35342 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35339) Consolidate unit tests for arithmetic operations

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35339: - Description: Unit tests for arithmetic operations are scattered in the codebase:  *

[jira] [Created] (SPARK-35339) Consolidate unit tests for arithmetic operations

2021-05-07 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35339: Summary: Consolidate unit tests for arithmetic operations Key: SPARK-35339 URL: https://issues.apache.org/jira/browse/SPARK-35339 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35340) Standardize TypeError messages for unsupported arithmetic operations

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35340: - Description: Inconsistent TypeError messages are shown for unsupported arithmetic operations.

[jira] [Updated] (SPARK-35340) Standardize TypeError messages for unsupported arithmetic operations

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35340: - Description: Inconsistent TypeError messages are shown for unsupported arithmetic operations.

[jira] [Created] (SPARK-35338) Separate arithmetic operations into data type based structures

2021-05-07 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35338: Summary: Separate arithmetic operations into data type based structures Key: SPARK-35338 URL: https://issues.apache.org/jira/browse/SPARK-35338 Project: Spark

[jira] [Created] (SPARK-35341) Introduce ExtentionDtypeOps

2021-05-07 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35341: Summary: Introduce ExtentionDtypeOps Key: SPARK-35341 URL: https://issues.apache.org/jira/browse/SPARK-35341 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35341) Introduce ExtentionDtypeOps

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35341: - Description: Now __and__, __or__,_ _rand__, and __ror__ are not data type based. So we would

[jira] [Updated] (SPARK-35341) Introduce ExtentionDtypeOps

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35341: - Description: Now __and__, __or__, __rand__, and __ror__ are not data type based. So we would

[jira] [Updated] (SPARK-35314) Support arithmetic operations against bool Series

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35314: - Summary: Support arithmetic operations against bool Series (was: Support arithmetic operations

[jira] [Updated] (SPARK-35314) Support arithmetic operations against bool Series in pandas-on-Spark

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35314: - Parent: SPARK-35337 Issue Type: Sub-task (was: Story) > Support arithmetic operations

[jira] [Updated] (SPARK-35337) pandas APIs on Spark: Separate basic operations into data type based structures

2021-05-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35337: - Description: Currently, the same basic operation of all data types is defined in one function,

[jira] [Created] (SPARK-35452) Introduce ComplexTypeOps

2021-05-19 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35452: Summary: Introduce ComplexTypeOps Key: SPARK-35452 URL: https://issues.apache.org/jira/browse/SPARK-35452 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35314) Support arithmetic operations against bool IndexOpsMixin

2021-05-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35314: - Summary: Support arithmetic operations against bool IndexOpsMixin (was: Support arithmetic

[jira] [Updated] (SPARK-35452) Introduce ComplexTypeOps

2021-05-24 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35452: - Description: StructType, ArrayType, and MapType are not accepted by DataTypeOps now. We should

[jira] [Updated] (SPARK-35452) Introduce ComplexOps for complex types

2021-05-24 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35452: - Summary: Introduce ComplexOps for complex types (was: Introduce ComplexTypeOps) > Introduce

[jira] [Updated] (SPARK-35452) ntroduce ArrayOps, MapOps and StructOps

2021-05-24 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35452: - Summary: ntroduce ArrayOps, MapOps and StructOps (was: Introduce ComplexOps for complex types)

[jira] [Created] (SPARK-35521) List Python 3.8 installed libraries in build_and_test workflow

2021-05-25 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35521: Summary: List Python 3.8 installed libraries in build_and_test workflow Key: SPARK-35521 URL: https://issues.apache.org/jira/browse/SPARK-35521 Project: Spark

[jira] [Updated] (SPARK-35341) Introduce BooleanExtensionOps

2021-05-28 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35341: - Description: {{Now ___and, __or,_ ___rand, and __ror___ are not data type based.}}

[jira] [Created] (SPARK-35606) List Python 3.9 installed libraries in build_and_test workflow

2021-06-01 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35606: Summary: List Python 3.9 installed libraries in build_and_test workflow Key: SPARK-35606 URL: https://issues.apache.org/jira/browse/SPARK-35606 Project: Spark

[jira] [Created] (SPARK-35616) Make astype data-type-based

2021-06-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35616: Summary: Make astype data-type-based Key: SPARK-35616 URL: https://issues.apache.org/jira/browse/SPARK-35616 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35341) Introduce ExtentionDtypeOps

2021-05-25 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35341: - Description: {{Now ___and, __or,_ ___rand, and __ror___ are not data type based.}}

[jira] [Created] (SPARK-35522) Introduce BinaryOps for BinaryType

2021-05-25 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35522: Summary: Introduce BinaryOps for BinaryType Key: SPARK-35522 URL: https://issues.apache.org/jira/browse/SPARK-35522 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35601) Support arithmetic operations against bool

2021-06-01 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35601: Summary: Support arithmetic operations against bool Key: SPARK-35601 URL: https://issues.apache.org/jira/browse/SPARK-35601 Project: Spark Issue Type:

[jira] [Created] (SPARK-35599) Introduce a way to compare series of array for older pandas

2021-06-01 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35599: Summary: Introduce a way to compare series of array for older pandas Key: SPARK-35599 URL: https://issues.apache.org/jira/browse/SPARK-35599 Project: Spark

[jira] [Updated] (SPARK-35343) Make conversion from/to pandas data-type-based

2021-06-01 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35343: - Description: Make the conversion from/to pandas data-type-based. > Make conversion from/to

[jira] [Created] (SPARK-35614) Make the conversion from/to pandas data-type-based for ExtensionDtypes

2021-06-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35614: Summary: Make the conversion from/to pandas data-type-based for ExtensionDtypes Key: SPARK-35614 URL: https://issues.apache.org/jira/browse/SPARK-35614 Project:

[jira] [Updated] (SPARK-35343) Make the conversion from/to pandas data-type-based for non-ExtensionDtypes

2021-06-02 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35343: - Summary: Make the conversion from/to pandas data-type-based for non-ExtensionDtypes (was: Make

[jira] [Updated] (SPARK-35343) Make the conversion from/to pandas data-type-based for non-ExtensionDtypes

2021-06-02 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35343: - Description: Make the conversion from/to pandas data-type-based.   Ops class per

[jira] [Updated] (SPARK-35343) Make the conversion from/to pandas data-type-based for non-ExtensionDtypes

2021-06-02 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35343: - Description: The conversion from/to pandas includes logic for checking data types and behaving

[jira] [Updated] (SPARK-35614) Make the conversion from/to pandas data-type-based for ExtensionDtypes

2021-06-02 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35614: - Description: The conversion from/to pandas includes logic for checking ExtensionDtypes data

[jira] [Updated] (SPARK-35614) Make the conversion from/to pandas data-type-based for ExtensionDtypes

2021-06-02 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35614: - Description: The conversion from/to pandas includes logic for checking ExtensionDtypes data

[jira] [Created] (SPARK-35615) Refactor python/pyspark/pandas/base.py

2021-06-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35615: Summary: Refactor python/pyspark/pandas/base.py Key: SPARK-35615 URL: https://issues.apache.org/jira/browse/SPARK-35615 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35615) Refactor python/pyspark/pandas/base.py for better abstraction

2021-06-02 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35615: - Summary: Refactor python/pyspark/pandas/base.py for better abstraction (was: Refactor

[jira] [Updated] (SPARK-35615) Refactor python/pyspark/pandas/base.py

2021-06-02 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35615: - Description: python/pyspark/pandas/data_type_ops/base.py and python/pyspark/pandas/base.py

[jira] [Created] (SPARK-36035) Make `test_astype` work with all pandas versions in pandas API on Spark

2021-07-07 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36035: Summary: Make `test_astype` work with all pandas versions in pandas API on Spark Key: SPARK-36035 URL: https://issues.apache.org/jira/browse/SPARK-36035 Project:

[jira] [Updated] (SPARK-36035) Adjust `test_astype`, `test_neg` for old pandas versions

2021-07-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36035: - Description: * test_astype For pandas < 1.1.0, declaring or converting to StringDtype was in

[jira] [Updated] (SPARK-36035) Adjust `test_astype`, `test_neg` for old pandas versions

2021-07-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36035: - Description: * test_astype For pandas < 1.1.0, declaring or converting to StringDtype was in

[jira] [Updated] (SPARK-36035) Adjust test_astype, test_neg for old pandas versions

2021-07-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36035: - Description: * test_astype For pandas < 1.1.0, declaring or converting to StringDtype was in

[jira] [Updated] (SPARK-36035) Adjust test_astype, test_neg for old pandas versions

2021-07-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36035: - Summary: Adjust test_astype, test_neg for old pandas versions (was: Make `test_astype` work

[jira] [Updated] (SPARK-36035) Adjust `test_astype`, `test_neg` for old pandas versions

2021-07-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36035: - Summary: Adjust `test_astype`, `test_neg` for old pandas versions (was: Adjust test_astype,

[jira] [Updated] (SPARK-36035) Adjust `test_astype`, `test_neg` for old pandas versions

2021-07-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36035: - Description: * test_astype For pandas < 1.1.0, declaring or converting to StringDtype was in

[jira] [Updated] (SPARK-35936) Deprecate Python 3.6 support

2021-06-29 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35936: - Affects Version/s: (was: 3.1.3) > Deprecate Python 3.6 support >

[jira] [Created] (SPARK-35936) Deprecate Python 3.6 support

2021-06-29 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35936: Summary: Deprecate Python 3.6 support Key: SPARK-35936 URL: https://issues.apache.org/jira/browse/SPARK-35936 Project: Spark Issue Type: Story

[jira] [Created] (SPARK-35997) Implement comparison operators for CategoricalDtype in pandas API on Spark

2021-07-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35997: Summary: Implement comparison operators for CategoricalDtype in pandas API on Spark Key: SPARK-35997 URL: https://issues.apache.org/jira/browse/SPARK-35997 Project:

[jira] [Created] (SPARK-36001) Assume result's index to be disordered in tests with operations on different Series

2021-07-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36001: Summary: Assume result's index to be disordered in tests with operations on different Series Key: SPARK-36001 URL: https://issues.apache.org/jira/browse/SPARK-36001

[jira] [Commented] (SPARK-36002) Consolidate tests for data-type-based operations of decimal Series

2021-07-02 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373839#comment-17373839 ] Xinrong Meng commented on SPARK-36002: -- How do you think about that? [~yikunkero] > Consolidate

[jira] [Created] (SPARK-36000) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

2021-07-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36000: Summary: Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled Key: SPARK-36000 URL: https://issues.apache.org/jira/browse/SPARK-36000

[jira] [Created] (SPARK-36002) Consolidate tests for data-type-based operations of decimal Series

2021-07-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36002: Summary: Consolidate tests for data-type-based operations of decimal Series Key: SPARK-36002 URL: https://issues.apache.org/jira/browse/SPARK-36002 Project: Spark

[jira] [Created] (SPARK-36003) Implement unary operator `invert` of numeric ps.Series/Index

2021-07-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36003: Summary: Implement unary operator `invert` of numeric ps.Series/Index Key: SPARK-36003 URL: https://issues.apache.org/jira/browse/SPARK-36003 Project: Spark

[jira] [Created] (SPARK-35939) Deprecate Python 3.6 in Spark documentation

2021-06-29 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35939: Summary: Deprecate Python 3.6 in Spark documentation Key: SPARK-35939 URL: https://issues.apache.org/jira/browse/SPARK-35939 Project: Spark Issue Type:

[jira] [Created] (SPARK-35938) Add deprecation warning for Python 3.6

2021-06-29 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35938: Summary: Add deprecation warning for Python 3.6 Key: SPARK-35938 URL: https://issues.apache.org/jira/browse/SPARK-35938 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35339) Improve unit tests for data-type-based basic operations

2021-06-25 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35339: - Summary: Improve unit tests for data-type-based basic operations (was: Consolidate unit tests

[jira] [Created] (SPARK-35976) Adjust `astype` method for ExtensionDtype in pandas API on Spark

2021-07-01 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35976: Summary: Adjust `astype` method for ExtensionDtype in pandas API on Spark Key: SPARK-35976 URL: https://issues.apache.org/jira/browse/SPARK-35976 Project: Spark

[jira] [Resolved] (SPARK-35337) pandas API on Spark: Separate basic operations into data type based structures

2021-07-12 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-35337. -- Resolution: Done > pandas API on Spark: Separate basic operations into data type based

[jira] [Updated] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36104: - Description: The spark data type and nullability must be the same as the original when 

[jira] [Created] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36104: Summary: Manage InternalField in DataTypeOps.neg/abs Key: SPARK-36104 URL: https://issues.apache.org/jira/browse/SPARK-36104 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35615) Make unary and comparison operators data-type-based

2021-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35615: - Description: We have been refactoring basic operators to be data-type-based for readability,

[jira] [Updated] (SPARK-35615) Make unary and comparison operators data-type-based

2021-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35615: - Summary: Make unary and comparison operators data-type-based (was: Refactor

[jira] [Commented] (SPARK-36002) Consolidate tests for data-type-based operations of decimal Series

2021-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17375826#comment-17375826 ] Xinrong Meng commented on SPARK-36002: -- Thank you [~yikunkero]! > Consolidate tests for

[jira] [Updated] (SPARK-35340) Standardize TypeError messages for unsupported basic operations

2021-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35340: - Description: Inconsistent TypeError messages are shown for unsupported data-type-based basic

[jira] [Updated] (SPARK-35340) Standardize TypeError messages for unsupported basic operations

2021-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35340: - Summary: Standardize TypeError messages for unsupported basic operations (was: Standardize

[jira] [Updated] (SPARK-35340) Standardize TypeError messages for unsupported basic operations

2021-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35340: - Description: Inconsistent TypeError messages are shown for unsupported data-type-based

[jira] [Updated] (SPARK-35340) Standardize TypeError messages for unsupported basic operations

2021-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35340: - Description: Inconsistent TypeError messages are shown for unsupported data-type-based basic

[jira] [Updated] (SPARK-35340) Standardize TypeError messages for unsupported basic operations

2021-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35340: - Description: Inconsistent TypeError messages are shown for unsupported data-type-based basic

[jira] [Created] (SPARK-36077) Support numpy literals as input for pandas-on-Spark APIs

2021-07-09 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36077: Summary: Support numpy literals as input for pandas-on-Spark APIs Key: SPARK-36077 URL: https://issues.apache.org/jira/browse/SPARK-36077 Project: Spark

[jira] [Created] (SPARK-36078) Complete mappings between numpy literals and Spark data types

2021-07-09 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36078: Summary: Complete mappings between numpy literals and Spark data types Key: SPARK-36078 URL: https://issues.apache.org/jira/browse/SPARK-36078 Project: Spark

[jira] [Updated] (SPARK-36003) Implement unary operator `invert` of integral ps.Series/Index

2021-07-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36003: - Summary: Implement unary operator `invert` of integral ps.Series/Index (was: Implement unary

[jira] [Updated] (SPARK-36003) Implement unary operator `invert` of integral ps.Series/Index

2021-07-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36003: - Description:   {code:java} >>> ~ps.Series([1, 2, 3]) Traceback (most recent call last): ...

[jira] [Updated] (SPARK-35615) Refactor python/pyspark/pandas/base.py for better abstraction

2021-06-30 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35615: - Description: python/pyspark/pandas/data_type_ops/base.py and python/pyspark/pandas/base.py

[jira] [Updated] (SPARK-35615) Refactor python/pyspark/pandas/base.py for better abstraction

2021-06-30 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35615: - Description: python/pyspark/pandas/data_type_ops/base.py and python/pyspark/pandas/base.py

[jira] [Updated] (SPARK-35976) Adjust `astype` method for ExtensionDtype in pandas API on Spark

2021-07-01 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35976: - Description: Currently, `astype` method for ExtensionDtype in pandas API on Spark is not

  1   2   3   4   5   6   7   8   9   10   >