[jira] [Created] (SPARK-36270) Change memory settings for enabling GA

2021-07-22 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-36270: --- Summary: Change memory settings for enabling GA Key: SPARK-36270 URL: https://issues.apache.org/jira/browse/SPARK-36270 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-35780) Support DATE/TIMESTAMP literals across the full range

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385939#comment-17385939 ] Apache Spark commented on SPARK-35780: -- User 'linhongliu-db' has created a pull request for this

[jira] [Resolved] (SPARK-36261) Add remove_unused_categories to CategoricalAccessor and CategoricalIndex.

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36261. -- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Fixed in

[jira] [Commented] (SPARK-36269) Fix only set data columns to Hive column names config

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385930#comment-17385930 ] Apache Spark commented on SPARK-36269: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36269) Fix only set data columns to Hive column names config

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36269: Assignee: Apache Spark > Fix only set data columns to Hive column names config >

[jira] [Assigned] (SPARK-36269) Fix only set data columns to Hive column names config

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36269: Assignee: (was: Apache Spark) > Fix only set data columns to Hive column names

[jira] [Created] (SPARK-36269) Fix only set data columns to Hive column names config

2021-07-22 Thread Cheng Su (Jira)
Cheng Su created SPARK-36269: Summary: Fix only set data columns to Hive column names config Key: SPARK-36269 URL: https://issues.apache.org/jira/browse/SPARK-36269 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-36268) Set the lowerbound of mypy version to 0.910

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36268: Assignee: Hyukjin Kwon > Set the lowerbound of mypy version to 0.910 >

[jira] [Resolved] (SPARK-36268) Set the lowerbound of mypy version to 0.910

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36268. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33487

[jira] [Resolved] (SPARK-36248) Add rename_categories to CategoricalAccessor and CategoricalIndex

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36248. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33471

[jira] [Assigned] (SPARK-36248) Add rename_categories to CategoricalAccessor and CategoricalIndex

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36248: Assignee: Xinrong Meng > Add rename_categories to CategoricalAccessor and

[jira] [Assigned] (SPARK-36189) Improve bool, string, numeric DataTypeOps tests by avoiding joins

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36189: Assignee: Xinrong Meng > Improve bool, string, numeric DataTypeOps tests by avoiding

[jira] [Resolved] (SPARK-36189) Improve bool, string, numeric DataTypeOps tests by avoiding joins

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36189. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33402

[jira] [Assigned] (SPARK-36241) support for creating table with void column datatype

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36241: Assignee: (was: Apache Spark) > support for creating table with void column datatype

[jira] [Commented] (SPARK-36241) support for creating table with void column datatype

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385900#comment-17385900 ] Apache Spark commented on SPARK-36241: -- User 'linhongliu-db' has created a pull request for this

[jira] [Assigned] (SPARK-36241) support for creating table with void column datatype

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36241: Assignee: Apache Spark > support for creating table with void column datatype >

[jira] [Updated] (SPARK-36268) Set the lowerbound of mypy version to 0.910

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36268: - Summary: Set the lowerbound of mypy version to 0.910 (was: Set the upperbound of mypy version

[jira] [Updated] (SPARK-36268) Set the upperbound of mypy version to 0.910

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36268: - Description: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141519/console

[jira] [Updated] (SPARK-36268) Set the upperbound of mypy version to 0.910

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36268: - Summary: Set the upperbound of mypy version to 0.910 (was: Make PySpark codebase compatible

[jira] [Resolved] (SPARK-36265) Use __getitem__ instead of getItem to suppress warnings.

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36265. -- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Fixed in

[jira] [Commented] (SPARK-36268) Make PySpark codebase compatible with other versions of mypy

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385858#comment-17385858 ] Apache Spark commented on SPARK-36268: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-36268) Make PySpark codebase compatible with other versions of mypy

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36268: Assignee: (was: Apache Spark) > Make PySpark codebase compatible with other versions

[jira] [Commented] (SPARK-36268) Make PySpark codebase compatible with other versions of mypy

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385854#comment-17385854 ] Apache Spark commented on SPARK-36268: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-36268) Make PySpark codebase compatible with other versions of mypy

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36268: Assignee: Apache Spark > Make PySpark codebase compatible with other versions of mypy >

[jira] [Created] (SPARK-36268) Make PySpark codebase compatible with other versions of mypy

2021-07-22 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-36268: Summary: Make PySpark codebase compatible with other versions of mypy Key: SPARK-36268 URL: https://issues.apache.org/jira/browse/SPARK-36268 Project: Spark

[jira] [Updated] (SPARK-36267) Clean up CategoricalAccessor and CategoricalIndex.

2021-07-22 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36267: -- Description: - Clean up the classes - Add deprecation warnings > Clean up

[jira] [Created] (SPARK-36267) Clean up CategoricalAccessor and CategoricalIndex.

2021-07-22 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36267: - Summary: Clean up CategoricalAccessor and CategoricalIndex. Key: SPARK-36267 URL: https://issues.apache.org/jira/browse/SPARK-36267 Project: Spark Issue

[jira] [Assigned] (SPARK-36266) Rename classes in shuffle RPC used for block push operations

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36266: Assignee: Apache Spark > Rename classes in shuffle RPC used for block push operations >

[jira] [Commented] (SPARK-36266) Rename classes in shuffle RPC used for block push operations

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385827#comment-17385827 ] Apache Spark commented on SPARK-36266: -- User 'Victsm' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36266) Rename classes in shuffle RPC used for block push operations

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36266: Assignee: (was: Apache Spark) > Rename classes in shuffle RPC used for block push

[jira] [Created] (SPARK-36266) Rename classes in shuffle RPC used for block push operations

2021-07-22 Thread Min Shen (Jira)
Min Shen created SPARK-36266: Summary: Rename classes in shuffle RPC used for block push operations Key: SPARK-36266 URL: https://issues.apache.org/jira/browse/SPARK-36266 Project: Spark Issue

[jira] [Commented] (SPARK-36265) Use __getitem__ instead of getItem to suppress warnings.

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385826#comment-17385826 ] Apache Spark commented on SPARK-36265: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36265) Use __getitem__ instead of getItem to suppress warnings.

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36265: Assignee: Apache Spark > Use __getitem__ instead of getItem to suppress warnings. >

[jira] [Commented] (SPARK-36265) Use __getitem__ instead of getItem to suppress warnings.

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385825#comment-17385825 ] Apache Spark commented on SPARK-36265: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36265) Use __getitem__ instead of getItem to suppress warnings.

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36265: Assignee: (was: Apache Spark) > Use __getitem__ instead of getItem to suppress

[jira] [Created] (SPARK-36265) Use __getitem__ instead of getItem to suppress warnings.

2021-07-22 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36265: - Summary: Use __getitem__ instead of getItem to suppress warnings. Key: SPARK-36265 URL: https://issues.apache.org/jira/browse/SPARK-36265 Project: Spark

[jira] [Commented] (SPARK-36264) Add reorder_categories to CategoricalAccessor and CategoricalIndex.

2021-07-22 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385785#comment-17385785 ] Takuya Ueshin commented on SPARK-36264: --- I'm working on this. > Add reorder_categories to

[jira] [Created] (SPARK-36264) Add reorder_categories to CategoricalAccessor and CategoricalIndex.

2021-07-22 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36264: - Summary: Add reorder_categories to CategoricalAccessor and CategoricalIndex. Key: SPARK-36264 URL: https://issues.apache.org/jira/browse/SPARK-36264 Project: Spark

[jira] [Commented] (SPARK-36261) Add remove_unused_categories to CategoricalAccessor and CategoricalIndex.

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385784#comment-17385784 ] Apache Spark commented on SPARK-36261: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36261) Add remove_unused_categories to CategoricalAccessor and CategoricalIndex.

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36261: Assignee: (was: Apache Spark) > Add remove_unused_categories to CategoricalAccessor

[jira] [Assigned] (SPARK-36261) Add remove_unused_categories to CategoricalAccessor and CategoricalIndex.

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36261: Assignee: Apache Spark > Add remove_unused_categories to CategoricalAccessor and

[jira] [Commented] (SPARK-36261) Add remove_unused_categories to CategoricalAccessor and CategoricalIndex.

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385783#comment-17385783 ] Apache Spark commented on SPARK-36261: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36262) Upgrade ZSTD-JNI to 1.5.0-4

2021-07-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36262: - Assignee: Dongjoon Hyun > Upgrade ZSTD-JNI to 1.5.0-4 > --- >

[jira] [Resolved] (SPARK-36262) Upgrade ZSTD-JNI to 1.5.0-4

2021-07-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36262. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33483

[jira] [Updated] (SPARK-36263) Add Dataset.observe(Observation, Column, Column*) to PySpark

2021-07-22 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-36263: -- Description: With SPARK-34806 we now have a way to use the `Dataset.observe` method without

[jira] [Commented] (SPARK-36263) Add Dataset.observe(Observation, Column, Column*) to PySpark

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385719#comment-17385719 ] Apache Spark commented on SPARK-36263: -- User 'EnricoMi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36263) Add Dataset.observe(Observation, Column, Column*) to PySpark

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36263: Assignee: Apache Spark > Add Dataset.observe(Observation, Column, Column*) to PySpark >

[jira] [Commented] (SPARK-36263) Add Dataset.observe(Observation, Column, Column*) to PySpark

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385718#comment-17385718 ] Apache Spark commented on SPARK-36263: -- User 'EnricoMi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36263) Add Dataset.observe(Observation, Column, Column*) to PySpark

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36263: Assignee: (was: Apache Spark) > Add Dataset.observe(Observation, Column, Column*) to

[jira] [Assigned] (SPARK-36262) Upgrade ZSTD-JNI to 1.5.0-4

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36262: Assignee: (was: Apache Spark) > Upgrade ZSTD-JNI to 1.5.0-4 >

[jira] [Assigned] (SPARK-36262) Upgrade ZSTD-JNI to 1.5.0-4

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36262: Assignee: Apache Spark > Upgrade ZSTD-JNI to 1.5.0-4 > --- > >

[jira] [Commented] (SPARK-36262) Upgrade ZSTD-JNI to 1.5.0-4

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385716#comment-17385716 ] Apache Spark commented on SPARK-36262: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-35848) Spark Bloom Filter, others using treeAggregate can throw OutOfMemoryError

2021-07-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-35848: - Issue Type: Improvement (was: Bug) > Spark Bloom Filter, others using treeAggregate can throw

[jira] [Resolved] (SPARK-35848) Spark Bloom Filter, others using treeAggregate can throw OutOfMemoryError

2021-07-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35848. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33443

[jira] [Assigned] (SPARK-35310) Bump Breeze from 1.0 to 1.2

2021-07-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35310: Assignee: Sean R. Owen > Bump Breeze from 1.0 to 1.2 > --- > >

[jira] [Resolved] (SPARK-35310) Bump Breeze from 1.0 to 1.2

2021-07-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35310. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33449

[jira] [Created] (SPARK-36263) Add Dataset.observe(Observation, Column, Column*) to PySpark

2021-07-22 Thread Enrico Minack (Jira)
Enrico Minack created SPARK-36263: - Summary: Add Dataset.observe(Observation, Column, Column*) to PySpark Key: SPARK-36263 URL: https://issues.apache.org/jira/browse/SPARK-36263 Project: Spark

[jira] [Created] (SPARK-36262) Upgrade ZSTD-JNI to 1.5.0-4

2021-07-22 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-36262: - Summary: Upgrade ZSTD-JNI to 1.5.0-4 Key: SPARK-36262 URL: https://issues.apache.org/jira/browse/SPARK-36262 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-36255) FileNotFoundException from the shuffle push can cause the executor to terminate

2021-07-22 Thread Chandni Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated SPARK-36255: -- Description: Once the shuffle is cleaned up by the {{ContextCleaner}}, the shuffle files are

[jira] [Updated] (SPARK-36255) FileNotFoundException from the shuffle push can cause the executor to terminate

2021-07-22 Thread Chandni Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated SPARK-36255: -- Summary: FileNotFoundException from the shuffle push can cause the executor to terminate

[jira] [Commented] (SPARK-36261) Add remove_unused_categories to CategoricalAccessor and CategoricalIndex.

2021-07-22 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385690#comment-17385690 ] Takuya Ueshin commented on SPARK-36261: --- I'm working on this. > Add remove_unused_categories to

[jira] [Created] (SPARK-36261) Add remove_unused_categories to CategoricalAccessor and CategoricalIndex.

2021-07-22 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36261: - Summary: Add remove_unused_categories to CategoricalAccessor and CategoricalIndex. Key: SPARK-36261 URL: https://issues.apache.org/jira/browse/SPARK-36261 Project:

[jira] [Commented] (SPARK-36260) Add set_categories to CategoricalAccessor and CategoricalIndex

2021-07-22 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385668#comment-17385668 ] Xinrong Meng commented on SPARK-36260: -- I am working on this. > Add set_categories to

[jira] [Created] (SPARK-36260) Add set_categories to CategoricalAccessor and CategoricalIndex

2021-07-22 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36260: Summary: Add set_categories to CategoricalAccessor and CategoricalIndex Key: SPARK-36260 URL: https://issues.apache.org/jira/browse/SPARK-36260 Project: Spark

[jira] [Commented] (SPARK-11100) HiveThriftServer HA issue,HiveThriftServer not registering with Zookeeper

2021-07-22 Thread suryansh agnihotri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385651#comment-17385651 ] suryansh agnihotri commented on SPARK-11100: [https://github.com/apache/spark/pull/16879]

[jira] [Commented] (SPARK-29183) Upgrade JDK 11 Installation to 11.0.6

2021-07-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385639#comment-17385639 ] Dongjoon Hyun commented on SPARK-29183: --- Thank you, [~shaneknapp]. > Upgrade JDK 11 Installation

[jira] [Assigned] (SPARK-35815) Allow delayThreshold for watermark to be represented as ANSI day-time/year-month interval literals

2021-07-22 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35815: Assignee: Kousuke Saruta > Allow delayThreshold for watermark to be represented as ANSI >

[jira] [Resolved] (SPARK-35815) Allow delayThreshold for watermark to be represented as ANSI day-time/year-month interval literals

2021-07-22 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35815. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33456

[jira] [Updated] (SPARK-36241) support for creating tablewith void column datatype

2021-07-22 Thread Linhong Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Linhong Liu updated SPARK-36241: Summary: support for creating tablewith void column datatype (was: support for creating

[jira] [Updated] (SPARK-36241) support for creating table with void column datatype

2021-07-22 Thread Linhong Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Linhong Liu updated SPARK-36241: Description: previously we blocked creating table with void column datatype to follow the hive

[jira] [Updated] (SPARK-36241) support for creating table/view with void column datatype

2021-07-22 Thread Linhong Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Linhong Liu updated SPARK-36241: Description: previously we blocked creating tablewith void column datatype to follow the hive

[jira] [Updated] (SPARK-36241) support for creating table with void column datatype

2021-07-22 Thread Linhong Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Linhong Liu updated SPARK-36241: Summary: support for creating table with void column datatype (was: support for creating

[jira] [Commented] (SPARK-36259) Expose localtimestamp in pyspark.sql.functions

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385528#comment-17385528 ] Apache Spark commented on SPARK-36259: -- User 'dominikgehl' has created a pull request for this

[jira] [Commented] (SPARK-36259) Expose localtimestamp in pyspark.sql.functions

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385524#comment-17385524 ] Apache Spark commented on SPARK-36259: -- User 'dominikgehl' has created a pull request for this

[jira] [Assigned] (SPARK-36259) Expose localtimestamp in pyspark.sql.functions

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36259: Assignee: (was: Apache Spark) > Expose localtimestamp in pyspark.sql.functions >

[jira] [Assigned] (SPARK-36259) Expose localtimestamp in pyspark.sql.functions

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36259: Assignee: Apache Spark > Expose localtimestamp in pyspark.sql.functions >

[jira] [Created] (SPARK-36259) Expose localtimestamp in pyspark.sql.functions

2021-07-22 Thread Dominik Gehl (Jira)
Dominik Gehl created SPARK-36259: Summary: Expose localtimestamp in pyspark.sql.functions Key: SPARK-36259 URL: https://issues.apache.org/jira/browse/SPARK-36259 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-36209) https://spark.apache.org/docs/latest/sql-programming-guide.html contains invalid link to Python doc

2021-07-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-36209. -- Fix Version/s: 3.1.3 3.2.0 Assignee: Dominik Gehl

[jira] [Updated] (SPARK-36209) https://spark.apache.org/docs/latest/sql-programming-guide.html contains invalid link to Python doc

2021-07-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-36209: - Priority: Minor (was: Major) > https://spark.apache.org/docs/latest/sql-programming-guide.html

[jira] [Resolved] (SPARK-36257) Updated the version of TimestampNTZ related changes as 3.3.0

2021-07-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-36257. Resolution: Fixed Issue resolved by pull request 33478

[jira] [Resolved] (SPARK-36256) Upgrade lz4-java to 1.8.0

2021-07-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-36256. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33476

[jira] [Commented] (SPARK-36258) Export functionExists in pyspark catalog

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385447#comment-17385447 ] Apache Spark commented on SPARK-36258: -- User 'dominikgehl' has created a pull request for this

[jira] [Assigned] (SPARK-36258) Export functionExists in pyspark catalog

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36258: Assignee: (was: Apache Spark) > Export functionExists in pyspark catalog >

[jira] [Assigned] (SPARK-36258) Export functionExists in pyspark catalog

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36258: Assignee: Apache Spark > Export functionExists in pyspark catalog >

[jira] [Commented] (SPARK-11100) HiveThriftServer HA issue,HiveThriftServer not registering with Zookeeper

2021-07-22 Thread suryansh agnihotri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385413#comment-17385413 ] suryansh agnihotri commented on SPARK-11100: Hello [~wangxiaoyu] I tried to use this patch

[jira] [Resolved] (SPARK-36239) Remove some APIs from documentation.

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36239. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33458

[jira] [Assigned] (SPARK-36239) Remove some APIs from documentation.

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36239: Assignee: Haejoon Lee > Remove some APIs from documentation. >

[jira] [Commented] (SPARK-36257) Updated the version of TimestampNTZ related changes as 3.3.0

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385393#comment-17385393 ] Apache Spark commented on SPARK-36257: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-36257) Updated the version of TimestampNTZ related changes as 3.3.0

2021-07-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385392#comment-17385392 ] Apache Spark commented on SPARK-36257: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-36258) Export functionExists in pyspark catalog

2021-07-22 Thread Dominik Gehl (Jira)
Dominik Gehl created SPARK-36258: Summary: Export functionExists in pyspark catalog Key: SPARK-36258 URL: https://issues.apache.org/jira/browse/SPARK-36258 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-36156) Row FORMAT SERDE should handle null value as `\N`

2021-07-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36156. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33363

[jira] [Assigned] (SPARK-36156) Row FORMAT SERDE should handle null value as `\N`

2021-07-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36156: --- Assignee: angerszhu > Row FORMAT SERDE should handle null value as `\N` >

[jira] [Resolved] (SPARK-36243) pyspark catalog.tableExists doesn't work for temporary views

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36243. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33461

[jira] [Assigned] (SPARK-36243) pyspark catalog.tableExists doesn't work for temporary views

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36243: Assignee: Dominik Gehl > pyspark catalog.tableExists doesn't work for temporary views >

[jira] [Assigned] (SPARK-35809) Add `index_col` argument for ps.sql.

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35809: Assignee: Haejoon Lee > Add `index_col` argument for ps.sql. >

[jira] [Resolved] (SPARK-35809) Add `index_col` argument for ps.sql.

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35809. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33450

[jira] [Resolved] (SPARK-36249) Add remove_categories to CategoricalAccessor and CategoricalIndex

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36249. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33474

[jira] [Assigned] (SPARK-36249) Add remove_categories to CategoricalAccessor and CategoricalIndex

2021-07-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36249: Assignee: Takuya Ueshin > Add remove_categories to CategoricalAccessor and

[jira] [Updated] (SPARK-36247) check string length for char/varchar and apply type coercion in UPDATE/MERGE command

2021-07-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-36247: Summary: check string length for char/varchar and apply type coercion in UPDATE/MERGE command

[jira] [Updated] (SPARK-36247) check string length for char/varchar apply type coercion in UPDATE/MERGE command

2021-07-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-36247: Summary: check string length for char/varchar apply type coercion in UPDATE/MERGE command (was:

  1   2   >