[jira] [Commented] (SPARK-29519) SHOW TBLPROPERTIES should look up catalog/table like v2 commands

[ https://issues.apache.org/jira/browse/SPARK-29519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381049#comment-17381049 ] Apache Spark commented on SPARK-29519: -- User 'Peng-Lei' has created a pull request for this issue:

[jira] [Commented] (SPARK-29519) SHOW TBLPROPERTIES should look up catalog/table like v2 commands

[ https://issues.apache.org/jira/browse/SPARK-29519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381050#comment-17381050 ] Apache Spark commented on SPARK-29519: -- User 'Peng-Lei' has created a pull request for this issue:

[jira] [Updated] (SPARK-36134) jackson-databind RCE vulnerability

[ https://issues.apache.org/jira/browse/SPARK-36134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit updated SPARK-36134: -- Summary: jackson-databind RCE vulnerability (was: jackson-databind RCE vulnerability [Need to upgrade to

[jira] [Updated] (SPARK-36134) jackson-databind RCE vulnerability [Need to upgrade to 2.9.3.1]

[ https://issues.apache.org/jira/browse/SPARK-36134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit updated SPARK-36134: -- Attachment: Screenshot 2021-07-15 at 1.00.55 PM.png > jackson-databind RCE vulnerability [Need to upgrade to

[jira] [Commented] (SPARK-36134) jackson-databind RCE vulnerability [Need to upgrade to 2.9.3.1]

[ https://issues.apache.org/jira/browse/SPARK-36134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381038#comment-17381038 ] Sumit commented on SPARK-36134: --- this is reported by the security team [Open-Source Scan - Nexus IQ Scan

[jira] [Updated] (SPARK-36134) jackson-databind RCE vulnerability [Need to upgrade to 2.9.3.1]

[ https://issues.apache.org/jira/browse/SPARK-36134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit updated SPARK-36134: -- Attachment: Screenshot 2021-07-15 at 1.00.55 PM.png > jackson-databind RCE vulnerability [Need to upgrade to

[jira] [Updated] (SPARK-36134) jackson-databind RCE vulnerability [Need to upgrade to 2.9.3.1]

[ https://issues.apache.org/jira/browse/SPARK-36134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit updated SPARK-36134: -- Attachment: (was: Screenshot 2021-07-15 at 1.00.55 PM.png) > jackson-databind RCE vulnerability [Need to

[jira] [Updated] (SPARK-36134) jackson-databind RCE vulnerability [Need to upgrade to 2.9.3.1]

[ https://issues.apache.org/jira/browse/SPARK-36134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit updated SPARK-36134: -- Attachment: (was: Screenshot 2021-07-15 at 1.00.55 PM.png) > jackson-databind RCE vulnerability [Need to

[jira] [Updated] (SPARK-36134) jackson-databind RCE vulnerability [Need to upgrade to 2.9.3.1]

[ https://issues.apache.org/jira/browse/SPARK-36134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit updated SPARK-36134: -- Attachment: Screenshot 2021-07-15 at 1.00.55 PM.png > jackson-databind RCE vulnerability [Need to upgrade to

[jira] [Created] (SPARK-36148) Missing validation of regexp_replace inputs

Linhong Liu created SPARK-36148: --- Summary: Missing validation of regexp_replace inputs Key: SPARK-36148 URL: https://issues.apache.org/jira/browse/SPARK-36148 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-35334) Spark should be more resilient to intermittent K8s flakiness

[ https://issues.apache.org/jira/browse/SPARK-35334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381013#comment-17381013 ] Dongjoon Hyun commented on SPARK-35334: --- Thank you for updating, [~attilapiros]. > Spark should

[jira] [Updated] (SPARK-36076) [SQL] ArrayIndexOutOfBounds in CAST string to timestamp

[ https://issues.apache.org/jira/browse/SPARK-36076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36076: -- Affects Version/s: 3.0.3 3.1.2 > [SQL] ArrayIndexOutOfBounds in CAST

[jira] [Resolved] (SPARK-36076) [SQL] ArrayIndexOutOfBounds in CAST string to timestamp

[ https://issues.apache.org/jira/browse/SPARK-36076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36076. --- Fix Version/s: 3.0.4 Resolution: Fixed Issue resolved by pull request 33325

[jira] [Updated] (SPARK-36076) [SQL] ArrayIndexOutOfBounds in CAST string to timestamp

[ https://issues.apache.org/jira/browse/SPARK-36076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36076: -- Fix Version/s: 3.1.3 > [SQL] ArrayIndexOutOfBounds in CAST string to timestamp >

[jira] [Commented] (SPARK-36147) [SQL] - log level should be warning if files not found in BasicWriteStatsTracker

[ https://issues.apache.org/jira/browse/SPARK-36147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381010#comment-17381010 ] Apache Spark commented on SPARK-36147: -- User 'tooptoop4' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36147) [SQL] - log level should be warning if files not found in BasicWriteStatsTracker

[ https://issues.apache.org/jira/browse/SPARK-36147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36147: Assignee: (was: Apache Spark) > [SQL] - log level should be warning if files not

[jira] [Commented] (SPARK-36147) [SQL] - log level should be warning if files not found in BasicWriteStatsTracker

[ https://issues.apache.org/jira/browse/SPARK-36147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381008#comment-17381008 ] Apache Spark commented on SPARK-36147: -- User 'tooptoop4' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36147) [SQL] - log level should be warning if files not found in BasicWriteStatsTracker

[ https://issues.apache.org/jira/browse/SPARK-36147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36147: Assignee: Apache Spark > [SQL] - log level should be warning if files not found in >

[jira] [Created] (SPARK-36147) [SQL] - log level should be warning if files not found in BasicWriteStatsTracker

t oo created SPARK-36147: Summary: [SQL] - log level should be warning if files not found in BasicWriteStatsTracker Key: SPARK-36147 URL: https://issues.apache.org/jira/browse/SPARK-36147 Project: Spark

[jira] [Commented] (SPARK-36145) Remove Python 3.6 support in codebase and CI

[ https://issues.apache.org/jira/browse/SPARK-36145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381006#comment-17381006 ] Dongjoon Hyun commented on SPARK-36145: --- Thank you, [~hyukjin.kwon]! > Remove Python 3.6 support

[jira] [Updated] (SPARK-36139) Remove Python 3.6 from `pyspark` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-36139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36139: -- Summary: Remove Python 3.6 from `pyspark` GitHub Action job (was: Remove Python 3.6 from

[jira] [Commented] (SPARK-36090) Support TimestampNTZ type in expression Sequence

[ https://issues.apache.org/jira/browse/SPARK-36090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381005#comment-17381005 ] jiaan.geng commented on SPARK-36090: I'm working on. > Support TimestampNTZ type in expression

[jira] [Assigned] (SPARK-36139) Remove Python 3.6 from `build and test` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-36139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36139: - Assignee: Dongjoon Hyun > Remove Python 3.6 from `build and test` GitHub Action job >

[jira] [Resolved] (SPARK-36139) Remove Python 3.6 from `build and test` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-36139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36139. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33349

[jira] [Updated] (SPARK-36094) Group SQL component error messages in Spark error class JSON file

[ https://issues.apache.org/jira/browse/SPARK-36094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-36094: --- Description: To improve auditing, reduce duplication, and improve quality of error messages thrown

[jira] [Updated] (SPARK-36106) Refactor a few query compilation errors to use error classes

[ https://issues.apache.org/jira/browse/SPARK-36106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-36106: --- Description: Refactor some exceptions in

[jira] [Commented] (SPARK-36146) Upgrade Python version from 3.6 to higher version in GitHub linter

[ https://issues.apache.org/jira/browse/SPARK-36146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380988#comment-17380988 ] Hyukjin Kwon commented on SPARK-36146: -- I'll work on this. FYI [~ueshin] [~dongjoon] > Upgrade

[jira] [Updated] (SPARK-36106) Add subset of error classes to QueryCompilationErrors

[ https://issues.apache.org/jira/browse/SPARK-36106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-36106: --- Summary: Add subset of error classes to QueryCompilationErrors (was: Add error classes to

[jira] [Updated] (SPARK-36146) Upgrade Python version from 3.6 to higher version in GitHub linter

[ https://issues.apache.org/jira/browse/SPARK-36146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36146: - Description: MyPy checks fails with higher Python versions. For example, with Python 3.8:

[jira] [Created] (SPARK-36146) Upgrade Python version from 3.6 to higher version in GitHub linter

Hyukjin Kwon created SPARK-36146: Summary: Upgrade Python version from 3.6 to higher version in GitHub linter Key: SPARK-36146 URL: https://issues.apache.org/jira/browse/SPARK-36146 Project: Spark

[jira] [Resolved] (SPARK-36144) Use Python 3.9 in `run-pip-tests` conda environment

[ https://issues.apache.org/jira/browse/SPARK-36144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36144. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33351

[jira] [Assigned] (SPARK-36144) Use Python 3.9 in `run-pip-tests` conda environment

[ https://issues.apache.org/jira/browse/SPARK-36144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36144: Assignee: Dongjoon Hyun > Use Python 3.9 in `run-pip-tests` conda environment >

[jira] [Commented] (SPARK-34952) Aggregate (Min/Max/Count) push down for Parquet

[ https://issues.apache.org/jira/browse/SPARK-34952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380963#comment-17380963 ] Apache Spark commented on SPARK-34952: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-34952) Aggregate (Min/Max/Count) push down for Parquet

[ https://issues.apache.org/jira/browse/SPARK-34952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380961#comment-17380961 ] Apache Spark commented on SPARK-34952: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Resolved] (SPARK-36106) Add error classes to QueryCompilationErrors

[ https://issues.apache.org/jira/browse/SPARK-36106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36106. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33309

[jira] [Assigned] (SPARK-36106) Add error classes to QueryCompilationErrors

[ https://issues.apache.org/jira/browse/SPARK-36106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36106: Assignee: Karen Feng > Add error classes to QueryCompilationErrors >

[jira] [Commented] (SPARK-35810) Deprecate ps.broadcast API

[ https://issues.apache.org/jira/browse/SPARK-35810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380956#comment-17380956 ] Haejoon Lee commented on SPARK-35810: - I'm working on this > Deprecate ps.broadcast API >

[jira] [Commented] (SPARK-36145) Remove Python 3.6 support in codebase and CI

[ https://issues.apache.org/jira/browse/SPARK-36145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380934#comment-17380934 ] Hyukjin Kwon commented on SPARK-36145: -- cc [~ueshin] [~itholic] [~XinrongM] [~smilegator] FYI >

[jira] [Updated] (SPARK-36145) Remove Python 3.6 support in codebase and CI

[ https://issues.apache.org/jira/browse/SPARK-36145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36145: - Target Version/s: 3.3.0 > Remove Python 3.6 support in codebase and CI >

[jira] [Commented] (SPARK-36145) Remove Python 3.6 support in codebase and CI

[ https://issues.apache.org/jira/browse/SPARK-36145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380933#comment-17380933 ] Hyukjin Kwon commented on SPARK-36145: -- [~dongjoon], I just made an umbrella ticket here to track

[jira] [Updated] (SPARK-36139) Remove Python 3.6 from `build and test` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-36139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36139: - Parent: SPARK-36145 Issue Type: Sub-task (was: Test) > Remove Python 3.6 from `build

[jira] [Updated] (SPARK-36145) Remove Python 3.6 support

[ https://issues.apache.org/jira/browse/SPARK-36145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36145: - Description: Python 3.6 is deprecated via SPARK-35938 at Apache Spark 3.2. We should remove it

[jira] [Updated] (SPARK-36145) Remove Python 3.6 support in codebase and CI

[ https://issues.apache.org/jira/browse/SPARK-36145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36145: - Summary: Remove Python 3.6 support in codebase and CI (was: Remove Python 3.6 support) >

[jira] [Updated] (SPARK-36144) Use Python 3.9 in `run-pip-tests` conda environment

[ https://issues.apache.org/jira/browse/SPARK-36144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36144: - Parent: SPARK-36145 Issue Type: Sub-task (was: Test) > Use Python 3.9 in

[jira] [Created] (SPARK-36145) Remove Python 3.6 support

Hyukjin Kwon created SPARK-36145: Summary: Remove Python 3.6 support Key: SPARK-36145 URL: https://issues.apache.org/jira/browse/SPARK-36145 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-36144) Use Python 3.9 in `run-pip-tests` conda environment

[ https://issues.apache.org/jira/browse/SPARK-36144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36144: Assignee: (was: Apache Spark) > Use Python 3.9 in `run-pip-tests` conda environment

[jira] [Assigned] (SPARK-36144) Use Python 3.9 in `run-pip-tests` conda environment

[ https://issues.apache.org/jira/browse/SPARK-36144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36144: Assignee: Apache Spark > Use Python 3.9 in `run-pip-tests` conda environment >

[jira] [Commented] (SPARK-36144) Use Python 3.9 in `run-pip-tests` conda environment

[ https://issues.apache.org/jira/browse/SPARK-36144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380925#comment-17380925 ] Apache Spark commented on SPARK-36144: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-36144) Use Python 3.9 in `run-pip-tests` conda environment

[ https://issues.apache.org/jira/browse/SPARK-36144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380924#comment-17380924 ] Apache Spark commented on SPARK-36144: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-36144) Use Python 3.9 in `run-pip-tests` conda environment

[ https://issues.apache.org/jira/browse/SPARK-36144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36144: -- Summary: Use Python 3.9 in `run-pip-tests` conda environment (was: Use Python 3.9 in

[jira] [Created] (SPARK-36144) Use Python 3.9 in `run-pip-tests` conda

Dongjoon Hyun created SPARK-36144: - Summary: Use Python 3.9 in `run-pip-tests` conda Key: SPARK-36144 URL: https://issues.apache.org/jira/browse/SPARK-36144 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-36136) Move PruneFileSourcePartitionsSuite out of org.apache.spark.sql.hive

[ https://issues.apache.org/jira/browse/SPARK-36136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380911#comment-17380911 ] Apache Spark commented on SPARK-36136: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36136) Move PruneFileSourcePartitionsSuite out of org.apache.spark.sql.hive

[ https://issues.apache.org/jira/browse/SPARK-36136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36136: Assignee: (was: Apache Spark) > Move PruneFileSourcePartitionsSuite out of

[jira] [Assigned] (SPARK-36136) Move PruneFileSourcePartitionsSuite out of org.apache.spark.sql.hive

[ https://issues.apache.org/jira/browse/SPARK-36136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36136: Assignee: Apache Spark > Move PruneFileSourcePartitionsSuite out of

[jira] [Commented] (SPARK-36136) Move PruneFileSourcePartitionsSuite out of org.apache.spark.sql.hive

[ https://issues.apache.org/jira/browse/SPARK-36136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380910#comment-17380910 ] Apache Spark commented on SPARK-36136: -- User 'sunchao' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380902#comment-17380902 ] Xinrong Meng edited comment on SPARK-36031 at 7/14/21, 11:52 PM: - Hi

[jira] [Commented] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380902#comment-17380902 ] Xinrong Meng commented on SPARK-36031: -- Hi [~yikunkero], I dispatched the examples in the

[jira] [Comment Edited] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380902#comment-17380902 ] Xinrong Meng edited comment on SPARK-36031 at 7/14/21, 11:51 PM: - Hi

[jira] [Created] (SPARK-36143) Adjust astype of Series of ExtensionDtype to follow pandas

Xinrong Meng created SPARK-36143: Summary: Adjust astype of Series of ExtensionDtype to follow pandas Key: SPARK-36143 URL: https://issues.apache.org/jira/browse/SPARK-36143 Project: Spark

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36031: - Description: There are many operations for series containing nan values that don't follow

[jira] [Created] (SPARK-36142) Adjust exponentiation between ExtentionDtypes and bools to follow pandas

Xinrong Meng created SPARK-36142: Summary: Adjust exponentiation between ExtentionDtypes and bools to follow pandas Key: SPARK-36142 URL: https://issues.apache.org/jira/browse/SPARK-36142 Project:

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36031: - Description: There are many operations for series doen't follow the pandas, such as:

[jira] [Created] (SPARK-36141) Support arithmetic operations of Series containing Decimal(np.nan)

Xinrong Meng created SPARK-36141: Summary: Support arithmetic operations of Series containing Decimal(np.nan) Key: SPARK-36141 URL: https://issues.apache.org/jira/browse/SPARK-36141 Project: Spark

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36031: - Description: There are many operations for series doen't follow the pandas, such as:

[jira] [Created] (SPARK-36140) Replace DataTypeOps tests that have operations on different Series

Xinrong Meng created SPARK-36140: Summary: Replace DataTypeOps tests that have operations on different Series Key: SPARK-36140 URL: https://issues.apache.org/jira/browse/SPARK-36140 Project: Spark

[jira] [Updated] (SPARK-36139) Remove Python 3.6 from `build and test` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-36139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36139: -- Priority: Minor (was: Major) > Remove Python 3.6 from `build and test` GitHub Action job >

[jira] [Assigned] (SPARK-36139) Remove Python 3.6 from `build and test` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-36139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36139: Assignee: Apache Spark > Remove Python 3.6 from `build and test` GitHub Action job >

[jira] [Commented] (SPARK-36139) Remove Python 3.6 from `build and test` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-36139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380879#comment-17380879 ] Apache Spark commented on SPARK-36139: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-36139) Remove Python 3.6 from `build and test` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-36139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380880#comment-17380880 ] Apache Spark commented on SPARK-36139: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-36139) Remove Python 3.6 from `build and test` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-36139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36139: Assignee: (was: Apache Spark) > Remove Python 3.6 from `build and test` GitHub

[jira] [Created] (SPARK-36139) Remove Python 3.6 from `build and test` GitHub Action job

Dongjoon Hyun created SPARK-36139: - Summary: Remove Python 3.6 from `build and test` GitHub Action job Key: SPARK-36139 URL: https://issues.apache.org/jira/browse/SPARK-36139 Project: Spark

[jira] [Updated] (SPARK-36138) Support for Slowly Changing Dimension Build Or Data Updates

[ https://issues.apache.org/jira/browse/SPARK-36138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nagharajan Raghavendran updated SPARK-36138: Summary: Support for Slowly Changing Dimension Build Or Data Updates

[jira] [Resolved] (SPARK-36125) Implement non-equality comparison operators between two Categoricals

[ https://issues.apache.org/jira/browse/SPARK-36125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36125. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Created] (SPARK-36138) Support for Slowly Changing Dimension Build

Nagharajan Raghavendran created SPARK-36138: --- Summary: Support for Slowly Changing Dimension Build Key: SPARK-36138 URL: https://issues.apache.org/jira/browse/SPARK-36138 Project: Spark

[jira] [Updated] (SPARK-36137) HiveShim always fallback to getAllPartitionsOf regardless of whether directSQL is enabled in remote HMS

[ https://issues.apache.org/jira/browse/SPARK-36137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-36137: - Description: At the moment {{getPartitionsByFilter}} in Hive shim only fallback to use

[jira] [Created] (SPARK-36137) HiveShim always fallback to getAllPartitionsOf regardless of whether directSQL is enabled in remote HMS

Chao Sun created SPARK-36137: Summary: HiveShim always fallback to getAllPartitionsOf regardless of whether directSQL is enabled in remote HMS Key: SPARK-36137 URL: https://issues.apache.org/jira/browse/SPARK-36137

[jira] [Assigned] (SPARK-36069) spark function from_json should output field name, field type and field value when FAILFAST mode throw exception

[ https://issues.apache.org/jira/browse/SPARK-36069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-36069: Assignee: Max Gekk > spark function from_json should output field name, field type and field

[jira] [Resolved] (SPARK-36069) spark function from_json should output field name, field type and field value when FAILFAST mode throw exception

[ https://issues.apache.org/jira/browse/SPARK-36069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36069. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33297

[jira] [Created] (SPARK-36136) Move PruneFileSourcePartitionsSuite out of org.apache.spark.sql.hive

Chao Sun created SPARK-36136: Summary: Move PruneFileSourcePartitionsSuite out of org.apache.spark.sql.hive Key: SPARK-36136 URL: https://issues.apache.org/jira/browse/SPARK-36136 Project: Spark

[jira] [Assigned] (SPARK-35639) Add metrics about coalesced partitions to CustomShuffleReader in AQE

[ https://issues.apache.org/jira/browse/SPARK-35639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35639: Assignee: Apache Spark (was: Eugene Koifman) > Add metrics about coalesced partitions

[jira] [Assigned] (SPARK-35639) Add metrics about coalesced partitions to CustomShuffleReader in AQE

[ https://issues.apache.org/jira/browse/SPARK-35639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35639: Assignee: Eugene Koifman (was: Apache Spark) > Add metrics about coalesced partitions

[jira] [Commented] (SPARK-36128) CatalogFileIndex.filterPartitions should respect spark.sql.hive.metastorePartitionPruning

[ https://issues.apache.org/jira/browse/SPARK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380737#comment-17380737 ] Apache Spark commented on SPARK-36128: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36128) CatalogFileIndex.filterPartitions should respect spark.sql.hive.metastorePartitionPruning

[ https://issues.apache.org/jira/browse/SPARK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36128: Assignee: (was: Apache Spark) > CatalogFileIndex.filterPartitions should respect >

[jira] [Commented] (SPARK-36128) CatalogFileIndex.filterPartitions should respect spark.sql.hive.metastorePartitionPruning

[ https://issues.apache.org/jira/browse/SPARK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380736#comment-17380736 ] Apache Spark commented on SPARK-36128: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36128) CatalogFileIndex.filterPartitions should respect spark.sql.hive.metastorePartitionPruning

[ https://issues.apache.org/jira/browse/SPARK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36128: Assignee: Apache Spark > CatalogFileIndex.filterPartitions should respect >

[jira] [Assigned] (SPARK-36034) Incorrect datetime filter when reading Parquet files written in legacy mode

[ https://issues.apache.org/jira/browse/SPARK-36034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36034: Assignee: Apache Spark > Incorrect datetime filter when reading Parquet files written in

[jira] [Assigned] (SPARK-36034) Incorrect datetime filter when reading Parquet files written in legacy mode

[ https://issues.apache.org/jira/browse/SPARK-36034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36034: Assignee: (was: Apache Spark) > Incorrect datetime filter when reading Parquet files

[jira] [Commented] (SPARK-36034) Incorrect datetime filter when reading Parquet files written in legacy mode

[ https://issues.apache.org/jira/browse/SPARK-36034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380727#comment-17380727 ] Apache Spark commented on SPARK-36034: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Reopened] (SPARK-35639) Add metrics about coalesced partitions to CustomShuffleReader in AQE

[ https://issues.apache.org/jira/browse/SPARK-35639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reopened SPARK-35639: [~cloud_fan] this is not done yet. The original PR that actually adds metrics is

[jira] [Commented] (SPARK-36134) jackson-databind RCE vulnerability [Need to upgrade to 2.9.3.1]

[ https://issues.apache.org/jira/browse/SPARK-36134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380695#comment-17380695 ] Erik Krogen commented on SPARK-36134: - Jackson is already 2.12.3 (from

[jira] [Commented] (SPARK-36037) Support ANSI SQL LOCALTIMESTAMP datetime value function

[ https://issues.apache.org/jira/browse/SPARK-36037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380691#comment-17380691 ] Apache Spark commented on SPARK-36037: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-36037) Support ANSI SQL LOCALTIMESTAMP datetime value function

[ https://issues.apache.org/jira/browse/SPARK-36037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380693#comment-17380693 ] Apache Spark commented on SPARK-36037: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-36121) Write data loss caused by stage retry when enable v2 FileOutputCommitter

[ https://issues.apache.org/jira/browse/SPARK-36121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380646#comment-17380646 ] gaoyajun02 commented on SPARK-36121: Hi [~hyukjin.kwon], Thank you very much. It seems to be

[jira] [Updated] (SPARK-36129) Upgrade commons-compress to 1.21 to deal with CVEs

[ https://issues.apache.org/jira/browse/SPARK-36129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36129: -- Fix Version/s: 3.0.4 3.1.3 > Upgrade commons-compress to 1.21 to deal with

[jira] [Assigned] (SPARK-36135) Support TimestampNTZ type in file partitioning

[ https://issues.apache.org/jira/browse/SPARK-36135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36135: Assignee: Gengliang Wang (was: Apache Spark) > Support TimestampNTZ type in file

[jira] [Commented] (SPARK-36135) Support TimestampNTZ type in file partitioning

[ https://issues.apache.org/jira/browse/SPARK-36135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380557#comment-17380557 ] Apache Spark commented on SPARK-36135: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-36135) Support TimestampNTZ type in file partitioning

[ https://issues.apache.org/jira/browse/SPARK-36135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36135: Assignee: Apache Spark (was: Gengliang Wang) > Support TimestampNTZ type in file

[jira] [Created] (SPARK-36135) Support TimestampNTZ type in file partitioning

Gengliang Wang created SPARK-36135: -- Summary: Support TimestampNTZ type in file partitioning Key: SPARK-36135 URL: https://issues.apache.org/jira/browse/SPARK-36135 Project: Spark Issue

[jira] [Commented] (SPARK-33898) Support SHOW CREATE TABLE in v2

[ https://issues.apache.org/jira/browse/SPARK-33898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380511#comment-17380511 ] Apache Spark commented on SPARK-33898: -- User 'sarutak' has created a pull request for this issue:

[jira] [Resolved] (SPARK-36123) Parquet vectorized reader doesn't skip null values correctly

[ https://issues.apache.org/jira/browse/SPARK-36123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36123. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 0

  1   2   >