[jira] [Commented] (SPARK-37031) Unify v1 and v2 DESCRIBE NAMESPACE tests

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434668#comment-17434668 ] Apache Spark commented on SPARK-37031: -- User 'imback82' has created a pull request for this issue:

[jira] [Commented] (SPARK-37031) Unify v1 and v2 DESCRIBE NAMESPACE tests

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434667#comment-17434667 ] Apache Spark commented on SPARK-37031: -- User 'imback82' has created a pull request for this issue:

[jira] [Resolved] (SPARK-37031) Unify v1 and v2 DESCRIBE NAMESPACE tests

2021-10-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37031. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34305

[jira] [Assigned] (SPARK-37031) Unify v1 and v2 DESCRIBE NAMESPACE tests

2021-10-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37031: --- Assignee: Terry Kim > Unify v1 and v2 DESCRIBE NAMESPACE tests >

[jira] [Created] (SPARK-37127) Support non-literal frame bound value for window functions

2021-10-26 Thread Kernel Force (Jira)
Kernel Force created SPARK-37127: Summary: Support non-literal frame bound value for window functions Key: SPARK-37127 URL: https://issues.apache.org/jira/browse/SPARK-37127 Project: Spark

[jira] [Commented] (SPARK-37125) Support AnsiInterval radix sort

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434654#comment-17434654 ] Apache Spark commented on SPARK-37125: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-37125) Support AnsiInterval radix sort

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434653#comment-17434653 ] Apache Spark commented on SPARK-37125: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-37125) Support AnsiInterval radix sort

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37125: Assignee: Apache Spark > Support AnsiInterval radix sort >

[jira] [Assigned] (SPARK-37125) Support AnsiInterval radix sort

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37125: Assignee: (was: Apache Spark) > Support AnsiInterval radix sort >

[jira] [Resolved] (SPARK-37126) Support TimestampNTZ in PySpark

2021-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37126. -- Fix Version/s: 3.3.0 Resolution: Done > Support TimestampNTZ in PySpark >

[jira] [Updated] (SPARK-37126) Support TimestampNTZ in PySpark

2021-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37126: - Issue Type: Improvement (was: Epic) > Support TimestampNTZ in PySpark >

[jira] [Assigned] (SPARK-36661) Support TimestampNTZ in Py4J

2021-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36661: Assignee: Hyukjin Kwon (was: Hyukjin Kwon) > Support TimestampNTZ in Py4J >

[jira] [Updated] (SPARK-37125) Support AnsiInterval radix sort

2021-10-26 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37125: -- Parent: SPARK-27790 Issue Type: Sub-task (was: Improvement) > Support AnsiInterval radix

[jira] [Created] (SPARK-37126) Support TimestampNTZ in PySpark

2021-10-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-37126: Summary: Support TimestampNTZ in PySpark Key: SPARK-37126 URL: https://issues.apache.org/jira/browse/SPARK-37126 Project: Spark Issue Type: Epic

[jira] [Created] (SPARK-37125) Support AnsiInterval radix sort

2021-10-26 Thread XiDuo You (Jira)
XiDuo You created SPARK-37125: - Summary: Support AnsiInterval radix sort Key: SPARK-37125 URL: https://issues.apache.org/jira/browse/SPARK-37125 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-37120) Add Java17 GitHub Action build and test job

2021-10-26 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434651#comment-17434651 ] Yang Jie commented on SPARK-37120: -- Thank [~dongjoon] > Add Java17 GitHub Action build and test job >

[jira] [Commented] (SPARK-36348) unexpected Index loaded: pd.Index([10, 20, None], name="x")

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434644#comment-17434644 ] Apache Spark commented on SPARK-36348: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35437) Use expressions to filter Hive partitions at client side

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35437: Assignee: (was: Apache Spark) > Use expressions to filter Hive partitions at client

[jira] [Assigned] (SPARK-35437) Use expressions to filter Hive partitions at client side

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35437: Assignee: Apache Spark > Use expressions to filter Hive partitions at client side >

[jira] [Commented] (SPARK-36348) unexpected Index loaded: pd.Index([10, 20, None], name="x")

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434643#comment-17434643 ] Apache Spark commented on SPARK-36348: -- User 'itholic' has created a pull request for this issue:

[jira] [Reopened] (SPARK-35437) Use expressions to filter Hive partitions at client side

2021-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-35437: -- Assignee: (was: dzcxzl) Reverted at

[jira] [Assigned] (SPARK-37124) Support Writable ArrowColumnarVector

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37124: Assignee: Apache Spark > Support Writable ArrowColumnarVector >

[jira] [Assigned] (SPARK-37124) Support Writable ArrowColumnarVector

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37124: Assignee: (was: Apache Spark) > Support Writable ArrowColumnarVector >

[jira] [Commented] (SPARK-37124) Support Writable ArrowColumnarVector

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434624#comment-17434624 ] Apache Spark commented on SPARK-37124: -- User 'xuechendi' has created a pull request for this issue:

[jira] [Created] (SPARK-37124) Support Writable ArrowColumnarVector

2021-10-26 Thread Chendi.Xue (Jira)
Chendi.Xue created SPARK-37124: -- Summary: Support Writable ArrowColumnarVector Key: SPARK-37124 URL: https://issues.apache.org/jira/browse/SPARK-37124 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-37123) Support Writable ArrowColumnarVector

2021-10-26 Thread Chendi.Xue (Jira)
Chendi.Xue created SPARK-37123: -- Summary: Support Writable ArrowColumnarVector Key: SPARK-37123 URL: https://issues.apache.org/jira/browse/SPARK-37123 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-37122) java.lang.IllegalArgumentException Related to Prometheus

2021-10-26 Thread Biswa Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biswa Singh updated SPARK-37122: Affects Version/s: (was: 3.0.2) > java.lang.IllegalArgumentException Related to Prometheus >

[jira] [Updated] (SPARK-37122) java.lang.IllegalArgumentException Related to Prometheus

2021-10-26 Thread Biswa Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biswa Singh updated SPARK-37122: Description: This issue is similar to

[jira] [Updated] (SPARK-37122) java.lang.IllegalArgumentException Related to Prometheus

2021-10-26 Thread Biswa Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biswa Singh updated SPARK-37122: Description: This issue is similar to

[jira] [Updated] (SPARK-37122) java.lang.IllegalArgumentException Related to Prometheus

2021-10-26 Thread Biswa Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biswa Singh updated SPARK-37122: Description: This issue is similar to

[jira] [Commented] (SPARK-37109) Install Java 17 on all of the Jenkins workers

2021-10-26 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434581#comment-17434581 ] Shane Knapp commented on SPARK-37109: - yep, jenkins is going away at the end of this year...  all

[jira] [Created] (SPARK-37122) java.lang.IllegalArgumentException Related to Prometheus

2021-10-26 Thread Biswa Singh (Jira)
Biswa Singh created SPARK-37122: --- Summary: java.lang.IllegalArgumentException Related to Prometheus Key: SPARK-37122 URL: https://issues.apache.org/jira/browse/SPARK-37122 Project: Spark Issue

[jira] [Commented] (SPARK-37120) Add Java17 GitHub Action build and test job

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434558#comment-17434558 ] Dongjoon Hyun commented on SPARK-37120: --- cc [~hyukjin.kwon] > Add Java17 GitHub Action build and

[jira] [Commented] (SPARK-37120) Add Java17 GitHub Action build and test job

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434557#comment-17434557 ] Dongjoon Hyun commented on SPARK-37120: --- I update the JIRA title to target GitHub Action job. >

[jira] [Updated] (SPARK-37120) Add Java17 GitHub Action build and test job

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37120: -- Summary: Add Java17 GitHub Action build and test job (was: Add a Jenkins build and test job

[jira] [Updated] (SPARK-37098) Alter table properties should invalidate cache

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37098: -- Fix Version/s: 3.0.4 > Alter table properties should invalidate cache >

[jira] [Updated] (SPARK-37114) Support Submitting Jobs to Cloud Providers Managed Spark Clusters

2021-10-26 Thread Naga Vijayapuram (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naga Vijayapuram updated SPARK-37114: - Priority: Minor (was: Trivial) > Support Submitting Jobs to Cloud Providers Managed

[jira] [Commented] (SPARK-36877) Calling ds.rdd with AQE enabled leads to jobs being run, eventually causing reruns

2021-10-26 Thread Shardul Mahadik (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434515#comment-17434515 ] Shardul Mahadik commented on SPARK-36877: - Was able to get around this by re-using the RDD for

[jira] [Resolved] (SPARK-36877) Calling ds.rdd with AQE enabled leads to jobs being run, eventually causing reruns

2021-10-26 Thread Shardul Mahadik (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shardul Mahadik resolved SPARK-36877. - Resolution: Not A Problem > Calling ds.rdd with AQE enabled leads to jobs being run,

[jira] [Resolved] (SPARK-36895) Add Create Index syntax support

2021-10-26 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-36895. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34148

[jira] [Assigned] (SPARK-36895) Add Create Index syntax support

2021-10-26 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-36895: --- Assignee: Huaxin Gao > Add Create Index syntax support > --- > >

[jira] [Assigned] (SPARK-37121) TestUtils.isPythonVersionAtLeast38 returns incorrect results

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37121: Assignee: Apache Spark > TestUtils.isPythonVersionAtLeast38 returns incorrect results >

[jira] [Assigned] (SPARK-37121) TestUtils.isPythonVersionAtLeast38 returns incorrect results

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37121: Assignee: (was: Apache Spark) > TestUtils.isPythonVersionAtLeast38 returns incorrect

[jira] [Commented] (SPARK-37121) TestUtils.isPythonVersionAtLeast38 returns incorrect results

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434503#comment-17434503 ] Apache Spark commented on SPARK-37121: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Created] (SPARK-37121) TestUtils.isPythonVersionAtLeast38 returns incorrect results

2021-10-26 Thread Erik Krogen (Jira)
Erik Krogen created SPARK-37121: --- Summary: TestUtils.isPythonVersionAtLeast38 returns incorrect results Key: SPARK-37121 URL: https://issues.apache.org/jira/browse/SPARK-37121 Project: Spark

[jira] [Commented] (SPARK-37118) Add KMeans distanceMeasure param to PythonMLLibAPI

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434380#comment-17434380 ] Apache Spark commented on SPARK-37118: -- User 'remykarem' has created a pull request for this issue:

[jira] [Commented] (SPARK-37119) parse_url can not handle `{` and `}` correctly

2021-10-26 Thread Liu Shuo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434326#comment-17434326 ] Liu Shuo commented on SPARK-37119: -- As discussed in `https://github.com/apache/spark/pull/30333` and

[jira] [Commented] (SPARK-37120) Add a Jenkins build and test job for Java 17

2021-10-26 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434319#comment-17434319 ] Yang Jie commented on SPARK-37120: -- ping [~sowen] [~dongjoon]  ,do we need to do this now? Who should

[jira] [Created] (SPARK-37120) Add a Jenkins build and test job for Java 17

2021-10-26 Thread Yang Jie (Jira)
Yang Jie created SPARK-37120: Summary: Add a Jenkins build and test job for Java 17 Key: SPARK-37120 URL: https://issues.apache.org/jira/browse/SPARK-37120 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-37110) Add Java 17 support for spark pull request builds

2021-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434307#comment-17434307 ] Hyukjin Kwon commented on SPARK-37110: -- [~yumwang], when you find some time, feel free to set up

[jira] [Updated] (SPARK-37119) parse_url can not handle `{` and `}` correctly

2021-10-26 Thread Liu Shuo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Shuo updated SPARK-37119: - Description: when we execute the follow sql command {code:java} select

[jira] [Created] (SPARK-37119) parse_url can not handle `{` and `}` correctly

2021-10-26 Thread Liu Shuo (Jira)
Liu Shuo created SPARK-37119: Summary: parse_url can not handle `{` and `}` correctly Key: SPARK-37119 URL: https://issues.apache.org/jira/browse/SPARK-37119 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-37118) Add KMeans distanceMeasure param to PythonMLLibAPI

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37118: Assignee: (was: Apache Spark) > Add KMeans distanceMeasure param to PythonMLLibAPI >

[jira] [Commented] (SPARK-37118) Add KMeans distanceMeasure param to PythonMLLibAPI

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434289#comment-17434289 ] Apache Spark commented on SPARK-37118: -- User 'remykarem' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37118) Add KMeans distanceMeasure param to PythonMLLibAPI

2021-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37118: Assignee: Apache Spark > Add KMeans distanceMeasure param to PythonMLLibAPI >

[jira] [Created] (SPARK-37118) Add KMeans distanceMeasure param to PythonMLLibAPI

2021-10-26 Thread Raimi bin Karim (Jira)
Raimi bin Karim created SPARK-37118: --- Summary: Add KMeans distanceMeasure param to PythonMLLibAPI Key: SPARK-37118 URL: https://issues.apache.org/jira/browse/SPARK-37118 Project: Spark

[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434269#comment-17434269 ] Dongjoon Hyun commented on SPARK-35181: --- If you want to get some help, please use the official

[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434263#comment-17434263 ] Dongjoon Hyun commented on SPARK-35181: --- I have no clue for those issues, but since it's JVM

[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

2021-10-26 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434258#comment-17434258 ] angerszhu commented on SPARK-35181: --- [~dongjoon] Yea, only when use ``spark.io.compression.codec=zstd`

[jira] [Comment Edited] (SPARK-35181) Use zstd for spark.io.compression.codec by default

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434257#comment-17434257 ] Dongjoon Hyun edited comment on SPARK-35181 at 10/26/21, 10:46 AM: ---

[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434257#comment-17434257 ] Dongjoon Hyun commented on SPARK-35181: --- BTW, according to the log, if you are trying to use

[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434252#comment-17434252 ] Dongjoon Hyun commented on SPARK-35181: --- I'm not sure how you build and configure your environment

[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

2021-10-26 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434241#comment-17434241 ] angerszhu commented on SPARK-35181: --- [~dongjoon] This issue resolve when we upgrade the zstd version,

[jira] [Resolved] (SPARK-37109) Install Java 17 on all of the Jenkins workers

2021-10-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-37109. - Resolution: Won't Do OK. I do not know this plan. > Install Java 17 on all of the Jenkins

[jira] [Resolved] (SPARK-37110) Add Java 17 support for spark pull request builds

2021-10-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-37110. - Resolution: Won't Do OK. I do not know this plan. > Add Java 17 support for spark pull request

[jira] [Commented] (SPARK-37051) The filter operator gets wrong results in char type

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434214#comment-17434214 ] Dongjoon Hyun commented on SPARK-37051: --- Got it. If that happens on Parquet, we had better drop

[jira] [Updated] (SPARK-37051) The filter operator gets wrong results in char type

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37051: -- Summary: The filter operator gets wrong results in char type (was: The filter operator gets

[jira] [Created] (SPARK-37117) Can't read files in one of Parquet encryption modes (external keymaterial)

2021-10-26 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created SPARK-37117: Summary: Can't read files in one of Parquet encryption modes (external keymaterial) Key: SPARK-37117 URL: https://issues.apache.org/jira/browse/SPARK-37117

[jira] [Comment Edited] (SPARK-37051) The filter operator gets wrong results in ORC's char type

2021-10-26 Thread frankli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434201#comment-17434201 ] frankli edited comment on SPARK-37051 at 10/26/21, 8:48 AM: This scenario

[jira] [Commented] (SPARK-37051) The filter operator gets wrong results in ORC's char type

2021-10-26 Thread frankli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434201#comment-17434201 ] frankli commented on SPARK-37051: - This scenario also occur on Parquet. Spark3.1 will do padding for

[jira] [Commented] (SPARK-37051) The filter operator gets wrong results in ORC's char type

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434196#comment-17434196 ] Dongjoon Hyun commented on SPARK-37051: --- Does Parquet work in those scenario, [~frankli] and

[jira] [Resolved] (SPARK-36989) Migrate type hint data tests

2021-10-26 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-36989. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-36989) Migrate type hint data tests

2021-10-26 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-36989: -- Assignee: Maciej Szymkiewicz > Migrate type hint data tests >

[jira] [Commented] (SPARK-37110) Add Java 17 support for spark pull request builds

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434190#comment-17434190 ] Dongjoon Hyun commented on SPARK-37110: --- +1 for [~hyukjin.kwon]'s comment to save the community

[jira] [Commented] (SPARK-37109) Install Java 17 on all of the Jenkins workers

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434189#comment-17434189 ] Dongjoon Hyun commented on SPARK-37109: --- +1 for [~hyukjin.kwon]'s comment. > Install Java 17 on

[jira] [Commented] (SPARK-37116) Allow sequences (tuples and lists) as pivot values argument in PySpark

2021-10-26 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434183#comment-17434183 ] Maciej Szymkiewicz commented on SPARK-37116: Sadly, this is not going to work. For example,

[jira] [Commented] (SPARK-37049) executorIdleTimeout is not working for pending pods on K8s

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434176#comment-17434176 ] Dongjoon Hyun commented on SPARK-37049: --- It's fixed now. > executorIdleTimeout is not working for

[jira] [Assigned] (SPARK-37049) executorIdleTimeout is not working for pending pods on K8s

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37049: - Assignee: Weiwei Yang > executorIdleTimeout is not working for pending pods on K8s >

[jira] [Assigned] (SPARK-37049) executorIdleTimeout is not working for pending pods on K8s

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37049: - Assignee: (was: wwei) > executorIdleTimeout is not working for pending pods on K8s

[jira] [Commented] (SPARK-37049) executorIdleTimeout is not working for pending pods on K8s

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434173#comment-17434173 ] Dongjoon Hyun commented on SPARK-37049: --- Oh, sure. Sorry, [~wwei]. > executorIdleTimeout is not

[jira] [Updated] (SPARK-37114) Support Submitting Jobs to Cloud Providers Managed Spark Clusters

2021-10-26 Thread Naga Vijayapuram (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naga Vijayapuram updated SPARK-37114: - Description: To be able to submit jobs to prominent cloud providers managed spark

[jira] [Updated] (SPARK-37114) Support Submitting Jobs to Cloud Providers Managed Spark Clusters

2021-10-26 Thread Naga Vijayapuram (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naga Vijayapuram updated SPARK-37114: - Summary: Support Submitting Jobs to Cloud Providers Managed Spark Clusters (was:

[jira] [Updated] (SPARK-37114) Support Submitting Jobs to Cloud Providers

2021-10-26 Thread Naga Vijayapuram (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naga Vijayapuram updated SPARK-37114: - Description: To be able to submit jobs to prominent cloud provider managed spark

[jira] [Updated] (SPARK-37098) Alter table properties should invalidate cache

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37098: -- Fix Version/s: 3.1.3 > Alter table properties should invalidate cache >

[jira] [Updated] (SPARK-37114) Support Submitting Jobs to Cloud Providers

2021-10-26 Thread Naga Vijayapuram (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naga Vijayapuram updated SPARK-37114: - Description: To be able to submit jobs to cloud providers, "spark-submit" can be

[jira] [Updated] (SPARK-36890) Use default WebsocketPingInterval for Kubernetes watches

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36890: -- Affects Version/s: (was: 3.0.3) (was: 3.1.2)

[jira] [Updated] (SPARK-36890) Use default WebsocketPingInterval for Kubernetes watches

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36890: -- Summary: Use default WebsocketPingInterval for Kubernetes watches (was: Websocket timeouts

[jira] [Resolved] (SPARK-36890) Websocket timeouts to K8s-API

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36890. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34143

[jira] [Assigned] (SPARK-36890) Websocket timeouts to K8s-API

2021-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36890: - Assignee: Philipp Dallig > Websocket timeouts to K8s-API >

[jira] [Updated] (SPARK-37110) Add Java 17 support for spark pull request builds

2021-10-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37110: Summary: Add Java 17 support for spark pull request builds (was: Add java17 support for spark

[jira] [Assigned] (SPARK-37087) merge three relation resolutions into one

2021-10-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37087: --- Assignee: Wenchen Fan > merge three relation resolutions into one >

[jira] [Resolved] (SPARK-37087) merge three relation resolutions into one

2021-10-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37087. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34358