[jira] [Updated] (SPARK-36195) Set MaxMetaspaceSize JVM option to 2g

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36195: -- Summary: Set MaxMetaspaceSize JVM option to 2g (was: Set MaxMetaspaceSize JVM option to 1g)

[jira] [Assigned] (SPARK-36205) Use set-env instead of set-output in GitHub Actions

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36205: Assignee: (was: Apache Spark) > Use set-env instead of set-output in GitHub Actions

[jira] [Commented] (SPARK-36205) Use set-env instead of set-output in GitHub Actions

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383021#comment-17383021 ] Apache Spark commented on SPARK-36205: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-36205) Use set-env instead of set-output in GitHub Actions

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36205: Assignee: Apache Spark > Use set-env instead of set-output in GitHub Actions >

[jira] [Resolved] (SPARK-36193) Recover SparkSubmit.runMain not to stop SparkContext in non-K8s env

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36193. --- Fix Version/s: 3.1.3 3.2.0 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-36193) Recover SparkSubmit.runMain not to stop SparkContext in non-K8s env

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36193: - Assignee: Dongjoon Hyun > Recover SparkSubmit.runMain not to stop SparkContext in

[jira] [Updated] (SPARK-36205) Use set-env instead of set-output in GitHub Actions

2021-07-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36205: - Environment: (was: Some places in GitHub Actions, we use set-output to set an environment

[jira] [Updated] (SPARK-36205) Use set-env instead of set-output in GitHub Actions

2021-07-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36205: - Description: Some places in GitHub Actions, we use set-output to set an environment variable.

[jira] [Updated] (SPARK-36205) Use set-env instead of set-output in GitHub Actions

2021-07-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36205: - Description: Some places in GitHub Actions, we use set-output to set an environment variable.

[jira] [Updated] (SPARK-36205) Use set-env instead of set-output in GitHub Actions

2021-07-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36205: - Description: Some places in GitHub Actions, we use set-output to set an environment vairable.

[jira] [Created] (SPARK-36205) Use set-env instead of set-output in GitHub Actions

2021-07-18 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-36205: Summary: Use set-env instead of set-output in GitHub Actions Key: SPARK-36205 URL: https://issues.apache.org/jira/browse/SPARK-36205 Project: Spark Issue

[jira] [Resolved] (SPARK-36199) Bump scalatest-maven-plugin to 2.0.2

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36199. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33408

[jira] [Updated] (SPARK-36199) Bump scalatest-maven-plugin to 2.0.2

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36199: -- Issue Type: Test (was: Improvement) > Bump scalatest-maven-plugin to 2.0.2 >

[jira] [Updated] (SPARK-36199) Bump scalatest-maven-plugin to 2.0.2

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36199: -- Affects Version/s: 3.2.0 > Bump scalatest-maven-plugin to 2.0.2 >

[jira] [Assigned] (SPARK-36199) Bump scalatest-maven-plugin to 2.0.2

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36199: - Assignee: William Hyun > Bump scalatest-maven-plugin to 2.0.2 >

[jira] [Commented] (SPARK-36166) Support Scala 2.13 test in `dev/run-tests.py`

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382999#comment-17382999 ] Apache Spark commented on SPARK-36166: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-36166) Support Scala 2.13 test in `dev/run-tests.py`

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383000#comment-17383000 ] Apache Spark commented on SPARK-36166: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-36204) Deduplicate Scala 2.13 daily build

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36204: Assignee: (was: Apache Spark) > Deduplicate Scala 2.13 daily build >

[jira] [Commented] (SPARK-36204) Deduplicate Scala 2.13 daily build

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382998#comment-17382998 ] Apache Spark commented on SPARK-36204: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-36204) Deduplicate Scala 2.13 daily build

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36204: Assignee: Apache Spark > Deduplicate Scala 2.13 daily build >

[jira] [Updated] (SPARK-36204) Deduplicate Scala 2.13 daily build

2021-07-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36204: - Affects Version/s: (was: 3.2.0) 3.3.0 > Deduplicate Scala 2.13 daily

[jira] [Created] (SPARK-36204) Deduplicate Scala 2.13 daily build

2021-07-18 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-36204: Summary: Deduplicate Scala 2.13 daily build Key: SPARK-36204 URL: https://issues.apache.org/jira/browse/SPARK-36204 Project: Spark Issue Type: Test

[jira] [Resolved] (SPARK-35807) Deprecate the `num_files` argument

2021-07-18 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-35807. - Resolution: Fixed > Deprecate the `num_files` argument > -- > >

[jira] [Comment Edited] (SPARK-35807) Deprecate the `num_files` argument

2021-07-18 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382984#comment-17382984 ] Haejoon Lee edited comment on SPARK-35807 at 7/19/21, 4:16 AM: --- This is

[jira] [Commented] (SPARK-35807) Deprecate the `num_files` argument

2021-07-18 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382984#comment-17382984 ] Haejoon Lee commented on SPARK-35807: - This resolved in https://github.com/apache/spark/pull/33379

[jira] [Reopened] (SPARK-35807) Deprecate the `num_files` argument

2021-07-18 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee reopened SPARK-35807: - Re open and change the title & description. We should deprecate, rather than remove for now to

[jira] [Issue Comment Deleted] (SPARK-35807) Deprecate the `num_files` argument

2021-07-18 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35807: Comment: was deleted (was: Re open and change the title & description. We should deprecate,

[jira] [Commented] (SPARK-35807) Deprecate the `num_files` argument

2021-07-18 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382982#comment-17382982 ] Haejoon Lee commented on SPARK-35807: - Re open and change the title & description. We should

[jira] [Updated] (SPARK-35807) Deprecate the `num_files` argument

2021-07-18 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35807: Summary: Deprecate the `num_files` argument (was: Rename the `num_files` argument) > Deprecate

[jira] [Updated] (SPARK-35807) Deprecate the `num_files` argument

2021-07-18 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35807: Description: We should deprecate the num_files argument in [DataFrame.to_csv 

[jira] [Commented] (SPARK-36021) Incorrect parsing of the start field in interval literals

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382979#comment-17382979 ] Apache Spark commented on SPARK-36021: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-36021) Incorrect parsing of the start field in interval literals

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382978#comment-17382978 ] Apache Spark commented on SPARK-36021: -- User 'AngersZh' has created a pull request for this

[jira] [Updated] (SPARK-36203) Spark SQL can't use "group by" on the column of map type.

2021-07-18 Thread Bruce Wong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Wong updated SPARK-36203: --- Description: I want to know why the 'group by' can't use in column of map tyep.     was:I want

[jira] [Updated] (SPARK-36203) Spark SQL can't use "group by" on the column of map type.

2021-07-18 Thread Bruce Wong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Wong updated SPARK-36203: --- Description: I want to know why the 'group by' can't use in column of map tyep.    

[jira] [Issue Comment Deleted] (SPARK-36203) Spark SQL can't use "group by" on the column of map type.

2021-07-18 Thread Bruce Wong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Wong updated SPARK-36203: --- Comment: was deleted (was: sql: select distinct idselect distinct id , cols , extend_value from

[jira] [Updated] (SPARK-36203) Spark SQL can't use "group by" on the column of map type.

2021-07-18 Thread Bruce Wong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Wong updated SPARK-36203: --- Description: I want to know why the 'group by' can't use in column of map tyep.   *sql:* select

[jira] [Commented] (SPARK-36203) Spark SQL can't use "group by" on the column of map type.

2021-07-18 Thread Bruce Wong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382971#comment-17382971 ] Bruce Wong commented on SPARK-36203: sql: select distinct idselect distinct id , cols , extend_value 

[jira] [Created] (SPARK-36203) Spark SQL can't use "group by" on the column of map type.

2021-07-18 Thread Bruce Wong (Jira)
Bruce Wong created SPARK-36203: -- Summary: Spark SQL can't use "group by" on the column of map type. Key: SPARK-36203 URL: https://issues.apache.org/jira/browse/SPARK-36203 Project: Spark Issue

[jira] [Created] (SPARK-36201) Add check for inner field of schema

2021-07-18 Thread angerszhu (Jira)
angerszhu created SPARK-36201: - Summary: Add check for inner field of schema Key: SPARK-36201 URL: https://issues.apache.org/jira/browse/SPARK-36201 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36202) Unify schema check API

2021-07-18 Thread angerszhu (Jira)
angerszhu created SPARK-36202: - Summary: Unify schema check API Key: SPARK-36202 URL: https://issues.apache.org/jira/browse/SPARK-36202 Project: Spark Issue Type: Sub-task Components:

[jira] [Updated] (SPARK-33865) When HiveDDL, we need check avro schema too like parquet & orc

2021-07-18 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-33865: -- Parent: SPARK-36200 Issue Type: Sub-task (was: Improvement) > When HiveDDL, we need check

[jira] [Created] (SPARK-36200) Improve schema check logic before execution

2021-07-18 Thread angerszhu (Jira)
angerszhu created SPARK-36200: - Summary: Improve schema check logic before execution Key: SPARK-36200 URL: https://issues.apache.org/jira/browse/SPARK-36200 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-35810) Deprecate ps.broadcast API

2021-07-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35810. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33379

[jira] [Assigned] (SPARK-35810) Deprecate ps.broadcast API

2021-07-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35810: Assignee: Haejoon Lee > Deprecate ps.broadcast API > -- > >

[jira] [Resolved] (SPARK-36198) Skip UNIDOC generation in pyspark GHA job

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36198. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33407

[jira] [Updated] (SPARK-36198) Skip UNIDOC generation in pyspark GHA job

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36198: -- Affects Version/s: 3.2.0 > Skip UNIDOC generation in pyspark GHA job >

[jira] [Assigned] (SPARK-36198) Skip UNIDOC generation in pyspark GHA job

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36198: - Assignee: William Hyun > Skip UNIDOC generation in pyspark GHA job >

[jira] [Commented] (SPARK-36199) Bump scalatest-maven-plugin to 2.0.2

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382940#comment-17382940 ] Apache Spark commented on SPARK-36199: -- User 'williamhyun' has created a pull request for this

[jira] [Commented] (SPARK-36199) Bump scalatest-maven-plugin to 2.0.2

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382939#comment-17382939 ] Apache Spark commented on SPARK-36199: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-36199) Bump scalatest-maven-plugin to 2.0.2

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36199: Assignee: Apache Spark > Bump scalatest-maven-plugin to 2.0.2 >

[jira] [Assigned] (SPARK-36199) Bump scalatest-maven-plugin to 2.0.2

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36199: Assignee: (was: Apache Spark) > Bump scalatest-maven-plugin to 2.0.2 >

[jira] [Created] (SPARK-36199) Bump scalatest-maven-plugin to 2.0.2

2021-07-18 Thread William Hyun (Jira)
William Hyun created SPARK-36199: Summary: Bump scalatest-maven-plugin to 2.0.2 Key: SPARK-36199 URL: https://issues.apache.org/jira/browse/SPARK-36199 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-36198) Skip UNIDOC generation in pyspark GHA job

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382937#comment-17382937 ] Apache Spark commented on SPARK-36198: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-36198) Skip UNIDOC generation in pyspark GHA job

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36198: Assignee: (was: Apache Spark) > Skip UNIDOC generation in pyspark GHA job >

[jira] [Assigned] (SPARK-36198) Skip UNIDOC generation in pyspark GHA job

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36198: Assignee: Apache Spark > Skip UNIDOC generation in pyspark GHA job >

[jira] [Commented] (SPARK-36198) Skip UNIDOC generation in pyspark GHA job

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382935#comment-17382935 ] Apache Spark commented on SPARK-36198: -- User 'williamhyun' has created a pull request for this

[jira] [Created] (SPARK-36198) Skip UNIDOC generation in pyspark GHA job

2021-07-18 Thread William Hyun (Jira)
William Hyun created SPARK-36198: Summary: Skip UNIDOC generation in pyspark GHA job Key: SPARK-36198 URL: https://issues.apache.org/jira/browse/SPARK-36198 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-36075) Support for specifiying executor/driver node selector

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36075. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33283

[jira] [Assigned] (SPARK-36075) Support for specifiying executor/driver node selector

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36075: - Assignee: Yikun Jiang > Support for specifiying executor/driver node selector >

[jira] [Assigned] (SPARK-35460) invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35460: - Assignee: Kent Yao > invalid `spark.kubernetes.executor.podNamePrefix` causes app to

[jira] [Resolved] (SPARK-35460) invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35460. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 32610

[jira] [Commented] (SPARK-35989) Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382925#comment-17382925 ] Dongjoon Hyun commented on SPARK-35989: --- This is assigned to [~ulysses]. > Do not remove

[jira] [Commented] (SPARK-35813) Add new adaptive config into sql-performance-tuning docs

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382924#comment-17382924 ] Dongjoon Hyun commented on SPARK-35813: --- This is assigned to @ulysses . > Add new adaptive config

[jira] [Assigned] (SPARK-35989) Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35989: - Assignee: XiDuo You > Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled >

[jira] [Comment Edited] (SPARK-35813) Add new adaptive config into sql-performance-tuning docs

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382924#comment-17382924 ] Dongjoon Hyun edited comment on SPARK-35813 at 7/18/21, 10:45 PM: -- This

[jira] [Assigned] (SPARK-35813) Add new adaptive config into sql-performance-tuning docs

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35813: - Assignee: XiDuo You > Add new adaptive config into sql-performance-tuning docs >

[jira] [Assigned] (SPARK-36014) Use uuid as app id in kubernetes client mode

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36014: - Assignee: XiDuo You > Use uuid as app id in kubernetes client mode >

[jira] [Resolved] (SPARK-36014) Use uuid as app id in kubernetes client mode

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36014. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33211

[jira] [Resolved] (SPARK-36040) Add reference to kubernetes-client's version

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36040. --- Fix Version/s: 3.1.3 3.2.0 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-36040) Add reference to kubernetes-client's version

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36040: - Assignee: Leona Yoda > Add reference to kubernetes-client's version >

[jira] [Assigned] (SPARK-36090) Support TimestampNTZType in expression Sequence

2021-07-18 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-36090: Assignee: jiaan.geng > Support TimestampNTZType in expression Sequence >

[jira] [Resolved] (SPARK-36090) Support TimestampNTZType in expression Sequence

2021-07-18 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36090. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33360

[jira] [Updated] (SPARK-36195) Set MaxMetaspaceSize JVM option to 1g

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36195: -- Affects Version/s: 3.1.2 > Set MaxMetaspaceSize JVM option to 1g >

[jira] [Updated] (SPARK-36195) Set MaxMetaspaceSize JVM option to 1g

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36195: -- Fix Version/s: 3.1.3 > Set MaxMetaspaceSize JVM option to 1g >

[jira] [Resolved] (SPARK-36195) Set MaxMetaspaceSize JVM option to 1g

2021-07-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36195. --- Fix Version/s: 3.2.0 Assignee: Dongjoon Hyun Resolution: Fixed This is

[jira] [Commented] (SPARK-36197) InputFormat of PartitionDesc is not respected

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382846#comment-17382846 ] Apache Spark commented on SPARK-36197: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36197) InputFormat of PartitionDesc is not respected

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36197: Assignee: (was: Apache Spark) > InputFormat of PartitionDesc is not respected >

[jira] [Assigned] (SPARK-36197) InputFormat of PartitionDesc is not respected

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36197: Assignee: Apache Spark > InputFormat of PartitionDesc is not respected >

[jira] [Commented] (SPARK-36197) InputFormat of PartitionDesc is not respected

2021-07-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382845#comment-17382845 ] Apache Spark commented on SPARK-36197: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Created] (SPARK-36197) InputFormat of PartitionDesc is not respected

2021-07-18 Thread Kent Yao (Jira)
Kent Yao created SPARK-36197: Summary: InputFormat of PartitionDesc is not respected Key: SPARK-36197 URL: https://issues.apache.org/jira/browse/SPARK-36197 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-36196) Spark FetchFailedException Stream is corrupted Error

2021-07-18 Thread Arghya Saha (Jira)
Arghya Saha created SPARK-36196: --- Summary: Spark FetchFailedException Stream is corrupted Error Key: SPARK-36196 URL: https://issues.apache.org/jira/browse/SPARK-36196 Project: Spark Issue