[jira] [Created] (SPARK-39103) SparkContext.addFiles trigger backend exception if it tries to add empty Hadoop directory

2022-05-04 Thread Daniel Dai (Jira)
Daniel Dai created SPARK-39103: -- Summary: SparkContext.addFiles trigger backend exception if it tries to add empty Hadoop directory Key: SPARK-39103 URL: https://issues.apache.org/jira/browse/SPARK-39103

[jira] [Created] (SPARK-39102) Replace the usage of guava's Files.createTempDir() with java.nio.file.Files.createTempDirectory()

2022-05-04 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-39102: Summary: Replace the usage of guava's Files.createTempDir() with java.nio.file.Files.createTempDirectory() Key: SPARK-39102 URL:

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

2022-05-04 Thread Shrikant (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532063#comment-17532063 ] Shrikant commented on SPARK-25355: -- The reason for no delegation token found is that the proxy user UGI

[jira] [Updated] (SPARK-25355) Support --proxy-user for Spark on K8s

2022-05-04 Thread Shrikant (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shrikant updated SPARK-25355: - Attachment: with_proxy_extradebugLogs.log > Support --proxy-user for Spark on K8s >

[jira] [Updated] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manAger in Spark

2022-05-04 Thread Junfan Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junfan Zhang updated SPARK-30873: - Summary: Handling Node Decommissioning for Yarn cluster manAger in Spark (was: Handling Node

[jira] [Updated] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manager in Spark

2022-05-04 Thread Junfan Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junfan Zhang updated SPARK-30873: - Summary: Handling Node Decommissioning for Yarn cluster manager in Spark (was: Handling Node

[jira] [Created] (SPARK-39101) Add longMessage and parameters to error-class

2022-05-04 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-39101: Summary: Add longMessage and parameters to error-class Key: SPARK-39101 URL: https://issues.apache.org/jira/browse/SPARK-39101 Project: Spark Issue Type:

[jira] [Commented] (SPARK-39100) RDD DeterministicLevel Needs Backward Compatibility

2022-05-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531975#comment-17531975 ] Apache Spark commented on SPARK-39100: -- User 'jiangzho' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39100) RDD DeterministicLevel Needs Backward Compatibility

2022-05-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39100: Assignee: (was: Apache Spark) > RDD DeterministicLevel Needs Backward Compatibility

[jira] [Commented] (SPARK-39100) RDD DeterministicLevel Needs Backward Compatibility

2022-05-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531974#comment-17531974 ] Apache Spark commented on SPARK-39100: -- User 'jiangzho' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39100) RDD DeterministicLevel Needs Backward Compatibility

2022-05-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39100: Assignee: Apache Spark > RDD DeterministicLevel Needs Backward Compatibility >

[jira] [Created] (SPARK-39100) RDD DeterministicLevel Needs Backward Compatibility

2022-05-04 Thread Zhou JIANG (Jira)
Zhou JIANG created SPARK-39100: -- Summary: RDD DeterministicLevel Needs Backward Compatibility Key: SPARK-39100 URL: https://issues.apache.org/jira/browse/SPARK-39100 Project: Spark Issue Type:

[jira] [Commented] (SPARK-39060) Typo in error messages of decimal overflow

2022-05-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531936#comment-17531936 ] Apache Spark commented on SPARK-39060: -- User 'vli-databricks' has created a pull request for this

[jira] [Commented] (SPARK-39099) Add dependencies to Dockerfile for building Spark releases

2022-05-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531923#comment-17531923 ] Apache Spark commented on SPARK-39099: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-39099) Add dependencies to Dockerfile for building Spark releases

2022-05-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531922#comment-17531922 ] Apache Spark commented on SPARK-39099: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39099) Add dependencies to Dockerfile for building Spark releases

2022-05-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39099: Assignee: (was: Apache Spark) > Add dependencies to Dockerfile for building Spark

[jira] [Assigned] (SPARK-39099) Add dependencies to Dockerfile for building Spark releases

2022-05-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39099: Assignee: Apache Spark > Add dependencies to Dockerfile for building Spark releases >

[jira] [Created] (SPARK-39099) Add dependencies to Dockerfile for building Spark releases

2022-05-04 Thread Max Gekk (Jira)
Max Gekk created SPARK-39099: Summary: Add dependencies to Dockerfile for building Spark releases Key: SPARK-39099 URL: https://issues.apache.org/jira/browse/SPARK-39099 Project: Spark Issue

[jira] [Updated] (SPARK-39098) Test the error class: PIVOT_VALUE_DATA_TYPE_MISMATCH

2022-05-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39098: - Fix Version/s: (was: 3.4.0) > Test the error class: PIVOT_VALUE_DATA_TYPE_MISMATCH >

[jira] [Assigned] (SPARK-39098) Test the error class: PIVOT_VALUE_DATA_TYPE_MISMATCH

2022-05-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-39098: Assignee: (was: panbingkun) > Test the error class: PIVOT_VALUE_DATA_TYPE_MISMATCH >

[jira] [Created] (SPARK-39098) Test the error class: PIVOT_VALUE_DATA_TYPE_MISMATCH

2022-05-04 Thread Max Gekk (Jira)
Max Gekk created SPARK-39098: Summary: Test the error class: PIVOT_VALUE_DATA_TYPE_MISMATCH Key: SPARK-39098 URL: https://issues.apache.org/jira/browse/SPARK-39098 Project: Spark Issue Type:

[jira] [Updated] (SPARK-39098) Test the error class: PIVOT_VALUE_DATA_TYPE_MISMATCH

2022-05-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39098: - Description: Add tests for the error classes *PIVOT_VALUE_DATA_TYPE_MISMATCH* to

[jira] [Resolved] (SPARK-38744) Test the pivot error classes

2022-05-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38744. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36431

[jira] [Assigned] (SPARK-38744) Test the pivot error classes

2022-05-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38744: Assignee: panbingkun > Test the pivot error classes > > >

[jira] [Updated] (SPARK-38891) Skipping allocating vector for repetition & definition levels when possible

2022-05-04 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-38891: - Fix Version/s: 3.3.0 > Skipping allocating vector for repetition & definition levels when possible >

[jira] [Resolved] (SPARK-38891) Skipping allocating vector for repetition & definition levels when possible

2022-05-04 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-38891. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36202

[jira] [Assigned] (SPARK-38891) Skipping allocating vector for repetition & definition levels when possible

2022-05-04 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-38891: Assignee: Chao Sun > Skipping allocating vector for repetition & definition levels when possible

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

2022-05-04 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531777#comment-17531777 ] Gabor Somogyi commented on SPARK-25355: --- Hmmm, seems like I've already debugged such thing and

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

2022-05-04 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531743#comment-17531743 ] Gabor Somogyi commented on SPARK-25355: --- Now I see where the driver blows up and why no Spark logs

[jira] [Commented] (SPARK-38262) Upgrade Google guava to version 30.0-jre

2022-05-04 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531723#comment-17531723 ] Bjørn Jørgensen commented on SPARK-38262: - [~pralabhkumar] have a look at this PR

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

2022-05-04 Thread Shrikant (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531696#comment-17531696 ] Shrikant commented on SPARK-25355: -- Haven't removed anything from log4j.properties. Only Changed log

[jira] [Resolved] (SPARK-38733) Test the error class: INCOMPATIBLE_DATASOURCE_REGISTER

2022-05-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38733. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36429

[jira] [Assigned] (SPARK-38733) Test the error class: INCOMPATIBLE_DATASOURCE_REGISTER

2022-05-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38733: Assignee: panbingkun > Test the error class: INCOMPATIBLE_DATASOURCE_REGISTER >

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

2022-05-04 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531646#comment-17531646 ] Gabor Somogyi commented on SPARK-25355: --- > (IllegalArgumentException: Empty cookie header string)

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

2022-05-04 Thread Shrikant (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531628#comment-17531628 ] Shrikant commented on SPARK-25355: -- We have to use the --proxy-user arg.So need to get Option 1

[jira] [Resolved] (SPARK-39093) Dividing interval by integral can result in codegen compilation error

2022-05-04 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-39093. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36442

[jira] [Assigned] (SPARK-39093) Dividing interval by integral can result in codegen compilation error

2022-05-04 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-39093: -- Assignee: Bruce Robbins > Dividing interval by integral can result in codegen

[jira] [Commented] (SPARK-38807) Error when starting spark shell on Windows system

2022-05-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531607#comment-17531607 ] Apache Spark commented on SPARK-38807: -- User '1104056452' has created a pull request for this

[jira] [Comment Edited] (SPARK-25355) Support --proxy-user for Spark on K8s

2022-05-04 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531523#comment-17531523 ] Gabor Somogyi edited comment on SPARK-25355 at 5/4/22 8:00 AM: --- After the

[jira] [Commented] (SPARK-38262) Upgrade Google guava to version 30.0-jre

2022-05-04 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531526#comment-17531526 ] pralabhkumar commented on SPARK-38262: -- [~bjornjorgensen]  So QQ , as part of this PR , it is not

[jira] [Resolved] (SPARK-38838) Support ALTER TABLE ALTER COLUMN commands with DEFAULT values

2022-05-04 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38838. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36398

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

2022-05-04 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531523#comment-17531523 ] Gabor Somogyi commented on SPARK-25355: --- After the attached logs now I see more.

[jira] [Resolved] (SPARK-39060) Typo in error messages of decimal overflow

2022-05-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-39060. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36397

[jira] [Assigned] (SPARK-39060) Typo in error messages of decimal overflow

2022-05-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-39060: Assignee: Vitalii Li > Typo in error messages of decimal overflow >