[jira] [Comment Edited] (SPARK-38061) security scan issue jackson-databinding HDFS dependency library

2022-02-02 Thread Sujit Biswas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486268#comment-17486268 ] Sujit Biswas edited comment on SPARK-38061 at 2/3/22, 7:49 AM: ---

[jira] [Commented] (SPARK-37936) Use error classes in the parsing errors of intervals

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486269#comment-17486269 ] Apache Spark commented on SPARK-37936: -- User 'senthh' has created a pull request fo

[jira] [Commented] (SPARK-38061) security scan issue jackson-databinding HDFS dependency library

2022-02-02 Thread Sujit Biswas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486268#comment-17486268 ] Sujit Biswas commented on SPARK-38061: -- also if some of the issues are resolved, ho

[jira] [Commented] (SPARK-38061) security scan issue jackson-databinding HDFS dependency library

2022-02-02 Thread Sujit Biswas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486264#comment-17486264 ] Sujit Biswas commented on SPARK-38061: -- not at all helpful, please refer to valid r

[jira] [Commented] (SPARK-38058) Writing a spark dataframe to Azure Sql Server is causing duplicate records intermittently

2022-02-02 Thread john (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486263#comment-17486263 ] john commented on SPARK-38058: -- since i am working in production env i cannot disclose any

[jira] [Commented] (SPARK-38061) security scan issue jackson-databinding HDFS dependency library

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486254#comment-17486254 ] Hyukjin Kwon commented on SPARK-38061: -- No, the security report here simply mention

[jira] [Commented] (SPARK-38061) security scan issue jackson-databinding HDFS dependency library

2022-02-02 Thread Sujit Biswas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486246#comment-17486246 ] Sujit Biswas commented on SPARK-38061: -- info is there in the attachment, you can do

[jira] [Assigned] (SPARK-38094) Parquet: enable matching schema columns by field id

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38094: Assignee: (was: Apache Spark) > Parquet: enable matching schema columns by field id >

[jira] [Assigned] (SPARK-38094) Parquet: enable matching schema columns by field id

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38094: Assignee: Apache Spark > Parquet: enable matching schema columns by field id > --

[jira] [Commented] (SPARK-38094) Parquet: enable matching schema columns by field id

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486237#comment-17486237 ] Apache Spark commented on SPARK-38094: -- User 'jackierwzhang' has created a pull req

[jira] [Commented] (SPARK-38061) security scan issue jackson-databinding HDFS dependency library

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486236#comment-17486236 ] Hyukjin Kwon commented on SPARK-38061: -- [~sujitbiswas] Let's separate a ticket for

[jira] [Commented] (SPARK-38061) security scan issue jackson-databinding HDFS dependency library

2022-02-02 Thread Sujit Biswas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486231#comment-17486231 ] Sujit Biswas commented on SPARK-38061: -- [~hyukjin.kwon]  note jackson-databind sol

[jira] [Commented] (SPARK-38073) NameError: name 'sc' is not defined when running driver with IPyhon and Pyhon > 3.7

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486219#comment-17486219 ] Hyukjin Kwon commented on SPARK-38073: -- I think we should fix this .. > NameError

[jira] [Resolved] (SPARK-38074) RuntimeError: Java gateway process exited before sending its port number

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38074. -- Resolution: Invalid > RuntimeError: Java gateway process exited before sending its port number

[jira] [Commented] (SPARK-38074) RuntimeError: Java gateway process exited before sending its port number

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486216#comment-17486216 ] Hyukjin Kwon commented on SPARK-38074: -- It's very likely your network configuration

[jira] [Updated] (SPARK-38074) RuntimeError: Java gateway process exited before sending its port number

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38074: - Priority: Major (was: Blocker) > RuntimeError: Java gateway process exited before sending its p

[jira] [Commented] (SPARK-38087) select doesnt validate if the column already exists

2022-02-02 Thread Deepa Vasanthkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486215#comment-17486215 ] Deepa Vasanthkumar commented on SPARK-38087: [~dongjoon]  Thank you, not sur

[jira] [Resolved] (SPARK-38095) HistoryServerDiskManager.appStorePath should use backend-based extensions

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38095. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35384 [https://

[jira] [Commented] (SPARK-38082) Update minimum numpy version

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486203#comment-17486203 ] Hyukjin Kwon commented on SPARK-38082: -- Yeah .. we should probably upgrade the mini

[jira] [Resolved] (SPARK-38061) security scan issue jackson-databinding HDFS dependency library

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38061. -- Resolution: Duplicate > security scan issue jackson-databinding HDFS dependency library >

[jira] [Commented] (SPARK-38061) security scan issue jackson-databinding HDFS dependency library

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486202#comment-17486202 ] Hyukjin Kwon commented on SPARK-38061: -- That's already upgraded at SPARK-35550 > s

[jira] [Commented] (SPARK-38058) Writing a spark dataframe to Azure Sql Server is causing duplicate records intermittently

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486201#comment-17486201 ] Hyukjin Kwon commented on SPARK-38058: -- spark.speculation has been disabled many ye

[jira] [Commented] (SPARK-38066) evaluateEquality should ignore attribute without min/max ColumnStat

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486200#comment-17486200 ] Apache Spark commented on SPARK-38066: -- User 'Stove-hust' has created a pull reques

[jira] [Assigned] (SPARK-38066) evaluateEquality should ignore attribute without min/max ColumnStat

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38066: Assignee: (was: Apache Spark) > evaluateEquality should ignore attribute without min/

[jira] [Assigned] (SPARK-38066) evaluateEquality should ignore attribute without min/max ColumnStat

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38066: Assignee: Apache Spark > evaluateEquality should ignore attribute without min/max ColumnS

[jira] [Commented] (SPARK-38095) HistoryServerDiskManager.appStorePath should use backend-based extensions

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486181#comment-17486181 ] Apache Spark commented on SPARK-38095: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-38095) HistoryServerDiskManager.appStorePath should use backend-based extensions

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38095: - Assignee: Dongjoon Hyun > HistoryServerDiskManager.appStorePath should use backend-base

[jira] [Assigned] (SPARK-38095) HistoryServerDiskManager.appStorePath should use backend-based extensions

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38095: - Assignee: Dongjoon Hyun (was: Apache Spark) > HistoryServerDiskManager.appStorePath sh

[jira] [Assigned] (SPARK-38095) HistoryServerDiskManager.appStorePath should use backend-based extensions

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38095: Assignee: (was: Dongjoon Hyun) > HistoryServerDiskManager.appStorePath should use bac

[jira] [Commented] (SPARK-38095) HistoryServerDiskManager.appStorePath should use backend-based extensions

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486180#comment-17486180 ] Apache Spark commented on SPARK-38095: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-38095) HistoryServerDiskManager.appStorePath should use backend-based extensions

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38095: Assignee: Apache Spark > HistoryServerDiskManager.appStorePath should use backend-based e

[jira] [Updated] (SPARK-38095) HistoryServerDiskManager.appStorePath should use backend-based extensions

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38095: -- Parent: SPARK-35781 Issue Type: Sub-task (was: Bug) > HistoryServerDiskManager.appSto

[jira] [Updated] (SPARK-38095) HistoryServerDiskManager.appStorePath should use backend-based extensions

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38095: -- Summary: HistoryServerDiskManager.appStorePath should use backend-based extensions (was: Hist

[jira] [Updated] (SPARK-38095) HistoryServerDiskManager.appStorePath should use backend extensions

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38095: -- Summary: HistoryServerDiskManager.appStorePath should use backend extensions (was: HistorySer

[jira] [Created] (SPARK-38095) HistoryServerDiskManager should use backend extensions for `apps` directory

2022-02-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-38095: - Summary: HistoryServerDiskManager should use backend extensions for `apps` directory Key: SPARK-38095 URL: https://issues.apache.org/jira/browse/SPARK-38095 Project

[jira] [Resolved] (SPARK-37958) Pyspark SparkContext.AddFile() does not respect spark.files.overwrite

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37958. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35377 [https://gi

[jira] [Assigned] (SPARK-37958) Pyspark SparkContext.AddFile() does not respect spark.files.overwrite

2022-02-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37958: Assignee: Leona Yoda > Pyspark SparkContext.AddFile() does not respect spark.files.overwr

[jira] [Created] (SPARK-38094) Parquet: enable matching schema columns by field id

2022-02-02 Thread Jackie Zhang (Jira)
Jackie Zhang created SPARK-38094: Summary: Parquet: enable matching schema columns by field id Key: SPARK-38094 URL: https://issues.apache.org/jira/browse/SPARK-38094 Project: Spark Issue Typ

[jira] [Updated] (SPARK-38087) select doesnt validate if the column already exists

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38087: -- Fix Version/s: (was: 3.3) > select doesnt validate if the column already exists >

[jira] [Commented] (SPARK-38087) select doesnt validate if the column already exists

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486167#comment-17486167 ] Dongjoon Hyun commented on SPARK-38087: --- I removed the fixed version field, [~deep

[jira] [Updated] (SPARK-30062) bug with DB2Driver using mode("overwrite") option("truncate",True)

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30062: -- Fix Version/s: 3.3.0 (was: 3.3) > bug with DB2Driver using mode("overwr

[jira] [Updated] (SPARK-38089) Show the root cause exception in TestUtils.assertExceptionMsg

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38089: -- Summary: Show the root cause exception in TestUtils.assertExceptionMsg (was: Improve assertio

[jira] [Resolved] (SPARK-38089) Improve assertion failure message in TestUtils.assertExceptionMsg

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38089. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35383 [https://

[jira] [Updated] (SPARK-38089) Show the root cause exception in TestUtils.assertExceptionMsg

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38089: -- Affects Version/s: 3.3.0 (was: 3.2.1) > Show the root cause excepti

[jira] [Assigned] (SPARK-38089) Improve assertion failure message in TestUtils.assertExceptionMsg

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38089: - Assignee: Erik Krogen > Improve assertion failure message in TestUtils.assertExceptionM

[jira] [Commented] (SPARK-37996) Contribution guide is stale

2022-02-02 Thread Khalid Mammadov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486139#comment-17486139 ] Khalid Mammadov commented on SPARK-37996: - Raised PR: https://github.com/apache/

[jira] [Commented] (SPARK-37771) Race condition in withHiveState and limited logic in IsolatedClientLoader result in ClassNotFoundException

2022-02-02 Thread Ivan Sadikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486107#comment-17486107 ] Ivan Sadikov commented on SPARK-37771: -- I could not manage to work around the issue

[jira] [Commented] (SPARK-38091) AvroSerializer can cause java.lang.ClassCastException at run time

2022-02-02 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486092#comment-17486092 ] Erik Krogen commented on SPARK-38091: - [~Zhen-hao] for formatting you need to use th

[jira] [Created] (SPARK-38093) Set shuffleMergeAllowed to false for a determinate stage after the stage is finalized

2022-02-02 Thread Venkata krishnan Sowrirajan (Jira)
Venkata krishnan Sowrirajan created SPARK-38093: --- Summary: Set shuffleMergeAllowed to false for a determinate stage after the stage is finalized Key: SPARK-38093 URL: https://issues.apache.org/jira/b

[jira] [Created] (SPARK-38092) Check if shuffleMergeId is the same as the current stage's shuffleMergeId before registering MergeStatus

2022-02-02 Thread Venkata krishnan Sowrirajan (Jira)
Venkata krishnan Sowrirajan created SPARK-38092: --- Summary: Check if shuffleMergeId is the same as the current stage's shuffleMergeId before registering MergeStatus Key: SPARK-38092 URL: https://issue

[jira] [Updated] (SPARK-38047) Add OUTLIER_NO_FALLBACK executor roll policy

2022-02-02 Thread Alex Holmes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Holmes updated SPARK-38047: Description: Currently executor rolling will always kill one executor every {{{}spark.kubernetes.

[jira] [Commented] (SPARK-38091) AvroSerializer can cause java.lang.ClassCastException at run time

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486041#comment-17486041 ] Apache Spark commented on SPARK-38091: -- User 'Zhen-hao' has created a pull request

[jira] [Assigned] (SPARK-38091) AvroSerializer can cause java.lang.ClassCastException at run time

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38091: Assignee: Apache Spark > AvroSerializer can cause java.lang.ClassCastException at run tim

[jira] [Assigned] (SPARK-38091) AvroSerializer can cause java.lang.ClassCastException at run time

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38091: Assignee: (was: Apache Spark) > AvroSerializer can cause java.lang.ClassCastException

[jira] [Commented] (SPARK-38091) AvroSerializer can cause java.lang.ClassCastException at run time

2022-02-02 Thread Zhenhao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486036#comment-17486036 ] Zhenhao Li commented on SPARK-38091: Can someone tell me how to let Jira render mark

[jira] [Created] (SPARK-38091) AvroSerializer can cause java.lang.ClassCastException at run time

2022-02-02 Thread Zhenhao Li (Jira)
Zhenhao Li created SPARK-38091: -- Summary: AvroSerializer can cause java.lang.ClassCastException at run time Key: SPARK-38091 URL: https://issues.apache.org/jira/browse/SPARK-38091 Project: Spark

[jira] [Assigned] (SPARK-38089) Improve assertion failure message in TestUtils.assertExceptionMsg

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38089: Assignee: (was: Apache Spark) > Improve assertion failure message in TestUtils.assert

[jira] [Commented] (SPARK-38089) Improve assertion failure message in TestUtils.assertExceptionMsg

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486025#comment-17486025 ] Apache Spark commented on SPARK-38089: -- User 'xkrogen' has created a pull request f

[jira] [Commented] (SPARK-38089) Improve assertion failure message in TestUtils.assertExceptionMsg

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486027#comment-17486027 ] Apache Spark commented on SPARK-38089: -- User 'xkrogen' has created a pull request f

[jira] [Assigned] (SPARK-38089) Improve assertion failure message in TestUtils.assertExceptionMsg

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38089: Assignee: Apache Spark > Improve assertion failure message in TestUtils.assertExceptionMs

[jira] [Created] (SPARK-38090) Make links for stderr/stdout on Spark on Kube configurable

2022-02-02 Thread Holden Karau (Jira)
Holden Karau created SPARK-38090: Summary: Make links for stderr/stdout on Spark on Kube configurable Key: SPARK-38090 URL: https://issues.apache.org/jira/browse/SPARK-38090 Project: Spark Is

[jira] [Created] (SPARK-38089) Improve assertion failure message in TestUtils.assertExceptionMsg

2022-02-02 Thread Erik Krogen (Jira)
Erik Krogen created SPARK-38089: --- Summary: Improve assertion failure message in TestUtils.assertExceptionMsg Key: SPARK-38089 URL: https://issues.apache.org/jira/browse/SPARK-38089 Project: Spark

[jira] [Resolved] (SPARK-37145) Improvement for extending pod feature steps with KubernetesConf

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37145. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35345 [https://

[jira] [Assigned] (SPARK-37145) Improvement for extending pod feature steps with KubernetesConf

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37145: - Assignee: Yikun Jiang > Improvement for extending pod feature steps with KubernetesConf

[jira] [Updated] (SPARK-37145) Add KubernetesCustom[Driver/Executor]FeatureConfigStep developer API

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37145: -- Summary: Add KubernetesCustom[Driver/Executor]FeatureConfigStep developer API (was: Improveme

[jira] [Commented] (SPARK-37771) Race condition in withHiveState and limited logic in IsolatedClientLoader result in ClassNotFoundException

2022-02-02 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485973#comment-17485973 ] Steve Loughran commented on SPARK-37771: [~ivan.sadikov] -any update here? > Ra

[jira] [Commented] (SPARK-28090) Spark hangs when an execution plan has many projections on nested structs

2022-02-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485917#comment-17485917 ] Apache Spark commented on SPARK-28090: -- User 'peter-toth' has created a pull reques

[jira] [Updated] (SPARK-38088) Kryo DataWritingSparkTaskResult registration error

2022-02-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michał Wieleba updated SPARK-38088: --- Attachment: (was: image-2022-02-02-10-08-51-144.png) > Kryo DataWritingSparkTaskResult r

[jira] [Updated] (SPARK-38088) Kryo DataWritingSparkTaskResult registration error

2022-02-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michał Wieleba updated SPARK-38088: --- Description: Spark 3.1.2, Scala 2.12 I'm registering classes with _sparkConf.registerKryoCl

[jira] [Updated] (SPARK-38088) Kryo DataWritingSparkTaskResult registration error

2022-02-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michał Wieleba updated SPARK-38088: --- Attachment: image-2022-02-02-10-09-14-858.png > Kryo DataWritingSparkTaskResult registration

[jira] [Updated] (SPARK-38088) Kryo DataWritingSparkTaskResult registration error

2022-02-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michał Wieleba updated SPARK-38088: --- Attachment: image-2022-02-02-10-08-51-144.png > Kryo DataWritingSparkTaskResult registration

[jira] [Updated] (SPARK-38088) Kryo DataWritingSparkTaskResult registration error

2022-02-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michał Wieleba updated SPARK-38088: --- Description: Spark 3.1.2, Scala 2.12 I'm registering classes with _sparkConf.registerKryoCl

[jira] [Created] (SPARK-38088) Kryo DataWritingSparkTaskResult registration error

2022-02-02 Thread Jira
Michał Wieleba created SPARK-38088: -- Summary: Kryo DataWritingSparkTaskResult registration error Key: SPARK-38088 URL: https://issues.apache.org/jira/browse/SPARK-38088 Project: Spark Issue

[jira] [Resolved] (SPARK-37908) Refactoring on pod label test in BasicFeatureStepSuite

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37908. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35209 [https://

[jira] [Assigned] (SPARK-37908) Refactoring on pod label test in BasicFeatureStepSuite

2022-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37908: - Assignee: Yikun Jiang > Refactoring on pod label test in BasicFeatureStepSuite > --