[jira] [Updated] (SPARK-43301) BlockStoreClient getHostLocalDirs RPC supports IOException retry

[ https://issues.apache.org/jira/browse/SPARK-43301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl updated SPARK-43301: --- Summary: BlockStoreClient getHostLocalDirs RPC supports IOException retry (was: BlockStoreClient

[jira] [Created] (SPARK-43377) Enable spark.sql.thriftServer.interruptOnCancel by defauly

XiDuo You created SPARK-43377: - Summary: Enable spark.sql.thriftServer.interruptOnCancel by defauly Key: SPARK-43377 URL: https://issues.apache.org/jira/browse/SPARK-43377 Project: Spark Issue

[jira] [Created] (SPARK-43375) Improve the error messages for INVALID_CONNECT_URL

Allison Wang created SPARK-43375: Summary: Improve the error messages for INVALID_CONNECT_URL Key: SPARK-43375 URL: https://issues.apache.org/jira/browse/SPARK-43375 Project: Spark Issue

[jira] [Commented] (SPARK-43346) Assign a name to the error class _LEGACY_ERROR_TEMP_1206

[ https://issues.apache.org/jira/browse/SPARK-43346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719207#comment-17719207 ] Jin Helin commented on SPARK-43346: --- I'd like to work on this. > Assign a name to the error class

[jira] [Commented] (SPARK-43368) Address DOI comments about /etc/passwd

[ https://issues.apache.org/jira/browse/SPARK-43368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719220#comment-17719220 ] Yikun Jiang commented on SPARK-43368: - Related original changes:

[jira] [Created] (SPARK-43376) Improve reuse subquery with table cache

XiDuo You created SPARK-43376: - Summary: Improve reuse subquery with table cache Key: SPARK-43376 URL: https://issues.apache.org/jira/browse/SPARK-43376 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-43377) Enable spark.sql.thriftServer.interruptOnCancel by default

[ https://issues.apache.org/jira/browse/SPARK-43377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-43377: -- Summary: Enable spark.sql.thriftServer.interruptOnCancel by default (was: Enable

[jira] [Commented] (SPARK-43342) Spark in Kubernetes mode throws IllegalArgumentException when using static PVC

[ https://issues.apache.org/jira/browse/SPARK-43342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719162#comment-17719162 ] Qian Sun commented on SPARK-43342: -- [~ofrenkel] Hello, I tried to reproduce using the configuration

[jira] [Commented] (SPARK-43369) Address comments about /etc/pam.d/su

[ https://issues.apache.org/jira/browse/SPARK-43369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719223#comment-17719223 ] Yikun Jiang commented on SPARK-43369: - Related original change:

[jira] [Resolved] (SPARK-42843) Assign a name to the error class _LEGACY_ERROR_TEMP_2007

[ https://issues.apache.org/jira/browse/SPARK-42843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-42843. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40955

[jira] [Assigned] (SPARK-42843) Assign a name to the error class _LEGACY_ERROR_TEMP_2007

[ https://issues.apache.org/jira/browse/SPARK-42843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42843: Assignee: Liang Yan > Assign a name to the error class _LEGACY_ERROR_TEMP_2007 >

[jira] [Updated] (SPARK-38682) Complex calculations with lead to driver oom

[ https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Fraison updated SPARK-38682: Attachment: 311.png > Complex calculations with lead to driver oom >

[jira] [Updated] (SPARK-38682) Complex calculations with lead to driver oom

[ https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Fraison updated SPARK-38682: Attachment: 331.png > Complex calculations with lead to driver oom >

[jira] [Commented] (SPARK-43279) Cleanup unused members from `SparkHadoopUtil`

[ https://issues.apache.org/jira/browse/SPARK-43279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719377#comment-17719377 ] Mike K commented on SPARK-43279: User 'LuciferYang' has created a pull request for this issue:

[jira] [Created] (SPARK-43378) SerializerHelper.deserializeFromChunkedBuffer

Emil Ejbyfeldt created SPARK-43378: -- Summary: SerializerHelper.deserializeFromChunkedBuffer Key: SPARK-43378 URL: https://issues.apache.org/jira/browse/SPARK-43378 Project: Spark Issue

[jira] [Updated] (SPARK-43340) Handle missing stack-trace field in eventlogs

[ https://issues.apache.org/jira/browse/SPARK-43340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein updated SPARK-43340: -- Description: Recently I was testing with some 3.0.2 eventlogs. The SHS-3.4+ does not

[jira] [Commented] (SPARK-41971) `toPandas` should support duplicate filed names when arrow-optimization is on

[ https://issues.apache.org/jira/browse/SPARK-41971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719463#comment-17719463 ] Xinrong Meng commented on SPARK-41971: -- Hi [~nikj] , the issue has been resolved. Feel free to pick

[jira] [Updated] (SPARK-42941) Add support for streaming listener in Python

[ https://issues.apache.org/jira/browse/SPARK-42941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Liu updated SPARK-42941: Summary: Add support for streaming listener in Python (was: Add support for streaming listener.) > Add

[jira] [Resolved] (SPARK-41971) `toPandas` should support duplicate filed names when arrow-optimization is on

[ https://issues.apache.org/jira/browse/SPARK-41971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-41971. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40988

[jira] [Assigned] (SPARK-41971) `toPandas` should support duplicate filed names when arrow-optimization is on

[ https://issues.apache.org/jira/browse/SPARK-41971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-41971: Assignee: Takuya Ueshin > `toPandas` should support duplicate filed names when

[jira] [Commented] (SPARK-38682) Complex calculations with lead to driver oom

[ https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719333#comment-17719333 ] Nicolas Fraison commented on SPARK-38682: - We are observing same pattern on one of our test job

[jira] [Updated] (SPARK-43380) Fix Avro data type conversion issues to avoid producing incorrect results

[ https://issues.apache.org/jira/browse/SPARK-43380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zerui Bao updated SPARK-43380: -- Target Version/s: 3.5.0 (was: 3.4.0) > Fix Avro data type conversion issues to avoid producing

[jira] [Assigned] (SPARK-43312) Protobuf: Allow converting Any fields to JSON

[ https://issues.apache.org/jira/browse/SPARK-43312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-43312: -- Assignee: Raghu Angadi > Protobuf: Allow converting Any fields to JSON >

[jira] [Resolved] (SPARK-43312) Protobuf: Allow converting Any fields to JSON

[ https://issues.apache.org/jira/browse/SPARK-43312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-43312. Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40983

[jira] [Resolved] (SPARK-43378) SerializerHelper.deserializeFromChunkedBuffer

[ https://issues.apache.org/jira/browse/SPARK-43378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-43378. -- Fix Version/s: 3.4.1 3.5.0 Assignee: Emil Ejbyfeldt

[jira] [Assigned] (SPARK-43363) Remove a workaround for pandas categorical type for pyarrow

[ https://issues.apache.org/jira/browse/SPARK-43363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43363: Assignee: Takuya Ueshin > Remove a workaround for pandas categorical type for pyarrow >

[jira] [Resolved] (SPARK-43363) Remove a workaround for pandas categorical type for pyarrow

[ https://issues.apache.org/jira/browse/SPARK-43363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43363. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41041

[jira] [Assigned] (SPARK-43375) Improve the error messages for INVALID_CONNECT_URL

[ https://issues.apache.org/jira/browse/SPARK-43375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43375: - Assignee: Allison Wang > Improve the error messages for INVALID_CONNECT_URL >

[jira] [Resolved] (SPARK-43375) Improve the error messages for INVALID_CONNECT_URL

[ https://issues.apache.org/jira/browse/SPARK-43375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43375. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41044

[jira] [Assigned] (SPARK-43040) Improve TimestampNTZ support in JDBC data source

[ https://issues.apache.org/jira/browse/SPARK-43040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-43040: --- Assignee: Tianhan Hu > Improve TimestampNTZ support in JDBC data source >

[jira] [Resolved] (SPARK-43040) Improve TimestampNTZ support in JDBC data source

[ https://issues.apache.org/jira/browse/SPARK-43040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-43040. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40678

[jira] [Created] (SPARK-43380) Fix Avro data type conversion issues to avoid producing incorrect results

Zerui Bao created SPARK-43380: - Summary: Fix Avro data type conversion issues to avoid producing incorrect results Key: SPARK-43380 URL: https://issues.apache.org/jira/browse/SPARK-43380 Project: Spark

[jira] [Assigned] (SPARK-43306) Migrate `ValueError` from Spark SQL types into error class

[ https://issues.apache.org/jira/browse/SPARK-43306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43306: - Assignee: Haejoon Lee > Migrate `ValueError` from Spark SQL types into error class >

[jira] [Resolved] (SPARK-43306) Migrate `ValueError` from Spark SQL types into error class

[ https://issues.apache.org/jira/browse/SPARK-43306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43306. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40975

[jira] [Commented] (SPARK-43364) Add docs for rocksdb state store memory mgmt to structured streaming guide

[ https://issues.apache.org/jira/browse/SPARK-43364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719498#comment-17719498 ] Hudson commented on SPARK-43364: User 'anishshri-db' has created a pull request for this issue:

[jira] [Assigned] (SPARK-43313) Adding missing default values for MERGE INSERT actions

[ https://issues.apache.org/jira/browse/SPARK-43313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-43313: -- Assignee: Daniel > Adding missing default values for MERGE INSERT actions >

[jira] [Resolved] (SPARK-43313) Adding missing default values for MERGE INSERT actions

[ https://issues.apache.org/jira/browse/SPARK-43313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-43313. Fix Version/s: 3.5.0 3.4.1 Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-43379) Deprecate old Java 8 versions prior to 8u371

Dongjoon Hyun created SPARK-43379: - Summary: Deprecate old Java 8 versions prior to 8u371 Key: SPARK-43379 URL: https://issues.apache.org/jira/browse/SPARK-43379 Project: Spark Issue Type:

[jira] [Updated] (SPARK-43338) Support modify the SESSION_CATALOG_NAME value

[ https://issues.apache.org/jira/browse/SPARK-43338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-43338: -- Description: {code:java} private[sql] object CatalogManager { val SESSION_CATALOG_NAME: String =

[jira] [Commented] (SPARK-43380) Fix Avro data type conversion issues to avoid producing incorrect results

[ https://issues.apache.org/jira/browse/SPARK-43380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719615#comment-17719615 ] Snoot.io commented on SPARK-43380: -- User 'zeruibao' has created a pull request for this issue:

[jira] [Resolved] (SPARK-43284) _metadata.file_path regression

[ https://issues.apache.org/jira/browse/SPARK-43284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-43284. - Fix Version/s: 3.5.0 3.4.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-43284) _metadata.file_path regression

[ https://issues.apache.org/jira/browse/SPARK-43284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-43284: --- Assignee: David Lewis > _metadata.file_path regression > -- >

[jira] [Created] (SPARK-43381) Make 'transformStatCov' lazy

Ruifeng Zheng created SPARK-43381: - Summary: Make 'transformStatCov' lazy Key: SPARK-43381 URL: https://issues.apache.org/jira/browse/SPARK-43381 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-43329) driver and executors shared same Kubernetes PVC in Spark 3.4+

[ https://issues.apache.org/jira/browse/SPARK-43329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719622#comment-17719622 ] Qian Sun commented on SPARK-43329: -- [~dongjoon] this ticket is duplicated with SPARK-43342 > driver

[jira] [Commented] (SPARK-43342) Spark in Kubernetes mode throws IllegalArgumentException when using static PVC

[ https://issues.apache.org/jira/browse/SPARK-43342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719621#comment-17719621 ] Snoot.io commented on SPARK-43342: -- User 'dcoliversun' has created a pull request for this issue:

[jira] [Commented] (SPARK-43342) Spark in Kubernetes mode throws IllegalArgumentException when using static PVC

[ https://issues.apache.org/jira/browse/SPARK-43342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719620#comment-17719620 ] Snoot.io commented on SPARK-43342: -- User 'dcoliversun' has created a pull request for this issue:

[jira] [Commented] (SPARK-43284) _metadata.file_path regression

[ https://issues.apache.org/jira/browse/SPARK-43284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719609#comment-17719609 ] ci-cassandra.apache.org commented on SPARK-43284: - User 'databricks-david-lewis' has

[jira] [Commented] (SPARK-43288) DataSourceV2: CREATE TABLE LIKE

[ https://issues.apache.org/jira/browse/SPARK-43288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719612#comment-17719612 ] Jia Fan commented on SPARK-43288: - I'm working on it. > DataSourceV2: CREATE TABLE LIKE >

[jira] [Commented] (SPARK-43337) Asc/desc arrow icons for sorting column does not get displayed in the table column

[ https://issues.apache.org/jira/browse/SPARK-43337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719617#comment-17719617 ] Snoot.io commented on SPARK-43337: -- User 'maytasm' has created a pull request for this issue:

[jira] [Commented] (SPARK-43381) Make 'transformStatCov' lazy

[ https://issues.apache.org/jira/browse/SPARK-43381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719618#comment-17719618 ] Snoot.io commented on SPARK-43381: -- User 'zhengruifeng' has created a pull request for this issue:

[jira] (SPARK-43342) Spark in Kubernetes mode throws IllegalArgumentException when using static PVC

[ https://issues.apache.org/jira/browse/SPARK-43342 ] Qian Sun deleted comment on SPARK-43342: -- was (Author: dcoliversun): [~ofrenkel] Hello, I tried to reproduce using the configuration you provided. There are some issues that I need to confirm