[jira] [Commented] (SPARK-36476) cloudpickle: ValueError: Cell is empty

[ https://issues.apache.org/jira/browse/SPARK-36476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482308#comment-17482308 ] Pedro Larroy commented on SPARK-36476: -- This seems to happen as an interaction with

[jira] [Commented] (SPARK-33326) Partition Parameters are not updated even after ANALYZE TABLE command

[ https://issues.apache.org/jira/browse/SPARK-33326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482294#comment-17482294 ] Apache Spark commented on SPARK-33326: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-33326) Partition Parameters are not updated even after ANALYZE TABLE command

[ https://issues.apache.org/jira/browse/SPARK-33326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33326: Assignee: Apache Spark > Partition Parameters are not updated even after ANALYZE TABLE co

[jira] [Assigned] (SPARK-33326) Partition Parameters are not updated even after ANALYZE TABLE command

[ https://issues.apache.org/jira/browse/SPARK-33326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33326: Assignee: (was: Apache Spark) > Partition Parameters are not updated even after ANALY

[jira] [Resolved] (SPARK-38032) Upgrade Arrow version < 7.0.0 for Python UDF tests in SQL

[ https://issues.apache.org/jira/browse/SPARK-38032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38032. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35331 [https://gi

[jira] [Assigned] (SPARK-38032) Upgrade Arrow version < 7.0.0 for Python UDF tests in SQL

[ https://issues.apache.org/jira/browse/SPARK-38032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38032: Assignee: Hyukjin Kwon > Upgrade Arrow version < 7.0.0 for Python UDF tests in SQL >

[jira] [Resolved] (SPARK-38031) Update document type conversion for Pandas UDFs (pyarrow 6.0.1, pandas 1.4.0, Python 3.9)

[ https://issues.apache.org/jira/browse/SPARK-38031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38031. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35330 [https://gi

[jira] [Assigned] (SPARK-38031) Update document type conversion for Pandas UDFs (pyarrow 6.0.1, pandas 1.4.0, Python 3.9)

[ https://issues.apache.org/jira/browse/SPARK-38031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38031: Assignee: Hyukjin Kwon > Update document type conversion for Pandas UDFs (pyarrow 6.0.1,

[jira] [Commented] (SPARK-37946) Use error classes in the execution errors related to partitions

[ https://issues.apache.org/jira/browse/SPARK-37946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482281#comment-17482281 ] Yuto Akutsu commented on SPARK-37946: - [~maxgekk] I will work on this. > Use error

[jira] [Commented] (SPARK-37937) Use error classes in the parsing errors of lateral join

[ https://issues.apache.org/jira/browse/SPARK-37937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482278#comment-17482278 ] Apache Spark commented on SPARK-37937: -- User 'imback82' has created a pull request

[jira] [Assigned] (SPARK-37937) Use error classes in the parsing errors of lateral join

[ https://issues.apache.org/jira/browse/SPARK-37937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37937: Assignee: Apache Spark > Use error classes in the parsing errors of lateral join > --

[jira] [Assigned] (SPARK-37937) Use error classes in the parsing errors of lateral join

[ https://issues.apache.org/jira/browse/SPARK-37937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37937: Assignee: (was: Apache Spark) > Use error classes in the parsing errors of lateral jo

[jira] [Commented] (SPARK-37937) Use error classes in the parsing errors of lateral join

[ https://issues.apache.org/jira/browse/SPARK-37937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482277#comment-17482277 ] Apache Spark commented on SPARK-37937: -- User 'imback82' has created a pull request

[jira] [Assigned] (SPARK-38030) Query with cast containing non-nullable columns fails with AQE on Spark 3.1.1

[ https://issues.apache.org/jira/browse/SPARK-38030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38030: Assignee: (was: Apache Spark) > Query with cast containing non-nullable columns fails

[jira] [Commented] (SPARK-38030) Query with cast containing non-nullable columns fails with AQE on Spark 3.1.1

[ https://issues.apache.org/jira/browse/SPARK-38030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482271#comment-17482271 ] Apache Spark commented on SPARK-38030: -- User 'shardulm94' has created a pull reques

[jira] [Assigned] (SPARK-38030) Query with cast containing non-nullable columns fails with AQE on Spark 3.1.1

[ https://issues.apache.org/jira/browse/SPARK-38030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38030: Assignee: Apache Spark > Query with cast containing non-nullable columns fails with AQE o

[jira] [Assigned] (SPARK-38032) Upgrade Arrow version < 7.0.0 for Python UDF tests in SQL

[ https://issues.apache.org/jira/browse/SPARK-38032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38032: Assignee: (was: Apache Spark) > Upgrade Arrow version < 7.0.0 for Python UDF tests in

[jira] [Commented] (SPARK-38032) Upgrade Arrow version < 7.0.0 for Python UDF tests in SQL

[ https://issues.apache.org/jira/browse/SPARK-38032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482242#comment-17482242 ] Apache Spark commented on SPARK-38032: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-38032) Upgrade Arrow version < 7.0.0 for Python UDF tests in SQL

[ https://issues.apache.org/jira/browse/SPARK-38032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38032: Assignee: Apache Spark > Upgrade Arrow version < 7.0.0 for Python UDF tests in SQL >

[jira] [Created] (SPARK-38032) Upgrade Arrow version < 7.0.0 for Python UDF tests in SQL

Hyukjin Kwon created SPARK-38032: Summary: Upgrade Arrow version < 7.0.0 for Python UDF tests in SQL Key: SPARK-38032 URL: https://issues.apache.org/jira/browse/SPARK-38032 Project: Spark Iss

[jira] [Commented] (SPARK-38031) Update document type conversion for Pandas UDFs (pyarrow 6.0.1, pandas 1.4.0, Python 3.9)

[ https://issues.apache.org/jira/browse/SPARK-38031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482239#comment-17482239 ] Apache Spark commented on SPARK-38031: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-38031) Update document type conversion for Pandas UDFs (pyarrow 6.0.1, pandas 1.4.0, Python 3.9)

[ https://issues.apache.org/jira/browse/SPARK-38031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38031: Assignee: (was: Apache Spark) > Update document type conversion for Pandas UDFs (pyar

[jira] [Assigned] (SPARK-38031) Update document type conversion for Pandas UDFs (pyarrow 6.0.1, pandas 1.4.0, Python 3.9)

[ https://issues.apache.org/jira/browse/SPARK-38031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38031: Assignee: Apache Spark > Update document type conversion for Pandas UDFs (pyarrow 6.0.1,

[jira] [Commented] (SPARK-38031) Update document type conversion for Pandas UDFs (pyarrow 6.0.1, pandas 1.4.0, Python 3.9)

[ https://issues.apache.org/jira/browse/SPARK-38031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482237#comment-17482237 ] Apache Spark commented on SPARK-38031: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-38003) Differentiate scalar and table function lookup in LookupFunctions

[ https://issues.apache.org/jira/browse/SPARK-38003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38003: --- Assignee: Allison Wang > Differentiate scalar and table function lookup in LookupFunctions

[jira] [Resolved] (SPARK-38003) Differentiate scalar and table function lookup in LookupFunctions

[ https://issues.apache.org/jira/browse/SPARK-38003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38003. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35304 [https://gith

[jira] [Created] (SPARK-38031) Update document type conversion for Pandas UDFs (pyarrow 6.0.1, pandas 1.4.0, Python 3.9)

Hyukjin Kwon created SPARK-38031: Summary: Update document type conversion for Pandas UDFs (pyarrow 6.0.1, pandas 1.4.0, Python 3.9) Key: SPARK-38031 URL: https://issues.apache.org/jira/browse/SPARK-38031

[jira] [Commented] (SPARK-38030) Query with cast containing non-nullable columns fails with AQE on Spark 3.1.1

[ https://issues.apache.org/jira/browse/SPARK-38030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482233#comment-17482233 ] Shardul Mahadik commented on SPARK-38030: - I plan to create a PR to change the c

[jira] [Resolved] (SPARK-37948) Disable mapreduce.fileoutputcommitter.algorithm.version=2 by default

[ https://issues.apache.org/jira/browse/SPARK-37948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37948. -- Resolution: Won't Fix > Disable mapreduce.fileoutputcommitter.algorithm.version=2 by default >

[jira] [Created] (SPARK-38030) Query with cast containing non-nullable columns fails with AQE on Spark 3.1.1

Shardul Mahadik created SPARK-38030: --- Summary: Query with cast containing non-nullable columns fails with AQE on Spark 3.1.1 Key: SPARK-38030 URL: https://issues.apache.org/jira/browse/SPARK-38030 P

[jira] [Assigned] (SPARK-33328) Fix Flaky HiveThriftHttpServerSuite

[ https://issues.apache.org/jira/browse/SPARK-33328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33328: Assignee: Apache Spark > Fix Flaky HiveThriftHttpServerSuite > --

[jira] [Commented] (SPARK-33328) Fix Flaky HiveThriftHttpServerSuite

[ https://issues.apache.org/jira/browse/SPARK-33328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482215#comment-17482215 ] Apache Spark commented on SPARK-33328: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-33328) Fix Flaky HiveThriftHttpServerSuite

[ https://issues.apache.org/jira/browse/SPARK-33328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33328: Assignee: (was: Apache Spark) > Fix Flaky HiveThriftHttpServerSuite > ---

[jira] [Resolved] (SPARK-38013) AQE can change bhj to smj if no extra shuffle introduce

[ https://issues.apache.org/jira/browse/SPARK-38013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You resolved SPARK-38013. --- Resolution: Won't Fix > AQE can change bhj to smj if no extra shuffle introduce > --

[jira] [Commented] (SPARK-38013) AQE can change bhj to smj if no extra shuffle introduce

[ https://issues.apache.org/jira/browse/SPARK-38013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482209#comment-17482209 ] XiDuo You commented on SPARK-38013: --- seems it is allowed in AQE, not a bug otherwise .

[jira] [Updated] (SPARK-38013) AQE can change bhj to smj if no extra shuffle introduce

[ https://issues.apache.org/jira/browse/SPARK-38013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-38013: -- Issue Type: Task (was: Bug) > AQE can change bhj to smj if no extra shuffle introduce > -

[jira] [Resolved] (SPARK-30062) bug with DB2Driver using mode("overwrite") option("truncate",True)

[ https://issues.apache.org/jira/browse/SPARK-30062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-30062. Fix Version/s: 3.2.2 3.3 Resolution: Fixed > bug with DB2Driver using mo

[jira] [Commented] (SPARK-37858) Throw Spark exceptions from AES functions

[ https://issues.apache.org/jira/browse/SPARK-37858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482197#comment-17482197 ] Apache Spark commented on SPARK-37858: -- User 'imback82' has created a pull request

[jira] [Commented] (SPARK-37858) Throw Spark exceptions from AES functions

[ https://issues.apache.org/jira/browse/SPARK-37858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482196#comment-17482196 ] Apache Spark commented on SPARK-37858: -- User 'imback82' has created a pull request

[jira] [Updated] (SPARK-38013) AQE can change bhj to smj if no extra shuffle introduce

[ https://issues.apache.org/jira/browse/SPARK-38013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-38013: -- Summary: AQE can change bhj to smj if no extra shuffle introduce (was: Fix AQE can change bhj to smj

[jira] [Commented] (SPARK-37995) TPCDS 1TB q72 fails when spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly is false

[ https://issues.apache.org/jira/browse/SPARK-37995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482194#comment-17482194 ] Hyukjin Kwon commented on SPARK-37995: -- cc [~maryannxue] FYI > TPCDS 1TB q72 fails

[jira] [Commented] (SPARK-37996) Contribution guide is stale

[ https://issues.apache.org/jira/browse/SPARK-37996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482193#comment-17482193 ] Hyukjin Kwon commented on SPARK-37996: -- Hm, yeah. I think now we always run the tes

[jira] [Commented] (SPARK-37997) Allow query parameters to be passed into spark.read

[ https://issues.apache.org/jira/browse/SPARK-37997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482192#comment-17482192 ] Hyukjin Kwon commented on SPARK-37997: -- Can we just format it before passing to spa

[jira] [Resolved] (SPARK-38000) Sort node incorrectly removed from the optimized logical plan

[ https://issues.apache.org/jira/browse/SPARK-38000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38000. -- Resolution: Cannot Reproduce > Sort node incorrectly removed from the optimized logical plan >

[jira] [Resolved] (SPARK-38028) Expose Arrow Vector from ArrowColumnVector

[ https://issues.apache.org/jira/browse/SPARK-38028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38028. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35326 [https://

[jira] [Commented] (SPARK-37980) Extend METADATA column to support row indices for file based data sources

[ https://issues.apache.org/jira/browse/SPARK-37980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482188#comment-17482188 ] Wenchen Fan commented on SPARK-37980: - I think it's possible for the parquet data so

[jira] [Comment Edited] (SPARK-37980) Extend METADATA column to support row indices for file based data sources

[ https://issues.apache.org/jira/browse/SPARK-37980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482142#comment-17482142 ] Prakhar Jain edited comment on SPARK-37980 at 1/26/22, 1:58 AM: --

[jira] [Assigned] (SPARK-38029) Support K8S integration test in SBT

[ https://issues.apache.org/jira/browse/SPARK-38029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38029: Assignee: (was: Apache Spark) > Support K8S integration test in SBT > ---

[jira] [Commented] (SPARK-38029) Support K8S integration test in SBT

[ https://issues.apache.org/jira/browse/SPARK-38029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482178#comment-17482178 ] Apache Spark commented on SPARK-38029: -- User 'williamhyun' has created a pull reque

[jira] [Assigned] (SPARK-38029) Support K8S integration test in SBT

[ https://issues.apache.org/jira/browse/SPARK-38029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38029: Assignee: Apache Spark > Support K8S integration test in SBT > --

[jira] [Assigned] (SPARK-38028) Expose Arrow Vector from ArrowColumnVector

[ https://issues.apache.org/jira/browse/SPARK-38028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38028: Assignee: Apache Spark (was: L. C. Hsieh) > Expose Arrow Vector from ArrowColumnVector >

[jira] [Assigned] (SPARK-38028) Expose Arrow Vector from ArrowColumnVector

[ https://issues.apache.org/jira/browse/SPARK-38028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38028: Assignee: L. C. Hsieh (was: Apache Spark) > Expose Arrow Vector from ArrowColumnVector >

[jira] [Commented] (SPARK-38028) Expose Arrow Vector from ArrowColumnVector

[ https://issues.apache.org/jira/browse/SPARK-38028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482176#comment-17482176 ] Apache Spark commented on SPARK-38028: -- User 'viirya' has created a pull request fo

[jira] [Created] (SPARK-38029) Support K8S integration test in SBT

William Hyun created SPARK-38029: Summary: Support K8S integration test in SBT Key: SPARK-38029 URL: https://issues.apache.org/jira/browse/SPARK-38029 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-38028) Expose Arrow Vector from ArrowColumnVector

[ https://issues.apache.org/jira/browse/SPARK-38028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-38028: --- Assignee: L. C. Hsieh > Expose Arrow Vector from ArrowColumnVector > --

[jira] [Created] (SPARK-38028) Expose Arrow Vector from ArrowColumnVector

L. C. Hsieh created SPARK-38028: --- Summary: Expose Arrow Vector from ArrowColumnVector Key: SPARK-38028 URL: https://issues.apache.org/jira/browse/SPARK-38028 Project: Spark Issue Type: Improvem

[jira] [Comment Edited] (SPARK-38004) read_excel's parameter - mangle_dupe_cols is used to handle duplicate columns but fails if the duplicate columns are case sensitive.

[ https://issues.apache.org/jira/browse/SPARK-38004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482174#comment-17482174 ] Haejoon Lee edited comment on SPARK-38004 at 1/26/22, 1:34 AM: ---

[jira] [Updated] (SPARK-38004) read_excel's parameter - mangle_dupe_cols is used to handle duplicate columns but fails if the duplicate columns are case sensitive.

[ https://issues.apache.org/jira/browse/SPARK-38004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-38004: Affects Version/s: 3.2.0 (was: 3.1.2) > read_excel's parameter - mangle

[jira] [Comment Edited] (SPARK-38004) read_excel's parameter - mangle_dupe_cols is used to handle duplicate columns but fails if the duplicate columns are case sensitive.

[ https://issues.apache.org/jira/browse/SPARK-38004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482174#comment-17482174 ] Haejoon Lee edited comment on SPARK-38004 at 1/26/22, 1:33 AM: ---

[jira] [Comment Edited] (SPARK-38004) read_excel's parameter - mangle_dupe_cols is used to handle duplicate columns but fails if the duplicate columns are case sensitive.

[ https://issues.apache.org/jira/browse/SPARK-38004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482174#comment-17482174 ] Haejoon Lee edited comment on SPARK-38004 at 1/26/22, 1:33 AM: ---

[jira] [Commented] (SPARK-38004) read_excel's parameter - mangle_dupe_cols is used to handle duplicate columns but fails if the duplicate columns are case sensitive.

[ https://issues.apache.org/jira/browse/SPARK-38004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482174#comment-17482174 ] Haejoon Lee commented on SPARK-38004: - [~Saikrishna_Pujari] Thanks for the report th

[jira] [Comment Edited] (SPARK-38004) read_excel's parameter - mangle_dupe_cols is used to handle duplicate columns but fails if the duplicate columns are case sensitive.

[ https://issues.apache.org/jira/browse/SPARK-38004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482174#comment-17482174 ] Haejoon Lee edited comment on SPARK-38004 at 1/26/22, 1:30 AM: ---

[jira] [Assigned] (SPARK-38015) Mark legacy file naming functions as deprecated in FileCommitProtocol

[ https://issues.apache.org/jira/browse/SPARK-38015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38015: Assignee: Cheng Su > Mark legacy file naming functions as deprecated in FileCommitProtoco

[jira] [Resolved] (SPARK-38015) Mark legacy file naming functions as deprecated in FileCommitProtocol

[ https://issues.apache.org/jira/browse/SPARK-38015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38015. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35311 [https://gi

[jira] [Commented] (SPARK-37793) Invalid LocalMergedBlockData cause task hang

[ https://issues.apache.org/jira/browse/SPARK-37793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482148#comment-17482148 ] Apache Spark commented on SPARK-37793: -- User 'otterc' has created a pull request fo

[jira] [Commented] (SPARK-37793) Invalid LocalMergedBlockData cause task hang

[ https://issues.apache.org/jira/browse/SPARK-37793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482149#comment-17482149 ] Apache Spark commented on SPARK-37793: -- User 'otterc' has created a pull request fo

[jira] [Commented] (SPARK-37793) Invalid LocalMergedBlockData cause task hang

[ https://issues.apache.org/jira/browse/SPARK-37793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482147#comment-17482147 ] Apache Spark commented on SPARK-37793: -- User 'otterc' has created a pull request fo

[jira] [Commented] (SPARK-37675) Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

[ https://issues.apache.org/jira/browse/SPARK-37675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482146#comment-17482146 ] Apache Spark commented on SPARK-37675: -- User 'otterc' has created a pull request fo

[jira] [Commented] (SPARK-37675) Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

[ https://issues.apache.org/jira/browse/SPARK-37675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482145#comment-17482145 ] Apache Spark commented on SPARK-37675: -- User 'otterc' has created a pull request fo

[jira] [Commented] (SPARK-37980) Extend METADATA column to support row indices for file based data sources

[ https://issues.apache.org/jira/browse/SPARK-37980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482142#comment-17482142 ] Prakhar Jain commented on SPARK-37980: -- Yes - this needs implementation in the unde

[jira] [Updated] (SPARK-38022) Use relativePath for K8s remote file test in BasicTestsSuite

[ https://issues.apache.org/jira/browse/SPARK-38022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38022: -- Fix Version/s: (was: 3.2.2) > Use relativePath for K8s remote file test in BasicTestsSuite

[jira] [Assigned] (SPARK-38023) ExecutorMonitor.onExecutorRemoved should handle ExecutorDecommission as finished

[ https://issues.apache.org/jira/browse/SPARK-38023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38023: - Assignee: Dongjoon Hyun > ExecutorMonitor.onExecutorRemoved should handle ExecutorDecom

[jira] [Updated] (SPARK-38023) ExecutorMonitor.onExecutorRemoved should handle ExecutorDecommission as finished

[ https://issues.apache.org/jira/browse/SPARK-38023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38023: -- Fix Version/s: 3.2.2 (was: 3.2.1) > ExecutorMonitor.onExecutorRemoved s

[jira] [Resolved] (SPARK-38023) ExecutorMonitor.onExecutorRemoved should handle ExecutorDecommission as finished

[ https://issues.apache.org/jira/browse/SPARK-38023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38023. --- Fix Version/s: 3.3.0 3.2.1 Resolution: Fixed Issue resolved by pul

[jira] [Commented] (SPARK-38027) Undefined link function causing error in GLM that uses Tweedie family

[ https://issues.apache.org/jira/browse/SPARK-38027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482128#comment-17482128 ] Evan Zamir commented on SPARK-38027: Looking into this further I think the issue is

[jira] [Commented] (SPARK-37896) ConstantColumnVector: a column vector with same values

[ https://issues.apache.org/jira/browse/SPARK-37896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482117#comment-17482117 ] Apache Spark commented on SPARK-37896: -- User 'c21' has created a pull request for t

[jira] [Created] (SPARK-38027) Undefined link function causing error in GLM that uses Tweedie family

Evan Zamir created SPARK-38027: -- Summary: Undefined link function causing error in GLM that uses Tweedie family Key: SPARK-38027 URL: https://issues.apache.org/jira/browse/SPARK-38027 Project: Spark

[jira] [Commented] (SPARK-38026) Sorting in Executors summary table in Stages Page is broken

[ https://issues.apache.org/jira/browse/SPARK-38026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482080#comment-17482080 ] Thejdeep Gudivada commented on SPARK-38026: --- Duplicate of https://issues.apach

[jira] [Resolved] (SPARK-38026) Sorting in Executors summary table in Stages Page is broken

[ https://issues.apache.org/jira/browse/SPARK-38026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejdeep Gudivada resolved SPARK-38026. --- Resolution: Duplicate > Sorting in Executors summary table in Stages Page is broken

[jira] [Updated] (SPARK-38026) Sorting in Executors summary table in Stages Page is broken

[ https://issues.apache.org/jira/browse/SPARK-38026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejdeep Gudivada updated SPARK-38026: -- Description: Sorting of certain columns in the Executors Summary table in the Stages P

[jira] [Created] (SPARK-38026) Sorting in Executors summary table in Stages Page is broken

Thejdeep Gudivada created SPARK-38026: - Summary: Sorting in Executors summary table in Stages Page is broken Key: SPARK-38026 URL: https://issues.apache.org/jira/browse/SPARK-38026 Project: Spark

[jira] [Updated] (SPARK-38026) Sorting in Executors summary table in Stages Page is broken

[ https://issues.apache.org/jira/browse/SPARK-38026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejdeep Gudivada updated SPARK-38026: -- Attachment: image (5).png > Sorting in Executors summary table in Stages Page is broke

[jira] [Commented] (SPARK-34372) Speculation results in broken CSV files in Amazon S3

[ https://issues.apache.org/jira/browse/SPARK-34372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482041#comment-17482041 ] Attila Zsolt Piros commented on SPARK-34372: hi [~daeheh]! Please look aroun

[jira] [Commented] (SPARK-38025) Improve test suite ExternalCatalogSuite

[ https://issues.apache.org/jira/browse/SPARK-38025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481992#comment-17481992 ] Apache Spark commented on SPARK-38025: -- User 'khalidmammadov' has created a pull re

[jira] [Assigned] (SPARK-38025) Improve test suite ExternalCatalogSuite

[ https://issues.apache.org/jira/browse/SPARK-38025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38025: Assignee: Apache Spark > Improve test suite ExternalCatalogSuite > --

[jira] [Assigned] (SPARK-38025) Improve test suite ExternalCatalogSuite

[ https://issues.apache.org/jira/browse/SPARK-38025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38025: Assignee: (was: Apache Spark) > Improve test suite ExternalCatalogSuite > ---

[jira] [Updated] (SPARK-38022) Use relativePath for K8s remote file test in BasicTestsSuite

[ https://issues.apache.org/jira/browse/SPARK-38022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38022: -- Fix Version/s: 3.2.2 (was: 3.2.1) > Use relativePath for K8s remote fil

[jira] [Resolved] (SPARK-38022) Use relativePath for K8s remote file test in BasicTestsSuite

[ https://issues.apache.org/jira/browse/SPARK-38022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38022. --- Fix Version/s: 3.3.0 3.2.1 Resolution: Fixed Issue resolved by pul

[jira] [Created] (SPARK-38025) Improve test suite ExternalCatalogSuite

Khalid Mammadov created SPARK-38025: --- Summary: Improve test suite ExternalCatalogSuite Key: SPARK-38025 URL: https://issues.apache.org/jira/browse/SPARK-38025 Project: Spark Issue Type: Imp

[jira] [Created] (SPARK-38024) add support for INFORMATION_SCHEMA or other catalog variant

Stephen Wilcoxon created SPARK-38024: Summary: add support for INFORMATION_SCHEMA or other catalog variant Key: SPARK-38024 URL: https://issues.apache.org/jira/browse/SPARK-38024 Project: Spark

[jira] [Commented] (SPARK-16452) basic INFORMATION_SCHEMA support

[ https://issues.apache.org/jira/browse/SPARK-16452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481935#comment-17481935 ] Stephen Wilcoxon commented on SPARK-16452: -- When will this be reexamined?  The

[jira] [Commented] (SPARK-38004) read_excel's parameter - mangle_dupe_cols is used to handle duplicate columns but fails if the duplicate columns are case sensitive.

[ https://issues.apache.org/jira/browse/SPARK-38004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481887#comment-17481887 ] Saikrishna Pujari commented on SPARK-38004: --- [~itholic] I suppose we are going

[jira] [Updated] (SPARK-38004) read_excel's parameter - mangle_dupe_cols is used to handle duplicate columns but fails if the duplicate columns are case sensitive.

[ https://issues.apache.org/jira/browse/SPARK-38004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saikrishna Pujari updated SPARK-38004: -- Description: mangle_dupe_cols - default is True So ideally it should have handled dupl

[jira] [Updated] (SPARK-38004) read_excel's parameter - mangle_dupe_cols is used to handle duplicate columns but fails if the duplicate columns are case sensitive.

[ https://issues.apache.org/jira/browse/SPARK-38004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saikrishna Pujari updated SPARK-38004: -- Issue Type: Documentation (was: Bug) > read_excel's parameter - mangle_dupe_cols is u

[jira] [Updated] (SPARK-38004) read_excel's parameter - mangle_dupe_cols is used to handle duplicate columns but fails if the duplicate columns are case sensitive.

[ https://issues.apache.org/jira/browse/SPARK-38004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saikrishna Pujari updated SPARK-38004: -- Priority: Minor (was: Major) > read_excel's parameter - mangle_dupe_cols is used to h

[jira] [Resolved] (SPARK-37479) Migrate DROP NAMESPACE to use V2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37479. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35202 [https://gith

[jira] [Assigned] (SPARK-37479) Migrate DROP NAMESPACE to use V2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37479: --- Assignee: dch nguyen > Migrate DROP NAMESPACE to use V2 command by default > --

[jira] [Updated] (SPARK-37999) Spark executor self-exiting due to driver disassociated in Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-37999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Petri updated SPARK-37999: -- Description: I have Spark driver running in a Kubernetes pod with client deploy-mode.I have created a headles

[jira] [Updated] (SPARK-37999) Spark executor self-exiting due to driver disassociated in Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-37999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Petri updated SPARK-37999: -- Description: I have Spark driver running in a Kubernetes pod with client deploy-mode.I have created a headles

[jira] [Assigned] (SPARK-38023) ExecutorMonitor.onExecutorRemoved should handle ExecutorDecommission as finished

[ https://issues.apache.org/jira/browse/SPARK-38023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38023: Assignee: (was: Apache Spark) > ExecutorMonitor.onExecutorRemoved should handle Execu

  1   2   >