[jira] [Commented] (SPARK-48238) Spark fail to start due to class o.a.h.yarn.server.webproxy.amfilter.AmIpFilter is not a jakarta.servlet.Filter

2024-05-12 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845781#comment-17845781 ] Cheng Pan commented on SPARK-48238: --- [~HF] javax => jakarta migration would be complex on the Hadoop

[jira] [Updated] (SPARK-48238) Spark fail to start due to class o.a.h.yarn.server.webproxy.amfilter.AmIpFilter is not a jakarta.servlet.Filter

2024-05-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48238: - Parent: SPARK-47970 Issue Type: Sub-task (was: Bug) > Spark fail to start due to class

[jira] [Updated] (SPARK-48250) Enable array inference tests at test_parity_types.py

2024-05-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48250: --- Labels: pull-request-available (was: ) > Enable array inference tests at

[jira] [Updated] (SPARK-48250) Enable array inference tests at test_parity_types.py

2024-05-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48250: - Issue Type: Test (was: Bug) > Enable array inference tests at test_parity_types.py >

[jira] [Updated] (SPARK-48250) Enable array inference tests at test_parity_types.py

2024-05-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48250: - Priority: Minor (was: Major) > Enable array inference tests at test_parity_types.py >

[jira] [Created] (SPARK-48250) Enable array inference tests at test_parity_types.py

2024-05-12 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48250: Summary: Enable array inference tests at test_parity_types.py Key: SPARK-48250 URL: https://issues.apache.org/jira/browse/SPARK-48250 Project: Spark Issue

[jira] [Deleted] (SPARK-48249) Use non-null value for legacy conf of inferArrayTypeFromFirstElement

2024-05-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon deleted SPARK-48249: - > Use non-null value for legacy conf of inferArrayTypeFromFirstElement >

[jira] [Updated] (SPARK-48249) Use non-null value for legacy conf of inferArrayTypeFromFirstElement

2024-05-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48249: - Description: {code} >>>

[jira] [Created] (SPARK-48249) Use non-null value for legacy conf of inferArrayTypeFromFirstElement

2024-05-12 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48249: Summary: Use non-null value for legacy conf of inferArrayTypeFromFirstElement Key: SPARK-48249 URL: https://issues.apache.org/jira/browse/SPARK-48249 Project: Spark

[jira] [Updated] (SPARK-48248) Fix nested array to respect legacy conf of inferArrayTypeFromFirstElement

2024-05-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48248: --- Labels: pull-request-available (was: ) > Fix nested array to respect legacy conf of

[jira] [Created] (SPARK-48248) Fix nested array to respect legacy conf of inferArrayTypeFromFirstElement

2024-05-12 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48248: Summary: Fix nested array to respect legacy conf of inferArrayTypeFromFirstElement Key: SPARK-48248 URL: https://issues.apache.org/jira/browse/SPARK-48248 Project:

[jira] [Updated] (SPARK-48247) Use all values in a python dict when inferring MapType schema

2024-05-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48247: --- Labels: pull-request-available (was: ) > Use all values in a python dict when inferring

[jira] [Created] (SPARK-48247) Use all values in a python dict when inferring MapType schema

2024-05-12 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48247: Summary: Use all values in a python dict when inferring MapType schema Key: SPARK-48247 URL: https://issues.apache.org/jira/browse/SPARK-48247 Project: Spark

[jira] [Updated] (SPARK-48246) Add a new ScalaStyle Rule `RedundantIfChecker`

2024-05-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48246: --- Labels: pull-request-available (was: ) > Add a new ScalaStyle Rule `RedundantIfChecker` >

[jira] [Updated] (SPARK-48244) Specify outputPartitioning of Full Outer Join to Avoid Shuffle

2024-05-12 Thread GANHONGNAN (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GANHONGNAN updated SPARK-48244: --- Description: h1. Demo SQL {code:java} select * from ( select coalesce(t1.a, t2.a) as

[jira] [Created] (SPARK-48246) Add a new ScalaStyle Rule `RedundantIfChecker`

2024-05-12 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-48246: --- Summary: Add a new ScalaStyle Rule `RedundantIfChecker` Key: SPARK-48246 URL: https://issues.apache.org/jira/browse/SPARK-48246 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-48245) Typo in `BadRecordException` class doc

2024-05-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48245. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46542

[jira] [Assigned] (SPARK-48245) Typo in `BadRecordException` class doc

2024-05-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48245: Assignee: Vladimir Golubev > Typo in `BadRecordException` class doc >

[jira] [Commented] (SPARK-44265) Built-in XML data source support

2024-05-12 Thread HiuFung Kwok (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845755#comment-17845755 ] HiuFung Kwok commented on SPARK-44265: -- Thanks! Let me have a look. > Built-in XML data source

[jira] [Resolved] (SPARK-48239) Update the release docker image to follow what we use in Github Action jobs

2024-05-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48239. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46534

[jira] [Commented] (SPARK-44265) Built-in XML data source support

2024-05-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845752#comment-17845752 ] Hyukjin Kwon commented on SPARK-44265: -- https://issues.apache.org/jira/browse/SPARK-45190 is not

[jira] [Commented] (SPARK-48238) Spark fail to start due to class o.a.h.yarn.server.webproxy.amfilter.AmIpFilter is not a jakarta.servlet.Filter

2024-05-12 Thread HiuFung Kwok (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845750#comment-17845750 ] HiuFung Kwok commented on SPARK-48238: -- I will open a few tickets on the Hadoop side to follow up

[jira] [Commented] (SPARK-48238) Spark fail to start due to class o.a.h.yarn.server.webproxy.amfilter.AmIpFilter is not a jakarta.servlet.Filter

2024-05-12 Thread HiuFung Kwok (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845749#comment-17845749 ] HiuFung Kwok commented on SPARK-48238: -- [~chengpan]  It's sad but you are right, we need to revert

[jira] [Commented] (SPARK-48238) Spark fail to start due to class o.a.h.yarn.server.webproxy.amfilter.AmIpFilter is not a jakarta.servlet.Filter

2024-05-12 Thread HiuFung Kwok (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845748#comment-17845748 ] HiuFung Kwok commented on SPARK-48238: -- Let me have a look. > Spark fail to start due to class >

[jira] [Comment Edited] (SPARK-47008) Spark to support S3 Express One Zone Storage

2024-05-12 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844968#comment-17844968 ] Leo Timofeyev edited comment on SPARK-47008 at 5/12/24 9:11 PM: Hey

[jira] [Comment Edited] (SPARK-47008) Spark to support S3 Express One Zone Storage

2024-05-12 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844968#comment-17844968 ] Leo Timofeyev edited comment on SPARK-47008 at 5/12/24 9:09 PM: Hey

[jira] [Comment Edited] (SPARK-47008) Spark to support S3 Express One Zone Storage

2024-05-12 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844968#comment-17844968 ] Leo Timofeyev edited comment on SPARK-47008 at 5/12/24 9:08 PM: Hey

[jira] [Comment Edited] (SPARK-47008) Spark to support S3 Express One Zone Storage

2024-05-12 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844968#comment-17844968 ] Leo Timofeyev edited comment on SPARK-47008 at 5/12/24 9:08 PM: Hey

[jira] [Comment Edited] (SPARK-47008) Spark to support S3 Express One Zone Storage

2024-05-12 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844968#comment-17844968 ] Leo Timofeyev edited comment on SPARK-47008 at 5/12/24 9:07 PM: Hey

[jira] [Comment Edited] (SPARK-47008) Spark to support S3 Express One Zone Storage

2024-05-12 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844968#comment-17844968 ] Leo Timofeyev edited comment on SPARK-47008 at 5/12/24 9:07 PM: Hey

[jira] [Comment Edited] (SPARK-47008) Spark to support S3 Express One Zone Storage

2024-05-12 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844968#comment-17844968 ] Leo Timofeyev edited comment on SPARK-47008 at 5/12/24 9:06 PM: Hey

[jira] [Updated] (SPARK-47429) Rename errorClass to errorCondition and subClass to subCondition

2024-05-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47429: --- Labels: pull-request-available (was: ) > Rename errorClass to errorCondition and subClass

[jira] [Updated] (SPARK-48245) Typo in `BadRecordException` class doc

2024-05-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48245: --- Labels: pull-request-available (was: ) > Typo in `BadRecordException` class doc >

[jira] [Updated] (SPARK-48245) Typo in `BadRecordException` class doc

2024-05-12 Thread Vladimir Golubev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Golubev updated SPARK-48245: - Summary: Typo in `BadRecordException` class doc (was: Typo in `BadRecordException`) >

[jira] [Created] (SPARK-48245) Typo in `BadRecordException`

2024-05-12 Thread Vladimir Golubev (Jira)
Vladimir Golubev created SPARK-48245: Summary: Typo in `BadRecordException` Key: SPARK-48245 URL: https://issues.apache.org/jira/browse/SPARK-48245 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-44789) XML: Spark connect support

2024-05-12 Thread Sandip Agarwala (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandip Agarwala resolved SPARK-44789. - Resolution: Fixed > XML: Spark connect support > -- > >

[jira] [Resolved] (SPARK-46108) XML: keepInnerXmlAsRaw option

2024-05-12 Thread Sandip Agarwala (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandip Agarwala resolved SPARK-46108. - Resolution: Not A Problem As discussed in the PR (#44022) comments , this is already

[jira] [Resolved] (SPARK-44810) XML: ArrayType and MapType support in from_xml

2024-05-12 Thread Sandip Agarwala (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandip Agarwala resolved SPARK-44810. - Resolution: Fixed > XML: ArrayType and MapType support in from_xml >

[jira] [Updated] (SPARK-48244) Specify outputPartitioning of Full Outer Join to Avoid Shuffle

2024-05-12 Thread GANHONGNAN (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GANHONGNAN updated SPARK-48244: --- Summary: Specify outputPartitioning of Full Outer Join to Avoid Shuffle (was: [improvement]

[jira] [Updated] (SPARK-48244) [improvement] Specify outputPartitioning of Full Outer Join to Avoid Shuffle

2024-05-12 Thread GANHONGNAN (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GANHONGNAN updated SPARK-48244: --- Summary: [improvement] Specify outputPartitioning of Full Outer Join to Avoid Shuffle (was:

[jira] [Updated] (SPARK-48244) [improvement]Specify outputPartitioning of Full Outer Join

2024-05-12 Thread GANHONGNAN (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GANHONGNAN updated SPARK-48244: --- Description: h1. Demo SQL {code:java} select * from ( select coalesce(t1.a, t2.a) as

[jira] [Updated] (SPARK-48244) [improvement]Specify outputPartitioning of Full Outer Join

2024-05-12 Thread GANHONGNAN (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GANHONGNAN updated SPARK-48244: --- Description: h1. Demo SQL {code:java} select * from ( select coalesce(t1.a, t2.a) as

[jira] [Created] (SPARK-48244) [improvement] Keep partitioning of Full Outer Join

2024-05-12 Thread GANHONGNAN (Jira)
GANHONGNAN created SPARK-48244: -- Summary: [improvement] Keep partitioning of Full Outer Join Key: SPARK-48244 URL: https://issues.apache.org/jira/browse/SPARK-48244 Project: Spark Issue Type:

[jira] [Updated] (SPARK-48217) Spark stdout and stderr getting removed at end of spark job triggered from cloudera hue workflow

2024-05-12 Thread Noam Shemesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noam Shemesh updated SPARK-48217: - Priority: Critical (was: Major) > Spark stdout and stderr getting removed at end of spark job