[jira] [Created] (SPARK-43588) Upgrade ASM to 9.5

Yang Jie created SPARK-43588: Summary: Upgrade ASM to 9.5 Key: SPARK-43588 URL: https://issues.apache.org/jira/browse/SPARK-43588 Project: Spark Issue Type: Improvement Components:

[jira] [Updated] (SPARK-43588) Upgrade ASM to 9.5

[ https://issues.apache.org/jira/browse/SPARK-43588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43588: - Description: ASM 9.5 is for Java 21   https://asm.ow2.io/versions.html was:ASM 9.5 is for Java 21

[jira] [Resolved] (SPARK-43582) Upgrade `sbt-pom-reader` to 2.4.0

[ https://issues.apache.org/jira/browse/SPARK-43582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43582. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41224

[jira] [Assigned] (SPARK-43582) Upgrade `sbt-pom-reader` to 2.4.0

[ https://issues.apache.org/jira/browse/SPARK-43582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43582: - Assignee: BingKun Pan > Upgrade `sbt-pom-reader` to 2.4.0 >

[jira] [Updated] (SPARK-43587) Run HealthTrackerIntegrationSuite in a dedicate JVM

[ https://issues.apache.org/jira/browse/SPARK-43587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43587: -- Affects Version/s: 3.4.0 3.3.2 (was: 3.5.0)

[jira] [Updated] (SPARK-43586) There will be many invalid tasks when `Range.numSlices` > `Range.numElements`

[ https://issues.apache.org/jira/browse/SPARK-43586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43586: - Priority: Minor (was: Major) > There will be many invalid tasks when `Range.numSlices` >

[jira] [Updated] (SPARK-43586) There will be many invalid tasks when `Range.numSlices` > `Range.numElements`

[ https://issues.apache.org/jira/browse/SPARK-43586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43586: - Description: For example, start a spark shell with `--master "local[100]"`, then run

[jira] [Updated] (SPARK-43586) There will be many invalid tasks when `Range.numSlices` > `Range.numElements`

[ https://issues.apache.org/jira/browse/SPARK-43586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43586: - Attachment: image-2023-05-19-13-01-19-589.png > There will be many invalid tasks when `Range.numSlices`

[jira] [Resolved] (SPARK-43572) Add a test for scrollable result set through thrift server

[ https://issues.apache.org/jira/browse/SPARK-43572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43572. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41213

[jira] [Assigned] (SPARK-43572) Add a test for scrollable result set through thrift server

[ https://issues.apache.org/jira/browse/SPARK-43572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-43572: --- Assignee: Kent Yao > Add a test for scrollable result set through thrift server >

[jira] [Updated] (SPARK-43586) There will be many invalid tasks when `Range.numSlices` > `Range.numElements`

[ https://issues.apache.org/jira/browse/SPARK-43586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43586: - Summary: There will be many invalid tasks when `Range.numSlices` > `Range.numElements` (was: there

[jira] [Created] (SPARK-43586) there will be many invalid tasks when `Range.numSlices` > `Range.numElements`

Yang Jie created SPARK-43586: Summary: there will be many invalid tasks when `Range.numSlices` > `Range.numElements` Key: SPARK-43586 URL: https://issues.apache.org/jira/browse/SPARK-43586 Project: Spark

[jira] [Created] (SPARK-43587) Run HealthTrackerIntegrationSuite in a dedicate JVM

Dongjoon Hyun created SPARK-43587: - Summary: Run HealthTrackerIntegrationSuite in a dedicate JVM Key: SPARK-43587 URL: https://issues.apache.org/jira/browse/SPARK-43587 Project: Spark Issue

[jira] [Commented] (SPARK-43530) Protobuf: Read descriptor file only once at the compile time

[ https://issues.apache.org/jira/browse/SPARK-43530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724104#comment-17724104 ] Snoot.io commented on SPARK-43530: -- User 'rangadi' has created a pull request for this issue:

[jira] [Commented] (SPARK-43510) Spark application hangs when YarnAllocator adds running executors after processing completed containers

[ https://issues.apache.org/jira/browse/SPARK-43510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724103#comment-17724103 ] Snoot.io commented on SPARK-43510: -- User 'manuzhang' has created a pull request for this issue:

[jira] [Commented] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724099#comment-17724099 ] Yuming Wang commented on SPARK-43526: - [~caican] Thank you for investigation. > when shuffle hash

[jira] [Commented] (SPARK-43516) Basic estimator / transformer / model / evaluator interfaces

[ https://issues.apache.org/jira/browse/SPARK-43516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724098#comment-17724098 ] Snoot.io commented on SPARK-43516: -- User 'WeichenXu123' has created a pull request for this issue:

[jira] [Updated] (SPARK-43585) Spark Connect client cannot read from Hive metastore

[ https://issues.apache.org/jira/browse/SPARK-43585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] roland updated SPARK-43585: --- Description: I created a Spark Connect shell in a pod using the following yaml. {code:java} apiVersion: v1

[jira] [Updated] (SPARK-43585) Spark Connect client cannot read from Hive metastore

[ https://issues.apache.org/jira/browse/SPARK-43585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] roland updated SPARK-43585: --- Description: I created a Spark Connect shell in a pod using the following yaml. {code:java} apiVersion: v1

[jira] [Created] (SPARK-43585) Spark Connect client cannot read from Hive metastore

roland created SPARK-43585: -- Summary: Spark Connect client cannot read from Hive metastore Key: SPARK-43585 URL: https://issues.apache.org/jira/browse/SPARK-43585 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-43585) Spark Connect client cannot read from Hive metastore

[ https://issues.apache.org/jira/browse/SPARK-43585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] roland updated SPARK-43585: --- Description: I created a Spark Connect shell in a pod using the following yaml. {code:java} apiVersion: v1

[jira] [Commented] (SPARK-43583) When encryption is enabled on the External Shuffle Service, then processing of push meta requests throws NPE

[ https://issues.apache.org/jira/browse/SPARK-43583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724095#comment-17724095 ] Snoot.io commented on SPARK-43583: -- User 'otterc' has created a pull request for this issue:

[jira] [Commented] (SPARK-43572) Add a test for scrollable result set through thrift server

[ https://issues.apache.org/jira/browse/SPARK-43572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724094#comment-17724094 ] Snoot.io commented on SPARK-43572: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-43572) Add a test for scrollable result set through thrift server

[ https://issues.apache.org/jira/browse/SPARK-43572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724092#comment-17724092 ] Snoot.io commented on SPARK-43572: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-43519) Bump Parquet to 1.13.1

[ https://issues.apache.org/jira/browse/SPARK-43519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724091#comment-17724091 ] Snoot.io commented on SPARK-43519: -- User 'pan3793' has created a pull request for this issue:

[jira] [Resolved] (SPARK-43577) Upgrade cyclonedx-maven-plugin to 2.7.9

[ https://issues.apache.org/jira/browse/SPARK-43577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43577. - Resolution: Won't Fix > Upgrade cyclonedx-maven-plugin to 2.7.9 >

[jira] [Updated] (SPARK-43519) Bump Parquet to 1.13.1

[ https://issues.apache.org/jira/browse/SPARK-43519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43519: -- Summary: Bump Parquet to 1.13.1 (was: Bump Parquet 1.13.1) > Bump Parquet to 1.13.1 >

[jira] [Resolved] (SPARK-43519) Bump Parquet 1.13.1

[ https://issues.apache.org/jira/browse/SPARK-43519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43519. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41178

[jira] [Assigned] (SPARK-43519) Bump Parquet 1.13.1

[ https://issues.apache.org/jira/browse/SPARK-43519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43519: - Assignee: Cheng Pan > Bump Parquet 1.13.1 > --- > >

[jira] [Created] (SPARK-43584) Update some sbt plugins

BingKun Pan created SPARK-43584: --- Summary: Update some sbt plugins Key: SPARK-43584 URL: https://issues.apache.org/jira/browse/SPARK-43584 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Affects Version/s: 3.3.2 > when shuffle hash join is enabled, q95 performance deteriorates >

[jira] [Comment Edited] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724080#comment-17724080 ] caican edited comment on SPARK-43526 at 5/19/23 2:51 AM: - gently ping [~yumwang] 

[jira] [Comment Edited] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724080#comment-17724080 ] caican edited comment on SPARK-43526 at 5/19/23 2:49 AM: - I find that the

[jira] [Comment Edited] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724080#comment-17724080 ] caican edited comment on SPARK-43526 at 5/19/23 2:48 AM: - I find that the

[jira] [Created] (SPARK-43583) When encryption is enabled on the External Shuffle Service, then processing of push meta requests throws NPE

Chandni Singh created SPARK-43583: - Summary: When encryption is enabled on the External Shuffle Service, then processing of push meta requests throws NPE Key: SPARK-43583 URL:

[jira] [Comment Edited] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724080#comment-17724080 ] caican edited comment on SPARK-43526 at 5/19/23 2:47 AM: - I find that the

[jira] [Resolved] (SPARK-43581) Upgrade `kubernetes-client` to 6.6.2

[ https://issues.apache.org/jira/browse/SPARK-43581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43581. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41223

[jira] [Assigned] (SPARK-43581) Upgrade `kubernetes-client` to 6.6.2

[ https://issues.apache.org/jira/browse/SPARK-43581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43581: - Assignee: Dongjoon Hyun > Upgrade `kubernetes-client` to 6.6.2 >

[jira] [Commented] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724080#comment-17724080 ] caican commented on SPARK-43526: I find that the shuffle hash join is slower than the sort merge join

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: sort2.png > when shuffle hash join is enabled, q95 performance deteriorates >

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-19-10-43-51-747.png > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: shuffle1.png > when shuffle hash join is enabled, q95 performance deteriorates >

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: sort1.png > when shuffle hash join is enabled, q95 performance deteriorates >

[jira] [Created] (SPARK-43582) Upgrade `sbt-pom-reader` to 2.4.0

BingKun Pan created SPARK-43582: --- Summary: Upgrade `sbt-pom-reader` to 2.4.0 Key: SPARK-43582 URL: https://issues.apache.org/jira/browse/SPARK-43582 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-43521) Support CREATE TABLE LIKE FILE for PARQUET

[ https://issues.apache.org/jira/browse/SPARK-43521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723900#comment-17723900 ] Jia Fan edited comment on SPARK-43521 at 5/19/23 1:31 AM: -- I'm working on this!

[jira] [Created] (SPARK-43581) Upgrade `kubernetes-client` to 6.6.2

Dongjoon Hyun created SPARK-43581: - Summary: Upgrade `kubernetes-client` to 6.6.2 Key: SPARK-43581 URL: https://issues.apache.org/jira/browse/SPARK-43581 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-43541) Incorrect column resolution on FULL OUTER JOIN with USING

[ https://issues.apache.org/jira/browse/SPARK-43541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43541: -- Fix Version/s: 3.3.3 > Incorrect column resolution on FULL OUTER JOIN with USING >

[jira] [Commented] (SPARK-43541) Incorrect column resolution on FULL OUTER JOIN with USING

[ https://issues.apache.org/jira/browse/SPARK-43541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724062#comment-17724062 ] Dongjoon Hyun commented on SPARK-43541: --- This is backported to branch-3.3 via

[jira] [Resolved] (SPARK-43580) Add https://dlcdn.apache.org/ to default_sites of get_preferred_mirrors

[ https://issues.apache.org/jira/browse/SPARK-43580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43580. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41222

[jira] [Assigned] (SPARK-43580) Add https://dlcdn.apache.org/ to default_sites of get_preferred_mirrors

[ https://issues.apache.org/jira/browse/SPARK-43580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43580: - Assignee: Dongjoon Hyun > Add https://dlcdn.apache.org/ to default_sites of

[jira] [Updated] (SPARK-43541) Incorrect column resolution on FULL OUTER JOIN with USING

[ https://issues.apache.org/jira/browse/SPARK-43541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43541: -- Fix Version/s: 3.4.1 > Incorrect column resolution on FULL OUTER JOIN with USING >

[jira] [Commented] (SPARK-43541) Incorrect column resolution on FULL OUTER JOIN with USING

[ https://issues.apache.org/jira/browse/SPARK-43541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724047#comment-17724047 ] Dongjoon Hyun commented on SPARK-43541: --- This is backported to branch-3.4 via

[jira] [Created] (SPARK-43580) Add https://dlcdn.apache.org/ to default_sites of get_preferred_mirrors

Dongjoon Hyun created SPARK-43580: - Summary: Add https://dlcdn.apache.org/ to default_sites of get_preferred_mirrors Key: SPARK-43580 URL: https://issues.apache.org/jira/browse/SPARK-43580 Project:

[jira] [Resolved] (SPARK-43574) Support to set Python executable in workers during runtime

[ https://issues.apache.org/jira/browse/SPARK-43574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43574. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41215

[jira] [Assigned] (SPARK-43574) Support to set Python executable in workers during runtime

[ https://issues.apache.org/jira/browse/SPARK-43574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43574: - Assignee: Hyukjin Kwon > Support to set Python executable in workers during runtime >

[jira] [Assigned] (SPARK-43575) Exclude duplicated classes from kafka assembly jar

[ https://issues.apache.org/jira/browse/SPARK-43575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43575: - Assignee: Cheng Pan > Exclude duplicated classes from kafka assembly jar >

[jira] [Resolved] (SPARK-43575) Exclude duplicated classes from kafka assembly jar

[ https://issues.apache.org/jira/browse/SPARK-43575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43575. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41217

[jira] [Commented] (SPARK-43358) Cover the resolution of insertion with the SupportsCustomSchemaWrite interface

[ https://issues.apache.org/jira/browse/SPARK-43358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723999#comment-17723999 ] Dongjoon Hyun commented on SPARK-43358: --- I converted this from a subtask of SPARK-38334 to a new

[jira] [Updated] (SPARK-43358) Cover the resolution of insertion with the SupportsCustomSchemaWrite interface

[ https://issues.apache.org/jira/browse/SPARK-43358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43358: -- Issue Type: Improvement (was: Bug) > Cover the resolution of insertion with the

[jira] [Commented] (SPARK-43313) Adding missing default values for MERGE INSERT actions

[ https://issues.apache.org/jira/browse/SPARK-43313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723998#comment-17723998 ] Dongjoon Hyun commented on SPARK-43313: --- To [~dtenedor] , SPARK-38334 was resolved with Fixed

[jira] [Updated] (SPARK-43358) Cover the resolution of insertion with the SupportsCustomSchemaWrite interface

[ https://issues.apache.org/jira/browse/SPARK-43358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43358: -- Parent: (was: SPARK-38334) Issue Type: Bug (was: Sub-task) > Cover the

[jira] [Updated] (SPARK-43313) Adding missing default values for MERGE INSERT actions

[ https://issues.apache.org/jira/browse/SPARK-43313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43313: -- Parent: (was: SPARK-38334) Issue Type: Bug (was: Sub-task) > Adding missing

[jira] [Updated] (SPARK-43313) Adding missing default values for MERGE INSERT actions

[ https://issues.apache.org/jira/browse/SPARK-43313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43313: -- Affects Version/s: 3.4.0 (was: 3.5.0) > Adding missing default

[jira] [Commented] (SPARK-43313) Adding missing default values for MERGE INSERT actions

[ https://issues.apache.org/jira/browse/SPARK-43313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723997#comment-17723997 ] Dongjoon Hyun commented on SPARK-43313: --- This is reverted from branch-3.4 via

[jira] [Updated] (SPARK-43313) Adding missing default values for MERGE INSERT actions

[ https://issues.apache.org/jira/browse/SPARK-43313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43313: -- Fix Version/s: (was: 3.4.1) > Adding missing default values for MERGE INSERT actions >

[jira] [Updated] (SPARK-43313) Adding missing default values for MERGE INSERT actions

[ https://issues.apache.org/jira/browse/SPARK-43313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43313: -- Affects Version/s: 3.5.0 (was: 3.4.0) > Adding missing default

[jira] [Resolved] (SPARK-43541) Incorrect column resolution on FULL OUTER JOIN with USING

[ https://issues.apache.org/jira/browse/SPARK-43541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-43541. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41204

[jira] [Created] (SPARK-43579) Cache the converter between Arrow and pandas for reuse

Xinrong Meng created SPARK-43579: Summary: Cache the converter between Arrow and pandas for reuse Key: SPARK-43579 URL: https://issues.apache.org/jira/browse/SPARK-43579 Project: Spark Issue

[jira] [Commented] (SPARK-43136) Scala mapGroup, coGroup

[ https://issues.apache.org/jira/browse/SPARK-43136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723976#comment-17723976 ] Hudson commented on SPARK-43136: User 'zhenlineo' has created a pull request for this issue:

[jira] [Commented] (SPARK-43511) Implemented State APIs for Spark Connect Scala

[ https://issues.apache.org/jira/browse/SPARK-43511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723974#comment-17723974 ] Hudson commented on SPARK-43511: User 'bogao007' has created a pull request for this issue:

[jira] [Commented] (SPARK-43317) Support combine adjacent aggregation

[ https://issues.apache.org/jira/browse/SPARK-43317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723977#comment-17723977 ] Hudson commented on SPARK-43317: User 'ulysses-you' has created a pull request for this issue:

[jira] [Commented] (SPARK-43267) Support creating data frame from a Postgres table that contains user-defined array column

[ https://issues.apache.org/jira/browse/SPARK-43267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723972#comment-17723972 ] Hudson commented on SPARK-43267: User 'Hisoka-X' has created a pull request for this issue:

[jira] [Commented] (SPARK-39420) Support ANALYZE TABLE on v2 tables

[ https://issues.apache.org/jira/browse/SPARK-39420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723975#comment-17723975 ] Hudson commented on SPARK-39420: User 'Hisoka-X' has created a pull request for this issue:

[jira] [Commented] (SPARK-43266) Move MergeScalarSubqueries to spark-sql

[ https://issues.apache.org/jira/browse/SPARK-43266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723973#comment-17723973 ] Hudson commented on SPARK-43266: User 'peter-toth' has created a pull request for this issue:

[jira] [Commented] (SPARK-43180) Upgrade mypy package in python pyspark dependencies

[ https://issues.apache.org/jira/browse/SPARK-43180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723971#comment-17723971 ] Hudson commented on SPARK-43180: User 'aimtsou' has created a pull request for this issue:

[jira] [Commented] (SPARK-43288) DataSourceV2: CREATE TABLE LIKE

[ https://issues.apache.org/jira/browse/SPARK-43288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723970#comment-17723970 ] Hudson commented on SPARK-43288: User 'jzhuge' has created a pull request for this issue:

[jira] [Created] (SPARK-43578) Protobuf : Use broadcast to distribute descriptor file content

Raghu Angadi created SPARK-43578: Summary: Protobuf : Use broadcast to distribute descriptor file content Key: SPARK-43578 URL: https://issues.apache.org/jira/browse/SPARK-43578 Project: Spark

[jira] [Commented] (SPARK-43577) Upgrade cyclonedx-maven-plugin to 2.7.9

[ https://issues.apache.org/jira/browse/SPARK-43577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723957#comment-17723957 ] Yuming Wang commented on SPARK-43577: - https://github.com/apache/spark/pull/41219 > Upgrade

[jira] [Created] (SPARK-43577) Upgrade cyclonedx-maven-plugin to 2.7.9

Yuming Wang created SPARK-43577: --- Summary: Upgrade cyclonedx-maven-plugin to 2.7.9 Key: SPARK-43577 URL: https://issues.apache.org/jira/browse/SPARK-43577 Project: Spark Issue Type:

[jira] [Created] (SPARK-43576) Remove unused declarations from Core module

BingKun Pan created SPARK-43576: --- Summary: Remove unused declarations from Core module Key: SPARK-43576 URL: https://issues.apache.org/jira/browse/SPARK-43576 Project: Spark Issue Type:

[jira] [Commented] (SPARK-43521) Support CREATE TABLE LIKE FILE for PARQUET

[ https://issues.apache.org/jira/browse/SPARK-43521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723900#comment-17723900 ] Jia Fan commented on SPARK-43521: - I'm working for this! > Support CREATE TABLE LIKE FILE for PARQUET >

[jira] [Created] (SPARK-43575) Exclude duplicated classes from kafka assembly jar

Cheng Pan created SPARK-43575: - Summary: Exclude duplicated classes from kafka assembly jar Key: SPARK-43575 URL: https://issues.apache.org/jira/browse/SPARK-43575 Project: Spark Issue Type:

[jira] [Updated] (SPARK-43549) Convert `_LEGACY_ERROR_TEMP_0036` to INVALID_SQL_SYNTAX

[ https://issues.apache.org/jira/browse/SPARK-43549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-43549: Summary: Convert `_LEGACY_ERROR_TEMP_0036` to INVALID_SQL_SYNTAX (was: Assign a name to the

[jira] [Created] (SPARK-43574) Support to set Python executable in workers during runtime

Hyukjin Kwon created SPARK-43574: Summary: Support to set Python executable in workers during runtime Key: SPARK-43574 URL: https://issues.apache.org/jira/browse/SPARK-43574 Project: Spark

[jira] [Updated] (SPARK-43573) Make SparkBuilder could config the heap size of test JVM.

[ https://issues.apache.org/jira/browse/SPARK-43573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-43573: --- Description: build/sbt "sql/Test/runMain --dsdgenDir --location --scaleFactor 1" causes OOM, if

[jira] [Updated] (SPARK-43573) Make SparkBuilder could config the heap size of test JVM.

[ https://issues.apache.org/jira/browse/SPARK-43573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-43573: --- Description: {code:java} build/sbt "sql/Test/runMain --dsdgenDir --location --scaleFactor 1"

[jira] [Commented] (SPARK-43573) Make SparkBuilder could config the heap size of test JVM.

[ https://issues.apache.org/jira/browse/SPARK-43573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723861#comment-17723861 ] ASF GitHub Bot commented on SPARK-43573: User 'beliefer' has created a pull request for this

[jira] [Updated] (SPARK-43573) Make SparkBuilder could config the heap size of test JVM.

[ https://issues.apache.org/jira/browse/SPARK-43573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-43573: --- Description: {code:java} [info] 16:43:41.618 ERROR

[jira] [Updated] (SPARK-43549) Assign a name to the error class _LEGACY_ERROR_TEMP_0036

[ https://issues.apache.org/jira/browse/SPARK-43549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-43549: Summary: Assign a name to the error class _LEGACY_ERROR_TEMP_0036 (was: Assign a name to the

[jira] [Reopened] (SPARK-43549) Assign a name to the error class _LEGACY_ERROR_TEMP_0035

[ https://issues.apache.org/jira/browse/SPARK-43549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan reopened SPARK-43549: - > Assign a name to the error class _LEGACY_ERROR_TEMP_0035 >

[jira] [Created] (SPARK-43573) Make SparkBuilder could config the heap size of test JVM.

jiaan.geng created SPARK-43573: -- Summary: Make SparkBuilder could config the heap size of test JVM. Key: SPARK-43573 URL: https://issues.apache.org/jira/browse/SPARK-43573 Project: Spark Issue

[jira] [Comment Edited] (SPARK-40964) Cannot run spark history server with shaded hadoop jar

[ https://issues.apache.org/jira/browse/SPARK-40964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723777#comment-17723777 ] Shuaipeng Lee edited comment on SPARK-40964 at 5/18/23 8:06 AM: Thanks

[jira] [Created] (SPARK-43572) Add a test for scrollable result set through thrift server

Kent Yao created SPARK-43572: Summary: Add a test for scrollable result set through thrift server Key: SPARK-43572 URL: https://issues.apache.org/jira/browse/SPARK-43572 Project: Spark Issue

[jira] [Resolved] (SPARK-43549) Assign a name to the error class _LEGACY_ERROR_TEMP_0035

[ https://issues.apache.org/jira/browse/SPARK-43549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan resolved SPARK-43549. - Resolution: Duplicate > Assign a name to the error class _LEGACY_ERROR_TEMP_0035 >

[jira] [Resolved] (SPARK-43569) Remove workaround for HADOOP-14067

[ https://issues.apache.org/jira/browse/SPARK-43569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43569. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41210

[jira] [Assigned] (SPARK-43569) Remove workaround for HADOOP-14067

[ https://issues.apache.org/jira/browse/SPARK-43569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43569: - Assignee: BingKun Pan > Remove workaround for HADOOP-14067 >

[jira] [Assigned] (SPARK-43450) Add more `_metadata` filter test cases

[ https://issues.apache.org/jira/browse/SPARK-43450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43450: - Assignee: Jan-Ole Sasse > Add more `_metadata` filter test cases >

[jira] [Resolved] (SPARK-43450) Add more `_metadata` filter test cases

[ https://issues.apache.org/jira/browse/SPARK-43450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43450. --- Fix Version/s: 3.5.0 3.4.1 Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-43450) Add more `_metadata` filter test cases

[ https://issues.apache.org/jira/browse/SPARK-43450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43450: -- Summary: Add more `_metadata` filter test cases (was: Additional tests for _metadata

[jira] [Updated] (SPARK-43491) In expression not compatible with EqualTo Expression

[ https://issues.apache.org/jira/browse/SPARK-43491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KuijianLiu updated SPARK-43491: --- Description: The query results of Spark SQL 3.1.1  and Hive SQL 3.1.0 are inconsistent with same

  1   2   >