[jira] [Created] (SPARK-46211) Directly writing the parquet table location will cause the query to be null

ming95 created SPARK-46211: -- Summary: Directly writing the parquet table location will cause the query to be null Key: SPARK-46211 URL: https://issues.apache.org/jira/browse/SPARK-46211 Project:

[jira] [Updated] (SPARK-40609) Casts types according to bucket info for Equality expression

[ https://issues.apache.org/jira/browse/SPARK-40609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-40609: --- Labels: pull-request-available (was: ) > Casts types according to bucket info for Equality

[jira] [Updated] (SPARK-46207) Support MergeInto in DataFrameWriterV2

[ https://issues.apache.org/jira/browse/SPARK-46207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46207: --- Labels: pull-request-available (was: ) > Support MergeInto in DataFrameWriterV2 >

[jira] [Assigned] (SPARK-46210) Update YuniKorn docs with v1.4

[ https://issues.apache.org/jira/browse/SPARK-46210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46210: - Assignee: Dongjoon Hyun > Update YuniKorn docs with v1.4 >

[jira] [Updated] (SPARK-46210) Update YuniKorn docs with v1.4

[ https://issues.apache.org/jira/browse/SPARK-46210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46210: --- Labels: pull-request-available (was: ) > Update YuniKorn docs with v1.4 >

[jira] [Created] (SPARK-46210) Update YuniKorn docs with v1.4

Dongjoon Hyun created SPARK-46210: - Summary: Update YuniKorn docs with v1.4 Key: SPARK-46210 URL: https://issues.apache.org/jira/browse/SPARK-46210 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-46205) Improve `PersistenceEngine` performance with `KryoSerializer`

[ https://issues.apache.org/jira/browse/SPARK-46205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46205. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44113

[jira] [Updated] (SPARK-46208) Use specific Pandas version for API specifications

[ https://issues.apache.org/jira/browse/SPARK-46208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46208: --- Labels: pull-request-available (was: ) > Use specific Pandas version for API

[jira] [Updated] (SPARK-46208) Use specific Pandas version for API specifications

[ https://issues.apache.org/jira/browse/SPARK-46208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-46208: Summary: Use specific Pandas version for API specifications (was: Use specific Pandas version

[jira] [Created] (SPARK-46209) Add docker java 11 only publish workflow for version before v3.5

Yikun Jiang created SPARK-46209: --- Summary: Add docker java 11 only publish workflow for version before v3.5 Key: SPARK-46209 URL: https://issues.apache.org/jira/browse/SPARK-46209 Project: Spark

[jira] [Updated] (SPARK-46209) Add docker java 11 only publish workflow for version before v3.5

[ https://issues.apache.org/jira/browse/SPARK-46209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46209: --- Labels: pull-request-available (was: ) > Add docker java 11 only publish workflow for

[jira] [Created] (SPARK-46208) Use specific Pandas version for API specifictations

Haejoon Lee created SPARK-46208: --- Summary: Use specific Pandas version for API specifictations Key: SPARK-46208 URL: https://issues.apache.org/jira/browse/SPARK-46208 Project: Spark Issue

[jira] [Created] (SPARK-46207) Support MergeInto in DataFrameWriterV2

Huaxin Gao created SPARK-46207: -- Summary: Support MergeInto in DataFrameWriterV2 Key: SPARK-46207 URL: https://issues.apache.org/jira/browse/SPARK-46207 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-46206) Use a narrower scope exception for SQL processor

[ https://issues.apache.org/jira/browse/SPARK-46206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46206: --- Labels: pull-request-available (was: ) > Use a narrower scope exception for SQL processor

[jira] [Created] (SPARK-46206) Use a narrower scope exception for SQL processor

Haejoon Lee created SPARK-46206: --- Summary: Use a narrower scope exception for SQL processor Key: SPARK-46206 URL: https://issues.apache.org/jira/browse/SPARK-46206 Project: Spark Issue Type:

[jira] [Updated] (SPARK-43536) Statsd sink reporter reports incorrect counter metrics.

[ https://issues.apache.org/jira/browse/SPARK-43536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43536: --- Labels: pull-request-available (was: ) > Statsd sink reporter reports incorrect counter

[jira] [Updated] (SPARK-39911) Optimize global Sort to RepartitionByExpression

[ https://issues.apache.org/jira/browse/SPARK-39911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-39911: --- Fix Version/s: 3.3.1 > Optimize global Sort to RepartitionByExpression >

[jira] [Updated] (SPARK-39911) Optimize global Sort to RepartitionByExpression

[ https://issues.apache.org/jira/browse/SPARK-39911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-39911: --- Labels: pull-request-available (was: ) > Optimize global Sort to RepartitionByExpression >

[jira] [Resolved] (SPARK-46199) Add PyPi link icon to PySpark doc header

[ https://issues.apache.org/jira/browse/SPARK-46199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46199. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44106

[jira] [Assigned] (SPARK-46199) Add PyPi link icon to PySpark doc header

[ https://issues.apache.org/jira/browse/SPARK-46199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46199: - Assignee: BingKun Pan > Add PyPi link icon to PySpark doc header >

[jira] [Assigned] (SPARK-46201) Correct the typing of schema_of_{csv, json, xml}

[ https://issues.apache.org/jira/browse/SPARK-46201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46201: - Assignee: Ruifeng Zheng > Correct the typing of schema_of_{csv, json, xml} >

[jira] [Resolved] (SPARK-46201) Correct the typing of schema_of_{csv, json, xml}

[ https://issues.apache.org/jira/browse/SPARK-46201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46201. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44108

[jira] [Resolved] (SPARK-46200) Re-org the testing dockerfile

[ https://issues.apache.org/jira/browse/SPARK-46200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46200. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44107

[jira] [Assigned] (SPARK-46200) Re-org the testing dockerfile

[ https://issues.apache.org/jira/browse/SPARK-46200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46200: - Assignee: Ruifeng Zheng > Re-org the testing dockerfile >

[jira] [Assigned] (SPARK-46204) Fix `dev-run-integration-tests.sh` to use 17 for `JAVA_VERSION`

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46204: - Assignee: Hannah Amundson > Fix `dev-run-integration-tests.sh` to use 17 for

[jira] [Resolved] (SPARK-46204) Fix `dev-run-integration-tests.sh` to use 17 for `JAVA_VERSION`

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46204. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44112

[jira] [Updated] (SPARK-46204) Fix `dev-run-integration-tests.sh` to use 17 for `JAVA_VERSION`

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46204: -- Summary: Fix `dev-run-integration-tests.sh` to use 17 for `JAVA_VERSION` (was: Kubernetes

[jira] [Commented] (SPARK-46204) Kubernetes dev-run-integration-tests is tagged to the wrong version

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17792182#comment-17792182 ] Dongjoon Hyun commented on SPARK-46204: --- Thank you, [~hannahkamundson]. I collected this as a

[jira] [Updated] (SPARK-46204) Kubernetes dev-run-integration-tests is tagged to the wrong version

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46204: -- Parent: SPARK-45315 Issue Type: Sub-task (was: Bug) > Kubernetes

[jira] [Updated] (SPARK-46204) Kubernetes dev-run-integration-tests is tagged to the wrong version

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46204: -- Affects Version/s: 4.0.0 (was: 3.5.0) > Kubernetes

[jira] [Updated] (SPARK-46204) Kubernetes dev-run-integration-tests is tagged to the wrong version

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46204: --- Labels: pull-request-available (was: ) > Kubernetes dev-run-integration-tests is tagged to

[jira] [Commented] (SPARK-46204) Kubernetes dev-run-integration-tests is tagged to the wrong version

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17792176#comment-17792176 ] Hannah Amundson commented on SPARK-46204: - I'm working on this > Kubernetes

[jira] [Updated] (SPARK-46204) Kubernetes dev-run-integration-tests is tagged to the wrong version

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hannah Amundson updated SPARK-46204: Description: The file dev-run-integration-tests.sh

[jira] [Updated] (SPARK-46204) Kubernetes dev-run-integration-tests is tagged to the wrong version

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hannah Amundson updated SPARK-46204: Description: The file dev-run-integration-tests.sh

[jira] [Updated] (SPARK-46204) Kubernetes dev-run-integration-tests is tagged to the wrong version

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hannah Amundson updated SPARK-46204: Description: The file dev-run-integration-tests.sh

[jira] [Created] (SPARK-46204) Kubernetes dev-run-integration-tests is tagged to the wrong version

Hannah Amundson created SPARK-46204: --- Summary: Kubernetes dev-run-integration-tests is tagged to the wrong version Key: SPARK-46204 URL: https://issues.apache.org/jira/browse/SPARK-46204 Project:

[jira] [Updated] (SPARK-46204) Kubernetes dev-run-integration-tests is tagged to the wrong version

[ https://issues.apache.org/jira/browse/SPARK-46204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hannah Amundson updated SPARK-46204: Attachment: Screenshot from 2023-12-01 13-14-47.png > Kubernetes

[jira] [Resolved] (SPARK-45314) Drop Scala 2.12 and make Scala 2.13 by default

[ https://issues.apache.org/jira/browse/SPARK-45314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45314. --- Fix Version/s: 4.0.0 Resolution: Fixed Hi, all. I marked this as `Resolved`.

[jira] [Updated] (SPARK-46203) Support get partitions by names in partition filter

[ https://issues.apache.org/jira/browse/SPARK-46203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46203: --- Labels: pull-request-available (was: ) > Support get partitions by names in partition

[jira] [Updated] (SPARK-46198) Unexpected Shuffle Read when using cached DataFrame

[ https://issues.apache.org/jira/browse/SPARK-46198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitaliy Savkin updated SPARK-46198: --- Description: When a computation is based on a cached data frame, I expect to see no Shuffle

[jira] [Commented] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

[ https://issues.apache.org/jira/browse/SPARK-21195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17792130#comment-17792130 ] Hudson commented on SPARK-21195: User 'robert3005' has created a pull request for this issue:

[jira] [Created] (SPARK-46203) Support get partitions by names in partition filter

Wechar created SPARK-46203: -- Summary: Support get partitions by names in partition filter Key: SPARK-46203 URL: https://issues.apache.org/jira/browse/SPARK-46203 Project: Spark Issue Type:

[jira] [Updated] (SPARK-42349) Support pandas cogroup with multiple df

[ https://issues.apache.org/jira/browse/SPARK-42349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42349: --- Labels: pull-request-available (was: ) > Support pandas cogroup with multiple df >

[jira] [Updated] (SPARK-46173) Skipping trimAll call in stringToDate functions to avoid needless string copy

[ https://issues.apache.org/jira/browse/SPARK-46173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46173: --- Labels: pull-request-available (was: ) > Skipping trimAll call in stringToDate functions

[jira] [Updated] (SPARK-46202) Expose new ArtifactManager API to support in-memory artifacts and sub-directory structure

[ https://issues.apache.org/jira/browse/SPARK-46202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46202: --- Labels: pull-request-available (was: ) > Expose new ArtifactManager API to support

[jira] [Created] (SPARK-46202) Expose new ArtifactManager API to support in-memory artifacts and sub-directory structure

Venkata Sai Akhil Gudesa created SPARK-46202: Summary: Expose new ArtifactManager API to support in-memory artifacts and sub-directory structure Key: SPARK-46202 URL:

[jira] [Resolved] (SPARK-46193) Add PersistenceEngineBenchmark

[ https://issues.apache.org/jira/browse/SPARK-46193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46193. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44102

[jira] [Comment Edited] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791996#comment-17791996 ] Faiz Halde edited comment on SPARK-45255 at 12/1/23 10:40 AM: -- it seems to

[jira] [Commented] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791996#comment-17791996 ] Faiz Halde commented on SPARK-45255: it seems to be failing on another issue now a fresh project

[jira] [Comment Edited] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791996#comment-17791996 ] Faiz Halde edited comment on SPARK-45255 at 12/1/23 10:39 AM: -- it seems to

[jira] [Comment Edited] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791996#comment-17791996 ] Faiz Halde edited comment on SPARK-45255 at 12/1/23 10:39 AM: -- it seems to

[jira] [Comment Edited] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791996#comment-17791996 ] Faiz Halde edited comment on SPARK-45255 at 12/1/23 10:38 AM: -- it seems to

[jira] [Updated] (SPARK-46201) Correct the typing of schema_of_{csv, json, xml}

[ https://issues.apache.org/jira/browse/SPARK-46201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46201: --- Labels: pull-request-available (was: ) > Correct the typing of schema_of_{csv, json, xml}

[jira] [Created] (SPARK-46201) Correct the typing of schema_of_{csv, json, xml}

Ruifeng Zheng created SPARK-46201: - Summary: Correct the typing of schema_of_{csv, json, xml} Key: SPARK-46201 URL: https://issues.apache.org/jira/browse/SPARK-46201 Project: Spark Issue

[jira] [Updated] (SPARK-46200) Re-org the testing dockerfile

[ https://issues.apache.org/jira/browse/SPARK-46200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46200: --- Labels: pull-request-available (was: ) > Re-org the testing dockerfile >

[jira] [Updated] (SPARK-46200) Re-org the testing dockerfile

[ https://issues.apache.org/jira/browse/SPARK-46200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46200: -- Parent: SPARK-44566 Issue Type: Sub-task (was: Improvement) > Re-org the testing

[jira] [Created] (SPARK-46200) Re-org the testing dockerfile

Ruifeng Zheng created SPARK-46200: - Summary: Re-org the testing dockerfile Key: SPARK-46200 URL: https://issues.apache.org/jira/browse/SPARK-46200 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-46187) Align codegen and non-codegen implementation of StringDecode

[ https://issues.apache.org/jira/browse/SPARK-46187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46187. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44094

[jira] [Resolved] (SPARK-46197) Upgrade memory-profiler>=0.61.0 for Python 3.12

[ https://issues.apache.org/jira/browse/SPARK-46197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46197. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44105

[jira] [Assigned] (SPARK-44556) Reuse `OrcTail` when enable vectorizedReader

[ https://issues.apache.org/jira/browse/SPARK-44556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-44556: -- Assignee: (was: Apache Spark) > Reuse `OrcTail` when enable vectorizedReader >

[jira] [Assigned] (SPARK-44556) Reuse `OrcTail` when enable vectorizedReader

[ https://issues.apache.org/jira/browse/SPARK-44556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-44556: -- Assignee: Apache Spark > Reuse `OrcTail` when enable vectorizedReader >

[jira] [Commented] (SPARK-44556) Reuse `OrcTail` when enable vectorizedReader

[ https://issues.apache.org/jira/browse/SPARK-44556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791965#comment-17791965 ] Mike K commented on SPARK-44556: User 'cxzl25' has created a pull request for this issue:

[jira] [Updated] (SPARK-46196) Add missing `toDegrees/toRadians/atan2/approxCountDistinct` function descriptions

[ https://issues.apache.org/jira/browse/SPARK-46196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46196: -- Summary: Add missing `toDegrees/toRadians/atan2/approxCountDistinct` function descriptions

[jira] [Resolved] (SPARK-46196) Add missing function descriptions

[ https://issues.apache.org/jira/browse/SPARK-46196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46196. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44104

[jira] [Assigned] (SPARK-46196) Add missing function descriptions

[ https://issues.apache.org/jira/browse/SPARK-46196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46196: - Assignee: Ruifeng Zheng > Add missing function descriptions >

[jira] [Updated] (SPARK-46199) Add PyPi link icon to PySpark doc header

[ https://issues.apache.org/jira/browse/SPARK-46199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46199: --- Labels: pull-request-available (was: ) > Add PyPi link icon to PySpark doc header >

[jira] [Created] (SPARK-46199) Add PyPi link icon to PySpark doc header

BingKun Pan created SPARK-46199: --- Summary: Add PyPi link icon to PySpark doc header Key: SPARK-46199 URL: https://issues.apache.org/jira/browse/SPARK-46199 Project: Spark Issue Type: