[jira] [Updated] (SPARK-48308) Unify getting data schema without partition columns in FileSourceStrategy

[ https://issues.apache.org/jira/browse/SPARK-48308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48308: -- Priority: Trivial (was: Blocker) > Unify getting data schema without partition columns in

[jira] [Updated] (SPARK-48308) Unify getting data schema without partition columns in FileSourceStrategy

[ https://issues.apache.org/jira/browse/SPARK-48308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48308: -- Target Version/s: (was: 3.5.2) > Unify getting data schema without partition columns in

[jira] [Commented] (SPARK-48308) Unify getting data schema without partition columns in FileSourceStrategy

[ https://issues.apache.org/jira/browse/SPARK-48308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868716#comment-17868716 ] Dongjoon Hyun commented on SPARK-48308: --- Since this causes RC2 failure, I updated the Jira

[jira] [Updated] (SPARK-48308) Unify getting data schema without partition columns in FileSourceStrategy

[ https://issues.apache.org/jira/browse/SPARK-48308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48308: -- Target Version/s: 3.5.2 > Unify getting data schema without partition columns in

[jira] [Updated] (SPARK-48308) Unify getting data schema without partition columns in FileSourceStrategy

[ https://issues.apache.org/jira/browse/SPARK-48308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48308: -- Priority: Blocker (was: Trivial) > Unify getting data schema without partition columns in

[jira] [Resolved] (SPARK-48984) Add Controller Metrics System and Utils

[ https://issues.apache.org/jira/browse/SPARK-48984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48984. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-48984) Add Controller Metrics System and Utils

[ https://issues.apache.org/jira/browse/SPARK-48984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48984: - Assignee: Zhou JIANG > Add Controller Metrics System and Utils >

[jira] [Resolved] (SPARK-48975) Remove unnecessary `ScalaReflectionLock` definition from `protobuf`

[ https://issues.apache.org/jira/browse/SPARK-48975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48975. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47459

[jira] [Assigned] (SPARK-48975) Remove unnecessary `ScalaReflectionLock` definition from `protobuf`

[ https://issues.apache.org/jira/browse/SPARK-48975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48975: - Assignee: Yang Jie > Remove unnecessary `ScalaReflectionLock` definition from

[jira] [Resolved] (SPARK-48971) Ban `org.apache.commons.collections` in favor of `collections4`

[ https://issues.apache.org/jira/browse/SPARK-48971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48971. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-48971) Ban `org.apache.commons.collections` in favor of `collections4`

[ https://issues.apache.org/jira/browse/SPARK-48971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48971: - Assignee: Dongjoon Hyun > Ban `org.apache.commons.collections` in favor of

[jira] [Created] (SPARK-48971) Ban `org.apache.commons.collections` in favor of `collections4`

Dongjoon Hyun created SPARK-48971: - Summary: Ban `org.apache.commons.collections` in favor of `collections4` Key: SPARK-48971 URL: https://issues.apache.org/jira/browse/SPARK-48971 Project: Spark

[jira] [Assigned] (SPARK-48969) Fix `spark-operator` module to define test framework explicitly

[ https://issues.apache.org/jira/browse/SPARK-48969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48969: - Assignee: Dongjoon Hyun > Fix `spark-operator` module to define test framework

[jira] [Resolved] (SPARK-48969) Fix `spark-operator` module to define test framework explicitly

[ https://issues.apache.org/jira/browse/SPARK-48969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48969. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-48969) Fix `spark-operator` module to define test framework explicitly

Dongjoon Hyun created SPARK-48969: - Summary: Fix `spark-operator` module to define test framework explicitly Key: SPARK-48969 URL: https://issues.apache.org/jira/browse/SPARK-48969 Project: Spark

[jira] [Resolved] (SPARK-48703) Upgrade `mssql-jdbc` to 12.6.3.jre11

[ https://issues.apache.org/jira/browse/SPARK-48703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48703. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47075

[jira] [Assigned] (SPARK-48968) Avoid unnecessary task configuration in `spark-operator-api`

[ https://issues.apache.org/jira/browse/SPARK-48968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48968: - Assignee: Dongjoon Hyun > Avoid unnecessary task configuration in `spark-operator-api`

[jira] [Resolved] (SPARK-48968) Avoid unnecessary task configuration in `spark-operator-api`

[ https://issues.apache.org/jira/browse/SPARK-48968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48968. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-48968) Avoid unnecessary task configuration in `spark-operator-api`

Dongjoon Hyun created SPARK-48968: - Summary: Avoid unnecessary task configuration in `spark-operator-api` Key: SPARK-48968 URL: https://issues.apache.org/jira/browse/SPARK-48968 Project: Spark

[jira] [Updated] (SPARK-48964) Fix the discrepancy between implementation, comment and documentation of option recursive.fields.max.depth in ProtoBuf connector

[ https://issues.apache.org/jira/browse/SPARK-48964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48964: -- Fix Version/s: (was: 3.5.0) (was: 4.0.0)

[jira] [Commented] (SPARK-48964) Fix the discrepancy between implementation, comment and documentation of option recursive.fields.max.depth in ProtoBuf connector

[ https://issues.apache.org/jira/browse/SPARK-48964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17867865#comment-17867865 ] Dongjoon Hyun commented on SPARK-48964: --- Thank you for reporting, [~eason-yuchen-liu]. According

[jira] [Assigned] (SPARK-48958) Upgrade `zstd-jni` to 1.5.6-4

[ https://issues.apache.org/jira/browse/SPARK-48958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48958: - Assignee: BingKun Pan > Upgrade `zstd-jni` to 1.5.6-4 > -

[jira] [Resolved] (SPARK-48958) Upgrade `zstd-jni` to 1.5.6-4

[ https://issues.apache.org/jira/browse/SPARK-48958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48958. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47432

[jira] [Resolved] (SPARK-48963) Support JIRA_ACCESS_TOKEN in translate-contributors.py

[ https://issues.apache.org/jira/browse/SPARK-48963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48963. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47440

[jira] [Assigned] (SPARK-48963) Support JIRA_ACCESS_TOKEN in translate-contributors.py

[ https://issues.apache.org/jira/browse/SPARK-48963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48963: - Assignee: Kent Yao > Support JIRA_ACCESS_TOKEN in translate-contributors.py >

[jira] [Assigned] (SPARK-48953) Upgrade Gradle to 8.9

[ https://issues.apache.org/jira/browse/SPARK-48953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48953: - Assignee: Dongjoon Hyun > Upgrade Gradle to 8.9 > - > >

[jira] [Resolved] (SPARK-48952) Use `BasePluginExtension` in `spark-operator/build.gradle`

[ https://issues.apache.org/jira/browse/SPARK-48952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48952. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-48953) Upgrade Gradle to 8.9

[ https://issues.apache.org/jira/browse/SPARK-48953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48953. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-48953) Upgrade Gradle to 8.9

Dongjoon Hyun created SPARK-48953: - Summary: Upgrade Gradle to 8.9 Key: SPARK-48953 URL: https://issues.apache.org/jira/browse/SPARK-48953 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-48952) Use `BasePluginExtension` in `spark-operator/build.gradle`

[ https://issues.apache.org/jira/browse/SPARK-48952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48952: - Assignee: Dongjoon Hyun > Use `BasePluginExtension` in `spark-operator/build.gradle` >

[jira] [Created] (SPARK-48952) Use `BasePluginExtension` in `spark-operator/build.gradle`

Dongjoon Hyun created SPARK-48952: - Summary: Use `BasePluginExtension` in `spark-operator/build.gradle` Key: SPARK-48952 URL: https://issues.apache.org/jira/browse/SPARK-48952 Project: Spark

[jira] [Updated] (SPARK-48946) NPE in DataSourceV2ScanExecBase redact method when session is null

[ https://issues.apache.org/jira/browse/SPARK-48946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48946: -- Issue Type: Improvement (was: Bug) > NPE in DataSourceV2ScanExecBase redact method when

[jira] [Resolved] (SPARK-48946) NPE in DataSourceV2ScanExecBase redact method when session is null

[ https://issues.apache.org/jira/browse/SPARK-48946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48946. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47419

[jira] [Resolved] (SPARK-48940) Upgrade `Arrow` to 17.0.0

[ https://issues.apache.org/jira/browse/SPARK-48940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48940. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47409

[jira] [Updated] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

[ https://issues.apache.org/jira/browse/SPARK-48791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48791: -- Fix Version/s: 3.4.4 > Perf regression due to accumulator registration overhead using >

[jira] [Resolved] (SPARK-48933) Upgrade `protobuf-java` to `3.25.3`

[ https://issues.apache.org/jira/browse/SPARK-48933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48933. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47397

[jira] [Updated] (SPARK-48921) ScalaUDF in subquery should run through analyzer

[ https://issues.apache.org/jira/browse/SPARK-48921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48921: -- Fix Version/s: 3.5.3 (was: 3.5.2) > ScalaUDF in subquery should run

[jira] [Resolved] (SPARK-48921) ScalaUDF in subquery should run through analyzer

[ https://issues.apache.org/jira/browse/SPARK-48921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48921. --- Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by

[jira] [Commented] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

[ https://issues.apache.org/jira/browse/SPARK-48791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17867038#comment-17867038 ] Dongjoon Hyun commented on SPARK-48791: --- I added a fix version, 3.5.3, for now because it arrives

[jira] [Updated] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

[ https://issues.apache.org/jira/browse/SPARK-48791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48791: -- Fix Version/s: 3.5.3 > Perf regression due to accumulator registration overhead using >

[jira] [Updated] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

[ https://issues.apache.org/jira/browse/SPARK-48791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48791: -- Issue Type: Bug (was: Improvement) > Perf regression due to accumulator registration

[jira] [Updated] (SPARK-48932) Upgrade `commons-lang3` to 3.15.0

[ https://issues.apache.org/jira/browse/SPARK-48932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48932: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Upgrade

[jira] [Resolved] (SPARK-48932) Upgrade `commons-lang3` to 3.15.0

[ https://issues.apache.org/jira/browse/SPARK-48932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48932. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47396

[jira] [Updated] (SPARK-48930) Redact `awsAccessKeyId` by including `accesskey` pattern

[ https://issues.apache.org/jira/browse/SPARK-48930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48930: -- Parent Issue: SPARK-45869 (was: SPARK-44111) > Redact `awsAccessKeyId` by including

[jira] [Updated] (SPARK-48930) Redact `awsAccessKeyId` by including `accesskey` pattern

[ https://issues.apache.org/jira/browse/SPARK-48930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48930: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Bug) > Redact `awsAccessKeyId` by

[jira] [Resolved] (SPARK-48930) Redact `awsAccessKeyId` by including `accesskey` pattern

[ https://issues.apache.org/jira/browse/SPARK-48930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48930. --- Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution:

[jira] [Updated] (SPARK-48930) Redact `awsAccessKeyId` by including `accesskey` pattern

[ https://issues.apache.org/jira/browse/SPARK-48930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48930: -- Labels: pull-request-available security (was: pull-request-available) > Redact

[jira] [Assigned] (SPARK-48930) Redact `awsAccessKeyId` by including `accesskey` pattern

[ https://issues.apache.org/jira/browse/SPARK-48930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48930: - Assignee: Dongjoon Hyun > Redact `awsAccessKeyId` by including `accesskey` pattern >

[jira] [Updated] (SPARK-48930) Redact `awsAccessKeyId` by including `accesskey` pattern

[ https://issues.apache.org/jira/browse/SPARK-48930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48930: -- Target Version/s: 3.5.2 > Redact `awsAccessKeyId` by including `accesskey` pattern >

[jira] [Updated] (SPARK-48930) Redact `awsAccessKeyId` by including `accesskey` pattern

[ https://issues.apache.org/jira/browse/SPARK-48930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48930: -- Priority: Blocker (was: Critical) > Redact `awsAccessKeyId` by including `accesskey` pattern

[jira] [Resolved] (SPARK-48927) Show the number of cached RDDs in StoragePage

[ https://issues.apache.org/jira/browse/SPARK-48927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48927. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47390

[jira] [Updated] (SPARK-48930) Redact `awsAccessKeyId` by including `accesskey` pattern

[ https://issues.apache.org/jira/browse/SPARK-48930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48930: -- Priority: Critical (was: Major) > Redact `awsAccessKeyId` by including `accesskey` pattern >

[jira] [Created] (SPARK-48930) Redact `awsAccessKeyId` by including `accesskey` pattern

Dongjoon Hyun created SPARK-48930: - Summary: Redact `awsAccessKeyId` by including `accesskey` pattern Key: SPARK-48930 URL: https://issues.apache.org/jira/browse/SPARK-48930 Project: Spark

[jira] [Assigned] (SPARK-48927) Show the number of cached RDDs in StoragePage

[ https://issues.apache.org/jira/browse/SPARK-48927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48927: - Assignee: Dongjoon Hyun > Show the number of cached RDDs in StoragePage >

[jira] [Updated] (SPARK-48927) Show the number of cached RDDs in StoragePage

[ https://issues.apache.org/jira/browse/SPARK-48927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48927: -- Summary: Show the number of cached RDDs in StoragePage (was: Show the total number of cached

[jira] [Created] (SPARK-48927) Show the total number of cached RDDs in StoragePage

Dongjoon Hyun created SPARK-48927: - Summary: Show the total number of cached RDDs in StoragePage Key: SPARK-48927 URL: https://issues.apache.org/jira/browse/SPARK-48927 Project: Spark Issue

[jira] [Resolved] (SPARK-48920) Upgrade ORC to 1.9.4

[ https://issues.apache.org/jira/browse/SPARK-48920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48920. --- Fix Version/s: 3.5.2 Resolution: Fixed Issue resolved by pull request 47379

[jira] [Created] (SPARK-48913) Avoid `com.sun.xml.txw2.output.IndentingXMLStreamWriter` usage

Dongjoon Hyun created SPARK-48913: - Summary: Avoid `com.sun.xml.txw2.output.IndentingXMLStreamWriter` usage Key: SPARK-48913 URL: https://issues.apache.org/jira/browse/SPARK-48913 Project: Spark

[jira] [Assigned] (SPARK-48909) Uses SparkSession over SparkContext when writing metadata

[ https://issues.apache.org/jira/browse/SPARK-48909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48909: - Assignee: Hyukjin Kwon > Uses SparkSession over SparkContext when writing metadata >

[jira] [Resolved] (SPARK-48909) Uses SparkSession over SparkContext when writing metadata

[ https://issues.apache.org/jira/browse/SPARK-48909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48909. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47366

[jira] [Resolved] (SPARK-48896) Remove repartition(1) in writing metadata ML

[ https://issues.apache.org/jira/browse/SPARK-48896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48896. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47347

[jira] [Assigned] (SPARK-48896) Remove repartition(1) in writing metadata ML

[ https://issues.apache.org/jira/browse/SPARK-48896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48896: - Assignee: Hyukjin Kwon > Remove repartition(1) in writing metadata ML >

[jira] [Assigned] (SPARK-48902) Upgrade `commons-codec` to 1.17.1

[ https://issues.apache.org/jira/browse/SPARK-48902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48902: - Assignee: BingKun Pan > Upgrade `commons-codec` to 1.17.1 >

[jira] [Resolved] (SPARK-48902) Upgrade `commons-codec` to 1.17.1

[ https://issues.apache.org/jira/browse/SPARK-48902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48902. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47362

[jira] [Resolved] (SPARK-48899) Fix `ENV` key value format in K8s Dockerfiles

[ https://issues.apache.org/jira/browse/SPARK-48899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48899. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47357

[jira] [Assigned] (SPARK-48899) Fix `ENV` key value format in K8s Dockerfiles

[ https://issues.apache.org/jira/browse/SPARK-48899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48899: - Assignee: Dongjoon Hyun > Fix `ENV` key value format in K8s Dockerfiles >

[jira] [Created] (SPARK-48899) Fix `ENV` key value format in K8s Dockerfiles

Dongjoon Hyun created SPARK-48899: - Summary: Fix `ENV` key value format in K8s Dockerfiles Key: SPARK-48899 URL: https://issues.apache.org/jira/browse/SPARK-48899 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-48887) Enable `spark.kubernetes.executor.checkAllContainers` by default

[ https://issues.apache.org/jira/browse/SPARK-48887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48887. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47337

[jira] [Created] (SPARK-48887) Enable `spark.kubernetes.executor.checkAllContainers` by default

Dongjoon Hyun created SPARK-48887: - Summary: Enable `spark.kubernetes.executor.checkAllContainers` by default Key: SPARK-48887 URL: https://issues.apache.org/jira/browse/SPARK-48887 Project: Spark

[jira] [Updated] (SPARK-48775) Replace SQLContext with SparkSession in STS

[ https://issues.apache.org/jira/browse/SPARK-48775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48775: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Replace SQLContext

[jira] [Resolved] (SPARK-48775) Replace SQLContext with SparkSession in STS

[ https://issues.apache.org/jira/browse/SPARK-48775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48775. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47176

[jira] [Resolved] (SPARK-46738) `Cast` displayed different results between Regular Spark and Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-46738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46738. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47302

[jira] [Assigned] (SPARK-46738) `Cast` displayed different results between Regular Spark and Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-46738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46738: - Assignee: BingKun Pan > `Cast` displayed different results between Regular Spark and

[jira] [Created] (SPARK-48860) Update `ui-test` to use `ws` 8.18.0

Dongjoon Hyun created SPARK-48860: - Summary: Update `ui-test` to use `ws` 8.18.0 Key: SPARK-48860 URL: https://issues.apache.org/jira/browse/SPARK-48860 Project: Spark Issue Type: Test

[jira] [Resolved] (SPARK-48844) Avoid throw UNSUPPORTED_DATASOURCE_FOR_DIRECT_QUERY when path is empty

[ https://issues.apache.org/jira/browse/SPARK-48844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48844. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47267

[jira] [Created] (SPARK-48855) Make ExecutorPodsAllocatorSuite independent from default allocation batch size

Dongjoon Hyun created SPARK-48855: - Summary: Make ExecutorPodsAllocatorSuite independent from default allocation batch size Key: SPARK-48855 URL: https://issues.apache.org/jira/browse/SPARK-48855

[jira] [Resolved] (SPARK-48786) Define Config Loading Framework for Spark Operator Controller

[ https://issues.apache.org/jira/browse/SPARK-48786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48786. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-48786) Define Config Loading Framework for Spark Operator Controller

[ https://issues.apache.org/jira/browse/SPARK-48786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48786: - Assignee: Zhou JIANG > Define Config Loading Framework for Spark Operator Controller >

[jira] [Assigned] (SPARK-48803) Throw internal error in OrcDeserializer to align with ParquetWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-48803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48803: - Assignee: Kent Yao > Throw internal error in OrcDeserializer to align with

[jira] [Resolved] (SPARK-48803) Throw internal error in OrcDeserializer to align with ParquetWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-48803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48803. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47208

[jira] [Resolved] (SPARK-48837) In CountVectorizer, only read binary parameter once per transform, not once per row

[ https://issues.apache.org/jira/browse/SPARK-48837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48837. --- Resolution: Fixed Issue resolved by pull request 47258

[jira] [Updated] (SPARK-48801) Upgrade `kubernetes-client` to 6.13.1

[ https://issues.apache.org/jira/browse/SPARK-48801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48801: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Upgrade

[jira] [Resolved] (SPARK-48801) Upgrade `kubernetes-client` to 6.13.1

[ https://issues.apache.org/jira/browse/SPARK-48801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48801. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47206

[jira] [Assigned] (SPARK-48801) Upgrade `kubernetes-client` to 6.13.1

[ https://issues.apache.org/jira/browse/SPARK-48801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48801: - Assignee: BingKun Pan > Upgrade `kubernetes-client` to 6.13.1 >

[jira] [Assigned] (SPARK-48787) Upgrade Kafka to 3.7.1

[ https://issues.apache.org/jira/browse/SPARK-48787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48787: - Assignee: BingKun Pan > Upgrade Kafka to 3.7.1 > -- > >

[jira] [Updated] (SPARK-48787) Upgrade Kafka to 3.7.1

[ https://issues.apache.org/jira/browse/SPARK-48787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48787: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Upgrade Kafka to 3.7.1

[jira] [Resolved] (SPARK-48787) Upgrade Kafka to 3.7.1

[ https://issues.apache.org/jira/browse/SPARK-48787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48787. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47191

[jira] [Resolved] (SPARK-48679) Upgrade checkstyle and spotbugs version

[ https://issues.apache.org/jira/browse/SPARK-48679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48679. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-48554) Use R 4.4.0 in `windows` R GitHub Action Windows job

[ https://issues.apache.org/jira/browse/SPARK-48554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48554. --- Fix Version/s: 4.0.0 Assignee: BingKun Pan Resolution: Fixed > Use R 4.4.0

[jira] [Resolved] (SPARK-48177) Upgrade `Parquet` to 1.14.1

[ https://issues.apache.org/jira/browse/SPARK-48177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48177. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46447

[jira] [Commented] (SPARK-48292) Revert [SPARK-39195][SQL] Spark OutputCommitCoordinator should abort stage when committed file not consistent with task status

[ https://issues.apache.org/jira/browse/SPARK-48292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17861276#comment-17861276 ] Dongjoon Hyun commented on SPARK-48292: --- This is backported via the following. -

[jira] [Updated] (SPARK-48292) Revert [SPARK-39195][SQL] Spark OutputCommitCoordinator should abort stage when committed file not consistent with task status

[ https://issues.apache.org/jira/browse/SPARK-48292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48292: -- Fix Version/s: 3.4.4 > Revert [SPARK-39195][SQL] Spark OutputCommitCoordinator should abort

[jira] [Updated] (SPARK-48292) Revert [SPARK-39195][SQL] Spark OutputCommitCoordinator should abort stage when committed file not consistent with task status

[ https://issues.apache.org/jira/browse/SPARK-48292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48292: -- Fix Version/s: 3.5.2 > Revert [SPARK-39195][SQL] Spark OutputCommitCoordinator should abort

[jira] [Resolved] (SPARK-48757) Make `IndexShuffleBlockResolver` have explicit constructors

[ https://issues.apache.org/jira/browse/SPARK-48757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48757. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47148

[jira] [Updated] (SPARK-48757) Make `IndexShuffleBlockResolver` have explicit constructors

[ https://issues.apache.org/jira/browse/SPARK-48757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48757: -- Description: Although `IndexShuffleBlockResolver` is `private` and there is no contract to

[jira] [Updated] (SPARK-48757) Make `IndexShuffleBlockResolver` have explicit constructors

[ https://issues.apache.org/jira/browse/SPARK-48757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48757: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Make

[jira] [Assigned] (SPARK-48757) Make `IndexShuffleBlockResolver` have explicit constructors

[ https://issues.apache.org/jira/browse/SPARK-48757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48757: - Assignee: Dongjoon Hyun > Make `IndexShuffleBlockResolver` have explicit constructors

[jira] [Created] (SPARK-48757) Make `IndexShuffleBlockResolver` have explicit constructors

Dongjoon Hyun created SPARK-48757: - Summary: Make `IndexShuffleBlockResolver` have explicit constructors Key: SPARK-48757 URL: https://issues.apache.org/jira/browse/SPARK-48757 Project: Spark

[jira] [Updated] (SPARK-47046) Apache Spark 4.0.0 Dependency Audit and Cleanup

[ https://issues.apache.org/jira/browse/SPARK-47046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47046: -- Labels: releasenotes (was: pull-request-available releasenotes) > Apache Spark 4.0.0

[jira] [Resolved] (SPARK-48642) False reported SparkOutOfMemoryError caused by killing task on spilling

[ https://issues.apache.org/jira/browse/SPARK-48642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48642. --- Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution:

<    1   2   3   4   5   6   7   8   9   10   >