[jira] [Updated] (SPARK-51414) Add the make_time() function

[ https://issues.apache.org/jira/browse/SPARK-51414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-51414: - Description: The make_time() function creates a time from hour, min, and sec fields. h4. Syntax {code:sq

[jira] [Updated] (SPARK-51415) Support the time type by make_timestamp()

[ https://issues.apache.org/jira/browse/SPARK-51415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-51415: - Summary: Support the time type by make_timestamp() (was: Support TIME by make_timestamp()) > Support t

[jira] [Created] (SPARK-51415) Support TIME by make_timestamp()

Max Gekk created SPARK-51415: Summary: Support TIME by make_timestamp() Key: SPARK-51415 URL: https://issues.apache.org/jira/browse/SPARK-51415 Project: Spark Issue Type: Sub-task Compo

[jira] [Created] (SPARK-51414) Add the make_time() function

Max Gekk created SPARK-51414: Summary: Add the make_time() function Key: SPARK-51414 URL: https://issues.apache.org/jira/browse/SPARK-51414 Project: Spark Issue Type: Sub-task Component

[jira] [Created] (SPARK-51413) Support TIME in Arrow

Max Gekk created SPARK-51413: Summary: Support TIME in Arrow Key: SPARK-51413 URL: https://issues.apache.org/jira/browse/SPARK-51413 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Commented] (SPARK-51365) OOM occurred during macOS daily tests

[ https://issues.apache.org/jira/browse/SPARK-51365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932823#comment-17932823 ] Yang Jie commented on SPARK-51365: -- On the other hand, I tried setting `ulimit -n 65536

[jira] [Updated] (SPARK-51365) OOM occurred during macOS daily tests

[ https://issues.apache.org/jira/browse/SPARK-51365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51365: --- Labels: pull-request-available (was: ) > OOM occurred during macOS daily tests > --

[jira] [Comment Edited] (SPARK-51365) OOM occurred during macOS daily tests

[ https://issues.apache.org/jira/browse/SPARK-51365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932822#comment-17932822 ] Yang Jie edited comment on SPARK-51365 at 3/6/25 4:15 AM: -- [~do

[jira] [Commented] (SPARK-51365) OOM occurred during macOS daily tests

[ https://issues.apache.org/jira/browse/SPARK-51365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932822#comment-17932822 ] Yang Jie commented on SPARK-51365: -- Sharing some findings:   When I run the  {code:ja

[jira] [Resolved] (SPARK-51346) Overall Audit on TransformWithState Scala API of calling `get()` on non-exist state

[ https://issues.apache.org/jira/browse/SPARK-51346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar resolved SPARK-51346. Resolution: Fixed > Overall Audit on TransformWithState Scala API of calling `get(

[jira] [Resolved] (SPARK-51412) Upgrade Gradle to 8.13

[ https://issues.apache.org/jira/browse/SPARK-51412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51412. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Assigned] (SPARK-51412) Upgrade Gradle to 8.13

[ https://issues.apache.org/jira/browse/SPARK-51412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51412: - Assignee: Dongjoon Hyun > Upgrade Gradle to 8.13 > -- > >

[jira] [Commented] (SPARK-47240) SPIP: Structured Logging Framework for Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-47240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932816#comment-17932816 ] Dongjoon Hyun commented on SPARK-47240: --- Thank you, [~Gengliang.Wang] .  > SPIP:

[jira] [Resolved] (SPARK-51151) Fix internal and public API naming for TimeMode.None() for TransformWithState

[ https://issues.apache.org/jira/browse/SPARK-51151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhan resolved SPARK-51151. --- Resolution: Won't Do See details on the linked PR comment. > Fix internal and public API naming for

[jira] [Assigned] (SPARK-47240) SPIP: Structured Logging Framework for Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-47240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47240: - Assignee: Gengliang Wang (was: Apache Spark) > SPIP: Structured Logging Framework for

[jira] [Updated] (SPARK-47240) SPIP: Structured Logging Framework for Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-47240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47240: -- Fix Version/s: 4.0.0 > SPIP: Structured Logging Framework for Apache Spark > -

[jira] [Updated] (SPARK-51290) Enable filling default values in DSv2 writes

[ https://issues.apache.org/jira/browse/SPARK-51290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51290: -- Parent: SPARK-51166 Issue Type: Sub-task (was: New Feature) > Enable filling default

[jira] [Updated] (SPARK-51281) DataFrameWriterV2 should respect the path option

[ https://issues.apache.org/jira/browse/SPARK-51281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51281: -- Fix Version/s: (was: 3.5.6) > DataFrameWriterV2 should respect the path option > -

[jira] [Updated] (SPARK-51412) Upgrade Gradle to 8.13

[ https://issues.apache.org/jira/browse/SPARK-51412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51412: --- Labels: pull-request-available (was: ) > Upgrade Gradle to 8.13 > -- >

[jira] [Updated] (SPARK-51411) Add documentation for transformWithState

[ https://issues.apache.org/jira/browse/SPARK-51411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51411: --- Labels: pull-request-available (was: ) > Add documentation for transformWithState > ---

[jira] [Commented] (SPARK-51281) DataFrameWriterV2 should respect the path option

[ https://issues.apache.org/jira/browse/SPARK-51281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932807#comment-17932807 ] Dongjoon Hyun commented on SPARK-51281: --- This is reverted from branch-3.5 via [ht

[jira] [Created] (SPARK-51412) Upgrade Gradle to 8.13

Dongjoon Hyun created SPARK-51412: - Summary: Upgrade Gradle to 8.13 Key: SPARK-51412 URL: https://issues.apache.org/jira/browse/SPARK-51412 Project: Spark Issue Type: Sub-task Compo

[jira] [Reopened] (SPARK-51097) Adding state store level metrics for last uploaded snapshot version in RocksDB

[ https://issues.apache.org/jira/browse/SPARK-51097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reopened SPARK-51097: -- Assignee: (was: Zeyu Chen) This was reverted as we observed UI being flooded with new me

[jira] [Updated] (SPARK-51407) Document missed `Spark Connect` configurations

[ https://issues.apache.org/jira/browse/SPARK-51407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51407: -- Fix Version/s: 3.5.6 > Document missed `Spark Connect` configurations > --

[jira] [Updated] (SPARK-51097) Adding state store level metrics for last uploaded snapshot version in RocksDB

[ https://issues.apache.org/jira/browse/SPARK-51097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-51097: - Fix Version/s: (was: 4.0.0) > Adding state store level metrics for last uploaded snapshot ve

[jira] [Comment Edited] (SPARK-51097) Adding state store level metrics for last uploaded snapshot version in RocksDB

[ https://issues.apache.org/jira/browse/SPARK-51097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932810#comment-17932810 ] Jungtaek Lim edited comment on SPARK-51097 at 3/6/25 2:50 AM:

[jira] [Resolved] (SPARK-51290) Enable filling default values in DSv2 writes

[ https://issues.apache.org/jira/browse/SPARK-51290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51290. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50044 [https://gith

[jira] [Assigned] (SPARK-49489) 'TTransportException: MaxMessageSize reached' occurs if get a table with a large number of partitions

[ https://issues.apache.org/jira/browse/SPARK-49489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-49489: --- Assignee: Cheng Pan > 'TTransportException: MaxMessageSize reached' occurs if get a table w

[jira] [Commented] (SPARK-49489) 'TTransportException: MaxMessageSize reached' occurs if get a table with a large number of partitions

[ https://issues.apache.org/jira/browse/SPARK-49489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932803#comment-17932803 ] Yuming Wang commented on SPARK-49489: - Since Spark 4.0, Please try to {{set spark.h

[jira] [Assigned] (SPARK-51410) Add `Spark Connect Plugin` example

[ https://issues.apache.org/jira/browse/SPARK-51410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51410: - Assignee: Dongjoon Hyun > Add `Spark Connect Plugin` example >

[jira] [Resolved] (SPARK-51410) Add `Spark Connect Plugin` example

[ https://issues.apache.org/jira/browse/SPARK-51410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51410. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Updated] (SPARK-51409) Add error classification for changelog writer related errors

[ https://issues.apache.org/jira/browse/SPARK-51409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51409: --- Labels: pull-request-available (was: ) > Add error classification for changelog writer rela

[jira] [Assigned] (SPARK-51393) Fallback to regular Python UDF when Arrow is not found but Arrow-optimized Python UDFs enabled

[ https://issues.apache.org/jira/browse/SPARK-51393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-51393: Assignee: Hyukjin Kwon > Fallback to regular Python UDF when Arrow is not found but Arrow

[jira] [Resolved] (SPARK-51393) Fallback to regular Python UDF when Arrow is not found but Arrow-optimized Python UDFs enabled

[ https://issues.apache.org/jira/browse/SPARK-51393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-51393. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50160 [https://gi

[jira] [Created] (SPARK-51410) Add `Spark Connect Plugin` example

Dongjoon Hyun created SPARK-51410: - Summary: Add `Spark Connect Plugin` example Key: SPARK-51410 URL: https://issues.apache.org/jira/browse/SPARK-51410 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-51410) Add `Spark Connect Plugin` example

[ https://issues.apache.org/jira/browse/SPARK-51410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51410: --- Labels: pull-request-available (was: ) > Add `Spark Connect Plugin` example > -

[jira] [Updated] (SPARK-50413) Add a flag to insert commands to know if they are part of CTAS

[ https://issues.apache.org/jira/browse/SPARK-50413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50413: --- Labels: pull-request-available (was: ) > Add a flag to insert commands to know if they are

[jira] [Created] (SPARK-51409) Add error classification for changelog writer related errors

Anish Shrigondekar created SPARK-51409: -- Summary: Add error classification for changelog writer related errors Key: SPARK-51409 URL: https://issues.apache.org/jira/browse/SPARK-51409 Project: Spa

[jira] [Updated] (SPARK-51393) Fallback to regular Python UDF when Arrow is not found but Arrow-optimized Python UDFs enabled

[ https://issues.apache.org/jira/browse/SPARK-51393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51393: -- Target Version/s: 4.0.0 > Fallback to regular Python UDF when Arrow is not found but Arrow-opt

[jira] [Updated] (SPARK-51393) Fallback to regular Python UDF when Arrow is not found but Arrow-optimized Python UDFs enabled

[ https://issues.apache.org/jira/browse/SPARK-51393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51393: -- Priority: Blocker (was: Major) > Fallback to regular Python UDF when Arrow is not found but A

[jira] [Updated] (SPARK-51207) SPIP: Constraints in DSv2

[ https://issues.apache.org/jira/browse/SPARK-51207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Okolnychyi updated SPARK-51207: - Shepherd: Gengliang Wang (was: Wenchen Fan) > SPIP: Constraints in DSv2 > -

[jira] [Commented] (SPARK-48238) Spark fail to start due to class o.a.h.yarn.server.webproxy.amfilter.AmIpFilter is not a jakarta.servlet.Filter

[ https://issues.apache.org/jira/browse/SPARK-48238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932777#comment-17932777 ] Chris Nauroth commented on SPARK-48238: --- Linking to SPARK-51408, which has a follo

[jira] [Assigned] (SPARK-51407) Document missed `Spark Connect` configurations

[ https://issues.apache.org/jira/browse/SPARK-51407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51407: - Assignee: Dongjoon Hyun > Document missed `Spark Connect` configurations >

[jira] [Resolved] (SPARK-48676) Structured Logging Framework Scala Style Migration [Part 2]

[ https://issues.apache.org/jira/browse/SPARK-48676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-48676. Resolution: Won't Do > Structured Logging Framework Scala Style Migration [Part 2] > -

[jira] [Resolved] (SPARK-47605) Enable structured logging in all the test log4j2.properties

[ https://issues.apache.org/jira/browse/SPARK-47605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-47605. Resolution: Won't Do Mark as won't do since it is disable by default > Enable structured

[jira] [Resolved] (SPARK-51407) Document missed `Spark Connect` configurations

[ https://issues.apache.org/jira/browse/SPARK-51407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51407. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50171 [https://

[jira] [Updated] (SPARK-51408) AmIpFilterSuite#testProxyUpdate fails in some networks

[ https://issues.apache.org/jira/browse/SPARK-51408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51408: --- Labels: pull-request-available (was: ) > AmIpFilterSuite#testProxyUpdate fails in some netw

[jira] [Resolved] (SPARK-47240) SPIP: Structured Logging Framework for Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-47240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-47240. Resolution: Done > SPIP: Structured Logging Framework for Apache Spark > -

[jira] [Commented] (SPARK-47240) SPIP: Structured Logging Framework for Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-47240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932776#comment-17932776 ] Gengliang Wang commented on SPARK-47240: [~dongjoon] Thanks, I have resolved mul

[jira] [Resolved] (SPARK-47688) Support `three` methods of the log `concatenation` in the `structured logging framework`

[ https://issues.apache.org/jira/browse/SPARK-47688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-47688. Resolution: Abandoned Mark as abandoned since the PR was closed > Support `three` methods

[jira] [Created] (SPARK-51408) AmIpFilterSuite#testProxyUpdate fails in some networks

Chris Nauroth created SPARK-51408: - Summary: AmIpFilterSuite#testProxyUpdate fails in some networks Key: SPARK-51408 URL: https://issues.apache.org/jira/browse/SPARK-51408 Project: Spark Issu

[jira] [Updated] (SPARK-45239) Reduce default spark.connect.jvmStacktrace.maxSize

[ https://issues.apache.org/jira/browse/SPARK-45239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45239: -- Epic Link: (was: SPARK-42554) > Reduce default spark.connect.jvmStacktrace.maxSize > -

[jira] [Updated] (SPARK-51407) Document missed `Spark Connect` configurations

[ https://issues.apache.org/jira/browse/SPARK-51407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51407: -- Summary: Document missed `Spark Connect` configurations (was: Document missed `spark.connect.

[jira] [Created] (SPARK-51407) Document missed `spark.connect.*` configurations

Dongjoon Hyun created SPARK-51407: - Summary: Document missed `spark.connect.*` configurations Key: SPARK-51407 URL: https://issues.apache.org/jira/browse/SPARK-51407 Project: Spark Issue Type

[jira] [Updated] (SPARK-51407) Document missed `spark.connect.*` configurations

[ https://issues.apache.org/jira/browse/SPARK-51407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51407: --- Labels: pull-request-available (was: ) > Document missed `spark.connect.*` configurations >

[jira] [Updated] (SPARK-51405) Upgrade `build-tools` to use Ubuntu 24.04 LTS instead of 22.04 LTS docker image

[ https://issues.apache.org/jira/browse/SPARK-51405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51405: --- Labels: pull-request-available (was: ) > Upgrade `build-tools` to use Ubuntu 24.04 LTS inst

[jira] [Assigned] (SPARK-51406) Remove no-op `spark.log.structuredLogging.enabled=false`

[ https://issues.apache.org/jira/browse/SPARK-51406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51406: - Assignee: Dongjoon Hyun > Remove no-op `spark.log.structuredLogging.enabled=false` > --

[jira] [Resolved] (SPARK-51406) Remove no-op `spark.log.structuredLogging.enabled=false`

[ https://issues.apache.org/jira/browse/SPARK-51406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51406. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Updated] (SPARK-51406) Remove no-op `spark.log.structuredLogging.enabled=false`

[ https://issues.apache.org/jira/browse/SPARK-51406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51406: --- Labels: pull-request-available (was: ) > Remove no-op `spark.log.structuredLogging.enabled=

[jira] [Created] (SPARK-51406) Remove no-op `spark.log.structuredLogging.enabled=false`

Dongjoon Hyun created SPARK-51406: - Summary: Remove no-op `spark.log.structuredLogging.enabled=false` Key: SPARK-51406 URL: https://issues.apache.org/jira/browse/SPARK-51406 Project: Spark Is

[jira] [Comment Edited] (SPARK-47240) SPIP: Structured Logging Framework for Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-47240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932741#comment-17932741 ] Dongjoon Hyun edited comment on SPARK-47240 at 3/5/25 8:52 PM: ---

[jira] [Commented] (SPARK-47240) SPIP: Structured Logging Framework for Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-47240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932741#comment-17932741 ] Dongjoon Hyun commented on SPARK-47240: --- I added a label, `releasenotes`, not to f

[jira] [Updated] (SPARK-47240) SPIP: Structured Logging Framework for Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-47240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47240: -- Labels: pull-request-available releasenotes (was: pull-request-available) > SPIP: Structured

[jira] [Resolved] (SPARK-51405) Upgrade `build-tools` to use Ubuntu 24.04 LTS instead of 22.04 LTS docker image

[ https://issues.apache.org/jira/browse/SPARK-51405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51405. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Updated] (SPARK-51405) Upgrade `build-tools` to use Ubuntu 24.04 LTS instead of 22.04 LTS docker image

[ https://issues.apache.org/jira/browse/SPARK-51405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51405: -- Summary: Upgrade `build-tools` to use Ubuntu 24.04 LTS instead of 22.04 LTS docker image (was

[jira] [Created] (SPARK-51405) Upgrade `build-tools` to use Ubuntu 24.04 LTS instead of 22.04 LTS

Dongjoon Hyun created SPARK-51405: - Summary: Upgrade `build-tools` to use Ubuntu 24.04 LTS instead of 22.04 LTS Key: SPARK-51405 URL: https://issues.apache.org/jira/browse/SPARK-51405 Project: Spark

[jira] [Updated] (SPARK-51389) Allow full customization of profiles in scalastyle

[ https://issues.apache.org/jira/browse/SPARK-51389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51389: -- Parent: SPARK-51166 Issue Type: Sub-task (was: Improvement) > Allow full customizatio

[jira] [Commented] (SPARK-51389) Allow full customization of profiles in scalastyle

[ https://issues.apache.org/jira/browse/SPARK-51389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932737#comment-17932737 ] Dongjoon Hyun commented on SPARK-51389: --- According to the discussion on the PR, I

[jira] [Updated] (SPARK-51389) Allow full customization of profiles in scalastyle

[ https://issues.apache.org/jira/browse/SPARK-51389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51389: -- Issue Type: Improvement (was: Bug) > Allow full customization of profiles in scalastyle > ---

[jira] [Created] (SPARK-51404) Support parser data type json "time(n)" as TimeType

Max Gekk created SPARK-51404: Summary: Support parser data type json "time(n)" as TimeType Key: SPARK-51404 URL: https://issues.apache.org/jira/browse/SPARK-51404 Project: Spark Issue Type: Sub-t

[jira] [Commented] (SPARK-51072) CallerContext to set Hadoop cloud client audit context

[ https://issues.apache.org/jira/browse/SPARK-51072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932644#comment-17932644 ] Steve Loughran commented on SPARK-51072: thanks! > CallerContext to set Hadoop

[jira] [Created] (SPARK-51403) Test TimeType as ordered and atomic type

Max Gekk created SPARK-51403: Summary: Test TimeType as ordered and atomic type Key: SPARK-51403 URL: https://issues.apache.org/jira/browse/SPARK-51403 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-51402) Test time in UDF

Max Gekk created SPARK-51402: Summary: Test time in UDF Key: SPARK-51402 URL: https://issues.apache.org/jira/browse/SPARK-51402 Project: Spark Issue Type: Sub-task Components: SQL A

[jira] [Assigned] (SPARK-51389) Allow full customization of profiles in scalastyle

[ https://issues.apache.org/jira/browse/SPARK-51389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51389: - Assignee: Chris Nauroth > Allow full customization of profiles in scalastyle >

[jira] [Resolved] (SPARK-51389) Allow full customization of profiles in scalastyle

[ https://issues.apache.org/jira/browse/SPARK-51389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51389. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50156 [https://

[jira] [Commented] (SPARK-51199) Valid CSV records considered malformed

[ https://issues.apache.org/jira/browse/SPARK-51199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932691#comment-17932691 ] Felix Wollschläger commented on SPARK-51199: This is probably an issue with

[jira] [Assigned] (SPARK-51401) Make `ExplainUtils.generateFieldString` reuse ` QueryPlan.generateFieldString`

[ https://issues.apache.org/jira/browse/SPARK-51401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51401: - Assignee: Yang Jie > Make `ExplainUtils.generateFieldString` reuse ` QueryPlan.generate

[jira] [Updated] (SPARK-51401) Make `ExplainUtils.generateFieldString` reuse ` QueryPlan.generateFieldString`

[ https://issues.apache.org/jira/browse/SPARK-51401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51401: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Make `ExplainUtils.gene

[jira] [Resolved] (SPARK-51401) Make `ExplainUtils.generateFieldString` reuse ` QueryPlan.generateFieldString`

[ https://issues.apache.org/jira/browse/SPARK-51401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51401. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50169 [https://

[jira] [Assigned] (SPARK-49507) Fix Expected only partition pruning predicates exception

[ https://issues.apache.org/jira/browse/SPARK-49507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-49507: --- Assignee: Tongwei (was: Apache Spark) > Fix Expected only partition pruning predicates exc

[jira] [Updated] (SPARK-49507) Fix Expected only partition pruning predicates exception

[ https://issues.apache.org/jira/browse/SPARK-49507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-49507: Fix Version/s: 4.0.0 3.5.6 > Fix Expected only partition pruning predicates exc

[jira] [Resolved] (SPARK-51371) Change toString to toPrettySQL when building Aliases in ResolveAggregateFunctions

[ https://issues.apache.org/jira/browse/SPARK-51371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51371. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50136 [https://gith

[jira] [Assigned] (SPARK-51371) Change toString to toPrettySQL when building Aliases in ResolveAggregateFunctions

[ https://issues.apache.org/jira/browse/SPARK-51371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-51371: --- Assignee: Mihailo Aleksic > Change toString to toPrettySQL when building Aliases in > Reso

[jira] [Updated] (SPARK-51400) Replace ArrayContains predicates with literal arguments with InSet predicates

[ https://issues.apache.org/jira/browse/SPARK-51400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51400: --- Labels: pull-request-available (was: ) > Replace ArrayContains predicates with literal argu

[jira] [Created] (SPARK-51401) Make `ExplainUtils.generateFieldString` reuse ` QueryPlan.generateFieldString`

Yang Jie created SPARK-51401: Summary: Make `ExplainUtils.generateFieldString` reuse ` QueryPlan.generateFieldString` Key: SPARK-51401 URL: https://issues.apache.org/jira/browse/SPARK-51401 Project: Spark

[jira] [Updated] (SPARK-51401) Make `ExplainUtils.generateFieldString` reuse ` QueryPlan.generateFieldString`

[ https://issues.apache.org/jira/browse/SPARK-51401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51401: --- Labels: pull-request-available (was: ) > Make `ExplainUtils.generateFieldString` reuse ` Qu

[jira] [Created] (SPARK-51400) Replace ArrayContains predicates with literal arguments with InSet predicates

Adrian Stratulat created SPARK-51400: Summary: Replace ArrayContains predicates with literal arguments with InSet predicates Key: SPARK-51400 URL: https://issues.apache.org/jira/browse/SPARK-51400

[jira] [Created] (SPARK-51399) Empty result for a stream-stream inner join to compute rolling averages

Alex C created SPARK-51399: -- Summary: Empty result for a stream-stream inner join to compute rolling averages Key: SPARK-51399 URL: https://issues.apache.org/jira/browse/SPARK-51399 Project: Spark

[jira] [Created] (SPARK-51398) SparkSQL supports sorting in the shuffle phase.

Chenyu Zheng created SPARK-51398: Summary: SparkSQL supports sorting in the shuffle phase. Key: SPARK-51398 URL: https://issues.apache.org/jira/browse/SPARK-51398 Project: Spark Issue Type: N

[jira] [Resolved] (SPARK-51388) Improve Cast SQL query context in to_timestamp and UNION

[ https://issues.apache.org/jira/browse/SPARK-51388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51388. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50154 [https://gith

[jira] [Assigned] (SPARK-51388) Improve Cast SQL query context in to_timestamp and UNION

[ https://issues.apache.org/jira/browse/SPARK-51388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-51388: --- Assignee: Vladimir Golubev > Improve Cast SQL query context in to_timestamp and UNION > ---

[jira] [Assigned] (SPARK-51394) Optimize out the additional shuffle in stats tests

[ https://issues.apache.org/jira/browse/SPARK-51394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-51394: - Assignee: Ruifeng Zheng > Optimize out the additional shuffle in stats tests >

[jira] [Resolved] (SPARK-51394) Optimize out the additional shuffle in stats tests

[ https://issues.apache.org/jira/browse/SPARK-51394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-51394. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50166 [https://