[jira] [Updated] (SPARK-49056) ErrorClassesJsonReader cannot handle null properly

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49056: --- Labels: pull-request-available (was: ) > ErrorClassesJsonReader cannot handle null properly

[jira] [Comment Edited] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869533#comment-17869533 ] Dongjoon Hyun edited comment on SPARK-49055 at 7/30/24 5:46 AM: --

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869533#comment-17869533 ] Dongjoon Hyun commented on SPARK-49055: --- Please see the discuss and conclusion on

[jira] [Closed] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-49055. - > Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS > environment > ---

[jira] [Resolved] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49055. --- Resolution: Not A Problem After investigating with [~LuciferYang], we concluded that there i

[jira] [Created] (SPARK-49056) ErrorClassesJsonReader cannot handle null properly

2024-07-29 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-49056: --- Summary: ErrorClassesJsonReader cannot handle null properly Key: SPARK-49056 URL: https://issues.apache.org/jira/browse/SPARK-49056 Project: Spark Issue Type:

[jira] [Updated] (SPARK-48829) Upgrade `RoaringBitmap` to 1.2.1

2024-07-29 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-48829: - Summary: Upgrade `RoaringBitmap` to 1.2.1 (was: Upgrade `RoaringBitmap` to 1.2.0) > Upgrade `Roarin

[jira] [Resolved] (SPARK-48829) Upgrade `RoaringBitmap` to 1.2.0

2024-07-29 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-48829. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47247 [https://github.com

[jira] [Assigned] (SPARK-49032) Add schema path to metadata table entry within operator metadata

2024-07-29 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-49032: Assignee: Anish Shrigondekar > Add schema path to metadata table entry within operator me

[jira] [Resolved] (SPARK-49032) Add schema path to metadata table entry within operator metadata

2024-07-29 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-49032. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47510 [https://gi

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869525#comment-17869525 ] Dongjoon Hyun commented on SPARK-49055: --- There is no functional change in Hadoop c

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869524#comment-17869524 ] Yang Jie commented on SPARK-49055: -- Yes, this did not cause the tests to fail, so what

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869523#comment-17869523 ] Dongjoon Hyun commented on SPARK-49055: --- To [~LuciferYang], this is a false alarm.

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869522#comment-17869522 ] Dongjoon Hyun commented on SPARK-49055: --- This seems to be a new feature of Hadoop

[jira] [Updated] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Environment: (was: MacOS on AppleSilicon) > Investigate OrcEncryptionSuite UnsatisfiedLink

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869521#comment-17869521 ] Dongjoon Hyun commented on SPARK-49055: --- Here is the Hadoop code. - https://githu

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869519#comment-17869519 ] Dongjoon Hyun commented on SPARK-49055: --- Ack. BTW, this is not a failure, isn't it

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869520#comment-17869520 ] Dongjoon Hyun commented on SPARK-49055: --- Does it mean any functional failure in Ha

[jira] [Updated] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Summary: Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

[jira] [Comment Edited] (SPARK-49055) Fix OrcEncryptionSuite failure on AppleSilicon MacOS environment

2024-07-29 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869518#comment-17869518 ] Yang Jie edited comment on SPARK-49055 at 7/30/24 4:59 AM: --- I

[jira] [Updated] (SPARK-49055) Fix OrcEncryptionSuite failure on Mac environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Environment: MacOS on AppleSilicon (was: MacOS) > Fix OrcEncryptionSuite failure on Mac envir

[jira] [Commented] (SPARK-49055) Fix OrcEncryptionSuite failure on AppleSilicon MacOS environment

2024-07-29 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869518#comment-17869518 ] Yang Jie commented on SPARK-49055: -- I apologize for providing misleading information. I

[jira] [Updated] (SPARK-49055) Fix OrcEncryptionSuite failure on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Summary: Fix OrcEncryptionSuite failure on AppleSilicon MacOS environment (was: Fix OrcEncryp

[jira] [Updated] (SPARK-49055) Fix OrcEncryptionSuite failure on Mac environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Environment: MacOS > Fix OrcEncryptionSuite failure on Mac environment > -

[jira] [Updated] (SPARK-49055) Fix OrcEncryptionSuite failure on Mac environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Reporter: Yang Jie (was: Dongjoon Hyun) > Fix OrcEncryptionSuite failure on Mac environment >

[jira] [Created] (SPARK-49055) Fix OrcEncryptionSuite failure on Mac environment

2024-07-29 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-49055: - Summary: Fix OrcEncryptionSuite failure on Mac environment Key: SPARK-49055 URL: https://issues.apache.org/jira/browse/SPARK-49055 Project: Spark Issue Typ

[jira] [Updated] (SPARK-46058) Add separate flag for privateKeyPassword

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46058: -- Summary: Add separate flag for privateKeyPassword (was: [CORE] Add separate flag for privateK

[jira] [Updated] (SPARK-45730) Make ReloadingX509TrustManagerSuite less flaky

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45730: -- Summary: Make ReloadingX509TrustManagerSuite less flaky (was: [CORE] Make ReloadingX509TrustM

[jira] [Updated] (SPARK-45544) Integrate SSL support into TransportContext

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45544: -- Summary: Integrate SSL support into TransportContext (was: [CORE] Integrate SSL support into

[jira] [Updated] (SPARK-46132) Support key password for JKS keys for RPC SSL

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46132: -- Summary: Support key password for JKS keys for RPC SSL (was: [CORE] Support key password for

[jira] [Updated] (SPARK-45541) Add SSLFactory

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45541: -- Summary: Add SSLFactory (was: [CORE] Add SSLFactory) > Add SSLFactory > -- > >

[jira] [Updated] (SPARK-45431) Document SSL RPC feature

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45431: -- Summary: Document SSL RPC feature (was: [DOCS] Document SSL RPC feature) > Document SSL RPC f

[jira] [Updated] (SPARK-45429) Add helper classes for RPC SSL communication

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45429: -- Summary: Add helper classes for RPC SSL communication (was: [CORE] Add helper classes for RPC

[jira] [Updated] (SPARK-45464) Fix yarn distribution build

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45464: -- Summary: Fix yarn distribution build (was: [CORE] Fix yarn distribution build) > Fix yarn di

[jira] [Updated] (SPARK-45377) Handle InputStream in NettyLogger

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45377: -- Summary: Handle InputStream in NettyLogger (was: [CORE] Handle InputStream in NettyLogger) >

[jira] [Updated] (SPARK-45408) Add RPC SSL settings to TransportConf

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45408: -- Summary: Add RPC SSL settings to TransportConf (was: [CORE] Add RPC SSL settings to Transport

[jira] [Updated] (SPARK-45378) Add convertToNettyForSsl to ManagedBuffer

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45378: -- Summary: Add convertToNettyForSsl to ManagedBuffer (was: [CORE] Add convertToNettyForSsl to M

[jira] [Updated] (SPARK-45426) Add support for a ReloadingTrustManager

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45426: -- Summary: Add support for a ReloadingTrustManager (was: [CORE] Add support for a ReloadingTrus

[jira] [Updated] (SPARK-45427) Add RPC SSL settings to SSLOptions and SparkTransportConf

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45427: -- Summary: Add RPC SSL settings to SSLOptions and SparkTransportConf (was: [CORE] Add RPC SSL s

[jira] [Updated] (SPARK-45375) Mark connection as timedOut in TransportClient.close

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45375: -- Summary: Mark connection as timedOut in TransportClient.close (was: [CORE] Mark connection as

[jira] [Updated] (SPARK-45374) Add test keys for SSL functionality

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45374: -- Summary: Add test keys for SSL functionality (was: [CORE] Add test keys for SSL functionality

[jira] [Resolved] (SPARK-48989) WholeStageCodeGen error resulting in NumberFormatException when calling SUBSTRING_INDEX

2024-07-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48989. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47481 [https://gith

[jira] [Updated] (SPARK-45911) Make TLS1.3 the default for RPC SSL support

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45911: -- Epic Link: SPARK-44937 > Make TLS1.3 the default for RPC SSL support > ---

[jira] [Updated] (SPARK-45911) Make TLS1.3 the default for RPC SSL support

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45911: -- Parent: (was: SPARK-44937) Issue Type: Task (was: Sub-task) > Make TLS1.3 the def

[jira] [Updated] (SPARK-49054) Column default value should support current_* functions

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49054: --- Labels: pull-request-available (was: ) > Column default value should support current_* func

[jira] [Updated] (SPARK-45911) Make TLS1.3 the default for RPC SSL support

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45911: -- Parent Issue: SPARK-44937 (was: SPARK-44111) > Make TLS1.3 the default for RPC SSL support >

[jira] [Created] (SPARK-49054) Column default value should support current_* functions

2024-07-29 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-49054: -- Summary: Column default value should support current_* functions Key: SPARK-49054 URL: https://issues.apache.org/jira/browse/SPARK-49054 Project: Spark I

[jira] [Updated] (SPARK-45911) Make TLS1.3 the default for RPC SSL support

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45911: -- Epic Link: (was: SPARK-44937) > Make TLS1.3 the default for RPC SSL support >

[jira] [Updated] (SPARK-45911) Make TLS1.3 the default for RPC SSL support

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45911: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Make TLS1.3 the default

[jira] [Updated] (SPARK-45911) Make TLS1.3 the default for RPC SSL support

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45911: -- Summary: Make TLS1.3 the default for RPC SSL support (was: [CORE] Make TLS1.3 the default for

[jira] [Resolved] (SPARK-49049) Document MasterPage custom title conf and REST API server-side env variable replacements

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49049. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47523 [https://

[jira] [Resolved] (SPARK-48608) Spark 3.5: fails to build with value defaultValueNotConstantError is not a member of object org.apache.spark.sql.errors.QueryCompilationErrors

2024-07-29 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48608. -- Fix Version/s: 3.5.2 Resolution: Fixed Issue resolved by https://github.com/apache/spark/pull/4

[jira] [Updated] (SPARK-49048) Support reading operator metadata for passed batchId

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49048: --- Labels: pull-request-available (was: ) > Support reading operator metadata for passed batch

[jira] [Updated] (SPARK-49053) Make model save/load helper functions accept spark session

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49053: --- Labels: pull-request-available (was: ) > Make model save/load helper functions accept spark

[jira] [Created] (SPARK-49053) Make model save/load helper functions accept spark session

2024-07-29 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-49053: - Summary: Make model save/load helper functions accept spark session Key: SPARK-49053 URL: https://issues.apache.org/jira/browse/SPARK-49053 Project: Spark

[jira] [Updated] (SPARK-48710) Incompatibilities with NumPy 2.0

2024-07-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48710: - Fix Version/s: (was: 3.5.2) (was: 3.4.4) > Incompatibilities with Num

[jira] [Assigned] (SPARK-48740) Catch missing window specification error early

2024-07-29 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-48740: -- Assignee: Amanda Liu > Catch missing window specification error early > -

[jira] [Resolved] (SPARK-48740) Catch missing window specification error early

2024-07-29 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-48740. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47129 [https:

[jira] [Updated] (SPARK-49052) Add SparkOperator main entry point class

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49052: --- Labels: pull-request-available (was: ) > Add SparkOperator main entry point class > ---

[jira] [Created] (SPARK-49052) Add SparkOperator main entry point class

2024-07-29 Thread Zhou JIANG (Jira)
Zhou JIANG created SPARK-49052: -- Summary: Add SparkOperator main entry point class Key: SPARK-49052 URL: https://issues.apache.org/jira/browse/SPARK-49052 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-49045) Add docker image build for operator

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49045: --- Labels: pull-request-available (was: ) > Add docker image build for operator >

[jira] [Updated] (SPARK-49051) Provide modifiedAfter and modifiedBefore options when filtering from a stream source

2024-07-29 Thread Jeff Steinmetz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Steinmetz updated SPARK-49051: --- Affects Version/s: 3.5.1 (was: 3.4.3) > Provide modifiedAfter and

[jira] [Created] (SPARK-49051) Provide modifiedAfter and modifiedBefore options when filtering from a stream source

2024-07-29 Thread Jeff Steinmetz (Jira)
Jeff Steinmetz created SPARK-49051: -- Summary: Provide modifiedAfter and modifiedBefore options when filtering from a stream source Key: SPARK-49051 URL: https://issues.apache.org/jira/browse/SPARK-49051

[jira] [Updated] (SPARK-49050) Integrate TransformWithState operator with Virtual Column Families

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49050: --- Labels: pull-request-available (was: ) > Integrate TransformWithState operator with Virtual

[jira] [Created] (SPARK-49050) Integrate TransformWithState operator with Virtual Column Families

2024-07-29 Thread Eric Marnadi (Jira)
Eric Marnadi created SPARK-49050: Summary: Integrate TransformWithState operator with Virtual Column Families Key: SPARK-49050 URL: https://issues.apache.org/jira/browse/SPARK-49050 Project: Spark

[jira] [Assigned] (SPARK-49049) Document MasterPage custom title conf and REST API server-side env variable replacements

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49049: - Assignee: Dongjoon Hyun > Document MasterPage custom title conf and REST API server-sid

[jira] [Updated] (SPARK-49049) Document MasterPage custom title conf and REST API server-side env variable replacements

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49049: --- Labels: pull-request-available (was: ) > Document MasterPage custom title conf and REST API

[jira] [Created] (SPARK-49049) Document MasterPage custom title conf and REST API server-side env variable replacements

2024-07-29 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-49049: - Summary: Document MasterPage custom title conf and REST API server-side env variable replacements Key: SPARK-49049 URL: https://issues.apache.org/jira/browse/SPARK-49049

[jira] [Updated] (SPARK-48740) Catch missing window specification error early

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48740: --- Labels: pull-request-available (was: ) > Catch missing window specification error early > -

[jira] [Assigned] (SPARK-49041) Raise proper error for dropDuplicates when wrong subset is given

2024-07-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49041: Assignee: Haejoon Lee > Raise proper error for dropDuplicates when wrong subset is given

[jira] [Resolved] (SPARK-49041) Raise proper error for dropDuplicates when wrong subset is given

2024-07-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49041. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47518 [https://gi

[jira] [Created] (SPARK-49048) Support reading operator metadata for passed batchId

2024-07-29 Thread Anish Shrigondekar (Jira)
Anish Shrigondekar created SPARK-49048: -- Summary: Support reading operator metadata for passed batchId Key: SPARK-49048 URL: https://issues.apache.org/jira/browse/SPARK-49048 Project: Spark

[jira] [Updated] (SPARK-49047) Hide data in LocalRelation message

2024-07-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-49047: - Summary: Hide data in LocalRelation message (was: Hide data in LocalRelation) > Hide data in L

[jira] [Created] (SPARK-49047) Hide data in LocalRelation

2024-07-29 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-49047: Summary: Hide data in LocalRelation Key: SPARK-49047 URL: https://issues.apache.org/jira/browse/SPARK-49047 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-49046) Run new batches only if timers are registered

2024-07-29 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869422#comment-17869422 ] Anish Shrigondekar commented on SPARK-49046: With processing time, we always

[jira] [Comment Edited] (SPARK-49046) Run new batches only if timers are registered

2024-07-29 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869422#comment-17869422 ] Anish Shrigondekar edited comment on SPARK-49046 at 7/29/24 6:12 PM: -

[jira] [Created] (SPARK-49046) Run new batches only if timers are registered

2024-07-29 Thread Anish Shrigondekar (Jira)
Anish Shrigondekar created SPARK-49046: -- Summary: Run new batches only if timers are registered Key: SPARK-49046 URL: https://issues.apache.org/jira/browse/SPARK-49046 Project: Spark Iss

[jira] [Created] (SPARK-49045) Add docker image build for operator

2024-07-29 Thread Zhou JIANG (Jira)
Zhou JIANG created SPARK-49045: -- Summary: Add docker image build for operator Key: SPARK-49045 URL: https://issues.apache.org/jira/browse/SPARK-49045 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-49030) Self join of a CTE seems non-deterministic

2024-07-29 Thread Gera Shegalov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869407#comment-17869407 ] Gera Shegalov commented on SPARK-49030: --- Hi [~jihoonson], can we generalize this i

[jira] [Updated] (SPARK-49044) Improve error message in ValidateExternalType

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49044: --- Labels: pull-request-available (was: ) > Improve error message in ValidateExternalType > --

[jira] [Created] (SPARK-49044) Improve error message in ValidateExternalType

2024-07-29 Thread Mark Andreev (Jira)
Mark Andreev created SPARK-49044: Summary: Improve error message in ValidateExternalType Key: SPARK-49044 URL: https://issues.apache.org/jira/browse/SPARK-49044 Project: Spark Issue Type: Imp

[jira] [Updated] (SPARK-49043) Fix group by on map containing collated strings

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49043: --- Labels: pull-request-available (was: ) > Fix group by on map containing collated strings >

[jira] [Created] (SPARK-49043) Fix group by on map containing collated strings

2024-07-29 Thread Marko Ilic (Jira)
Marko Ilic created SPARK-49043: -- Summary: Fix group by on map containing collated strings Key: SPARK-49043 URL: https://issues.apache.org/jira/browse/SPARK-49043 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-49036) Exclude `JUnitAssertionsShouldIncludeMessage/JUnitTestContainsTooManyAsserts` PMD rules and simplify test code

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49036. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Resolved] (SPARK-49040) Fix doc `sql-ref-syntax-aux-exec-imm.md`

2024-07-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49040. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47517 [https://gi

[jira] [Assigned] (SPARK-49040) Fix doc `sql-ref-syntax-aux-exec-imm.md`

2024-07-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49040: Assignee: BingKun Pan > Fix doc `sql-ref-syntax-aux-exec-imm.md` > --

[jira] [Resolved] (SPARK-48910) Slow linear searches in PreprocessTableCreation

2024-07-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48910. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47484 [https://gith

[jira] [Assigned] (SPARK-48910) Slow linear searches in PreprocessTableCreation

2024-07-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48910: --- Assignee: Vladimir Golubev > Slow linear searches in PreprocessTableCreation >

[jira] [Created] (SPARK-49042) CodeGenerator: Error calculating stats of compiled class. java.lang.UnsupportedOperationException: empty.max

2024-07-29 Thread Arnaud Nauwynck (Jira)
Arnaud Nauwynck created SPARK-49042: --- Summary: CodeGenerator: Error calculating stats of compiled class. java.lang.UnsupportedOperationException: empty.max Key: SPARK-49042 URL: https://issues.apache.org/jira/br

[jira] [Updated] (SPARK-49039) Reset checkbox when executor metrics are loaded in the Stages tab

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49039: --- Labels: pull-request-available (was: ) > Reset checkbox when executor metrics are loaded in

[jira] [Commented] (SPARK-49000) Aggregation with DISTINCT gives wrong results when dealing with literals

2024-07-29 Thread GridGain Integration (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869348#comment-17869348 ] GridGain Integration commented on SPARK-49000: -- User 'uros-db' has created

[jira] [Updated] (SPARK-49041) Raise proper error for dropDuplicates when wrong subset is given

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49041: --- Labels: pull-request-available (was: ) > Raise proper error for dropDuplicates when wrong s

[jira] [Created] (SPARK-49041) Raise proper error for dropDuplicates when wrong subset is given

2024-07-29 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-49041: --- Summary: Raise proper error for dropDuplicates when wrong subset is given Key: SPARK-49041 URL: https://issues.apache.org/jira/browse/SPARK-49041 Project: Spark

[jira] [Created] (SPARK-49040) Fix doc `sql-ref-syntax-aux-exec-imm.md`

2024-07-29 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-49040: --- Summary: Fix doc `sql-ref-syntax-aux-exec-imm.md` Key: SPARK-49040 URL: https://issues.apache.org/jira/browse/SPARK-49040 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-49040) Fix doc `sql-ref-syntax-aux-exec-imm.md`

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49040: --- Labels: pull-request-available (was: ) > Fix doc `sql-ref-syntax-aux-exec-imm.md` > ---

[jira] [Created] (SPARK-49039) Reset checkbox when executor metrics are loaded in the Stages tab

2024-07-29 Thread dzcxzl (Jira)
dzcxzl created SPARK-49039: -- Summary: Reset checkbox when executor metrics are loaded in the Stages tab Key: SPARK-49039 URL: https://issues.apache.org/jira/browse/SPARK-49039 Project: Spark Issue

[jira] [Updated] (SPARK-49038) Spark UI is showing wrong timing metrics for min and median time.

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49038: --- Labels: pull-request-available (was: ) > Spark UI is showing wrong timing metrics for min a

[jira] [Assigned] (SPARK-49016) Spark DataSet.isEmpty behaviour is different on CSV than JSON

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49016: -- Assignee: (was: Apache Spark) > Spark DataSet.isEmpty behaviour is different on C

[jira] [Assigned] (SPARK-49016) Spark DataSet.isEmpty behaviour is different on CSV than JSON

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49016: -- Assignee: Apache Spark > Spark DataSet.isEmpty behaviour is different on CSV than JSO

[jira] [Assigned] (SPARK-49016) Spark DataSet.isEmpty behaviour is different on CSV than JSON

2024-07-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49016: -- Assignee: (was: Apache Spark) > Spark DataSet.isEmpty behaviour is different on C

  1   2   >