[jira] [Updated] (SPARK-45274) Implementation of a new DAG drawing approach to avoid fork

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45274: --- Labels: pull-request-available (was: ) > Implementation of a new DAG drawing approach to a

[jira] [Created] (SPARK-45274) Implementation of a new DAG drawing approach to avoid fork

2023-09-21 Thread Kent Yao (Jira)
Kent Yao created SPARK-45274: Summary: Implementation of a new DAG drawing approach to avoid fork Key: SPARK-45274 URL: https://issues.apache.org/jira/browse/SPARK-45274 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-45270) Upgrade `Volcano` to 1.8.0

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45270: - Assignee: Dongjoon Hyun > Upgrade `Volcano` to 1.8.0 > -- > >

[jira] [Resolved] (SPARK-45270) Upgrade `Volcano` to 1.8.0

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45270. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43050 [https://

[jira] [Resolved] (SPARK-45269) Use Java 21-jre in K8s Dockerfile

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45269. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43048 [https://

[jira] [Assigned] (SPARK-45269) Use Java 21-jre in K8s Dockerfile

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45269: - Assignee: Dongjoon Hyun > Use Java 21-jre in K8s Dockerfile > -

[jira] [Created] (SPARK-45273) Http header Attack【HttpSecurityFilter】

2023-09-21 Thread chenyu (Jira)
chenyu created SPARK-45273: -- Summary: Http header Attack【HttpSecurityFilter】 Key: SPARK-45273 URL: https://issues.apache.org/jira/browse/SPARK-45273 Project: Spark Issue Type: Bug Componen

[jira] [Updated] (SPARK-43655) Enable NamespaceParityTests.test_get_index_map

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43655: --- Labels: pull-request-available (was: ) > Enable NamespaceParityTests.test_get_index_map > -

[jira] [Updated] (SPARK-43877) Fix behavior difference for compare binary functions.

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43877: Epic Link: SPARK-39375 > Fix behavior difference for compare binary functions. > -

[jira] [Updated] (SPARK-43877) Fix behavior difference for compare binary functions.

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43877: Parent: (was: SPARK-42497) Issue Type: Improvement (was: Sub-task) > Fix behavior dif

[jira] [Updated] (SPARK-45209) Flame Graph Support For Executor Thread Dump Page

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45209: --- Labels: pull-request-available (was: ) > Flame Graph Support For Executor Thread Dump Page

[jira] [Comment Edited] (SPARK-45227) Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend where an executor process randomly gets stuck

2023-09-21 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767843#comment-17767843 ] Bo Xiong edited comment on SPARK-45227 at 9/22/23 6:12 AM: --- I'

[jira] [Comment Edited] (SPARK-45227) Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend where an executor process randomly gets stuck

2023-09-21 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767843#comment-17767843 ] Bo Xiong edited comment on SPARK-45227 at 9/22/23 6:11 AM: --- I'

[jira] [Commented] (SPARK-45227) Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend where an executor process randomly gets stuck

2023-09-21 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767843#comment-17767843 ] Bo Xiong commented on SPARK-45227: -- I've submitted a fix.  Please help get it merged.

[jira] [Resolved] (SPARK-43623) Enable DefaultIndexParityTests.test_index_distributed_sequence_cleanup.

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43623. - Resolution: Duplicate > Enable DefaultIndexParityTests.test_index_distributed_sequence_cleanup.

[jira] [Updated] (SPARK-45227) Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend where an executor process randomly gets stuck

2023-09-21 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Summary: Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend where an executor process ra

[jira] [Updated] (SPARK-45272) Remove Scala version specific comments, and scala-2.13 profile usage

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45272: --- Labels: pull-request-available (was: ) > Remove Scala version specific comments, and scala-

[jira] [Updated] (SPARK-45269) Use Java 21-jre in K8s Dockerfile

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45269: -- Summary: Use Java 21-jre in K8s Dockerfile (was: Use 21-jre in K8s Dockerfile) > Use Java 21

[jira] [Resolved] (SPARK-43159) Refine `column_op` to use lambda function instead of Column API.

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43159. - Resolution: Won't Fix > Refine `column_op` to use lambda function instead of Column API. > -

[jira] [Created] (SPARK-45272) Remove Scala version specific comments, and scala-2.13 profile usage

2023-09-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45272: Summary: Remove Scala version specific comments, and scala-2.13 profile usage Key: SPARK-45272 URL: https://issues.apache.org/jira/browse/SPARK-45272 Project: Spark

[jira] [Updated] (SPARK-43711) Support `pyspark.ml.feature.Bucketizer` and `pyspark.mllib.stat.KernelDensity` to work with Spark Connect.

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43711: --- Labels: pull-request-available (was: ) > Support `pyspark.ml.feature.Bucketizer` and > `py

[jira] [Updated] (SPARK-45271) Merge _LEGACY_ERROR_TEMP_1113 into TABLE_OPERATION & delete some unused method in QueryCompilationErrors

2023-09-21 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-45271: Summary: Merge _LEGACY_ERROR_TEMP_1113 into TABLE_OPERATION & delete some unused method in QueryCo

[jira] [Created] (SPARK-45271) Merge _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_FEATURE.TABLE_OPERATION & delete some unused method in QueryCompilationErrors

2023-09-21 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-45271: --- Summary: Merge _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_FEATURE.TABLE_OPERATION & delete some unused method in QueryCompilationErrors Key: SPARK-45271 URL: https://issues.apache.or

[jira] [Updated] (SPARK-42965) metadata mismatch for StructField when running some tests.

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42965: Parent: (was: SPARK-42497) Issue Type: Improvement (was: Sub-task) > metadata mismatc

[jira] [Updated] (SPARK-42965) metadata mismatch for StructField when running some tests.

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42965: Epic Link: SPARK-39375 > metadata mismatch for StructField when running some tests. >

[jira] [Updated] (SPARK-45270) Upgrade `Volcano` to 1.8.0

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45270: --- Labels: pull-request-available (was: ) > Upgrade `Volcano` to 1.8.0 > -

[jira] [Created] (SPARK-45270) Upgrade `Volcano` to 1.8.0

2023-09-21 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45270: - Summary: Upgrade `Volcano` to 1.8.0 Key: SPARK-45270 URL: https://issues.apache.org/jira/browse/SPARK-45270 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-45269) Use 21-jre in K8s Dockerfile

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45269: --- Labels: pull-request-available (was: ) > Use 21-jre in K8s Dockerfile > ---

[jira] [Created] (SPARK-45269) Use 21-jre in K8s Dockerfile

2023-09-21 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45269: - Summary: Use 21-jre in K8s Dockerfile Key: SPARK-45269 URL: https://issues.apache.org/jira/browse/SPARK-45269 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-44113) Make Scala 2.13+ as default Scala version

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44113. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43008 [https://

[jira] [Updated] (SPARK-45268) python function categories should be consistent with SQL function groups

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45268: --- Labels: pull-request-available (was: ) > python function categories should be consistent wi

[jira] [Created] (SPARK-45268) python function categories should be consistent with SQL function groups

2023-09-21 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-45268: - Summary: python function categories should be consistent with SQL function groups Key: SPARK-45268 URL: https://issues.apache.org/jira/browse/SPARK-45268 Project: S

[jira] [Resolved] (SPARK-45251) Add client_type field for FetchErrorDetails

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45251. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43031 [https://gi

[jira] [Assigned] (SPARK-45251) Add client_type field for FetchErrorDetails

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45251: Assignee: Yihong He > Add client_type field for FetchErrorDetails > -

[jira] [Assigned] (SPARK-45253) Correct the group of `ShiftLeft` and `ArraySize`

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45253: Assignee: Ruifeng Zheng > Correct the group of `ShiftLeft` and `ArraySize` >

[jira] [Resolved] (SPARK-45253) Correct the group of `ShiftLeft` and `ArraySize`

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45253. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43033 [https://gi

[jira] [Resolved] (SPARK-43433) Match `GroupBy.nth` behavior with new pandas behavior

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43433. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42994 [https://gi

[jira] [Assigned] (SPARK-43433) Match `GroupBy.nth` behavior with new pandas behavior

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43433: Assignee: Haejoon Lee > Match `GroupBy.nth` behavior with new pandas behavior > -

[jira] [Updated] (SPARK-43433) Match `GroupBy.nth` behavior with new pandas behavior

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43433: --- Labels: pull-request-available (was: ) > Match `GroupBy.nth` behavior with new pandas behav

[jira] [Commented] (SPARK-44444) Enabled ANSI mode by default

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767804#comment-17767804 ] Dongjoon Hyun commented on SPARK-4: --- +1 > Enabled ANSI mode by default >

[jira] [Commented] (SPARK-44442) Drop mesos support

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767803#comment-17767803 ] Dongjoon Hyun commented on SPARK-2: --- +1 for the removal. We need some discussi

[jira] [Resolved] (SPARK-45093) AddArtifacts should give proper error messages if it fails

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45093. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42949 [https://gi

[jira] [Assigned] (SPARK-45093) AddArtifacts should give proper error messages if it fails

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45093: Assignee: Alice Sayutina > AddArtifacts should give proper error messages if it fails > -

[jira] [Resolved] (SPARK-45257) Enable spark.eventLog.compress by default

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45257. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43036 [https://

[jira] [Assigned] (SPARK-45257) Enable spark.eventLog.compress by default

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45257: - Assignee: Dongjoon Hyun > Enable spark.eventLog.compress by default > -

[jira] [Resolved] (SPARK-41086) Consolidate SecondArgumentXXX error to INVALID_PARAMETER_VALUE

2023-09-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41086. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Updated] (SPARK-45267) Change the default value for `numeric_only`.

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45267: --- Labels: pull-request-available (was: ) > Change the default value for `numeric_only`. > ---

[jira] [Updated] (SPARK-45267) Change the default value for `numeric_only`.

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-45267: Summary: Change the default value for `numeric_only`. (was: Changed the default value for `numeri

[jira] [Created] (SPARK-45267) Changed the default value for `numeric_only`.

2023-09-21 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-45267: --- Summary: Changed the default value for `numeric_only`. Key: SPARK-45267 URL: https://issues.apache.org/jira/browse/SPARK-45267 Project: Spark Issue Type: Sub-t

[jira] [Assigned] (SPARK-45244) Correct spelling in VolcanoTestsSuite

2023-09-21 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You reassigned SPARK-45244: - Assignee: Binjie Yang > Correct spelling in VolcanoTestsSuite > ---

[jira] [Resolved] (SPARK-45244) Correct spelling in VolcanoTestsSuite

2023-09-21 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You resolved SPARK-45244. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43026 [https://github.c

[jira] [Updated] (SPARK-45266) Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45266: --- Labels: pull-request-available (was: ) > Refactor ResolveFunctions analyzer rule to delay m

[jira] [Resolved] (SPARK-45191) InMemoryTableScanExec simpleStringWithNodeId adds columnar info

2023-09-21 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You resolved SPARK-45191. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42967 [https://github.c

[jira] [Updated] (SPARK-44307) Bloom filter is not added for left outer join if the left side table is smaller than broadcast threshold.

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44307: --- Labels: pull-request-available (was: ) > Bloom filter is not added for left outer join if t

[jira] [Assigned] (SPARK-45163) Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_TABLE_OPERATION and refactor some logic

2023-09-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45163: --- Assignee: BingKun Pan > Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into > UNSUPPORTED

[jira] [Resolved] (SPARK-45163) Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_TABLE_OPERATION and refactor some logic

2023-09-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45163. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42917 [https://gith

[jira] [Created] (SPARK-45266) Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used

2023-09-21 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-45266: - Summary: Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used Key: SPARK-45266 URL: https://issues.apache.org/jira/browse/SPARK-452

[jira] [Comment Edited] (SPARK-45264) Configurable error when generating Python docs

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767764#comment-17767764 ] Haejoon Lee edited comment on SPARK-45264 at 9/22/23 12:28 AM: ---

[jira] [Commented] (SPARK-45264) Configurable error when generating Python docs

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767764#comment-17767764 ] Haejoon Lee commented on SPARK-45264: - Currently the PySpark documentation build req

[jira] [Updated] (SPARK-43152) User-defined output metadata path (_spark_metadata)

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43152: --- Labels: pull-request-available (was: ) > User-defined output metadata path (_spark_metadata

[jira] [Commented] (SPARK-44838) Enhance raise_error() to exploit the new error framework

2023-09-21 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767762#comment-17767762 ] Hudson commented on SPARK-44838: User 'srielau' has created a pull request for this issu

[jira] [Created] (SPARK-45265) Support Hive 4.0 metastore

2023-09-21 Thread Attila Zsolt Piros (Jira)
Attila Zsolt Piros created SPARK-45265: -- Summary: Support Hive 4.0 metastore Key: SPARK-45265 URL: https://issues.apache.org/jira/browse/SPARK-45265 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-45264) Configurable error when generating Python docs

2023-09-21 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767758#comment-17767758 ] Ruifeng Zheng commented on SPARK-45264: --- I think we already support 2.1.0? [~ithol

[jira] [Updated] (SPARK-45262) Improve the description for `LIKE`

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45262: --- Labels: pull-request-available (was: ) > Improve the description for `LIKE` > -

[jira] [Updated] (SPARK-44937) Add SSL/TLS support for RPC and Shuffle communications

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44937: --- Labels: pull-request-available (was: ) > Add SSL/TLS support for RPC and Shuffle communicat

[jira] [Assigned] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45252: - Assignee: Yang Jie > `sbt doc` execution failed. > --- > >

[jira] [Resolved] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45252. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43032 [https://

[jira] [Resolved] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45263. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43040 [https://

[jira] [Assigned] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45263: - Assignee: Dongjoon Hyun > Make EventLoggingListenerSuite independent from spark.eventLo

[jira] [Commented] (SPARK-45264) Configurable error when generating Python docs

2023-09-21 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767743#comment-17767743 ] Allison Wang commented on SPARK-45264: -- [~podongfeng] do we have ways to bypass suc

[jira] [Created] (SPARK-45264) Configurable error when generating Python docs

2023-09-21 Thread Allison Wang (Jira)
Allison Wang created SPARK-45264: Summary: Configurable error when generating Python docs Key: SPARK-45264 URL: https://issues.apache.org/jira/browse/SPARK-45264 Project: Spark Issue Type: Su

[jira] [Updated] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45263: -- Affects Version/s: (was: 3.5.0) (was: 3.4.1) > Make EventLoggin

[jira] [Updated] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45263: -- Affects Version/s: 3.5.0 > Make EventLoggingListenerSuite independent from spark.eventLog.comp

[jira] [Assigned] (SPARK-45261) Fix EventLogFileWriters to handle `none` codec case

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45261: - Assignee: Dongjoon Hyun > Fix EventLogFileWriters to handle `none` codec case > ---

[jira] [Resolved] (SPARK-45261) Fix EventLogFileWriters to handle `none` codec case

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45261. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43038 [https://

[jira] [Updated] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45263: --- Labels: pull-request-available (was: ) > Make EventLoggingListenerSuite independent from sp

[jira] [Created] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45263: - Summary: Make EventLoggingListenerSuite independent from spark.eventLog.compress conf Key: SPARK-45263 URL: https://issues.apache.org/jira/browse/SPARK-45263 Projec

[jira] [Updated] (SPARK-45220) Refine docstring of `DataFrame.join`

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45220: --- Labels: pull-request-available (was: ) > Refine docstring of `DataFrame.join` > ---

[jira] [Created] (SPARK-45262) Improve the description for `LIKE`

2023-09-21 Thread Max Gekk (Jira)
Max Gekk created SPARK-45262: Summary: Improve the description for `LIKE` Key: SPARK-45262 URL: https://issues.apache.org/jira/browse/SPARK-45262 Project: Spark Issue Type: Documentation

[jira] [Updated] (SPARK-45261) Fix EventLogFileWriters to handle `none` codec case

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45261: --- Labels: pull-request-available (was: ) > Fix EventLogFileWriters to handle `none` codec cas

[jira] [Created] (SPARK-45261) Fix EventLogFileWriters to handle `none` codec case

2023-09-21 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45261: - Summary: Fix EventLogFileWriters to handle `none` codec case Key: SPARK-45261 URL: https://issues.apache.org/jira/browse/SPARK-45261 Project: Spark Issue T

[jira] [Created] (SPARK-45260) Refine docstring of count_distinct

2023-09-21 Thread Allison Wang (Jira)
Allison Wang created SPARK-45260: Summary: Refine docstring of count_distinct Key: SPARK-45260 URL: https://issues.apache.org/jira/browse/SPARK-45260 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45259) Refine docstring of `count`

2023-09-21 Thread Allison Wang (Jira)
Allison Wang created SPARK-45259: Summary: Refine docstring of `count` Key: SPARK-45259 URL: https://issues.apache.org/jira/browse/SPARK-45259 Project: Spark Issue Type: Sub-task Co

[jira] [Created] (SPARK-45258) Refine docstring of `sum`

2023-09-21 Thread Allison Wang (Jira)
Allison Wang created SPARK-45258: Summary: Refine docstring of `sum` Key: SPARK-45258 URL: https://issues.apache.org/jira/browse/SPARK-45258 Project: Spark Issue Type: Sub-task Comp

[jira] [Updated] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45255: --- Description: java 1.8, sbt 1.9, scala 2.12   I have a very simple repo with the following dependen

[jira] [Updated] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45255: --- Description: java 1.8, sbt 1.9, scala 2.12   I have a very simple repo with the following dependen

[jira] [Assigned] (SPARK-44113) Make Scala 2.13+ as default Scala version

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44113: - Assignee: Yang Jie > Make Scala 2.13+ as default Scala version > --

[jira] [Updated] (SPARK-45257) Enable spark.eventLog.compress by default

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45257: --- Labels: pull-request-available (was: ) > Enable spark.eventLog.compress by default > --

[jira] [Created] (SPARK-45257) Enable spark.eventLog.compress by default

2023-09-21 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45257: - Summary: Enable spark.eventLog.compress by default Key: SPARK-45257 URL: https://issues.apache.org/jira/browse/SPARK-45257 Project: Spark Issue Type: Sub-t

[jira] [Updated] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45255: --- Issue Type: Bug (was: New Feature) > Spark connect client failing with java.lang.NoClassDefFoundErr

[jira] [Comment Edited] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-09-21 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767622#comment-17767622 ] Sebastian Daberdaku edited comment on SPARK-45201 at 9/21/23 4:59 PM:

[jira] [Updated] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-09-21 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Daberdaku updated SPARK-45201: Attachment: spark-3.5.0.patch > NoClassDefFoundError: InternalFutureFailureAccess

[jira] [Comment Edited] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-09-21 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767622#comment-17767622 ] Sebastian Daberdaku edited comment on SPARK-45201 at 9/21/23 4:57 PM:

[jira] [Comment Edited] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-09-21 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767622#comment-17767622 ] Sebastian Daberdaku edited comment on SPARK-45201 at 9/21/23 4:56 PM:

[jira] [Updated] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45255: --- Description: I have a very simple repo with the following dependency in `build.sbt` ``` {{libraryD

[jira] [Commented] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-09-21 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767622#comment-17767622 ] Sebastian Daberdaku commented on SPARK-45201: - After spending hours analyzin

[jira] [Updated] (SPARK-45256) Arrow DurationWriter fails when vector is at capacity

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45256: --- Labels: pull-request-available (was: ) > Arrow DurationWriter fails when vector is at capac

[jira] [Created] (SPARK-45256) Arrow DurationWriter fails when vector is at capacity

2023-09-21 Thread Sander Goos (Jira)
Sander Goos created SPARK-45256: --- Summary: Arrow DurationWriter fails when vector is at capacity Key: SPARK-45256 URL: https://issues.apache.org/jira/browse/SPARK-45256 Project: Spark Issue Typ

[jira] [Updated] (SPARK-45240) Implement Error Enrichment for Python Client

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45240: --- Labels: pull-request-available (was: ) > Implement Error Enrichment for Python Client > ---

[jira] [Updated] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45255: --- Description: I have a very simple repo with the following dependency in `build.sbt` ``` {{libraryD

  1   2   >