[jira] [Updated] (SPARK-45272) Remove Scala version specific comments, and scala-2.13 profile usage

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45272: --- Labels: pull-request-available (was: ) > Remove Scala version specific comments, and

[jira] [Updated] (SPARK-45269) Use Java 21-jre in K8s Dockerfile

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45269: -- Summary: Use Java 21-jre in K8s Dockerfile (was: Use 21-jre in K8s Dockerfile) > Use Java

[jira] [Resolved] (SPARK-43159) Refine `column_op` to use lambda function instead of Column API.

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43159. - Resolution: Won't Fix > Refine `column_op` to use lambda function instead of Column API. >

[jira] [Created] (SPARK-45272) Remove Scala version specific comments, and scala-2.13 profile usage

2023-09-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45272: Summary: Remove Scala version specific comments, and scala-2.13 profile usage Key: SPARK-45272 URL: https://issues.apache.org/jira/browse/SPARK-45272 Project: Spark

[jira] [Updated] (SPARK-43711) Support `pyspark.ml.feature.Bucketizer` and `pyspark.mllib.stat.KernelDensity` to work with Spark Connect.

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43711: --- Labels: pull-request-available (was: ) > Support `pyspark.ml.feature.Bucketizer` and >

[jira] [Updated] (SPARK-45271) Merge _LEGACY_ERROR_TEMP_1113 into TABLE_OPERATION & delete some unused method in QueryCompilationErrors

2023-09-21 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-45271: Summary: Merge _LEGACY_ERROR_TEMP_1113 into TABLE_OPERATION & delete some unused method in

[jira] [Created] (SPARK-45271) Merge _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_FEATURE.TABLE_OPERATION & delete some unused method in QueryCompilationErrors

2023-09-21 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-45271: --- Summary: Merge _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_FEATURE.TABLE_OPERATION & delete some unused method in QueryCompilationErrors Key: SPARK-45271 URL:

[jira] [Updated] (SPARK-42965) metadata mismatch for StructField when running some tests.

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42965: Parent: (was: SPARK-42497) Issue Type: Improvement (was: Sub-task) > metadata

[jira] [Updated] (SPARK-42965) metadata mismatch for StructField when running some tests.

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42965: Epic Link: SPARK-39375 > metadata mismatch for StructField when running some tests. >

[jira] [Updated] (SPARK-45270) Upgrade `Volcano` to 1.8.0

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45270: --- Labels: pull-request-available (was: ) > Upgrade `Volcano` to 1.8.0 >

[jira] [Created] (SPARK-45270) Upgrade `Volcano` to 1.8.0

2023-09-21 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45270: - Summary: Upgrade `Volcano` to 1.8.0 Key: SPARK-45270 URL: https://issues.apache.org/jira/browse/SPARK-45270 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-45269) Use 21-jre in K8s Dockerfile

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45269: --- Labels: pull-request-available (was: ) > Use 21-jre in K8s Dockerfile >

[jira] [Created] (SPARK-45269) Use 21-jre in K8s Dockerfile

2023-09-21 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45269: - Summary: Use 21-jre in K8s Dockerfile Key: SPARK-45269 URL: https://issues.apache.org/jira/browse/SPARK-45269 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-44113) Make Scala 2.13+ as default Scala version

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44113. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43008

[jira] [Updated] (SPARK-45268) python function categories should be consistent with SQL function groups

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45268: --- Labels: pull-request-available (was: ) > python function categories should be consistent

[jira] [Created] (SPARK-45268) python function categories should be consistent with SQL function groups

2023-09-21 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-45268: - Summary: python function categories should be consistent with SQL function groups Key: SPARK-45268 URL: https://issues.apache.org/jira/browse/SPARK-45268 Project:

[jira] [Resolved] (SPARK-45251) Add client_type field for FetchErrorDetails

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45251. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43031

[jira] [Assigned] (SPARK-45251) Add client_type field for FetchErrorDetails

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45251: Assignee: Yihong He > Add client_type field for FetchErrorDetails >

[jira] [Assigned] (SPARK-45253) Correct the group of `ShiftLeft` and `ArraySize`

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45253: Assignee: Ruifeng Zheng > Correct the group of `ShiftLeft` and `ArraySize` >

[jira] [Resolved] (SPARK-45253) Correct the group of `ShiftLeft` and `ArraySize`

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45253. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43033

[jira] [Resolved] (SPARK-43433) Match `GroupBy.nth` behavior with new pandas behavior

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43433. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42994

[jira] [Assigned] (SPARK-43433) Match `GroupBy.nth` behavior with new pandas behavior

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43433: Assignee: Haejoon Lee > Match `GroupBy.nth` behavior with new pandas behavior >

[jira] [Updated] (SPARK-43433) Match `GroupBy.nth` behavior with new pandas behavior

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43433: --- Labels: pull-request-available (was: ) > Match `GroupBy.nth` behavior with new pandas

[jira] [Commented] (SPARK-44444) Enabled ANSI mode by default

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767804#comment-17767804 ] Dongjoon Hyun commented on SPARK-4: --- +1 > Enabled ANSI mode by default >

[jira] [Commented] (SPARK-44442) Drop mesos support

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767803#comment-17767803 ] Dongjoon Hyun commented on SPARK-2: --- +1 for the removal. We need some discussions as the final

[jira] [Resolved] (SPARK-45093) AddArtifacts should give proper error messages if it fails

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45093. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42949

[jira] [Assigned] (SPARK-45093) AddArtifacts should give proper error messages if it fails

2023-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45093: Assignee: Alice Sayutina > AddArtifacts should give proper error messages if it fails >

[jira] [Resolved] (SPARK-45257) Enable spark.eventLog.compress by default

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45257. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43036

[jira] [Assigned] (SPARK-45257) Enable spark.eventLog.compress by default

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45257: - Assignee: Dongjoon Hyun > Enable spark.eventLog.compress by default >

[jira] [Resolved] (SPARK-41086) Consolidate SecondArgumentXXX error to INVALID_PARAMETER_VALUE

2023-09-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41086. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-45267) Change the default value for `numeric_only`.

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45267: --- Labels: pull-request-available (was: ) > Change the default value for `numeric_only`. >

[jira] [Updated] (SPARK-45267) Change the default value for `numeric_only`.

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-45267: Summary: Change the default value for `numeric_only`. (was: Changed the default value for

[jira] [Created] (SPARK-45267) Changed the default value for `numeric_only`.

2023-09-21 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-45267: --- Summary: Changed the default value for `numeric_only`. Key: SPARK-45267 URL: https://issues.apache.org/jira/browse/SPARK-45267 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-45244) Correct spelling in VolcanoTestsSuite

2023-09-21 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You reassigned SPARK-45244: - Assignee: Binjie Yang > Correct spelling in VolcanoTestsSuite >

[jira] [Resolved] (SPARK-45244) Correct spelling in VolcanoTestsSuite

2023-09-21 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You resolved SPARK-45244. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43026

[jira] [Updated] (SPARK-45266) Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45266: --- Labels: pull-request-available (was: ) > Refactor ResolveFunctions analyzer rule to delay

[jira] [Resolved] (SPARK-45191) InMemoryTableScanExec simpleStringWithNodeId adds columnar info

2023-09-21 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You resolved SPARK-45191. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42967

[jira] [Updated] (SPARK-44307) Bloom filter is not added for left outer join if the left side table is smaller than broadcast threshold.

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44307: --- Labels: pull-request-available (was: ) > Bloom filter is not added for left outer join if

[jira] [Assigned] (SPARK-45163) Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_TABLE_OPERATION and refactor some logic

2023-09-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45163: --- Assignee: BingKun Pan > Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into >

[jira] [Resolved] (SPARK-45163) Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_TABLE_OPERATION and refactor some logic

2023-09-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45163. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42917

[jira] [Created] (SPARK-45266) Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used

2023-09-21 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-45266: - Summary: Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used Key: SPARK-45266 URL:

[jira] [Comment Edited] (SPARK-45264) Configurable error when generating Python docs

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767764#comment-17767764 ] Haejoon Lee edited comment on SPARK-45264 at 9/22/23 12:28 AM: --- Currently

[jira] [Commented] (SPARK-45264) Configurable error when generating Python docs

2023-09-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767764#comment-17767764 ] Haejoon Lee commented on SPARK-45264: - Currently the PySpark documentation build requires installing

[jira] [Updated] (SPARK-43152) User-defined output metadata path (_spark_metadata)

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43152: --- Labels: pull-request-available (was: ) > User-defined output metadata path

[jira] [Commented] (SPARK-44838) Enhance raise_error() to exploit the new error framework

2023-09-21 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767762#comment-17767762 ] Hudson commented on SPARK-44838: User 'srielau' has created a pull request for this issue:

[jira] [Created] (SPARK-45265) Support Hive 4.0 metastore

2023-09-21 Thread Attila Zsolt Piros (Jira)
Attila Zsolt Piros created SPARK-45265: -- Summary: Support Hive 4.0 metastore Key: SPARK-45265 URL: https://issues.apache.org/jira/browse/SPARK-45265 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-45264) Configurable error when generating Python docs

2023-09-21 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767758#comment-17767758 ] Ruifeng Zheng commented on SPARK-45264: --- I think we already support 2.1.0? [~itholic]  >

[jira] [Updated] (SPARK-45262) Improve the description for `LIKE`

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45262: --- Labels: pull-request-available (was: ) > Improve the description for `LIKE` >

[jira] [Updated] (SPARK-44937) Add SSL/TLS support for RPC and Shuffle communications

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44937: --- Labels: pull-request-available (was: ) > Add SSL/TLS support for RPC and Shuffle

[jira] [Assigned] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45252: - Assignee: Yang Jie > `sbt doc` execution failed. > --- > >

[jira] [Resolved] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45252. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43032

[jira] [Resolved] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45263. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43040

[jira] [Assigned] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45263: - Assignee: Dongjoon Hyun > Make EventLoggingListenerSuite independent from

[jira] [Commented] (SPARK-45264) Configurable error when generating Python docs

2023-09-21 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767743#comment-17767743 ] Allison Wang commented on SPARK-45264: -- [~podongfeng] do we have ways to bypass such pandas version

[jira] [Created] (SPARK-45264) Configurable error when generating Python docs

2023-09-21 Thread Allison Wang (Jira)
Allison Wang created SPARK-45264: Summary: Configurable error when generating Python docs Key: SPARK-45264 URL: https://issues.apache.org/jira/browse/SPARK-45264 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45263: -- Affects Version/s: (was: 3.5.0) (was: 3.4.1) > Make

[jira] [Updated] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45263: -- Affects Version/s: 3.5.0 > Make EventLoggingListenerSuite independent from

[jira] [Assigned] (SPARK-45261) Fix EventLogFileWriters to handle `none` codec case

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45261: - Assignee: Dongjoon Hyun > Fix EventLogFileWriters to handle `none` codec case >

[jira] [Resolved] (SPARK-45261) Fix EventLogFileWriters to handle `none` codec case

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45261. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43038

[jira] [Updated] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45263: --- Labels: pull-request-available (was: ) > Make EventLoggingListenerSuite independent from

[jira] [Created] (SPARK-45263) Make EventLoggingListenerSuite independent from spark.eventLog.compress conf

2023-09-21 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45263: - Summary: Make EventLoggingListenerSuite independent from spark.eventLog.compress conf Key: SPARK-45263 URL: https://issues.apache.org/jira/browse/SPARK-45263

[jira] [Updated] (SPARK-45220) Refine docstring of `DataFrame.join`

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45220: --- Labels: pull-request-available (was: ) > Refine docstring of `DataFrame.join` >

[jira] [Created] (SPARK-45262) Improve the description for `LIKE`

2023-09-21 Thread Max Gekk (Jira)
Max Gekk created SPARK-45262: Summary: Improve the description for `LIKE` Key: SPARK-45262 URL: https://issues.apache.org/jira/browse/SPARK-45262 Project: Spark Issue Type: Documentation

[jira] [Updated] (SPARK-45261) Fix EventLogFileWriters to handle `none` codec case

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45261: --- Labels: pull-request-available (was: ) > Fix EventLogFileWriters to handle `none` codec

[jira] [Created] (SPARK-45261) Fix EventLogFileWriters to handle `none` codec case

2023-09-21 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45261: - Summary: Fix EventLogFileWriters to handle `none` codec case Key: SPARK-45261 URL: https://issues.apache.org/jira/browse/SPARK-45261 Project: Spark Issue

[jira] [Created] (SPARK-45260) Refine docstring of count_distinct

2023-09-21 Thread Allison Wang (Jira)
Allison Wang created SPARK-45260: Summary: Refine docstring of count_distinct Key: SPARK-45260 URL: https://issues.apache.org/jira/browse/SPARK-45260 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45259) Refine docstring of `count`

2023-09-21 Thread Allison Wang (Jira)
Allison Wang created SPARK-45259: Summary: Refine docstring of `count` Key: SPARK-45259 URL: https://issues.apache.org/jira/browse/SPARK-45259 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45258) Refine docstring of `sum`

2023-09-21 Thread Allison Wang (Jira)
Allison Wang created SPARK-45258: Summary: Refine docstring of `sum` Key: SPARK-45258 URL: https://issues.apache.org/jira/browse/SPARK-45258 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45255: --- Description: java 1.8, sbt 1.9, scala 2.12   I have a very simple repo with the following

[jira] [Updated] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45255: --- Description: java 1.8, sbt 1.9, scala 2.12   I have a very simple repo with the following

[jira] [Assigned] (SPARK-44113) Make Scala 2.13+ as default Scala version

2023-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44113: - Assignee: Yang Jie > Make Scala 2.13+ as default Scala version >

[jira] [Updated] (SPARK-45257) Enable spark.eventLog.compress by default

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45257: --- Labels: pull-request-available (was: ) > Enable spark.eventLog.compress by default >

[jira] [Created] (SPARK-45257) Enable spark.eventLog.compress by default

2023-09-21 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45257: - Summary: Enable spark.eventLog.compress by default Key: SPARK-45257 URL: https://issues.apache.org/jira/browse/SPARK-45257 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45255: --- Issue Type: Bug (was: New Feature) > Spark connect client failing with

[jira] [Comment Edited] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-09-21 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767622#comment-17767622 ] Sebastian Daberdaku edited comment on SPARK-45201 at 9/21/23 4:59 PM:

[jira] [Updated] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-09-21 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Daberdaku updated SPARK-45201: Attachment: spark-3.5.0.patch > NoClassDefFoundError: InternalFutureFailureAccess

[jira] [Comment Edited] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-09-21 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767622#comment-17767622 ] Sebastian Daberdaku edited comment on SPARK-45201 at 9/21/23 4:57 PM:

[jira] [Comment Edited] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-09-21 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767622#comment-17767622 ] Sebastian Daberdaku edited comment on SPARK-45201 at 9/21/23 4:56 PM:

[jira] [Updated] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45255: --- Description: I have a very simple repo with the following dependency in `build.sbt` ```

[jira] [Commented] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-09-21 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767622#comment-17767622 ] Sebastian Daberdaku commented on SPARK-45201: - After spending hours analyzing the project

[jira] [Updated] (SPARK-45256) Arrow DurationWriter fails when vector is at capacity

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45256: --- Labels: pull-request-available (was: ) > Arrow DurationWriter fails when vector is at

[jira] [Created] (SPARK-45256) Arrow DurationWriter fails when vector is at capacity

2023-09-21 Thread Sander Goos (Jira)
Sander Goos created SPARK-45256: --- Summary: Arrow DurationWriter fails when vector is at capacity Key: SPARK-45256 URL: https://issues.apache.org/jira/browse/SPARK-45256 Project: Spark Issue

[jira] [Updated] (SPARK-45240) Implement Error Enrichment for Python Client

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45240: --- Labels: pull-request-available (was: ) > Implement Error Enrichment for Python Client >

[jira] [Updated] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45255: --- Description: I have a very simple repo with the following dependency in `build.sbt` ```

[jira] [Created] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
Faiz Halde created SPARK-45255: -- Summary: Spark connect client failing with java.lang.NoClassDefFoundError Key: SPARK-45255 URL: https://issues.apache.org/jira/browse/SPARK-45255 Project: Spark

[jira] [Updated] (SPARK-45255) Spark connect client failing with java.lang.NoClassDefFoundError

2023-09-21 Thread Faiz Halde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faiz Halde updated SPARK-45255: --- Description: I have a very simple repo with the following dependency in `build.sbt` ```

[jira] [Created] (SPARK-45254) Non-nullable schema is not effective in DF from JSON

2023-09-21 Thread Michele Rastelli (Jira)
Michele Rastelli created SPARK-45254: Summary: Non-nullable schema is not effective in DF from JSON Key: SPARK-45254 URL: https://issues.apache.org/jira/browse/SPARK-45254 Project: Spark

[jira] [Updated] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45252: - Affects Version/s: 3.3.4 > `sbt doc` execution failed. > --- > >

[jira] [Updated] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45252: - Affects Version/s: 3.4.2 > `sbt doc` execution failed. > --- > >

[jira] [Updated] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45252: - Description: run    {code:java} build/sbt clean doc -Phadoop-3 -Phadoop-cloud -Pmesos -Pyarn

[jira] [Updated] (SPARK-45253) Correct the group of `ShiftLeft` and `ArraySize`

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45253: --- Labels: pull-request-available (was: ) > Correct the group of `ShiftLeft` and `ArraySize`

[jira] [Created] (SPARK-45253) Correct the group of `ShiftLeft` and `ArraySize`

2023-09-21 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-45253: - Summary: Correct the group of `ShiftLeft` and `ArraySize` Key: SPARK-45253 URL: https://issues.apache.org/jira/browse/SPARK-45253 Project: Spark Issue

[jira] [Updated] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45252: --- Labels: pull-request-available (was: ) > `sbt doc` execution failed. >

[jira] [Updated] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45252: - Affects Version/s: 3.5.1 > `sbt doc` execution failed. > --- > >

[jira] [Updated] (SPARK-45251) Add client_type field for FetchErrorDetails

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45251: --- Labels: pull-request-available (was: ) > Add client_type field for FetchErrorDetails >

[jira] [Created] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread Yang Jie (Jira)
Yang Jie created SPARK-45252: Summary: `sbt doc` execution failed. Key: SPARK-45252 URL: https://issues.apache.org/jira/browse/SPARK-45252 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45252: - Issue Type: Bug (was: Improvement) > `sbt doc` execution failed. > --- > >

[jira] [Updated] (SPARK-45252) `sbt doc` execution failed.

2023-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45252: - Priority: Minor (was: Major) > `sbt doc` execution failed. > --- > >

[jira] [Created] (SPARK-45251) Add client_type field for FetchErrorDetails

2023-09-21 Thread Yihong He (Jira)
Yihong He created SPARK-45251: - Summary: Add client_type field for FetchErrorDetails Key: SPARK-45251 URL: https://issues.apache.org/jira/browse/SPARK-45251 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-45200) Spark 3.4.0 always using default log4j profile

2023-09-21 Thread Jitin Dominic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitin Dominic updated SPARK-45200: -- Description: I've been using Spark core 3.2.2 and was upgrading to 3.4.0 On execution of my

  1   2   >