[jira] [Commented] (SPARK-42209) Introduce test tag ExtendedConnectTest

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681218#comment-17681218 ] Apache Spark commented on SPARK-42209: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-42209) Introduce test tag ExtendedConnectTest

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42209: Assignee: Apache Spark > Introduce test tag ExtendedConnectTest >

[jira] [Created] (SPARK-42210) Standardize registered pickled Python UDFs

2023-01-26 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42210: Summary: Standardize registered pickled Python UDFs Key: SPARK-42210 URL: https://issues.apache.org/jira/browse/SPARK-42210 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-42209) Introduce test tag ExtendedConnectTest

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42209: Assignee: (was: Apache Spark) > Introduce test tag ExtendedConnectTest >

[jira] [Commented] (SPARK-42209) Introduce test tag ExtendedConnectTest

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681217#comment-17681217 ] Apache Spark commented on SPARK-42209: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-42209) Introduce test tag ExtendedConnectTest

2023-01-26 Thread Yang Jie (Jira)
Yang Jie created SPARK-42209: Summary: Introduce test tag ExtendedConnectTest Key: SPARK-42209 URL: https://issues.apache.org/jira/browse/SPARK-42209 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-42206) Omit "Task Executor Metrics" field in JsonProtocol output if values are all zero

2023-01-26 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-42206: -- Assignee: Josh Rosen > Omit "Task Executor Metrics" field in JsonProtocol output if values

[jira] [Created] (SPARK-42208) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-26 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42208: Summary: Reuse UDF test cases under `pyspark.sql.tests` Key: SPARK-42208 URL: https://issues.apache.org/jira/browse/SPARK-42208 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42205) Remove logging of Accumulables in Task/Stage start events in JsonProtocol

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681216#comment-17681216 ] Apache Spark commented on SPARK-42205: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Resolved] (SPARK-41662) Minimal support for pickled Python UDFs

2023-01-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-41662. -- Resolution: Duplicate > Minimal support for pickled Python UDFs >

[jira] [Resolved] (SPARK-42158) Assign name to _LEGACY_ERROR_TEMP_1003

2023-01-26 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-42158. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39706

[jira] [Assigned] (SPARK-42158) Assign name to _LEGACY_ERROR_TEMP_1003

2023-01-26 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42158: Assignee: Haejoon Lee > Assign name to _LEGACY_ERROR_TEMP_1003 >

[jira] [Assigned] (SPARK-42205) Remove logging of Accumulables in Task/Stage start events in JsonProtocol

2023-01-26 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-42205: -- Assignee: Josh Rosen > Remove logging of Accumulables in Task/Stage start events in

[jira] [Commented] (SPARK-41875) Throw proper errors in Dataset.to()

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681186#comment-17681186 ] Apache Spark commented on SPARK-41875: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-41875) Throw proper errors in Dataset.to()

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681185#comment-17681185 ] Apache Spark commented on SPARK-41875: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-41849) Implement DataFrameReader.text

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681183#comment-17681183 ] Apache Spark commented on SPARK-41849: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-41830) Fix DataFrame.sample parameters

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681184#comment-17681184 ] Apache Spark commented on SPARK-41830: -- User 'zhengruifeng' has created a pull request for this

[jira] [Resolved] (SPARK-42207) Upgrade `build_and_test.yml` to `Ubuntu 22.04`

2023-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42207. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39762

[jira] [Assigned] (SPARK-42207) Upgrade `build_and_test.yml` to `Ubuntu 22.04`

2023-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42207: - Assignee: Dongjoon Hyun > Upgrade `build_and_test.yml` to `Ubuntu 22.04` >

[jira] [Assigned] (SPARK-42204) Remove redundant logging of TaskMetrics internal accumulators in JsonProtocol event logs

2023-01-26 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-42204: -- Assignee: Josh Rosen > Remove redundant logging of TaskMetrics internal accumulators in

[jira] [Assigned] (SPARK-42204) Remove redundant logging of TaskMetrics internal accumulators in JsonProtocol event logs

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42204: Assignee: (was: Apache Spark) > Remove redundant logging of TaskMetrics internal

[jira] [Commented] (SPARK-42204) Remove redundant logging of TaskMetrics internal accumulators in JsonProtocol event logs

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681177#comment-17681177 ] Apache Spark commented on SPARK-42204: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42204) Remove redundant logging of TaskMetrics internal accumulators in JsonProtocol event logs

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42204: Assignee: Apache Spark > Remove redundant logging of TaskMetrics internal accumulators

[jira] [Resolved] (SPARK-42200) Enable GA build and test for connect-client-jvm module

2023-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42200. --- Resolution: Duplicate > Enable GA build and test for connect-client-jvm module >

[jira] [Assigned] (SPARK-42207) Upgrade `build_and_test.yml` to `Ubuntu 22.04`

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42207: Assignee: Apache Spark > Upgrade `build_and_test.yml` to `Ubuntu 22.04` >

[jira] [Commented] (SPARK-42207) Upgrade `build_and_test.yml` to `Ubuntu 22.04`

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681168#comment-17681168 ] Apache Spark commented on SPARK-42207: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-42207) Upgrade `build_and_test.yml` to `Ubuntu 22.04`

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42207: Assignee: (was: Apache Spark) > Upgrade `build_and_test.yml` to `Ubuntu 22.04` >

[jira] [Commented] (SPARK-42207) Upgrade `build_and_test.yml` to `Ubuntu 22.04`

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681169#comment-17681169 ] Apache Spark commented on SPARK-42207: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-42207) Upgrade `build_and_test.yml` to `Ubuntu 22.04`

2023-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42207: -- Priority: Minor (was: Major) > Upgrade `build_and_test.yml` to `Ubuntu 22.04` >

[jira] [Created] (SPARK-42207) Upgrade `build_and_test.yml` to `Ubuntu 22.04`

2023-01-26 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-42207: - Summary: Upgrade `build_and_test.yml` to `Ubuntu 22.04` Key: SPARK-42207 URL: https://issues.apache.org/jira/browse/SPARK-42207 Project: Spark Issue Type:

[jira] [Created] (SPARK-42206) Omit "Task Executor Metrics" field in JsonProtocol output if values are all zero

2023-01-26 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-42206: -- Summary: Omit "Task Executor Metrics" field in JsonProtocol output if values are all zero Key: SPARK-42206 URL: https://issues.apache.org/jira/browse/SPARK-42206

[jira] [Assigned] (SPARK-42201) `build/sbt` should allow SBT_OPTS to override JVM memory setting

2023-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42201: - Assignee: Dongjoon Hyun > `build/sbt` should allow SBT_OPTS to override JVM memory

[jira] [Resolved] (SPARK-42201) `build/sbt` should allow SBT_OPTS to override JVM memory setting

2023-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42201. --- Fix Version/s: 3.2.4 3.3.2 3.4.0 Resolution:

[jira] [Created] (SPARK-42205) Remove logging of Accumulables in Task/Stage start events in JsonProtocol

2023-01-26 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-42205: -- Summary: Remove logging of Accumulables in Task/Stage start events in JsonProtocol Key: SPARK-42205 URL: https://issues.apache.org/jira/browse/SPARK-42205 Project: Spark

[jira] [Created] (SPARK-42204) Remove redundant logging of TaskMetrics internal accumulators in JsonProtocol event logs

2023-01-26 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-42204: -- Summary: Remove redundant logging of TaskMetrics internal accumulators in JsonProtocol event logs Key: SPARK-42204 URL: https://issues.apache.org/jira/browse/SPARK-42204

[jira] [Commented] (SPARK-41757) Compatibility of string representation in Column

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681140#comment-17681140 ] Apache Spark commented on SPARK-41757: -- User 'techaddict' has created a pull request for this

[jira] [Commented] (SPARK-42203) JsonProtocol should skip logging of push-based shuffle read metrics when push-based shuffle is disabled

2023-01-26 Thread Thejdeep Gudivada (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681126#comment-17681126 ] Thejdeep Gudivada commented on SPARK-42203: --- Thanks for filing this Josh. I will begin work on

[jira] [Assigned] (SPARK-42202) Scala Client E2E test stop the server gracefully

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42202: Assignee: Apache Spark > Scala Client E2E test stop the server gracefully >

[jira] [Assigned] (SPARK-42202) Scala Client E2E test stop the server gracefully

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42202: Assignee: (was: Apache Spark) > Scala Client E2E test stop the server gracefully >

[jira] [Commented] (SPARK-42202) Scala Client E2E test stop the server gracefully

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681125#comment-17681125 ] Apache Spark commented on SPARK-42202: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Created] (SPARK-42203) JsonProtocol should skip logging of push-based shuffle read metrics when push-based shuffle is disabled

2023-01-26 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-42203: -- Summary: JsonProtocol should skip logging of push-based shuffle read metrics when push-based shuffle is disabled Key: SPARK-42203 URL:

[jira] [Updated] (SPARK-42203) JsonProtocol should skip logging of push-based shuffle read metrics when push-based shuffle is disabled

2023-01-26 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-42203: --- Description: This is a followup to SPARK-36620: When push-based shuffle is disabled (the default),

[jira] [Created] (SPARK-42202) Scala Client E2E test stop the server gracefully

2023-01-26 Thread Zhen Li (Jira)
Zhen Li created SPARK-42202: --- Summary: Scala Client E2E test stop the server gracefully Key: SPARK-42202 URL: https://issues.apache.org/jira/browse/SPARK-42202 Project: Spark Issue Type:

[jira] [Closed] (SPARK-38378) ANTLR grammar definition in separate Parser and Lexer files

2023-01-26 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li closed SPARK-38378. --- > ANTLR grammar definition in separate Parser and Lexer files >

[jira] [Closed] (SPARK-38646) Pull a trait out for Python functions

2023-01-26 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li closed SPARK-38646. --- > Pull a trait out for Python functions > - > > Key:

[jira] [Commented] (SPARK-36124) Support set operators to be on correlation paths

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681120#comment-17681120 ] Apache Spark commented on SPARK-36124: -- User 'jchen5' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42201) `build/sbt` should allow SBT_OPTS to override JVM memory setting

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42201: Assignee: (was: Apache Spark) > `build/sbt` should allow SBT_OPTS to override JVM

[jira] [Assigned] (SPARK-42201) `build/sbt` should allow SBT_OPTS to override JVM memory setting

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42201: Assignee: Apache Spark > `build/sbt` should allow SBT_OPTS to override JVM memory

[jira] [Commented] (SPARK-42201) `build/sbt` should allow SBT_OPTS to override JVM memory setting

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681106#comment-17681106 ] Apache Spark commented on SPARK-42201: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-42201) `build/sbt` should allow SBT_OPTS to override JVM memory setting

2023-01-26 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-42201: - Summary: `build/sbt` should allow SBT_OPTS to override JVM memory setting Key: SPARK-42201 URL: https://issues.apache.org/jira/browse/SPARK-42201 Project: Spark

[jira] [Commented] (SPARK-41585) The Spark exclude node functionality for YARN should work independently of dynamic allocation

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681088#comment-17681088 ] Apache Spark commented on SPARK-41585: -- User 'LucaCanali' has created a pull request for this

[jira] [Commented] (SPARK-41585) The Spark exclude node functionality for YARN should work independently of dynamic allocation

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681087#comment-17681087 ] Apache Spark commented on SPARK-41585: -- User 'LucaCanali' has created a pull request for this

[jira] [Updated] (SPARK-41585) The Spark exclude node functionality for YARN should work independently of dynamic allocation

2023-01-26 Thread Luca Canali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-41585: Affects Version/s: 3.2.2 3.1.3 3.0.3 > The Spark

[jira] [Assigned] (SPARK-42200) Enable GA build and test for connect-client-jvm module

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42200: Assignee: Apache Spark > Enable GA build and test for connect-client-jvm module >

[jira] [Assigned] (SPARK-42200) Enable GA build and test for connect-client-jvm module

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42200: Assignee: (was: Apache Spark) > Enable GA build and test for connect-client-jvm

[jira] [Created] (SPARK-42200) Enable GA build and test for connect-client-jvm module

2023-01-26 Thread Yang Jie (Jira)
Yang Jie created SPARK-42200: Summary: Enable GA build and test for connect-client-jvm module Key: SPARK-42200 URL: https://issues.apache.org/jira/browse/SPARK-42200 Project: Spark Issue Type:

[jira] [Updated] (SPARK-41585) The Spark exclude node functionality for YARN should work independently of dynamic allocation

2023-01-26 Thread Luca Canali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-41585: Description: The Spark exclude node functionality for Spark on YARN, introduced in SPARK-26688,

[jira] [Updated] (SPARK-41585) The Spark exclude node functionality for YARN should work independently of dynamic allocation

2023-01-26 Thread Luca Canali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-41585: Issue Type: Improvement (was: Bug) > The Spark exclude node functionality for YARN should work

[jira] [Updated] (SPARK-41585) The Spark exclude node functionality for YARN should work independently of dynamic allocation

2023-01-26 Thread Luca Canali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-41585: Description: The Spark exclude node functionality for Spark on YARN, introduced in SPARK-26688,

[jira] [Commented] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681054#comment-17681054 ] Apache Spark commented on SPARK-21195: -- User 'robert3005' has created a pull request for this

[jira] [Commented] (SPARK-42173) IPv6 address mapping can fail with sparse addresses

2023-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681051#comment-17681051 ] Dongjoon Hyun commented on SPARK-42173: --- I collected this as a subtask of SPARK-39457 to give more

[jira] [Updated] (SPARK-42173) IPv6 address mapping can fail with sparse addresses

2023-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42173: -- Parent: SPARK-39457 Issue Type: Sub-task (was: Bug) > IPv6 address mapping can fail

[jira] [Resolved] (SPARK-42173) IPv6 address mapping can fail with sparse addresses

2023-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42173. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39728

[jira] [Updated] (SPARK-42132) DeduplicateRelations rule breaks plan when co-grouping the same DataFrame

2023-01-26 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-42132: -- Affects Version/s: 3.5.0 > DeduplicateRelations rule breaks plan when co-grouping the same

[jira] [Commented] (SPARK-42199) groupByKey creates columns that may conflict with exising columns

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681026#comment-17681026 ] Apache Spark commented on SPARK-42199: -- User 'EnricoMi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42199) groupByKey creates columns that may conflict with exising columns

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42199: Assignee: Apache Spark > groupByKey creates columns that may conflict with exising

[jira] [Assigned] (SPARK-42199) groupByKey creates columns that may conflict with exising columns

2023-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42199: Assignee: (was: Apache Spark) > groupByKey creates columns that may conflict with

[jira] [Updated] (SPARK-42199) groupByKey creates columns that may conflict with exising columns

2023-01-26 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-42199: -- Description: Calling {{ds.groupByKey(func: V => K)}} creates columns to store the key value.

[jira] [Created] (SPARK-42199) groupByKey creates columns that may conflict with exising columns

2023-01-26 Thread Enrico Minack (Jira)
Enrico Minack created SPARK-42199: - Summary: groupByKey creates columns that may conflict with exising columns Key: SPARK-42199 URL: https://issues.apache.org/jira/browse/SPARK-42199 Project: Spark

[jira] [Resolved] (SPARK-42195) Add Github action test job for branch-3.4

2023-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42195. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39749

[jira] [Assigned] (SPARK-42195) Add Github action test job for branch-3.4

2023-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42195: Assignee: Yang Jie > Add Github action test job for branch-3.4 >

[jira] [Assigned] (SPARK-42197) Reuses JVM initialization, and separate configuration groups to set in remote local mode

2023-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42197: Assignee: Hyukjin Kwon > Reuses JVM initialization, and separate configuration groups to

[jira] [Resolved] (SPARK-42197) Reuses JVM initialization, and separate configuration groups to set in remote local mode

2023-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42197. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39751

[jira] [Commented] (SPARK-42126) Accept return type in DDL strings for Python Scalar UDFs in Spark Connect

2023-01-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680954#comment-17680954 ] Xinrong Meng commented on SPARK-42126: -- Resolved by https://github.com/apache/spark/pull/39739. >

[jira] [Resolved] (SPARK-42126) Accept return type in DDL strings for Python Scalar UDFs in Spark Connect

2023-01-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42126. -- Resolution: Resolved > Accept return type in DDL strings for Python Scalar UDFs in Spark

[jira] [Assigned] (SPARK-42126) Accept return type in DDL strings for Python Scalar UDFs in Spark Connect

2023-01-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42126: Assignee: Xinrong Meng > Accept return type in DDL strings for Python Scalar UDFs in

[jira] [Updated] (SPARK-42198) spark.read fails to read filenames with accented characters

2023-01-26 Thread Tarique Anwer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tarique Anwer updated SPARK-42198: -- Description: Unable to read filenames with accented characters in the filename. *Sample

[jira] [Updated] (SPARK-42198) spark.read fails to read filenames with accented characters

2023-01-26 Thread Tarique Anwer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tarique Anwer updated SPARK-42198: -- Description: Unable to read filenames with accented characters in the filename. *Sample

[jira] [Updated] (SPARK-42198) spark.read fails to read filenames with accented characters

2023-01-26 Thread Tarique Anwer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tarique Anwer updated SPARK-42198: -- Description: Unable to read filenames with accented characters in the filename. {*}Sample

[jira] [Created] (SPARK-42198) spark.read fails to read filenames with accented characters

2023-01-26 Thread Tarique Anwer (Jira)
Tarique Anwer created SPARK-42198: - Summary: spark.read fails to read filenames with accented characters Key: SPARK-42198 URL: https://issues.apache.org/jira/browse/SPARK-42198 Project: Spark