[jira] [Updated] (SPARK-45189) Creating UnresolvedRelation from TableIdentifier should include the catalog field

2023-09-19 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-45189: --- Affects Version/s: 3.5.1 > Creating UnresolvedRelation from TableIdentifier should include

[jira] [Updated] (SPARK-45189) Creating UnresolvedRelation from TableIdentifier should include the catalog field

2023-09-19 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-45189: --- Fix Version/s: 3.5.1 > Creating UnresolvedRelation from TableIdentifier should include the

[jira] [Created] (SPARK-45229) Show the number of drivers waiting in SUBMITTED status

2023-09-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45229: - Summary: Show the number of drivers waiting in SUBMITTED status Key: SPARK-45229 URL: https://issues.apache.org/jira/browse/SPARK-45229 Project: Spark

[jira] [Updated] (SPARK-44112) Drop Java 8 and 11 support

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44112: -- Summary: Drop Java 8 and 11 support (was: Drop Java 8 Support) > Drop Java 8 and 11 support

[jira] [Commented] (SPARK-45225) XML: XSD file URL support

2023-09-19 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766983#comment-17766983 ] Snoot.io commented on SPARK-45225: -- User 'sandip-db' has created a pull request for this issue:

[jira] [Resolved] (SPARK-45225) XML: XSD file URL support

2023-09-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45225. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43000

[jira] [Assigned] (SPARK-45225) XML: XSD file URL support

2023-09-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45225: Assignee: Sandip Agarwala > XML: XSD file URL support > - > >

[jira] [Resolved] (SPARK-44622) Implement FetchErrorDetails RPC

2023-09-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-44622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44622. --- Fix Version/s: 4.0.0 Assignee: Yihong He Resolution: Fixed >

[jira] [Commented] (SPARK-45218) Refine docstring of `Column.isin`

2023-09-19 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766980#comment-17766980 ] Snoot.io commented on SPARK-45218: -- User 'allisonwang-db' has created a pull request for this issue:

[jira] [Commented] (SPARK-44112) Drop Java 8 Support

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766979#comment-17766979 ] Dongjoon Hyun commented on SPARK-44112: --- Never mind because you already made a PR, [~LuciferYang].

[jira] [Commented] (SPARK-45226) Refine docstring of `rand/randn`

2023-09-19 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766977#comment-17766977 ] Snoot.io commented on SPARK-45226: -- User 'panbingkun' has created a pull request for this issue:

[jira] [Commented] (SPARK-45226) Refine docstring of `rand/randn`

2023-09-19 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766975#comment-17766975 ] Snoot.io commented on SPARK-45226: -- User 'panbingkun' has created a pull request for this issue:

[jira] [Resolved] (SPARK-44463) Improve error handling in Connect foreachBatch worker.

2023-09-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44463. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42986

[jira] [Commented] (SPARK-43498) Enable StatsTests.test_axis_on_dataframe for pandas 2.0.0.

2023-09-19 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766972#comment-17766972 ] Snoot.io commented on SPARK-43498: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-43498) Enable StatsTests.test_axis_on_dataframe for pandas 2.0.0.

2023-09-19 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766971#comment-17766971 ] Snoot.io commented on SPARK-43498: -- User 'itholic' has created a pull request for this issue:

[jira] [Updated] (SPARK-45228) Update `test_axis_on_dataframe` when Pandas regression is fixed

2023-09-19 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-45228: Summary: Update `test_axis_on_dataframe` when Pandas regression is fixed (was: Restore

[jira] [Created] (SPARK-45228) Restore `test_axis_on_dataframe` in normal state when Pandas regression is fixed

2023-09-19 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-45228: --- Summary: Restore `test_axis_on_dataframe` in normal state when Pandas regression is fixed Key: SPARK-45228 URL: https://issues.apache.org/jira/browse/SPARK-45228

[jira] [Commented] (SPARK-44112) Drop Java 8 Support

2023-09-19 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766970#comment-17766970 ] Snoot.io commented on SPARK-44112: -- User 'LuciferYang' has created a pull request for this issue:

[jira] [Commented] (SPARK-45134) Data duplication may occur when fallback to origin shuffle block

2023-09-19 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766969#comment-17766969 ] Snoot.io commented on SPARK-45134: -- User 'gaoyajun02' has created a pull request for this issue:

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck, by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Description: h2. Symptom Our Spark 3 app running on EMR 6.10.0 with Spark 3.3.1 got stuck in the very

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck, by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Fix Version/s: (was: 4.0.0) (was: 3.5.1) Target Version/s: (was:

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck, by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Summary: Fix an issue where an executor process randomly gets stuck, by making

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Description: h2. Symptom Our Spark 3 app running on EMR 6.10.0 with Spark 3.3.1 got stuck in the very

[jira] [Comment Edited] (SPARK-44112) Drop Java 8 Support

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766966#comment-17766966 ] Dongjoon Hyun edited comment on SPARK-44112 at 9/20/23 3:03 AM: Oh.

[jira] [Commented] (SPARK-44112) Drop Java 8 Support

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766967#comment-17766967 ] Dongjoon Hyun commented on SPARK-44112: --- My bad! You're right in that part, but please keep Java

[jira] [Commented] (SPARK-44112) Drop Java 8 Support

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766966#comment-17766966 ] Dongjoon Hyun commented on SPARK-44112: --- Oh. Thanks. It seems that I was outdated and missed the

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Description: h3. Symptom Our Spark 3 app running on EMR 6.10.0 with Spark 3.3.1 got stuck in the very

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Attachment: hashtable1.png > Fix an issue where an executor process randomly gets stuck by making >

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Attachment: hashtable2.png > Fix an issue where an executor process randomly gets stuck by making >

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Attachment: (was: Screenshot 2023-09-19 at 7.55.37 PM.png) > Fix an issue where an executor process

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Attachment: (was: Screenshot 2023-09-19 at 7.55.31 PM.png) > Fix an issue where an executor process

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Description: h3. Symptom Our Spark 3 app running on EMR 6.10.0 with Spark 3.3.1 got stuck in the very

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Attachment: Screenshot 2023-09-19 at 7.55.37 PM.png > Fix an issue where an executor process randomly

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Attachment: Screenshot 2023-09-19 at 7.55.31 PM.png > Fix an issue where an executor process randomly

[jira] [Commented] (SPARK-44112) Drop Java 8 Support

2023-09-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766965#comment-17766965 ] Yang Jie commented on SPARK-44112: -- !image-2023-09-20-10-53-34-956.png|width=729,height=161!   I have

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Description: h3. Symptom Our Spark 3 app running on EMR 6.10.0 with Spark 3.3.1 got stuck in the very

[jira] [Updated] (SPARK-44112) Drop Java 8 Support

2023-09-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44112: - Attachment: image-2023-09-20-10-53-34-956.png > Drop Java 8 Support > --- > >

[jira] [Updated] (SPARK-44112) Drop Java 8 Support

2023-09-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44112: - Attachment: image-2023-09-20-10-52-59-327.png > Drop Java 8 Support > --- > >

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Description: h3. Symptom Our Spark 3 app running on EMR 6.10.0 with Spark 3.3.1 got stuck in the very

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Description: h3. Symptom Our Spark 3 app running on EMR 6.10.0 with Spark 3.3.1 got stuck in the very

[jira] [Updated] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Xiong updated SPARK-45227: - Fix Version/s: 4.0.0 Description: h3. Symptom Our Spark 3 app running on EMR 6.10.0 with Spark

[jira] [Created] (SPARK-45227) Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe

2023-09-19 Thread Bo Xiong (Jira)
Bo Xiong created SPARK-45227: Summary: Fix an issue where an executor process randomly gets stuck by making CoarseGrainedExecutorBackend.taskResources thread-safe Key: SPARK-45227 URL:

[jira] [Updated] (SPARK-44112) Drop Java 8 Support

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44112: -- Summary: Drop Java 8 Support (was: Drop Java 8 and Java 11 Support) > Drop Java 8 Support >

[jira] [Commented] (SPARK-44112) Drop Java 8 and Java 11 Support

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766963#comment-17766963 ] Dongjoon Hyun commented on SPARK-44112: --- Sorry, but don't change this JIRA, [~LuciferYang]. If we

[jira] [Updated] (SPARK-44112) Drop Java 8 and Java 11 Support

2023-09-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44112: - Summary: Drop Java 8 and Java 11 Support (was: Drop Java 8 Support) > Drop Java 8 and Java 11 Support

[jira] [Resolved] (SPARK-45178) Fallback to use single batch executor for Trigger.AvailableNow with unsupported sources rather than using wrapper

2023-09-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-45178. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42940

[jira] [Assigned] (SPARK-45178) Fallback to use single batch executor for Trigger.AvailableNow with unsupported sources rather than using wrapper

2023-09-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-45178: Assignee: Jungtaek Lim > Fallback to use single batch executor for Trigger.AvailableNow

[jira] [Resolved] (SPARK-45192) lineInterpolate for graphviz edge is overdue

2023-09-19 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-45192. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42969

[jira] [Assigned] (SPARK-45192) lineInterpolate for graphviz edge is overdue

2023-09-19 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-45192: Assignee: Kent Yao > lineInterpolate for graphviz edge is overdue >

[jira] [Created] (SPARK-45226) Refine docstring of `rand/randn`

2023-09-19 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-45226: --- Summary: Refine docstring of `rand/randn` Key: SPARK-45226 URL: https://issues.apache.org/jira/browse/SPARK-45226 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-43432) Fix `min_periods` for Rolling to work same as pandas

2023-09-19 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43432: Parent: (was: SPARK-44101) Issue Type: Improvement (was: Sub-task) > Fix

[jira] [Updated] (SPARK-45208) Website doesn't have horizontal scrollbar

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45208: -- Summary: Website doesn't have horizontal scrollbar (was: Kubernetes Configuration in Spark

[jira] [Updated] (SPARK-45208) Kubernetes Configuration in Spark Community Website doesn't have horizontal scrollbar

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45208: -- Description: I find a recent issue with the official Spark documentation on the website.

[jira] [Created] (SPARK-45225) XML: XSD file URL support

2023-09-19 Thread Sandip Agarwala (Jira)
Sandip Agarwala created SPARK-45225: --- Summary: XML: XSD file URL support Key: SPARK-45225 URL: https://issues.apache.org/jira/browse/SPARK-45225 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45224) Add examples w/ map and array as parameters of sql()

2023-09-19 Thread Max Gekk (Jira)
Max Gekk created SPARK-45224: Summary: Add examples w/ map and array as parameters of sql() Key: SPARK-45224 URL: https://issues.apache.org/jira/browse/SPARK-45224 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45220) Refine docstring of `DataFrame.join`

2023-09-19 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-45220: - Description: Refine the docstring of `DataFrame.join`. The examples should also include: left

[jira] [Created] (SPARK-45223) Refine docstring of `Column.when`

2023-09-19 Thread Allison Wang (Jira)
Allison Wang created SPARK-45223: Summary: Refine docstring of `Column.when` Key: SPARK-45223 URL: https://issues.apache.org/jira/browse/SPARK-45223 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45222) Refine docstring of `DataFrameReader.json`

2023-09-19 Thread Allison Wang (Jira)
Allison Wang created SPARK-45222: Summary: Refine docstring of `DataFrameReader.json` Key: SPARK-45222 URL: https://issues.apache.org/jira/browse/SPARK-45222 Project: Spark Issue Type:

[jira] [Created] (SPARK-45221) Refine docstring of `DataFrameReader.parquet`

2023-09-19 Thread Allison Wang (Jira)
Allison Wang created SPARK-45221: Summary: Refine docstring of `DataFrameReader.parquet` Key: SPARK-45221 URL: https://issues.apache.org/jira/browse/SPARK-45221 Project: Spark Issue Type:

[jira] [Created] (SPARK-45220) Refine docstring of `DataFrame.join`

2023-09-19 Thread Allison Wang (Jira)
Allison Wang created SPARK-45220: Summary: Refine docstring of `DataFrame.join` Key: SPARK-45220 URL: https://issues.apache.org/jira/browse/SPARK-45220 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45219) Refine docstring of `DataFrame.withColumnRenamed`

2023-09-19 Thread Allison Wang (Jira)
Allison Wang created SPARK-45219: Summary: Refine docstring of `DataFrame.withColumnRenamed` Key: SPARK-45219 URL: https://issues.apache.org/jira/browse/SPARK-45219 Project: Spark Issue

[jira] [Created] (SPARK-45218) Refine docstring of `Column.isin`

2023-09-19 Thread Allison Wang (Jira)
Allison Wang created SPARK-45218: Summary: Refine docstring of `Column.isin` Key: SPARK-45218 URL: https://issues.apache.org/jira/browse/SPARK-45218 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45217) Support change log level of specific package or class

2023-09-19 Thread Zhongwei Zhu (Jira)
Zhongwei Zhu created SPARK-45217: Summary: Support change log level of specific package or class Key: SPARK-45217 URL: https://issues.apache.org/jira/browse/SPARK-45217 Project: Spark Issue

[jira] [Updated] (SPARK-45207) Implement Error Enrichment for Scala Client

2023-09-19 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-45207: -- Summary: Implement Error Enrichment for Scala Client (was: Implement FetchErrorDetails RPC) >

[jira] [Resolved] (SPARK-44306) Group FileStatus with few RPC calls within Yarn Client

2023-09-19 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-44306. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-44306) Group FileStatus with few RPC calls within Yarn Client

2023-09-19 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-44306: --- Assignee: SHU WANG > Group FileStatus with few RPC calls within Yarn

[jira] [Resolved] (SPARK-45215) Combine HiveCatalogedDDLSuite and HiveDDLSuite

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45215. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42992

[jira] [Assigned] (SPARK-45215) Combine HiveCatalogedDDLSuite and HiveDDLSuite

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45215: - Assignee: BingKun Pan > Combine HiveCatalogedDDLSuite and HiveDDLSuite >

[jira] [Resolved] (SPARK-43453) Enable OpsOnDiffFramesEnabledTests.test_concat_column_axis for pandas 2.0.0.

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43453. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42991

[jira] [Assigned] (SPARK-43453) Enable OpsOnDiffFramesEnabledTests.test_concat_column_axis for pandas 2.0.0.

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43453: - Assignee: Haejoon Lee > Enable OpsOnDiffFramesEnabledTests.test_concat_column_axis for

[jira] [Updated] (SPARK-43453) Ignore the names of MultiIndex when axis=1 for concat

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43453: -- Summary: Ignore the names of MultiIndex when axis=1 for concat (was: Enable

[jira] [Commented] (SPARK-38200) [SQL] Spark JDBC Savemode Supports Upsert

2023-09-19 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766881#comment-17766881 ] Enrico Minack commented on SPARK-38200: --- Sadly, still no feedback from reviewers. > [SQL] Spark

[jira] [Updated] (SPARK-45200) Spark 3.4.0 always using default log4j profile

2023-09-19 Thread Jitin Dominic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitin Dominic updated SPARK-45200: -- Flags: Important > Spark 3.4.0 always using default log4j profile >

[jira] [Commented] (SPARK-38200) [SQL] Spark JDBC Savemode Supports Upsert

2023-09-19 Thread Yair Ofek (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766815#comment-17766815 ] Yair Ofek commented on SPARK-38200: --- [~EnricoMi] any news on when this important feature going to be

[jira] [Updated] (SPARK-44622) Implement FetchErrorDetails RPC

2023-09-19 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-44622: -- Summary: Implement FetchErrorDetails RPC (was: Implement error enrichment and JVM stacktrace) >

[jira] [Commented] (SPARK-45183) ExecutorPodsLifecycleManager delete a pod multi times.

2023-09-19 Thread hgs (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766802#comment-17766802 ] hgs commented on SPARK-45183: - I have compared  Spark 3.2.0 with Spark 3.5.0. The deletion of pod is nothing

[jira] [Updated] (SPARK-45216) Fix non-deterministic seeded Dataset APIs

2023-09-19 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-45216: --- Description: If we run the following example the result is the expected equal 2 columns:

[jira] [Created] (SPARK-45216) Fix non-deterministic seeded Dataset APIs

2023-09-19 Thread Peter Toth (Jira)
Peter Toth created SPARK-45216: -- Summary: Fix non-deterministic seeded Dataset APIs Key: SPARK-45216 URL: https://issues.apache.org/jira/browse/SPARK-45216 Project: Spark Issue Type: Bug

[jira] [Reopened] (SPARK-43433) Match `GroupBy.nth` behavior with new pandas behavior

2023-09-19 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee reopened SPARK-43433: - > Match `GroupBy.nth` behavior with new pandas behavior >

[jira] [Created] (SPARK-45215) Combine HiveCatalogedDDLSuite and HiveDDLSuite

2023-09-19 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-45215: --- Summary: Combine HiveCatalogedDDLSuite and HiveDDLSuite Key: SPARK-45215 URL: https://issues.apache.org/jira/browse/SPARK-45215 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-45211) Scala 2.13 daily test failed

2023-09-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-45211: Assignee: Yang Jie > Scala 2.13 daily test failed > - > >

[jira] [Resolved] (SPARK-45211) Scala 2.13 daily test failed

2023-09-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-45211. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Updated] (SPARK-45213) Assign name to _LEGACY_ERROR_TEMP_2151

2023-09-19 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deng Ziming updated SPARK-45213: Description: in DatasetSuite test("CLASS_UNSUPPORTED_BY_MAP_OBJECTS when creating dataset") , we

[jira] [Updated] (SPARK-45213) Assign name to _LEGACY_ERROR_TEMP_2151

2023-09-19 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deng Ziming updated SPARK-45213: Affects Version/s: 3.5.0 (was: 3.4.0) > Assign name to

[jira] [Updated] (SPARK-45213) Assign name to _LEGACY_ERROR_TEMP_2151

2023-09-19 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deng Ziming updated SPARK-45213: Fix Version/s: (was: 3.4.0) > Assign name to _LEGACY_ERROR_TEMP_2151 >

[jira] [Created] (SPARK-45214) Columns should not be visible for filter after projection

2023-09-19 Thread Jakub Wozniak (Jira)
Jakub Wozniak created SPARK-45214: - Summary: Columns should not be visible for filter after projection Key: SPARK-45214 URL: https://issues.apache.org/jira/browse/SPARK-45214 Project: Spark

[jira] [Created] (SPARK-45213) Assign name to _LEGACY_ERROR_TEMP_2151

2023-09-19 Thread Deng Ziming (Jira)
Deng Ziming created SPARK-45213: --- Summary: Assign name to _LEGACY_ERROR_TEMP_2151 Key: SPARK-45213 URL: https://issues.apache.org/jira/browse/SPARK-45213 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-45101) Spark UI: A stage is still active even when all of it's tasks are succeeded

2023-09-19 Thread RickyMa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RickyMa updated SPARK-45101: Priority: Critical (was: Major) > Spark UI: A stage is still active even when all of it's tasks are

[jira] [Comment Edited] (SPARK-43182) Mutilple tables join with limit when AE is enabled and one table is skewed

2023-09-19 Thread Qian Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766690#comment-17766690 ] Qian Sun edited comment on SPARK-43182 at 9/19/23 8:14 AM: --- Hi [~Resol1992] I

[jira] [Commented] (SPARK-43182) Mutilple tables join with limit when AE is enabled and one table is skewed

2023-09-19 Thread Qian Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766690#comment-17766690 ] Qian Sun commented on SPARK-43182: -- Hi [~Resol1992] I ran your sql, tried different configuration

[jira] [Updated] (SPARK-45101) Spark UI: A stage is still active even when all of it's tasks are succeeded

2023-09-19 Thread RickyMa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RickyMa updated SPARK-45101: Affects Version/s: 3.5.0 4.0.0 > Spark UI: A stage is still active even when all

[jira] [Resolved] (SPARK-45212) Install independent Python linter dependencies for branch-3.5

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45212. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42990

[jira] [Assigned] (SPARK-45212) Install independent Python linter dependencies for branch-3.5

2023-09-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45212: - Assignee: Yang Jie > Install independent Python linter dependencies for branch-3.5 >

[jira] [Updated] (SPARK-45200) Spark 3.4.0 always using default log4j profile

2023-09-19 Thread Jitin Dominic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitin Dominic updated SPARK-45200: -- Description: I've been using Spark core 3.2.2 and was upgrading to 3.4.0 On execution of my

[jira] [Updated] (SPARK-45200) Spark 3.4.0 always using default log4j profile

2023-09-19 Thread Jitin Dominic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitin Dominic updated SPARK-45200: -- Description: I've been using Spark core 3.2.2 and was upgrading to 3.4.0   On execution of

[jira] [Updated] (SPARK-45200) Spark 3.4.0 always using default log4j profile

2023-09-19 Thread Jitin Dominic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitin Dominic updated SPARK-45200: -- Summary: Spark 3.4.0 always using default log4j profile (was: Ignore/Disable Spark's default

[jira] [Updated] (SPARK-45200) Ignore/Disable Spark's default log4j profile

2023-09-19 Thread Jitin Dominic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitin Dominic updated SPARK-45200: -- Issue Type: Bug (was: Question) > Ignore/Disable Spark's default log4j profile >