[jira] [Assigned] (SPARK-46875) When the `mode` is null, a `NullPointException` should `not` be thrown

[ https://issues.apache.org/jira/browse/SPARK-46875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-46875: Assignee: BingKun Pan > When the `mode` is null, a `NullPointException` should `not` be thrown >

[jira] [Resolved] (SPARK-46875) When the `mode` is null, a `NullPointException` should `not` be thrown

[ https://issues.apache.org/jira/browse/SPARK-46875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46875. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44900

[jira] [Updated] (SPARK-46883) Support `/json/clusterutilization` API

[ https://issues.apache.org/jira/browse/SPARK-46883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46883: --- Labels: pull-request-available (was: ) > Support `/json/clusterutilization` API >

[jira] [Created] (SPARK-46883) Make Master expose `/json/clusterutilization` endpoint

Dongjoon Hyun created SPARK-46883: - Summary: Make Master expose `/json/clusterutilization` endpoint Key: SPARK-46883 URL: https://issues.apache.org/jira/browse/SPARK-46883 Project: Spark

[jira] [Updated] (SPARK-46883) Support `/json/clusterutilization` API

[ https://issues.apache.org/jira/browse/SPARK-46883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46883: -- Summary: Support `/json/clusterutilization` API (was: Make Master expose

[jira] [Comment Edited] (SPARK-46810) Clarify error class terminology

[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811470#comment-17811470 ] Nicholas Chammas edited comment on SPARK-46810 at 1/27/24 5:00 AM: ---

[jira] [Updated] (SPARK-46810) Clarify error class terminology

[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-46810: - Description: We use inconsistent terminology when talking about error classes. I'd like

[jira] [Commented] (SPARK-46810) Clarify error class terminology

[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811470#comment-17811470 ] Nicholas Chammas commented on SPARK-46810: -- [~srielau] - What do you think of the problem and

[jira] [Updated] (SPARK-46810) Clarify error class terminology

[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-46810: - Description: We use inconsistent terminology when talking about error classes. I'd like

[jira] [Updated] (SPARK-46810) Clarify error class terminology

[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-46810: - Description: We use inconsistent terminology when talking about error classes. I'd like

[jira] [Updated] (SPARK-46882) Remove unnecessary AtomicInteger

[ https://issues.apache.org/jira/browse/SPARK-46882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-46882: --- Summary: Remove unnecessary AtomicInteger (was: Remove unnessary AtomicInteger) > Remove

[jira] [Created] (SPARK-46882) Remove unnessary AtomicInteger

Jiaan Geng created SPARK-46882: -- Summary: Remove unnessary AtomicInteger Key: SPARK-46882 URL: https://issues.apache.org/jira/browse/SPARK-46882 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-46881) Support `spark.deploy.workerSelectionPolicy`

[ https://issues.apache.org/jira/browse/SPARK-46881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46881. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44906

[jira] [Resolved] (SPARK-46880) Improve and test warning for Arrow-optimized Python UDF

[ https://issues.apache.org/jira/browse/SPARK-46880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46880. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44905

[jira] [Updated] (SPARK-46881) Support `spark.deploy.workerSelectionPolicy`

[ https://issues.apache.org/jira/browse/SPARK-46881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46881: --- Labels: pull-request-available (was: ) > Support `spark.deploy.workerSelectionPolicy` >

[jira] [Assigned] (SPARK-46881) Support `spark.deploy.workerSelectionPolicy`

[ https://issues.apache.org/jira/browse/SPARK-46881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46881: - Assignee: Dongjoon Hyun > Support `spark.deploy.workerSelectionPolicy` >

[jira] [Created] (SPARK-46881) Support `spark.deploy.workerSelectionPolicy`

Dongjoon Hyun created SPARK-46881: - Summary: Support `spark.deploy.workerSelectionPolicy` Key: SPARK-46881 URL: https://issues.apache.org/jira/browse/SPARK-46881 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46880) Improve and test warning for Arrow-optimized Python UDF

[ https://issues.apache.org/jira/browse/SPARK-46880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46880: --- Labels: pull-request-available (was: ) > Improve and test warning for Arrow-optimized

[jira] [Created] (SPARK-46880) Improve and test warning for Arrow-optimized Python UDF

Xinrong Meng created SPARK-46880: Summary: Improve and test warning for Arrow-optimized Python UDF Key: SPARK-46880 URL: https://issues.apache.org/jira/browse/SPARK-46880 Project: Spark

[jira] [Updated] (SPARK-46879) Run optimizer on REPLACE TABLE column defaults

[ https://issues.apache.org/jira/browse/SPARK-46879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46879: --- Labels: pull-request-available (was: ) > Run optimizer on REPLACE TABLE column defaults >

[jira] [Updated] (SPARK-46879) Run optimizer on REPLACE TABLE column defaults

[ https://issues.apache.org/jira/browse/SPARK-46879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-46879: --- Summary: Run optimizer on REPLACE TABLE column defaults (was: Run optimizer on CREATE TABLE column

[jira] [Created] (SPARK-46879) Run optimizer on CREATE TABLE column defaults

Daniel created SPARK-46879: -- Summary: Run optimizer on CREATE TABLE column defaults Key: SPARK-46879 URL: https://issues.apache.org/jira/browse/SPARK-46879 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-46849) Run optimizer on CREATE TABLE column defaults

[ https://issues.apache.org/jira/browse/SPARK-46849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46849: - Assignee: Daniel > Run optimizer on CREATE TABLE column defaults >

[jira] [Resolved] (SPARK-46849) Run optimizer on CREATE TABLE column defaults

[ https://issues.apache.org/jira/browse/SPARK-46849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46849. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44876

[jira] [Updated] (SPARK-46798) Kafka custom partition location assignment in Spark Structured Streaming (rack awareness)

[ https://issues.apache.org/jira/browse/SPARK-46798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Schwager updated SPARK-46798: - Description: I'd like to propose, and implement if approved, support for custom

[jira] [Updated] (SPARK-46810) Clarify error class terminology

[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46810: --- Labels: pull-request-available (was: ) > Clarify error class terminology >

[jira] [Resolved] (SPARK-46819) Port error class data to automation-friendly format

[ https://issues.apache.org/jira/browse/SPARK-46819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46819. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44863

[jira] [Assigned] (SPARK-46819) Port error class data to automation-friendly format

[ https://issues.apache.org/jira/browse/SPARK-46819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-46819: Assignee: Nicholas Chammas > Port error class data to automation-friendly format >

[jira] [Updated] (SPARK-46831) Extend StringType and PhysicalStringType with collation id

[ https://issues.apache.org/jira/browse/SPARK-46831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46831: --- Labels: pull-request-available (was: ) > Extend StringType and PhysicalStringType with

[jira] [Updated] (SPARK-46830) Introducing collation concept into Spark

[ https://issues.apache.org/jira/browse/SPARK-46830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandar Tomic updated SPARK-46830: - Attachment: Collation Support in Spark.docx > Introducing collation concept into Spark

[jira] [Updated] (SPARK-46830) Introducing collation concept into Spark

[ https://issues.apache.org/jira/browse/SPARK-46830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandar Tomic updated SPARK-46830: - Description: This feature will introduce collation support to the Spark engine. This

[jira] [Created] (SPARK-46878) Invalid Mima report for StringType extension

Aleksandar Tomic created SPARK-46878: Summary: Invalid Mima report for StringType extension Key: SPARK-46878 URL: https://issues.apache.org/jira/browse/SPARK-46878 Project: Spark Issue

[jira] [Commented] (SPARK-46876) Data is silently lost in Tab separated CSV with empty (whitespace) rows

[ https://issues.apache.org/jira/browse/SPARK-46876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811211#comment-17811211 ] Martin Rueckl commented on SPARK-46876: --- Any resolution of this should probaly consider the mode.

[jira] [Updated] (SPARK-46876) Data is silently lost in Tab separated CSV with empty (whitespace) rows

[ https://issues.apache.org/jira/browse/SPARK-46876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Rueckl updated SPARK-46876: -- Description: When reading a tab separated file that contains lines that only contain tabs

[jira] [Updated] (SPARK-46876) Data is silently lost in Tab separated CSV with empty (whitespace) rows

[ https://issues.apache.org/jira/browse/SPARK-46876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Rueckl updated SPARK-46876: -- Description: When reading a tab separated file that contains lines that only contain tabs

[jira] [Created] (SPARK-46876) Data is silently lost in Tab separated CSV with empty (whitespace) rows

Martin Rueckl created SPARK-46876: - Summary: Data is silently lost in Tab separated CSV with empty (whitespace) rows Key: SPARK-46876 URL: https://issues.apache.org/jira/browse/SPARK-46876 Project:

[jira] [Assigned] (SPARK-46873) PySpark spark.streams should not recreate new StreamingQueryManager

[ https://issues.apache.org/jira/browse/SPARK-46873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46873: -- Assignee: (was: Apache Spark) > PySpark spark.streams should not recreate new

[jira] [Assigned] (SPARK-46873) PySpark spark.streams should not recreate new StreamingQueryManager

[ https://issues.apache.org/jira/browse/SPARK-46873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46873: -- Assignee: Apache Spark > PySpark spark.streams should not recreate new

[jira] [Assigned] (SPARK-46874) Remove pyspark.pandas dependency from assertDataFrameEqual

[ https://issues.apache.org/jira/browse/SPARK-46874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46874: -- Assignee: Apache Spark > Remove pyspark.pandas dependency from assertDataFrameEqual

[jira] [Assigned] (SPARK-46874) Remove pyspark.pandas dependency from assertDataFrameEqual

[ https://issues.apache.org/jira/browse/SPARK-46874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46874: -- Assignee: (was: Apache Spark) > Remove pyspark.pandas dependency from

[jira] [Updated] (SPARK-46874) Remove pyspark.pandas dependency from assertDataFrameEqual

[ https://issues.apache.org/jira/browse/SPARK-46874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46874: --- Labels: pull-request-available (was: ) > Remove pyspark.pandas dependency from

[jira] [Created] (SPARK-46874) Remove pyspark.pandas dependency from assertDataFrameEqual

Haejoon Lee created SPARK-46874: --- Summary: Remove pyspark.pandas dependency from assertDataFrameEqual Key: SPARK-46874 URL: https://issues.apache.org/jira/browse/SPARK-46874 Project: Spark

[jira] [Resolved] (SPARK-46862) Incorrect count() of a dataframe loaded from CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-46862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46862. -- Fix Version/s: 3.4.3 3.5.1 4.0.0 Resolution: Fixed Issue