[jira] [Updated] (SPARK-46968) Replace UnsupportedOperationException by SparkUnsupportedOperationException in sql

2024-02-02 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-46968: - Description: Replace all UnsupportedOperationException by SparkUnsupportedOperationException in the

[jira] [Updated] (SPARK-46967) Hide `Thread Dump` and `Heap Histogram` of `Dead` executors in `Executors` UI

2024-02-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46967: --- Labels: pull-request-available (was: ) > Hide `Thread Dump` and `Heap Histogram` of `Dead`

[jira] [Created] (SPARK-46968) Replace UnsupportedOperationException by SparkUnsupportedOperationException in sql

2024-02-02 Thread Max Gekk (Jira)
Max Gekk created SPARK-46968: Summary: Replace UnsupportedOperationException by SparkUnsupportedOperationException in sql Key: SPARK-46968 URL: https://issues.apache.org/jira/browse/SPARK-46968 Project:

[jira] [Created] (SPARK-46967) Hide `Thread Dump` and `Heap Histogram` of `Dead` executors in `Executors` UI

2024-02-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-46967: - Summary: Hide `Thread Dump` and `Heap Histogram` of `Dead` executors in `Executors` UI Key: SPARK-46967 URL: https://issues.apache.org/jira/browse/SPARK-46967

[jira] [Updated] (SPARK-46967) Hide `Thread Dump` and `Heap Histogram` of `Dead` executors in `Executors` UI

2024-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46967: -- Component/s: Web UI > Hide `Thread Dump` and `Heap Histogram` of `Dead` executors in

[jira] [Assigned] (SPARK-46950) Align `not available codec` error-class

2024-02-02 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-46950: Assignee: BingKun Pan > Align `not available codec` error-class >

[jira] [Resolved] (SPARK-46950) Align `not available codec` error-class

2024-02-02 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46950. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44992

[jira] [Updated] (SPARK-44111) Prepare Apache Spark 4.0.0

2024-02-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44111: --- Labels: pull-request-available (was: ) > Prepare Apache Spark 4.0.0 >

[jira] [Resolved] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-02-02 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46890. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44939

[jira] [Assigned] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-02-02 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-46890: Assignee: Daniel > CSV fails on a column with default and without enforcing schema >

[jira] [Resolved] (SPARK-46965) Check logType in Utils.getLog

2024-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46965. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45006

[jira] [Updated] (SPARK-46966) Create API for 'analyze' method to indicate subset of input table columns to select

2024-02-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46966: --- Labels: pull-request-available (was: ) > Create API for 'analyze' method to indicate

[jira] [Created] (SPARK-46966) Create API for 'analyze' method to indicate subset of input table columns to select

2024-02-02 Thread Daniel (Jira)
Daniel created SPARK-46966: -- Summary: Create API for 'analyze' method to indicate subset of input table columns to select Key: SPARK-46966 URL: https://issues.apache.org/jira/browse/SPARK-46966 Project:

[jira] [Resolved] (SPARK-46638) Create API to acquire execution memory for 'eval' and 'terminate' methods

2024-02-02 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel resolved SPARK-46638. Resolution: Won't Fix > Create API to acquire execution memory for 'eval' and 'terminate' methods >

[jira] [Assigned] (SPARK-46965) Check logType in Utils.getLog

2024-02-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46965: - Assignee: Dongjoon Hyun > Check logType in Utils.getLog >

[jira] [Updated] (SPARK-46965) Check logType in Utils.getLog

2024-02-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46965: --- Labels: pull-request-available (was: ) > Check logType in Utils.getLog >

[jira] [Created] (SPARK-46965) Check logType in Utils.getLog

2024-02-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-46965: - Summary: Check logType in Utils.getLog Key: SPARK-46965 URL: https://issues.apache.org/jira/browse/SPARK-46965 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-46964) Change the signature of the hllInvalidLgK query execution error to take an integer as 4th argument

2024-02-02 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-46964: -- Assignee: Menelaos Karavelas > Change the signature of the hllInvalidLgK query

[jira] [Resolved] (SPARK-46964) Change the signature of the hllInvalidLgK query execution error to take an integer as 4th argument

2024-02-02 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-46964. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44995

[jira] [Updated] (SPARK-46964) Change the signature of the hllInvalidLgK query execution error to take an integer as 4th argument

2024-02-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46964: --- Labels: pull-request-available (was: ) > Change the signature of the hllInvalidLgK query

[jira] [Created] (SPARK-46964) Change the signature of the hllInvalidLgK query execution error to take an integer as 4th argument

2024-02-02 Thread Menelaos Karavelas (Jira)
Menelaos Karavelas created SPARK-46964: -- Summary: Change the signature of the hllInvalidLgK query execution error to take an integer as 4th argument Key: SPARK-46964 URL:

[jira] [Updated] (SPARK-46963) Verify AQE is not enabled for Structured Streaming

2024-02-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46963: --- Labels: pull-request-available (was: ) > Verify AQE is not enabled for Structured

[jira] [Created] (SPARK-46963) Verify AQE is not enabled for Structured Streaming

2024-02-02 Thread Bo Gao (Jira)
Bo Gao created SPARK-46963: -- Summary: Verify AQE is not enabled for Structured Streaming Key: SPARK-46963 URL: https://issues.apache.org/jira/browse/SPARK-46963 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-46962) Implement python worker to run python streaming data source

2024-02-02 Thread Chaoqin Li (Jira)
Chaoqin Li created SPARK-46962: -- Summary: Implement python worker to run python streaming data source Key: SPARK-46962 URL: https://issues.apache.org/jira/browse/SPARK-46962 Project: Spark

[jira] [Updated] (SPARK-46866) Streaming python data source API

2024-02-02 Thread Chaoqin Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoqin Li updated SPARK-46866: --- Issue Type: Epic (was: Improvement) > Streaming python data source API >

[jira] [Created] (SPARK-46961) Adding processorHandle as a Context Variable

2024-02-02 Thread Eric Marnadi (Jira)
Eric Marnadi created SPARK-46961: Summary: Adding processorHandle as a Context Variable Key: SPARK-46961 URL: https://issues.apache.org/jira/browse/SPARK-46961 Project: Spark Issue Type:

[jira] [Created] (SPARK-46960) Testing Multiple Input Streams for TransformWithState operator

2024-02-02 Thread Eric Marnadi (Jira)
Eric Marnadi created SPARK-46960: Summary: Testing Multiple Input Streams for TransformWithState operator Key: SPARK-46960 URL: https://issues.apache.org/jira/browse/SPARK-46960 Project: Spark

[jira] [Commented] (SPARK-40549) PYSPARK: Observation computes the wrong results when using `corr` function

2024-02-02 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813780#comment-17813780 ] Nicholas Chammas commented on SPARK-40549: -- I think this is just a consequence of floating

[jira] [Resolved] (SPARK-46915) Simplify `UnaryMinus` and align error class

2024-02-02 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46915. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44942

[jira] [Assigned] (SPARK-46915) Simplify `UnaryMinus` and align error class

2024-02-02 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-46915: Assignee: BingKun Pan > Simplify `UnaryMinus` and align error class >

[jira] [Commented] (SPARK-45786) Inaccurate Decimal multiplication and division results

2024-02-02 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813766#comment-17813766 ] Nicholas Chammas commented on SPARK-45786: -- [~kazuyukitanimura] - I'm just curious: How did you

[jira] [Commented] (SPARK-38167) CSV parsing error when using escape='"'

2024-02-02 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813741#comment-17813741 ] Nicholas Chammas commented on SPARK-38167: -- [~marnixvandenbroek] - Could you link to the bug

[jira] [Updated] (SPARK-42399) CONV() silently overflows returning wrong results

2024-02-02 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-42399: - Affects Version/s: (was: 3.5.0) > CONV() silently overflows returning wrong results

[jira] [Commented] (SPARK-42399) CONV() silently overflows returning wrong results

2024-02-02 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813733#comment-17813733 ] Nicholas Chammas commented on SPARK-42399: -- This issue does indeed appear to be resolved on

[jira] [Updated] (SPARK-42399) CONV() silently overflows returning wrong results

2024-02-02 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-42399: - Affects Version/s: 3.5.0 > CONV() silently overflows returning wrong results >

[jira] [Updated] (SPARK-42399) CONV() silently overflows returning wrong results

2024-02-02 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-42399: - Labels: correctness pull-request-available (was: pull-request-available) > CONV()

[jira] [Resolved] (SPARK-46911) Add deleteIfExists operator to StatefulProcessorHandle

2024-02-02 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-46911. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44903

[jira] [Assigned] (SPARK-46911) Add deleteIfExists operator to StatefulProcessorHandle

2024-02-02 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-46911: Assignee: Eric Marnadi > Add deleteIfExists operator to StatefulProcessorHandle >

[jira] [Updated] (SPARK-46959) CSV reader reads data inconsistently depending on column position

2024-02-02 Thread Martin Rueckl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Rueckl updated SPARK-46959: -- Description: Reading the following CSV {code:java} "a";"b";"c";"d"

[jira] [Updated] (SPARK-46959) CSV reader reads data inconsistently depending on column position

2024-02-02 Thread Martin Rueckl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Rueckl updated SPARK-46959: -- Description: Reading the following CSV {code:java} "a";"b";"c";"d"

[jira] [Updated] (SPARK-46959) CSV reader reads data inconsistently depending on column position

2024-02-02 Thread Martin Rueckl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Rueckl updated SPARK-46959: -- Description: Reading the following CSV {code:java} "a";"b";"c";"d"

[jira] [Updated] (SPARK-46959) CSV reader reads data inconsistently depending on column position

2024-02-02 Thread Martin Rueckl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Rueckl updated SPARK-46959: -- Description: Reading the following CSV {code:java} "a";"b";"c";"d"

[jira] [Updated] (SPARK-46959) CSV reader reads data inconsistently depending on column position

2024-02-02 Thread Martin Rueckl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Rueckl updated SPARK-46959: -- Description: Reading the following CSV {code:java} "a";"b";"c";"d"

[jira] [Created] (SPARK-46959) CSV reader reads data inconsistently depending on column position

2024-02-02 Thread Martin Rueckl (Jira)
Martin Rueckl created SPARK-46959: - Summary: CSV reader reads data inconsistently depending on column position Key: SPARK-46959 URL: https://issues.apache.org/jira/browse/SPARK-46959 Project: Spark

[jira] [Resolved] (SPARK-46946) Supporting broadcast of multiple filtering keys in DynamicPruning

2024-02-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46946. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44988

[jira] [Assigned] (SPARK-46946) Supporting broadcast of multiple filtering keys in DynamicPruning

2024-02-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46946: --- Assignee: Thang Long Vu > Supporting broadcast of multiple filtering keys in

[jira] [Updated] (SPARK-46958) FIx coerceDefaultValue when canUpCast

2024-02-02 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-46958: - Affects Version/s: 3.5.0 > FIx coerceDefaultValue when canUpCast >

[jira] [Created] (SPARK-46958) FIx coerceDefaultValue when canUpCast

2024-02-02 Thread Kent Yao (Jira)
Kent Yao created SPARK-46958: Summary: FIx coerceDefaultValue when canUpCast Key: SPARK-46958 URL: https://issues.apache.org/jira/browse/SPARK-46958 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-46949) Support CHAR/VARCHAR through ResolveDefaultColumns

2024-02-02 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-46949: Assignee: Kent Yao > Support CHAR/VARCHAR through ResolveDefaultColumns >

[jira] [Resolved] (SPARK-46949) Support CHAR/VARCHAR through ResolveDefaultColumns

2024-02-02 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-46949. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44991

[jira] [Commented] (SPARK-20624) SPIP: Add better handling for node shutdown

2024-02-02 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813592#comment-17813592 ] Yu-Jhe Li commented on SPARK-20624: --- Hi, we found the migrated shuffle files from the decommissioned

[jira] [Updated] (SPARK-46957) Migrated shuffle data files from the decommissioned node should be removed when job completed

2024-02-02 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-46957: -- Description: Hi, we have a long-lived Spark application run on a standalone cluster on GCP and we

[jira] [Updated] (SPARK-46957) Migrated shuffle data files from the decommissioned node should be removed when job completed

2024-02-02 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-46957: -- Description: Hi, we have a long-lived Spark application run on a standalone cluster on GCP and we

[jira] [Updated] (SPARK-46957) Migrated shuffle data files from the decommissioned node should be removed when job completed

2024-02-02 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-46957: -- Description: Hi, we have a long-lived Spark application run on a standalone cluster on GCP and we

[jira] [Created] (SPARK-46957) Migrated shuffle data files from the decommissioned node should be removed when job completed

2024-02-02 Thread Yu-Jhe Li (Jira)
Yu-Jhe Li created SPARK-46957: - Summary: Migrated shuffle data files from the decommissioned node should be removed when job completed Key: SPARK-46957 URL: https://issues.apache.org/jira/browse/SPARK-46957

[jira] [Updated] (SPARK-46954) XML: Perf optimizations

2024-02-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46954: --- Labels: pull-request-available (was: ) > XML: Perf optimizations > ---