[jira] [Commented] (SPARK-36418) Use CAST in parsing of dates/timestamps with default pattern

[ https://issues.apache.org/jira/browse/SPARK-36418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399098#comment-17399098 ] Max Gekk commented on SPARK-36418: -- [~Gengliang.Wang] Yes. > Use CAST in parsing of dates/timestamps

[jira] [Commented] (SPARK-36508) Disallow binary operations between Interval and String literal

[ https://issues.apache.org/jira/browse/SPARK-36508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399071#comment-17399071 ] Gengliang Wang commented on SPARK-36508: The issue is resolved in

[jira] [Resolved] (SPARK-36508) Disallow binary operations between Interval and String literal

[ https://issues.apache.org/jira/browse/SPARK-36508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-36508. Resolution: Done > Disallow binary operations between Interval and String literal >

[jira] [Commented] (SPARK-36418) Use CAST in parsing of dates/timestamps with default pattern

[ https://issues.apache.org/jira/browse/SPARK-36418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399068#comment-17399068 ] Gengliang Wang commented on SPARK-36418: [~maxgekk]shall we target this to 3.3.0 only? cc

[jira] [Created] (SPARK-36511) Remove ColumnIO once PARQUET-2050 is released in Parquet 1.13

Chao Sun created SPARK-36511: Summary: Remove ColumnIO once PARQUET-2050 is released in Parquet 1.13 Key: SPARK-36511 URL: https://issues.apache.org/jira/browse/SPARK-36511 Project: Spark Issue

[jira] [Assigned] (SPARK-36510) Missing spark.redaction.string.regex property

[ https://issues.apache.org/jira/browse/SPARK-36510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36510: Assignee: Apache Spark > Missing spark.redaction.string.regex property >

[jira] [Assigned] (SPARK-36510) Missing spark.redaction.string.regex property

[ https://issues.apache.org/jira/browse/SPARK-36510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36510: Assignee: (was: Apache Spark) > Missing spark.redaction.string.regex property >

[jira] [Commented] (SPARK-36510) Missing spark.redaction.string.regex property

[ https://issues.apache.org/jira/browse/SPARK-36510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398979#comment-17398979 ] Apache Spark commented on SPARK-36510: -- User 'dnskr' has created a pull request for this issue:

[jira] [Updated] (SPARK-36510) Missing spark.redaction.string.regex property

[ https://issues.apache.org/jira/browse/SPARK-36510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Krivenko updated SPARK-36510: --- Description: The property *spark.redaction.string.regex* is missing in  [Runtime

[jira] [Created] (SPARK-36510) Missing spark.redaction.string.regex property

Denis Krivenko created SPARK-36510: -- Summary: Missing spark.redaction.string.regex property Key: SPARK-36510 URL: https://issues.apache.org/jira/browse/SPARK-36510 Project: Spark Issue

[jira] [Resolved] (SPARK-36471) Implement DatetimeIndex.map

[ https://issues.apache.org/jira/browse/SPARK-36471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-36471. -- Resolution: Won't Do Map is not implemented for DatetimeIndex in pandas yet. > Implement

[jira] [Commented] (SPARK-34952) DS V2 Aggregate push down

[ https://issues.apache.org/jira/browse/SPARK-34952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398897#comment-17398897 ] Apache Spark commented on SPARK-34952: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36399) Implement DataFrame.combine_first

[ https://issues.apache.org/jira/browse/SPARK-36399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36399: Assignee: (was: Apache Spark) > Implement DataFrame.combine_first >

[jira] [Assigned] (SPARK-36399) Implement DataFrame.combine_first

[ https://issues.apache.org/jira/browse/SPARK-36399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36399: Assignee: Apache Spark > Implement DataFrame.combine_first >

[jira] [Commented] (SPARK-36399) Implement DataFrame.combine_first

[ https://issues.apache.org/jira/browse/SPARK-36399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398839#comment-17398839 ] Apache Spark commented on SPARK-36399: -- User 'xinrong-databricks' has created a pull request for

[jira] [Resolved] (SPARK-36475) Add doc about spark.shuffle.service.fetch.rdd.enabled

[ https://issues.apache.org/jira/browse/SPARK-36475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36475. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33700

[jira] [Assigned] (SPARK-36475) Add doc about spark.shuffle.service.fetch.rdd.enabled

[ https://issues.apache.org/jira/browse/SPARK-36475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36475: - Assignee: angerszhu > Add doc about spark.shuffle.service.fetch.rdd.enabled >

[jira] [Resolved] (SPARK-36503) Add RowToColumnConverter for BinaryType

[ https://issues.apache.org/jira/browse/SPARK-36503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36503. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33733

[jira] [Assigned] (SPARK-36503) Add RowToColumnConverter for BinaryType

[ https://issues.apache.org/jira/browse/SPARK-36503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36503: - Assignee: Huaxin Gao > Add RowToColumnConverter for BinaryType >

[jira] [Assigned] (SPARK-36487) modify exit executor log logic

[ https://issues.apache.org/jira/browse/SPARK-36487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-36487: Assignee: binjie yang > modify exit executor log logic > --

[jira] [Resolved] (SPARK-36487) modify exit executor log logic

[ https://issues.apache.org/jira/browse/SPARK-36487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-36487. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33718

[jira] [Created] (SPARK-36509) Executors don't get rescheduled in standalone mode when worker dies

Peter Kaiser created SPARK-36509: Summary: Executors don't get rescheduled in standalone mode when worker dies Key: SPARK-36509 URL: https://issues.apache.org/jira/browse/SPARK-36509 Project: Spark

[jira] [Commented] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398617#comment-17398617 ] Apache Spark commented on SPARK-32920: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Commented] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398616#comment-17398616 ] Apache Spark commented on SPARK-32920: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Commented] (SPARK-35324) Spark SQL configs not respected in RDDs

[ https://issues.apache.org/jira/browse/SPARK-35324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398610#comment-17398610 ] Willi Raschkowski commented on SPARK-35324: --- [~mgekk], apologies for the direct ping. Do you

[jira] [Comment Edited] (SPARK-35324) Spark SQL configs not respected in RDDs

[ https://issues.apache.org/jira/browse/SPARK-35324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398607#comment-17398607 ] Willi Raschkowski edited comment on SPARK-35324 at 8/13/21, 11:44 AM:

[jira] [Comment Edited] (SPARK-35324) Spark SQL configs not respected in RDDs

[ https://issues.apache.org/jira/browse/SPARK-35324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398607#comment-17398607 ] Willi Raschkowski edited comment on SPARK-35324 at 8/13/21, 11:43 AM:

[jira] [Comment Edited] (SPARK-35324) Spark SQL configs not respected in RDDs

[ https://issues.apache.org/jira/browse/SPARK-35324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398607#comment-17398607 ] Willi Raschkowski edited comment on SPARK-35324 at 8/13/21, 11:43 AM:

[jira] [Commented] (SPARK-35324) Spark SQL configs not respected in RDDs

[ https://issues.apache.org/jira/browse/SPARK-35324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398607#comment-17398607 ] Willi Raschkowski commented on SPARK-35324: --- Here's an example of that: {code:title=Rebase

[jira] [Resolved] (SPARK-36500) The temp_shuffle files are not cleaned up when a task is interrupted

[ https://issues.apache.org/jira/browse/SPARK-36500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36500. -- Fix Version/s: 3.1.3 3.2.0 3.0.4 Resolution:

[jira] [Resolved] (SPARK-36495) Use Type match to simplify CatalystTypeConverter.toCatalyst

[ https://issues.apache.org/jira/browse/SPARK-36495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36495. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33722

[jira] [Assigned] (SPARK-36495) Use Type match to simplify CatalystTypeConverter.toCatalyst

[ https://issues.apache.org/jira/browse/SPARK-36495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36495: Assignee: Yang Jie > Use Type match to simplify CatalystTypeConverter.toCatalyst >

[jira] [Resolved] (SPARK-36507) Remove/Replace missing links to AMP Camp materials from index.md

[ https://issues.apache.org/jira/browse/SPARK-36507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36507. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33734

[jira] [Assigned] (SPARK-36508) Disallow binary operations between Interval and String literal

[ https://issues.apache.org/jira/browse/SPARK-36508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36508: Assignee: Gengliang Wang (was: Apache Spark) > Disallow binary operations between

[jira] [Commented] (SPARK-36508) Disallow binary operations between Interval and String literal

[ https://issues.apache.org/jira/browse/SPARK-36508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398563#comment-17398563 ] Apache Spark commented on SPARK-36508: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-36508) Disallow binary operations between Interval and String literal

[ https://issues.apache.org/jira/browse/SPARK-36508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36508: Assignee: Apache Spark (was: Gengliang Wang) > Disallow binary operations between

[jira] [Updated] (SPARK-36508) Disallow binary operations between Interval and String literal

[ https://issues.apache.org/jira/browse/SPARK-36508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-36508: --- Description: If a binary operation contains interval type and string literal, we can't

[jira] [Updated] (SPARK-36508) Disallow binary operations between Interval and String literal

[ https://issues.apache.org/jira/browse/SPARK-36508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-36508: --- Summary: Disallow binary operations between Interval and String literal (was: Disallow

[jira] [Created] (SPARK-36508) Disallow Interval add/subtract String literal

Gengliang Wang created SPARK-36508: -- Summary: Disallow Interval add/subtract String literal Key: SPARK-36508 URL: https://issues.apache.org/jira/browse/SPARK-36508 Project: Spark Issue

[jira] [Updated] (SPARK-34247) Add end-to-end tests for ANSI implicit cast rules

[ https://issues.apache.org/jira/browse/SPARK-34247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-34247: --- Parent: (was: SPARK-34246) Issue Type: Task (was: Sub-task) > Add end-to-end

[jira] [Updated] (SPARK-34247) Add end-to-end tests for ANSI implicit cast rules

[ https://issues.apache.org/jira/browse/SPARK-34247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-34247: --- Affects Version/s: (was: 3.1.2) 3.3.0 > Add end-to-end tests for

[jira] [Updated] (SPARK-34247) Add end-to-end tests for ANSI implicit cast rules

[ https://issues.apache.org/jira/browse/SPARK-34247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-34247: --- Priority: Minor (was: Major) > Add end-to-end tests for ANSI implicit cast rules >

[jira] [Resolved] (SPARK-34413) Clean up AnsiTypeCoercionSuite and TypeCoercionSuite

[ https://issues.apache.org/jira/browse/SPARK-34413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-34413. Resolution: Later > Clean up AnsiTypeCoercionSuite and TypeCoercionSuite >

[jira] [Resolved] (SPARK-36490) Make from_csv/to_csv to handle timestamp_ntz type properly

[ https://issues.apache.org/jira/browse/SPARK-36490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36490. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33719

[jira] [Assigned] (SPARK-35991) Add PlanStability suite for TPCH

[ https://issues.apache.org/jira/browse/SPARK-35991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35991: Assignee: Apache Spark > Add PlanStability suite for TPCH >

[jira] [Assigned] (SPARK-35991) Add PlanStability suite for TPCH

[ https://issues.apache.org/jira/browse/SPARK-35991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35991: Assignee: (was: Apache Spark) > Add PlanStability suite for TPCH >

[jira] [Commented] (SPARK-35991) Add PlanStability suite for TPCH

[ https://issues.apache.org/jira/browse/SPARK-35991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398520#comment-17398520 ] Apache Spark commented on SPARK-35991: -- User 'AngersZh' has created a pull request for this

[jira] [Updated] (SPARK-36353) RemoveNoopOperators should keep output schema

[ https://issues.apache.org/jira/browse/SPARK-36353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-36353: Fix Version/s: 3.0.4 3.1.3 > RemoveNoopOperators should keep output schema >

[jira] [Assigned] (SPARK-36473) Add non-unified memory metrics

[ https://issues.apache.org/jira/browse/SPARK-36473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36473: Assignee: (was: Apache Spark) > Add non-unified memory metrics >

[jira] [Commented] (SPARK-36473) Add non-unified memory metrics

[ https://issues.apache.org/jira/browse/SPARK-36473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398495#comment-17398495 ] Apache Spark commented on SPARK-36473: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-36473) Add non-unified memory metrics

[ https://issues.apache.org/jira/browse/SPARK-36473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36473: Assignee: Apache Spark > Add non-unified memory metrics > --

[jira] [Resolved] (SPARK-36499) Test Interval multiply / divide null

[ https://issues.apache.org/jira/browse/SPARK-36499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36499. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33729

[jira] [Assigned] (SPARK-36387) Fix Series.astype from datetime to nullable string

[ https://issues.apache.org/jira/browse/SPARK-36387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36387: Assignee: Apache Spark > Fix Series.astype from datetime to nullable string >

[jira] [Commented] (SPARK-36387) Fix Series.astype from datetime to nullable string

[ https://issues.apache.org/jira/browse/SPARK-36387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398474#comment-17398474 ] Apache Spark commented on SPARK-36387: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36387) Fix Series.astype from datetime to nullable string

[ https://issues.apache.org/jira/browse/SPARK-36387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36387: Assignee: (was: Apache Spark) > Fix Series.astype from datetime to nullable string >

[jira] [Commented] (SPARK-36507) Remove/Replace missing links to AMP Camp materials from index.md

[ https://issues.apache.org/jira/browse/SPARK-36507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398456#comment-17398456 ] Apache Spark commented on SPARK-36507: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36507) Remove/Replace missing links to AMP Camp materials from index.md

[ https://issues.apache.org/jira/browse/SPARK-36507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36507: Assignee: Apache Spark (was: Kousuke Saruta) > Remove/Replace missing links to AMP Camp

[jira] [Assigned] (SPARK-36507) Remove/Replace missing links to AMP Camp materials from index.md

[ https://issues.apache.org/jira/browse/SPARK-36507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36507: Assignee: Kousuke Saruta (was: Apache Spark) > Remove/Replace missing links to AMP Camp