[jira] [Assigned] (SPARK-36044) Suport TimestampNTZ in functions unix_timestamp/to_unix_timestamp

[ https://issues.apache.org/jira/browse/SPARK-36044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-36044: Assignee: jiaan.geng > Suport TimestampNTZ in functions unix_timestamp/to_unix_timestamp > --

[jira] [Resolved] (SPARK-36044) Suport TimestampNTZ in functions unix_timestamp/to_unix_timestamp

[ https://issues.apache.org/jira/browse/SPARK-36044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36044. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33278 [https://github.com

[jira] [Assigned] (SPARK-36003) Implement unary operator `invert` of integral ps.Series/Index

[ https://issues.apache.org/jira/browse/SPARK-36003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36003: Assignee: Xinrong Meng > Implement unary operator `invert` of integral ps.Series/Index >

[jira] [Resolved] (SPARK-36003) Implement unary operator `invert` of integral ps.Series/Index

[ https://issues.apache.org/jira/browse/SPARK-36003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36003. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33285 [https://gi

[jira] [Commented] (SPARK-36045) TO_UTC_TIMESTAMP: return different result based on the default timestamp type

[ https://issues.apache.org/jira/browse/SPARK-36045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378978#comment-17378978 ] jiaan.geng commented on SPARK-36045: I'm working on. > TO_UTC_TIMESTAMP: return dif

[jira] [Commented] (SPARK-33679) Enable spark.sql.adaptive.enabled true by default

[ https://issues.apache.org/jira/browse/SPARK-33679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378976#comment-17378976 ] Apache Spark commented on SPARK-33679: -- User 'ulysses-you' has created a pull reque

[jira] [Resolved] (SPARK-34745) Unify overflow exception error message of integral types

[ https://issues.apache.org/jira/browse/SPARK-34745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-34745. - Resolution: Later > Unify overflow exception error message of integral types > -

[jira] [Commented] (SPARK-34402) Group exception about data format schema

[ https://issues.apache.org/jira/browse/SPARK-34402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378962#comment-17378962 ] Xiao Li commented on SPARK-34402: - [~angerszhuuu] any update? > Group exception about d

[jira] [Created] (SPARK-36086) The case of the delta table is inconsistent with parquet

Yuming Wang created SPARK-36086: --- Summary: The case of the delta table is inconsistent with parquet Key: SPARK-36086 URL: https://issues.apache.org/jira/browse/SPARK-36086 Project: Spark Issue

[jira] [Resolved] (SPARK-35813) Add new adaptive config into sql-performance-tuning docs

[ https://issues.apache.org/jira/browse/SPARK-35813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35813. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32960 [https://gith

[jira] [Resolved] (SPARK-36071) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-36071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36071. -- Resolution: Cannot Reproduce > Spark driver requires large memory space for serialized results

[jira] [Commented] (SPARK-36071) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-36071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378935#comment-17378935 ] Hyukjin Kwon commented on SPARK-36071: -- [~vcshashank] can you show your codes? > S

[jira] [Resolved] (SPARK-36084) spark kafka offset missed some partition offset describle

[ https://issues.apache.org/jira/browse/SPARK-36084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36084. -- Resolution: Incomplete > spark kafka offset missed some partition offset describle > -

[jira] [Updated] (SPARK-36071) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-36071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36071: - Priority: Major (was: Critical) > Spark driver requires large memory space for serialized resul

[jira] [Commented] (SPARK-36084) spark kafka offset missed some partition offset describle

[ https://issues.apache.org/jira/browse/SPARK-36084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378934#comment-17378934 ] Hyukjin Kwon commented on SPARK-36084: -- [~geekyouth] Spark 2.4 is EOL, and there wi

[jira] [Updated] (SPARK-36084) spark kafka offset missed some partition offset describle

[ https://issues.apache.org/jira/browse/SPARK-36084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36084: - Priority: Major (was: Critical) > spark kafka offset missed some partition offset describle > -

[jira] [Assigned] (SPARK-36085) Make broadcast query stage executionContext isolation from AQE

[ https://issues.apache.org/jira/browse/SPARK-36085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36085: Assignee: (was: Apache Spark) > Make broadcast query stage executionContext isolation

[jira] [Commented] (SPARK-36085) Make broadcast query stage executionContext isolation from AQE

[ https://issues.apache.org/jira/browse/SPARK-36085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378913#comment-17378913 ] Apache Spark commented on SPARK-36085: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-36085) Make broadcast query stage executionContext isolation from AQE

[ https://issues.apache.org/jira/browse/SPARK-36085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36085: Assignee: Apache Spark > Make broadcast query stage executionContext isolation from AQE >

[jira] [Assigned] (SPARK-36076) [SQL] ArrayIndexOutOfBounds in CAST string to timestamp

[ https://issues.apache.org/jira/browse/SPARK-36076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36076: Assignee: (was: Apache Spark) > [SQL] ArrayIndexOutOfBounds in CAST string to timesta

[jira] [Assigned] (SPARK-36076) [SQL] ArrayIndexOutOfBounds in CAST string to timestamp

[ https://issues.apache.org/jira/browse/SPARK-36076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36076: Assignee: Apache Spark > [SQL] ArrayIndexOutOfBounds in CAST string to timestamp > --

[jira] [Commented] (SPARK-36076) [SQL] ArrayIndexOutOfBounds in CAST string to timestamp

[ https://issues.apache.org/jira/browse/SPARK-36076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378911#comment-17378911 ] Apache Spark commented on SPARK-36076: -- User 'dgd-contributor' has created a pull r

[jira] [Created] (SPARK-36085) Make broadcast query stage executionContext isolation from AQE

XiDuo You created SPARK-36085: - Summary: Make broadcast query stage executionContext isolation from AQE Key: SPARK-36085 URL: https://issues.apache.org/jira/browse/SPARK-36085 Project: Spark Iss

[jira] [Commented] (SPARK-35561) partition result is incorrect when insert into partition table with int datatype partition column

[ https://issues.apache.org/jira/browse/SPARK-35561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378876#comment-17378876 ] Apache Spark commented on SPARK-35561: -- User 'dgd-contributor' has created a pull r

[jira] [Commented] (SPARK-36069) spark function from_json should output field name, field type and field value when FAILFAST mode throw exception

[ https://issues.apache.org/jira/browse/SPARK-36069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1737#comment-1737 ] geekyouth commented on SPARK-36069: --- I also want to merge this feature into version 2.

[jira] [Commented] (SPARK-36069) spark function from_json should output field name, field type and field value when FAILFAST mode throw exception

[ https://issues.apache.org/jira/browse/SPARK-36069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378882#comment-17378882 ] Apache Spark commented on SPARK-36069: -- User 'geekyouth' has created a pull request

[jira] [Commented] (SPARK-36069) spark function from_json should output field name, field type and field value when FAILFAST mode throw exception

[ https://issues.apache.org/jira/browse/SPARK-36069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378883#comment-17378883 ] Apache Spark commented on SPARK-36069: -- User 'geekyouth' has created a pull request

[jira] [Updated] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zengrui updated SPARK-27396: Description: *strong text**SPIP: Columnar Processing Without Arrow Formatting Guarantees.*   *Q1.* What

[jira] [Assigned] (SPARK-36069) spark function from_json should output field name, field type and field value when FAILFAST mode throw exception

[ https://issues.apache.org/jira/browse/SPARK-36069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36069: Assignee: (was: Apache Spark) > spark function from_json should output field name, fi

[jira] [Assigned] (SPARK-36069) spark function from_json should output field name, field type and field value when FAILFAST mode throw exception

[ https://issues.apache.org/jira/browse/SPARK-36069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36069: Assignee: Apache Spark > spark function from_json should output field name, field type an

[jira] [Assigned] (SPARK-35561) partition result is incorrect when insert into partition table with int datatype partition column

[ https://issues.apache.org/jira/browse/SPARK-35561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35561: Assignee: (was: Apache Spark) > partition result is incorrect when insert into partit

[jira] [Commented] (SPARK-35561) partition result is incorrect when insert into partition table with int datatype partition column

[ https://issues.apache.org/jira/browse/SPARK-35561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378875#comment-17378875 ] Apache Spark commented on SPARK-35561: -- User 'dgd-contributor' has created a pull r

[jira] [Assigned] (SPARK-35561) partition result is incorrect when insert into partition table with int datatype partition column

[ https://issues.apache.org/jira/browse/SPARK-35561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35561: Assignee: Apache Spark > partition result is incorrect when insert into partition table w

[jira] [Created] (SPARK-36084) spark kafka offset missed some partition offset describle

geekyouth created SPARK-36084: - Summary: spark kafka offset missed some partition offset describle Key: SPARK-36084 URL: https://issues.apache.org/jira/browse/SPARK-36084 Project: Spark Issue Typ

[jira] [Assigned] (SPARK-36064) Manage InternalField more in DataTypeOps.

[ https://issues.apache.org/jira/browse/SPARK-36064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36064: Assignee: Takuya Ueshin > Manage InternalField more in DataTypeOps. > ---

[jira] [Resolved] (SPARK-36064) Manage InternalField more in DataTypeOps.

[ https://issues.apache.org/jira/browse/SPARK-36064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36064. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33275 [https://gi

[jira] [Updated] (SPARK-36037) Support ANSI SQL LOCALTIMESTAMP datetime value function

[ https://issues.apache.org/jira/browse/SPARK-36037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-36037: --- Summary: Support ANSI SQL LOCALTIMESTAMP datetime value function (was: Support new function localti

[jira] [Commented] (SPARK-35508) job group and description do not apply on broadcasts

[ https://issues.apache.org/jira/browse/SPARK-35508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378815#comment-17378815 ] Hyukjin Kwon commented on SPARK-35508: -- I think we should probably have a way to se

[jira] [Resolved] (SPARK-36083) make_timestamp: return different result based on the default timestamp type

[ https://issues.apache.org/jira/browse/SPARK-36083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36083. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33290 [https://github.com

[jira] [Resolved] (SPARK-36036) Regression: Remote blocks stored on disk by BlockManager are not deleted

[ https://issues.apache.org/jira/browse/SPARK-36036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-36036. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33251 [https://gi

[jira] [Assigned] (SPARK-36036) Regression: Remote blocks stored on disk by BlockManager are not deleted

[ https://issues.apache.org/jira/browse/SPARK-36036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-36036: Assignee: Denis Tarima > Regression: Remote blocks stored on disk by BlockManager are not

[jira] [Updated] (SPARK-35743) Improve Parquet vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-35743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-35743: - Labels: parquet (was: ) > Improve Parquet vectorized reader > - > >

[jira] [Commented] (SPARK-36083) make_timestamp: return different result based on the default timestamp type

[ https://issues.apache.org/jira/browse/SPARK-36083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378605#comment-17378605 ] Apache Spark commented on SPARK-36083: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-36083) make_timestamp: return different result based on the default timestamp type

[ https://issues.apache.org/jira/browse/SPARK-36083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378604#comment-17378604 ] Apache Spark commented on SPARK-36083: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-36083) make_timestamp: return different result based on the default timestamp type

[ https://issues.apache.org/jira/browse/SPARK-36083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36083: Assignee: Apache Spark (was: Gengliang Wang) > make_timestamp: return different result b

[jira] [Assigned] (SPARK-36083) make_timestamp: return different result based on the default timestamp type

[ https://issues.apache.org/jira/browse/SPARK-36083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36083: Assignee: Gengliang Wang (was: Apache Spark) > make_timestamp: return different result b

[jira] [Commented] (SPARK-36046) Support new function make_timestamp_ntz

[ https://issues.apache.org/jira/browse/SPARK-36046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378603#comment-17378603 ] Gengliang Wang commented on SPARK-36046: I will work on this after https://issue

[jira] [Updated] (SPARK-36046) Support new functions make_timestamp_ntz and make_timestamp_ltz

[ https://issues.apache.org/jira/browse/SPARK-36046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-36046: --- Summary: Support new functions make_timestamp_ntz and make_timestamp_ltz (was: Support new

[jira] [Created] (SPARK-36083) make_timestamp: return different result based on the default timestamp type

Gengliang Wang created SPARK-36083: -- Summary: make_timestamp: return different result based on the default timestamp type Key: SPARK-36083 URL: https://issues.apache.org/jira/browse/SPARK-36083 Proje

[jira] [Commented] (SPARK-36082) when the right side is small enough to use SingleColumn Null Aware Anti Join

[ https://issues.apache.org/jira/browse/SPARK-36082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378594#comment-17378594 ] Apache Spark commented on SPARK-36082: -- User 'mcdull-zhang' has created a pull requ

[jira] [Assigned] (SPARK-36082) when the right side is small enough to use SingleColumn Null Aware Anti Join

[ https://issues.apache.org/jira/browse/SPARK-36082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36082: Assignee: Apache Spark > when the right side is small enough to use SingleColumn Null Awa

[jira] [Assigned] (SPARK-36082) when the right side is small enough to use SingleColumn Null Aware Anti Join

[ https://issues.apache.org/jira/browse/SPARK-36082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36082: Assignee: (was: Apache Spark) > when the right side is small enough to use SingleColu