[jira] [Assigned] (SPARK-31521) The fetch size is not correct when merging blocks into a merged block

[ https://issues.apache.org/jira/browse/SPARK-31521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31521: - Assignee: wuyi > The fetch size is not correct when merging blocks into a merged block

[jira] [Resolved] (SPARK-31521) The fetch size is not correct when merging blocks into a merged block

[ https://issues.apache.org/jira/browse/SPARK-31521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31521. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28301 [https://

[jira] [Resolved] (SPARK-31516) Non-existed metric hiveClientCalls.count of CodeGenerator in Monitoring Doc

[ https://issues.apache.org/jira/browse/SPARK-31516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31516. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28292 [https://

[jira] [Assigned] (SPARK-31516) Non-existed metric hiveClientCalls.count of CodeGenerator in Monitoring Doc

[ https://issues.apache.org/jira/browse/SPARK-31516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31516: - Assignee: ZHANG Wei > Non-existed metric hiveClientCalls.count of CodeGenerator in Moni

[jira] [Resolved] (SPARK-31560) Add V1/V2 tests for TextSuite and WholeTextFileSuite

[ https://issues.apache.org/jira/browse/SPARK-31560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31560. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28335 [https://

[jira] [Reopened] (SPARK-20732) Copy cache data when node is being shut down

[ https://issues.apache.org/jira/browse/SPARK-20732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-20732: --- > Copy cache data when node is being shut down > >

[jira] [Updated] (SPARK-20732) Copy cache data when node is being shut down

[ https://issues.apache.org/jira/browse/SPARK-20732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-20732: -- Fix Version/s: (was: 3.1.0) > Copy cache data when node is being shut down > -

[jira] [Commented] (SPARK-31554) Flaky test suite org.apache.spark.sql.hive.thriftserver.CliSuite

[ https://issues.apache.org/jira/browse/SPARK-31554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092012#comment-17092012 ] Jungtaek Lim commented on SPARK-31554: -- There're two existing PRs addressing the te

[jira] [Created] (SPARK-31561) Add QUALIFY Clause

Yuming Wang created SPARK-31561: --- Summary: Add QUALIFY Clause Key: SPARK-31561 URL: https://issues.apache.org/jira/browse/SPARK-31561 Project: Spark Issue Type: Improvement Components

[jira] [Created] (SPARK-31560) Add V1/V2 tests for TextSuite and WholeTextFileSuite

Gengliang Wang created SPARK-31560: -- Summary: Add V1/V2 tests for TextSuite and WholeTextFileSuite Key: SPARK-31560 URL: https://issues.apache.org/jira/browse/SPARK-31560 Project: Spark Issu

[jira] [Created] (SPARK-31559) AM starts with initial fetched tokens in any attempt

Jungtaek Lim created SPARK-31559: Summary: AM starts with initial fetched tokens in any attempt Key: SPARK-31559 URL: https://issues.apache.org/jira/browse/SPARK-31559 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-31552) Fix potential ClassCastException in ScalaReflection arrayClassFor

[ https://issues.apache.org/jira/browse/SPARK-31552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31552. --- Fix Version/s: 3.0.0 Assignee: Kent Yao Resolution: Fixed This is resolved v

[jira] [Assigned] (SPARK-31533) Enable DB2IntegrationSuite test and upgrade the DB2 docker inside

[ https://issues.apache.org/jira/browse/SPARK-31533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31533: - Assignee: Gabor Somogyi > Enable DB2IntegrationSuite test and upgrade the DB2 docker in

[jira] [Resolved] (SPARK-31533) Enable DB2IntegrationSuite test and upgrade the DB2 docker inside

[ https://issues.apache.org/jira/browse/SPARK-31533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31533. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28325 [https://

[jira] [Commented] (SPARK-31546) Backport SPARK-25595 Ignore corrupt Avro file if flag IGNORE_CORRUPT_FILES enabled

[ https://issues.apache.org/jira/browse/SPARK-31546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091991#comment-17091991 ] Gengliang Wang commented on SPARK-31546: I have created backport PR for this: ht

[jira] [Updated] (SPARK-31558) Code cleanup in spark-sql-viz.js

[ https://issues.apache.org/jira/browse/SPARK-31558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-31558: --- Summary: Code cleanup in spark-sql-viz.js (was: Code clean up in spark-sql-viz.js) > Code

[jira] [Created] (SPARK-31558) Code clean up in spark-sql-viz.js

Gengliang Wang created SPARK-31558: -- Summary: Code clean up in spark-sql-viz.js Key: SPARK-31558 URL: https://issues.apache.org/jira/browse/SPARK-31558 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-31557) Legacy parser incorrectly interprets pre-Gregorian dates

Bruce Robbins created SPARK-31557: - Summary: Legacy parser incorrectly interprets pre-Gregorian dates Key: SPARK-31557 URL: https://issues.apache.org/jira/browse/SPARK-31557 Project: Spark Is

[jira] [Resolved] (SPARK-31491) Re-arrange Data Types page to document Floating Point Special Values

[ https://issues.apache.org/jira/browse/SPARK-31491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-31491. -- Fix Version/s: 3.0.0 Assignee: Huaxin Gao Resolution: Fixed Resolved b

[jira] [Resolved] (SPARK-31532) SparkSessionBuilder shoud not propagate static sql configurations to the existing active/default SparkSession

[ https://issues.apache.org/jira/browse/SPARK-31532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-31532. -- Fix Version/s: 2.4.6 Assignee: Kent Yao Resolution: Fixed Resolved by

[jira] [Commented] (SPARK-31556) Document LIKE clause in SQL Reference

[ https://issues.apache.org/jira/browse/SPARK-31556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091965#comment-17091965 ] Huaxin Gao commented on SPARK-31556: https://github.com/apache/spark/pull/28332 > D

[jira] [Created] (SPARK-31556) Document LIKE clause in SQL Reference

Huaxin Gao created SPARK-31556: -- Summary: Document LIKE clause in SQL Reference Key: SPARK-31556 URL: https://issues.apache.org/jira/browse/SPARK-31556 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-31364) Benchmark Nested Parquet Predicate Pushdown

[ https://issues.apache.org/jira/browse/SPARK-31364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-31364. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28319 [https://github.com/a

[jira] [Updated] (SPARK-31364) Benchmark Nested Parquet Predicate Pushdown

[ https://issues.apache.org/jira/browse/SPARK-31364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-31364: Summary: Benchmark Nested Parquet Predicate Pushdown (was: Benchmark Parquet Predicate Pushdown) > Bench

[jira] [Updated] (SPARK-31377) Add unit tests for "number of output rows" metric for joins in SQLMetricsSuite

[ https://issues.apache.org/jira/browse/SPARK-31377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srinivas Rishindra Pothireddi updated SPARK-31377: -- Description: For some combinations of join algorithm and join

[jira] [Commented] (SPARK-31500) collect_set() of BinaryType returns duplicate elements

[ https://issues.apache.org/jira/browse/SPARK-31500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091833#comment-17091833 ] Pablo Langa Blanco commented on SPARK-31500: Hi [~ewasserman], This is a sc

[jira] [Resolved] (SPARK-20732) Copy cache data when node is being shut down

[ https://issues.apache.org/jira/browse/SPARK-20732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-20732. -- Fix Version/s: 3.1.0 Target Version/s: 3.1.0 Resolution: Fixed Fixed, thank y

[jira] [Assigned] (SPARK-20732) Copy cache data when node is being shut down

[ https://issues.apache.org/jira/browse/SPARK-20732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau reassigned SPARK-20732: Assignee: Prakhar Jain > Copy cache data when node is being shut down > -

[jira] [Created] (SPARK-31555) Improve cache block migration

Holden Karau created SPARK-31555: Summary: Improve cache block migration Key: SPARK-31555 URL: https://issues.apache.org/jira/browse/SPARK-31555 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-31007) KMeans optimization based on triangle-inequality

[ https://issues.apache.org/jira/browse/SPARK-31007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31007. -- Fix Version/s: 3.1.0 Resolution: Fixed Resolved by https://github.com/apache/spark/pull

[jira] [Commented] (SPARK-31539) Backport SPARK-27138 Remove AdminUtils calls (fixes deprecation)

[ https://issues.apache.org/jira/browse/SPARK-31539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091705#comment-17091705 ] Dylan Guedes commented on SPARK-31539: -- Agreed, I think it is not worth it. > Back

[jira] [Commented] (SPARK-31538) Backport SPARK-25338 Ensure to call super.beforeAll() and super.afterAll() in test cases

[ https://issues.apache.org/jira/browse/SPARK-31538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091699#comment-17091699 ] Kazuaki Ishizaki commented on SPARK-31538: -- We could backport this. On the othe

[jira] [Updated] (SPARK-31553) Wrong result of isInCollection for large collections

[ https://issues.apache.org/jira/browse/SPARK-31553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31553: -- Labels: correctness (was: ) > Wrong result of isInCollection for large collections >

[jira] [Commented] (SPARK-31554) Flaky test suite org.apache.spark.sql.hive.thriftserver.CliSuite

[ https://issues.apache.org/jira/browse/SPARK-31554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091636#comment-17091636 ] Wenchen Fan commented on SPARK-31554: - [~Qin Yao] do you have any clue? > Flaky tes

[jira] [Commented] (SPARK-31552) Fix potential ClassCastException in ScalaReflection arrayClassFor

[ https://issues.apache.org/jira/browse/SPARK-31552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091633#comment-17091633 ] Dongjoon Hyun commented on SPARK-31552: --- Hi, [~Qin Yao]. I updated the Affected Ve

[jira] [Updated] (SPARK-31552) Fix potential ClassCastException in ScalaReflection arrayClassFor

[ https://issues.apache.org/jira/browse/SPARK-31552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31552: -- Affects Version/s: 2.0.2 2.1.3 > Fix potential ClassCastException in Sc

[jira] [Updated] (SPARK-31552) Fix potential ClassCastException in ScalaReflection arrayClassFor

[ https://issues.apache.org/jira/browse/SPARK-31552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31552: -- Affects Version/s: 2.2.3 > Fix potential ClassCastException in ScalaReflection arrayClassFor >

[jira] [Updated] (SPARK-31552) Fix potential ClassCastException in ScalaReflection arrayClassFor

[ https://issues.apache.org/jira/browse/SPARK-31552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31552: -- Affects Version/s: 2.3.4 > Fix potential ClassCastException in ScalaReflection arrayClassFor >

[jira] [Updated] (SPARK-31552) Fix potential ClassCastException in ScalaReflection arrayClassFor

[ https://issues.apache.org/jira/browse/SPARK-31552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31552: -- Description: arrayClassFor and dataTypeFor in ScalaReflection call each other circularly, the

[jira] [Updated] (SPARK-31552) Fix potential ClassCastException in ScalaReflection arrayClassFor

[ https://issues.apache.org/jira/browse/SPARK-31552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31552: -- Affects Version/s: 2.4.5 > Fix potential ClassCastException in ScalaReflection arrayClassFor >

[jira] [Updated] (SPARK-31552) Fix potential ClassCastException in ScalaReflection arrayClassFor

[ https://issues.apache.org/jira/browse/SPARK-31552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31552: -- Description: arrayClassFor and dataTypeFor in ScalaReflection call each other circularly, the

[jira] [Updated] (SPARK-31552) Fix potential ClassCastException in ScalaReflection arrayClassFor

[ https://issues.apache.org/jira/browse/SPARK-31552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31552: -- Description: arrayClassFor and dataTypeFor in ScalaReflection call each other circularly, the

[jira] [Commented] (SPARK-31554) Flaky test suite org.apache.spark.sql.hive.thriftserver.CliSuite

[ https://issues.apache.org/jira/browse/SPARK-31554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091614#comment-17091614 ] Maxim Gekk commented on SPARK-31554: [~cloud_fan] [~hyukjin.kwon] Can I we disable t

[jira] [Created] (SPARK-31554) Flaky test suite org.apache.spark.sql.hive.thriftserver.CliSuite

Maxim Gekk created SPARK-31554: -- Summary: Flaky test suite org.apache.spark.sql.hive.thriftserver.CliSuite Key: SPARK-31554 URL: https://issues.apache.org/jira/browse/SPARK-31554 Project: Spark

[jira] [Issue Comment Deleted] (SPARK-31550) nondeterministic configurations with general meanings in sql configuration doc

[ https://issues.apache.org/jira/browse/SPARK-31550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31550: --- Comment: was deleted (was: try to specify conf in spark-defaults.conf spark.sql.warehouse.dir /tmp

[jira] [Issue Comment Deleted] (SPARK-31532) SparkSessionBuilder shoud not propagate static sql configurations to the existing active/default SparkSession

[ https://issues.apache.org/jira/browse/SPARK-31532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31532: --- Comment: was deleted (was: Thanks for your issue, these followings may not be allowed to modify aft

[jira] [Resolved] (SPARK-30724) Support 'like any' and 'like all' operators

[ https://issues.apache.org/jira/browse/SPARK-30724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-30724. -- Fix Version/s: 3.1.0 Assignee: Yuming Wang Resolution: Fixed Resolved

[jira] [Commented] (SPARK-31553) Wrong result of isInCollection for large collections

[ https://issues.apache.org/jira/browse/SPARK-31553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091490#comment-17091490 ] Maxim Gekk commented on SPARK-31553: I am working on the issue > Wrong result of is

[jira] [Created] (SPARK-31553) Wrong result of isInCollection for large collections

Maxim Gekk created SPARK-31553: -- Summary: Wrong result of isInCollection for large collections Key: SPARK-31553 URL: https://issues.apache.org/jira/browse/SPARK-31553 Project: Spark Issue Type:

[jira] [Commented] (SPARK-31463) Enhance JsonDataSource by replacing jackson with simdjson

[ https://issues.apache.org/jira/browse/SPARK-31463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091389#comment-17091389 ] Maxim Gekk commented on SPARK-31463: Parsing itself takes 10-20%. JSON datasource sp

[jira] [Created] (SPARK-31552) Fix potential ClassCastException in ScalaReflection arrayClassFor

Kent Yao created SPARK-31552: Summary: Fix potential ClassCastException in ScalaReflection arrayClassFor Key: SPARK-31552 URL: https://issues.apache.org/jira/browse/SPARK-31552 Project: Spark Is

[jira] [Assigned] (SPARK-31502) document identifier in SQL Reference

[ https://issues.apache.org/jira/browse/SPARK-31502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31502: --- Assignee: Huaxin Gao > document identifier in SQL Reference > -

[jira] [Resolved] (SPARK-31502) document identifier in SQL Reference

[ https://issues.apache.org/jira/browse/SPARK-31502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31502. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28277 [https://gith

[jira] [Updated] (SPARK-31449) Investigate the difference between JDK and Spark's time zone offset calculation

[ https://issues.apache.org/jira/browse/SPARK-31449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31449: --- Summary: Investigate the difference between JDK and Spark's time zone offset calculation (was: Is t

[jira] [Updated] (SPARK-31449) Investigate the difference between JDK and Spark's time zone offset calculation

[ https://issues.apache.org/jira/browse/SPARK-31449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31449: --- Issue Type: Improvement (was: Question) > Investigate the difference between JDK and Spark's time z

[jira] [Commented] (SPARK-31535) Fix nested CTE substitution

[ https://issues.apache.org/jira/browse/SPARK-31535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091299#comment-17091299 ] Peter Toth commented on SPARK-31535: Hmm, for some reason my PR ([https://github.com