[jira] [Commented] (SPARK-35905) Adding withTable in SQLQuerySuite about test("SPARK-33338: GROUP BY using literal map should not fail")

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369813#comment-17369813 ] Apache Spark commented on SPARK-35905: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-35905) Adding withTable in SQLQuerySuite about test("SPARK-33338: GROUP BY using literal map should not fail")

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369812#comment-17369812 ] Apache Spark commented on SPARK-35905: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-35905) Adding withTable in SQLQuerySuite about test("SPARK-33338: GROUP BY using literal map should not fail")

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35905: Assignee: (was: Apache Spark) > Adding withTable in SQLQuerySuite about test("SPAR

[jira] [Assigned] (SPARK-35905) Adding withTable in SQLQuerySuite about test("SPARK-33338: GROUP BY using literal map should not fail")

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35905: Assignee: Apache Spark > Adding withTable in SQLQuerySuite about test("SPARK-8: GR

[jira] [Created] (SPARK-35905) Adding withTable in SQLQuerySuite about test("SPARK-33338: GROUP BY using literal map should not fail")

2021-06-25 Thread angerszhu (Jira)
angerszhu created SPARK-35905: - Summary: Adding withTable in SQLQuerySuite about test("SPARK-8: GROUP BY using literal map should not fail") Key: SPARK-35905 URL: https://issues.apache.org/jira/browse/SPARK-359

[jira] [Resolved] (SPARK-35879) Fix performance regression caused by collectFetchRequests

2021-06-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-35879. -- Fix Version/s: 3.1.3 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-35879) Fix performance regression caused by collectFetchRequests

2021-06-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-35879: Assignee: Kent Yao > Fix performance regression caused by collectFetchRequests >

[jira] [Assigned] (SPARK-35904) Collapse above RebalancePartitions

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35904: Assignee: (was: Apache Spark) > Collapse above RebalancePartitions >

[jira] [Assigned] (SPARK-35904) Collapse above RebalancePartitions

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35904: Assignee: Apache Spark > Collapse above RebalancePartitions > ---

[jira] [Commented] (SPARK-35904) Collapse above RebalancePartitions

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369796#comment-17369796 ] Apache Spark commented on SPARK-35904: -- User 'wangyum' has created a pull request f

[jira] [Updated] (SPARK-35904) Collapse above RebalancePartitions

2021-06-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-35904: Description: Make RebalancePartitions extends RepartitionOperation. > Collapse above RebalancePart

[jira] [Created] (SPARK-35904) Collapse above RebalancePartitions

2021-06-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-35904: --- Summary: Collapse above RebalancePartitions Key: SPARK-35904 URL: https://issues.apache.org/jira/browse/SPARK-35904 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-35903) Parameterize `master` in TPCDSQueryBenchmark

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35903: Assignee: (was: Apache Spark) > Parameterize `master` in TPCDSQueryBenchmark > -

[jira] [Assigned] (SPARK-35903) Parameterize `master` in TPCDSQueryBenchmark

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35903: Assignee: Apache Spark > Parameterize `master` in TPCDSQueryBenchmark >

[jira] [Commented] (SPARK-35903) Parameterize `master` in TPCDSQueryBenchmark

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369786#comment-17369786 ] Apache Spark commented on SPARK-35903: -- User 'dongjoon-hyun' has created a pull req

[jira] [Updated] (SPARK-35884) EXPLAIN FORMATTED for AQE

2021-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35884: -- Parent: SPARK-33828 Issue Type: Sub-task (was: Bug) > EXPLAIN FORMATTED for AQE > ---

[jira] [Created] (SPARK-35903) Parameterize `master` in TPCDSQueryBenchmark

2021-06-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35903: - Summary: Parameterize `master` in TPCDSQueryBenchmark Key: SPARK-35903 URL: https://issues.apache.org/jira/browse/SPARK-35903 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-35902) spark.driver.log.dfsDir with hdfs scheme failed

2021-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369783#comment-17369783 ] Dongjoon Hyun commented on SPARK-35902: --- Thank you for reporting, [~yghu]. Go for

[jira] [Updated] (SPARK-35902) spark.driver.log.dfsDir with hdfs scheme failed

2021-06-25 Thread YuanGuanhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YuanGuanhu updated SPARK-35902: --- Description: when i set spark.driver.log.dfsDir value with hdfs scheme path, it throw an exception:

[jira] [Commented] (SPARK-35902) spark.driver.log.dfsDir with hdfs scheme failed

2021-06-25 Thread YuanGuanhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369780#comment-17369780 ] YuanGuanhu commented on SPARK-35902: I'd like to work on this. > spark.driver.log.d

[jira] [Updated] (SPARK-35902) spark.driver.log.dfsDir with hdfs scheme failed

2021-06-25 Thread YuanGuanhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YuanGuanhu updated SPARK-35902: --- Description: when i set spark.driver.log.dfsDir value with hdfs scheme path, it throw an exception:

[jira] [Created] (SPARK-35902) spark.driver.log.dfsDir with hdfs scheme failed

2021-06-25 Thread YuanGuanhu (Jira)
YuanGuanhu created SPARK-35902: -- Summary: spark.driver.log.dfsDir with hdfs scheme failed Key: SPARK-35902 URL: https://issues.apache.org/jira/browse/SPARK-35902 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-35466) Enable disallow_untyped_defs mypy check for pyspark.pandas.data_type_ops.

2021-06-25 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35466. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Assigned] (SPARK-35901) Refine type hints in pyspark.pandas.window

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35901: Assignee: (was: Apache Spark) > Refine type hints in pyspark.pandas.window >

[jira] [Assigned] (SPARK-35901) Refine type hints in pyspark.pandas.window

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35901: Assignee: Apache Spark > Refine type hints in pyspark.pandas.window > ---

[jira] [Commented] (SPARK-35901) Refine type hints in pyspark.pandas.window

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369776#comment-17369776 ] Apache Spark commented on SPARK-35901: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-35899) Add a utility to convert connector expressions to Catalyst expressions

2021-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35899: - Assignee: Anton Okolnychyi > Add a utility to convert connector expressions to Catalyst

[jira] [Resolved] (SPARK-35899) Add a utility to convert connector expressions to Catalyst expressions

2021-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35899. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33096 [https://

[jira] [Resolved] (SPARK-35894) Introduce new style enforce to not import scala.collection.Seq/IndexedSeq

2021-06-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-35894. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33085 [https://gi

[jira] [Assigned] (SPARK-35894) Introduce new style enforce to not import scala.collection.Seq/IndexedSeq

2021-06-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-35894: Assignee: Jungtaek Lim > Introduce new style enforce to not import scala.collection.Seq/I

[jira] [Created] (SPARK-35901) Refine type hints in pyspark.pandas.window

2021-06-25 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35901: - Summary: Refine type hints in pyspark.pandas.window Key: SPARK-35901 URL: https://issues.apache.org/jira/browse/SPARK-35901 Project: Spark Issue Type: Sub-

[jira] [Assigned] (SPARK-35899) Add a utility to convert connector expressions to Catalyst expressions

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35899: Assignee: Apache Spark > Add a utility to convert connector expressions to Catalyst expre

[jira] [Commented] (SPARK-35899) Add a utility to convert connector expressions to Catalyst expressions

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369760#comment-17369760 ] Apache Spark commented on SPARK-35899: -- User 'aokolnychyi' has created a pull reque

[jira] [Assigned] (SPARK-35899) Add a utility to convert connector expressions to Catalyst expressions

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35899: Assignee: (was: Apache Spark) > Add a utility to convert connector expressions to Cat

[jira] [Commented] (SPARK-35339) Improve unit tests for data-type-based basic operations

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369753#comment-17369753 ] Apache Spark commented on SPARK-35339: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-35339) Improve unit tests for data-type-based basic operations

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35339: Assignee: (was: Apache Spark) > Improve unit tests for data-type-based basic operatio

[jira] [Assigned] (SPARK-35339) Improve unit tests for data-type-based basic operations

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35339: Assignee: Apache Spark > Improve unit tests for data-type-based basic operations > --

[jira] [Updated] (SPARK-35339) Improve unit tests for data-type-based basic operations

2021-06-25 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35339: - Summary: Improve unit tests for data-type-based basic operations (was: Consolidate unit tests f

[jira] [Assigned] (SPARK-35466) Enable disallow_untyped_defs mypy check for pyspark.pandas.data_type_ops.

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35466: Assignee: Apache Spark > Enable disallow_untyped_defs mypy check for pyspark.pandas.data_

[jira] [Assigned] (SPARK-35466) Enable disallow_untyped_defs mypy check for pyspark.pandas.data_type_ops.

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35466: Assignee: (was: Apache Spark) > Enable disallow_untyped_defs mypy check for pyspark.p

[jira] [Commented] (SPARK-35466) Enable disallow_untyped_defs mypy check for pyspark.pandas.data_type_ops.

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369654#comment-17369654 ] Apache Spark commented on SPARK-35466: -- User 'ueshin' has created a pull request fo

[jira] [Created] (SPARK-35900) Consider whether we need to broadcast dynamic filtering subquery results for v2 tables

2021-06-25 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-35900: Summary: Consider whether we need to broadcast dynamic filtering subquery results for v2 tables Key: SPARK-35900 URL: https://issues.apache.org/jira/browse/SPARK-35900

[jira] [Commented] (SPARK-35896) [SS] Include more granular metrics for stateful operators in StreamingQueryProgress

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369627#comment-17369627 ] Apache Spark commented on SPARK-35896: -- User 'vkorukanti' has created a pull reques

[jira] [Commented] (SPARK-35896) [SS] Include more granular metrics for stateful operators in StreamingQueryProgress

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369626#comment-17369626 ] Apache Spark commented on SPARK-35896: -- User 'vkorukanti' has created a pull reques

[jira] [Assigned] (SPARK-35897) Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35897: Assignee: (was: Apache Spark) > Support user defined initial state with flatMapGroups

[jira] [Assigned] (SPARK-35897) Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35897: Assignee: Apache Spark > Support user defined initial state with flatMapGroupsWithState i

[jira] [Commented] (SPARK-35897) Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369625#comment-17369625 ] Apache Spark commented on SPARK-35897: -- User 'rahulsmahadev' has created a pull req

[jira] [Commented] (SPARK-33338) GROUP BY using literal map should not fail

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369624#comment-17369624 ] Apache Spark commented on SPARK-8: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-35896) [SS] Include more granular metrics for stateful operators in StreamingQueryProgress

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35896: Assignee: (was: Apache Spark) > [SS] Include more granular metrics for stateful opera

[jira] [Assigned] (SPARK-35896) [SS] Include more granular metrics for stateful operators in StreamingQueryProgress

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35896: Assignee: Apache Spark > [SS] Include more granular metrics for stateful operators in >

[jira] [Created] (SPARK-35899) Add a utility to convert connector expressions to Catalyst expressions

2021-06-25 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-35899: Summary: Add a utility to convert connector expressions to Catalyst expressions Key: SPARK-35899 URL: https://issues.apache.org/jira/browse/SPARK-35899 Projec

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369565#comment-17369565 ] Apache Spark commented on SPARK-35672: -- User 'xkrogen' has created a pull request f

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-06-25 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369564#comment-17369564 ] Erik Krogen commented on SPARK-35672: - #32810 went into master. Put up #33090 for b

[jira] [Created] (SPARK-35898) Converting arrays with RowToColumnConverter triggers assertion

2021-06-25 Thread Tom van Bussel (Jira)
Tom van Bussel created SPARK-35898: -- Summary: Converting arrays with RowToColumnConverter triggers assertion Key: SPARK-35898 URL: https://issues.apache.org/jira/browse/SPARK-35898 Project: Spark

[jira] [Commented] (SPARK-35898) Converting arrays with RowToColumnConverter triggers assertion

2021-06-25 Thread Tom van Bussel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369553#comment-17369553 ] Tom van Bussel commented on SPARK-35898: I will open a PR with the fix. > Conve

[jira] [Created] (SPARK-35897) Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-06-25 Thread Rahul Shivu Mahadev (Jira)
Rahul Shivu Mahadev created SPARK-35897: --- Summary: Support user defined initial state with flatMapGroupsWithState in Structured Streaming Key: SPARK-35897 URL: https://issues.apache.org/jira/browse/SPARK-358

[jira] [Resolved] (SPARK-35863) Upgrade Ivy to 2.5.0

2021-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35863. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33088 [https://

[jira] [Assigned] (SPARK-35863) Upgrade Ivy to 2.5.0

2021-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35863: - Assignee: Adam Binford > Upgrade Ivy to 2.5.0 > > >

[jira] [Created] (SPARK-35896) [SS] Include more granular metrics for stateful operators in StreamingQueryProgress

2021-06-25 Thread Venki Korukanti (Jira)
Venki Korukanti created SPARK-35896: --- Summary: [SS] Include more granular metrics for stateful operators in StreamingQueryProgress Key: SPARK-35896 URL: https://issues.apache.org/jira/browse/SPARK-35896

[jira] [Commented] (SPARK-35892) numPartitions does not work when saves the RDD to database

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369487#comment-17369487 ] Apache Spark commented on SPARK-35892: -- User 'zengruios' has created a pull request

[jira] [Commented] (SPARK-35892) numPartitions does not work when saves the RDD to database

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369490#comment-17369490 ] Apache Spark commented on SPARK-35892: -- User 'zengruios' has created a pull request

[jira] [Assigned] (SPARK-35892) numPartitions does not work when saves the RDD to database

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35892: Assignee: (was: Apache Spark) > numPartitions does not work when saves the RDD to dat

[jira] [Assigned] (SPARK-35892) numPartitions does not work when saves the RDD to database

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35892: Assignee: Apache Spark > numPartitions does not work when saves the RDD to database > ---

[jira] [Commented] (SPARK-34927) Support TPCDSQueryBenchmark in Benchmarks

2021-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369486#comment-17369486 ] Dongjoon Hyun commented on SPARK-34927: --- No problem~ I was just curious if we can

[jira] [Resolved] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-06-25 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-35672. --- Fix Version/s: 3.2.0 Assignee: Erik Krogen Resolution: Fixed > Spark fails t

[jira] [Assigned] (SPARK-35893) No Unit Test case for MySQLDialect.getCatalystType

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35893: Assignee: (was: Apache Spark) > No Unit Test case for MySQLDialect.getCatalystType >

[jira] [Commented] (SPARK-35863) Upgrade Ivy to 2.5.0

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369473#comment-17369473 ] Apache Spark commented on SPARK-35863: -- User 'Kimahriman' has created a pull reques

[jira] [Assigned] (SPARK-35863) Upgrade Ivy to 2.5.0

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35863: Assignee: Apache Spark > Upgrade Ivy to 2.5.0 > > >

[jira] [Commented] (SPARK-35893) No Unit Test case for MySQLDialect.getCatalystType

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369474#comment-17369474 ] Apache Spark commented on SPARK-35893: -- User 'zengruios' has created a pull request

[jira] [Assigned] (SPARK-35863) Upgrade Ivy to 2.5.0

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35863: Assignee: (was: Apache Spark) > Upgrade Ivy to 2.5.0 > > >

[jira] [Assigned] (SPARK-35893) No Unit Test case for MySQLDialect.getCatalystType

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35893: Assignee: Apache Spark > No Unit Test case for MySQLDialect.getCatalystType > --

[jira] [Assigned] (SPARK-35878) add fs.s3a.endpoint if unset and fs.s3a.endpoint.region is null

2021-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35878: - Assignee: Steve Loughran > add fs.s3a.endpoint if unset and fs.s3a.endpoint.region is n

[jira] [Resolved] (SPARK-35878) add fs.s3a.endpoint if unset and fs.s3a.endpoint.region is null

2021-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35878. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33064 [https://

[jira] [Assigned] (SPARK-35895) Support subtracting Intervals from TimestampWithoutTZ

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35895: Assignee: Gengliang Wang (was: Apache Spark) > Support subtracting Intervals from Timest

[jira] [Commented] (SPARK-35895) Support subtracting Intervals from TimestampWithoutTZ

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369436#comment-17369436 ] Apache Spark commented on SPARK-35895: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-35895) Support subtracting Intervals from TimestampWithoutTZ

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35895: Assignee: Apache Spark (was: Gengliang Wang) > Support subtracting Intervals from Timest

[jira] [Assigned] (SPARK-35895) Support subtracting Intervals from TimestampWithoutTZ

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35895: Assignee: Apache Spark (was: Gengliang Wang) > Support subtracting Intervals from Timest

[jira] [Assigned] (SPARK-35894) Introduce new style enforce to not import scala.collection.Seq/IndexedSeq

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35894: Assignee: (was: Apache Spark) > Introduce new style enforce to not import scala.colle

[jira] [Commented] (SPARK-35894) Introduce new style enforce to not import scala.collection.Seq/IndexedSeq

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369435#comment-17369435 ] Apache Spark commented on SPARK-35894: -- User 'HeartSaVioR' has created a pull reque

[jira] [Updated] (SPARK-35895) Support subtracting Intervals from TimestampWithoutTZ

2021-06-25 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-35895: --- Summary: Support subtracting Intervals from TimestampWithoutTZ (was: Support subtract Inter

[jira] [Assigned] (SPARK-35894) Introduce new style enforce to not import scala.collection.Seq/IndexedSeq

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35894: Assignee: Apache Spark > Introduce new style enforce to not import scala.collection.Seq/I

[jira] [Created] (SPARK-35895) Support subtract Intervals from TimestampWithoutTZ

2021-06-25 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35895: -- Summary: Support subtract Intervals from TimestampWithoutTZ Key: SPARK-35895 URL: https://issues.apache.org/jira/browse/SPARK-35895 Project: Spark Issue

[jira] [Updated] (SPARK-35894) Introduce new style enforce to not import scala.collection.Seq/IndexedSeq

2021-06-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-35894: - Summary: Introduce new style enforce to not import scala.collection.Seq/IndexedSeq (was: Introd

[jira] [Created] (SPARK-35894) Introduce new style enforce to not import scala.collection.Seq

2021-06-25 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-35894: Summary: Introduce new style enforce to not import scala.collection.Seq Key: SPARK-35894 URL: https://issues.apache.org/jira/browse/SPARK-35894 Project: Spark

[jira] [Resolved] (SPARK-35889) Support adding TimestampWithoutTZ with Interval types

2021-06-25 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35889. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33076 [https:

[jira] [Commented] (SPARK-35628) RocksDBFileManager - load checkpoint from DFS

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369423#comment-17369423 ] Apache Spark commented on SPARK-35628: -- User 'xuanyuanking' has created a pull requ

[jira] [Created] (SPARK-35893) No Unit Test case for MySQLDialect.getCatalystType

2021-06-25 Thread zengrui (Jira)
zengrui created SPARK-35893: --- Summary: No Unit Test case for MySQLDialect.getCatalystType Key: SPARK-35893 URL: https://issues.apache.org/jira/browse/SPARK-35893 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-35886) Codegen issue for decimal type

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369363#comment-17369363 ] Apache Spark commented on SPARK-35886: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-35886) Codegen issue for decimal type

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35886: Assignee: (was: Apache Spark) > Codegen issue for decimal type >

[jira] [Commented] (SPARK-35886) Codegen issue for decimal type

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369362#comment-17369362 ] Apache Spark commented on SPARK-35886: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-35886) Codegen issue for decimal type

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35886: Assignee: Apache Spark > Codegen issue for decimal type > --

[jira] [Assigned] (SPARK-35628) RocksDBFileManager - load checkpoint from DFS

2021-06-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-35628: Assignee: Yuanjian Li > RocksDBFileManager - load checkpoint from DFS > -

[jira] [Resolved] (SPARK-35628) RocksDBFileManager - load checkpoint from DFS

2021-06-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-35628. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32767 [https://gi

[jira] [Assigned] (SPARK-34893) Support native session window

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34893: Assignee: Apache Spark > Support native session window > - >

[jira] [Assigned] (SPARK-34893) Support native session window

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34893: Assignee: (was: Apache Spark) > Support native session window > -

[jira] [Commented] (SPARK-34893) Support native session window

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369340#comment-17369340 ] Apache Spark commented on SPARK-34893: -- User 'HeartSaVioR' has created a pull reque

[jira] [Updated] (SPARK-35892) numPartitions does not work when saves the RDD to database

2021-06-25 Thread zengrui (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zengrui updated SPARK-35892: Description: When use SQL to insert data in spark to database, suppose the original RDD's partition num i

[jira] [Commented] (SPARK-35778) Check multiply/divide of year-month intervals of any fields by numeric

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369323#comment-17369323 ] Apache Spark commented on SPARK-35778: -- User 'Peng-Lei' has created a pull request

[jira] [Commented] (SPARK-35728) Check multiply/divide of day-time intervals of any fields by numeric

2021-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17369321#comment-17369321 ] Apache Spark commented on SPARK-35728: -- User 'Peng-Lei' has created a pull request

[jira] [Created] (SPARK-35892) numPartitions does not work when saves the RDD to database

2021-06-25 Thread zengrui (Jira)
zengrui created SPARK-35892: --- Summary: numPartitions does not work when saves the RDD to database Key: SPARK-35892 URL: https://issues.apache.org/jira/browse/SPARK-35892 Project: Spark Issue Type:

  1   2   >