[jira] [Commented] (SPARK-33230) FileOutputWriter jobs have duplicate JobIDs if launched in same second

2020-10-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221963#comment-17221963 ] Dongjoon Hyun commented on SPARK-33230: --- Thank you for sharing the status, [~ste...@apache.org]!

[jira] [Resolved] (SPARK-33183) Bug in optimizer rule EliminateSorts

2020-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33183. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30093

[jira] [Assigned] (SPARK-33183) Bug in optimizer rule EliminateSorts

2020-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33183: --- Assignee: Allison Wang > Bug in optimizer rule EliminateSorts >

[jira] [Assigned] (SPARK-33174) Migrate DROP TABLE to new resolution framework

2020-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33174: --- Assignee: Terry Kim > Migrate DROP TABLE to new resolution framework >

[jira] [Resolved] (SPARK-33174) Migrate DROP TABLE to new resolution framework

2020-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33174. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30079

[jira] [Commented] (SPARK-33240) Fail fast when fails to instantiate configured v2 session catalog

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221955#comment-17221955 ] Apache Spark commented on SPARK-33240: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Commented] (SPARK-33240) Fail fast when fails to instantiate configured v2 session catalog

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221954#comment-17221954 ] Apache Spark commented on SPARK-33240: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Resolved] (SPARK-33240) Fail fast when fails to instantiate configured v2 session catalog

2020-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33240. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30147

[jira] [Assigned] (SPARK-33240) Fail fast when fails to instantiate configured v2 session catalog

2020-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33240: --- Assignee: Jungtaek Lim > Fail fast when fails to instantiate configured v2 session catalog

[jira] [Updated] (SPARK-33266) Add total duration, read duration, and write duration as task level metrics

2020-10-27 Thread Noritaka Sekiyama (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noritaka Sekiyama updated SPARK-33266: -- Description: Sometimes we need to identify performance bottlenecks, for example, how

[jira] [Created] (SPARK-33266) Add total duration, read duration, and write duration as task level metrics

2020-10-27 Thread Noritaka Sekiyama (Jira)
Noritaka Sekiyama created SPARK-33266: - Summary: Add total duration, read duration, and write duration as task level metrics Key: SPARK-33266 URL: https://issues.apache.org/jira/browse/SPARK-33266

[jira] [Resolved] (SPARK-33264) Add a dedicated page for SQL-on-file in SQL documents

2020-10-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-33264. -- Fix Version/s: 3.1.0 3.0.2 Assignee: Takeshi Yamamuro

[jira] [Assigned] (SPARK-33265) Rename classOf[Seq] to classOf[scala.collection.Seq] in PostgresIntegrationSuite for Scala 2.13

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33265: Assignee: Apache Spark (was: Kousuke Saruta) > Rename classOf[Seq] to

[jira] [Commented] (SPARK-33265) Rename classOf[Seq] to classOf[scala.collection.Seq] in PostgresIntegrationSuite for Scala 2.13

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221898#comment-17221898 ] Apache Spark commented on SPARK-33265: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33265) Rename classOf[Seq] to classOf[scala.collection.Seq] in PostgresIntegrationSuite for Scala 2.13

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33265: Assignee: Apache Spark (was: Kousuke Saruta) > Rename classOf[Seq] to

[jira] [Assigned] (SPARK-33265) Rename classOf[Seq] to classOf[scala.collection.Seq] in PostgresIntegrationSuite for Scala 2.13

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33265: Assignee: Kousuke Saruta (was: Apache Spark) > Rename classOf[Seq] to

[jira] [Commented] (SPARK-33265) Rename classOf[Seq] to classOf[scala.collection.Seq] in PostgresIntegrationSuite for Scala 2.13

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221899#comment-17221899 ] Apache Spark commented on SPARK-33265: -- User 'sarutak' has created a pull request for this issue:

[jira] [Updated] (SPARK-33265) Rename classOf[Seq] to classOf[scala.collection.Seq] in PostgresIntegrationSuite for Scala 2.13

2020-10-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-33265: --- Priority: Minor (was: Major) > Rename classOf[Seq] to classOf[scala.collection.Seq] in >

[jira] [Created] (SPARK-33265) Rename classOf[Seq] to classOf[scala.collection.Seq] in PostgresIntegrationSuite for Scala 2.13

2020-10-27 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-33265: -- Summary: Rename classOf[Seq] to classOf[scala.collection.Seq] in PostgresIntegrationSuite for Scala 2.13 Key: SPARK-33265 URL:

[jira] [Assigned] (SPARK-33204) `Event Timeline` in Spark Job UI sometimes cannot be opened

2020-10-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta reassigned SPARK-33204: -- Assignee: akiyamaneko (was: Apache Spark) > `Event Timeline` in Spark Job UI

[jira] [Assigned] (SPARK-33264) Add a dedicated page for SQL-on-file in SQL documents

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33264: Assignee: (was: Apache Spark) > Add a dedicated page for SQL-on-file in SQL

[jira] [Commented] (SPARK-33264) Add a dedicated page for SQL-on-file in SQL documents

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221880#comment-17221880 ] Apache Spark commented on SPARK-33264: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33264) Add a dedicated page for SQL-on-file in SQL documents

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33264: Assignee: Apache Spark > Add a dedicated page for SQL-on-file in SQL documents >

[jira] [Created] (SPARK-33264) Add a dedicated page for SQL-on-file in SQL documents

2020-10-27 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-33264: Summary: Add a dedicated page for SQL-on-file in SQL documents Key: SPARK-33264 URL: https://issues.apache.org/jira/browse/SPARK-33264 Project: Spark

[jira] [Assigned] (SPARK-33258) Add asc_nulls_* and desc_nulls_* methods to SparkR

2020-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33258: Assignee: Maciej Szymkiewicz > Add asc_nulls_* and desc_nulls_* methods to SparkR >

[jira] [Resolved] (SPARK-33258) Add asc_nulls_* and desc_nulls_* methods to SparkR

2020-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33258. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30159

[jira] [Updated] (SPARK-33228) Don't uncache data when replacing an existing view having the same plan

2020-10-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-33228: - Fix Version/s: 2.4.8 > Don't uncache data when replacing an existing view having the

[jira] [Updated] (SPARK-32090) UserDefinedType.equal() does not have symmetry

2020-10-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32090: - Fix Version/s: 3.0.2 2.4.8 > UserDefinedType.equal() does not have

[jira] [Resolved] (SPARK-33246) Spark SQL null semantics documentation is incorrect

2020-10-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-33246. -- Fix Version/s: 3.1.0 3.0.2 Assignee: Stuart White

[jira] [Updated] (SPARK-33246) Spark SQL null semantics documentation is incorrect

2020-10-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-33246: - Affects Version/s: (was: 3.0.1) 3.1.0

[jira] [Updated] (SPARK-33246) Spark SQL null semantics documentation is incorrect

2020-10-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-33246: - Component/s: SQL > Spark SQL null semantics documentation is incorrect >

[jira] [Updated] (SPARK-33183) Bug in optimizer rule EliminateSorts

2020-10-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-33183: - Issue Type: Bug (was: Improvement) > Bug in optimizer rule EliminateSorts >

[jira] [Updated] (SPARK-33183) Bug in optimizer rule EliminateSorts

2020-10-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-33183: - Issue Type: Improvement (was: Bug) > Bug in optimizer rule EliminateSorts >

[jira] [Assigned] (SPARK-32919) Add support in Spark driver to coordinate the shuffle map stage in push-based shuffle by selecting external shuffle services for merging shuffle partitions

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32919: Assignee: (was: Apache Spark) > Add support in Spark driver to coordinate the

[jira] [Assigned] (SPARK-32919) Add support in Spark driver to coordinate the shuffle map stage in push-based shuffle by selecting external shuffle services for merging shuffle partitions

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32919: Assignee: Apache Spark > Add support in Spark driver to coordinate the shuffle map stage

[jira] [Assigned] (SPARK-32918) RPC implementation to support control plane coordination for push-based shuffle

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32918: Assignee: (was: Apache Spark) > RPC implementation to support control plane

[jira] [Commented] (SPARK-32919) Add support in Spark driver to coordinate the shuffle map stage in push-based shuffle by selecting external shuffle services for merging shuffle partitions

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221822#comment-17221822 ] Apache Spark commented on SPARK-32919: -- User 'venkata91' has created a pull request for this issue:

[jira] [Commented] (SPARK-32918) RPC implementation to support control plane coordination for push-based shuffle

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221821#comment-17221821 ] Apache Spark commented on SPARK-32918: -- User 'zhouyejoe' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32918) RPC implementation to support control plane coordination for push-based shuffle

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32918: Assignee: Apache Spark > RPC implementation to support control plane coordination for

[jira] [Commented] (SPARK-32919) Add support in Spark driver to coordinate the shuffle map stage in push-based shuffle by selecting external shuffle services for merging shuffle partitions

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221824#comment-17221824 ] Apache Spark commented on SPARK-32919: -- User 'venkata91' has created a pull request for this issue:

[jira] [Commented] (SPARK-33263) Configurable StateStore compression codec

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221770#comment-17221770 ] Apache Spark commented on SPARK-33263: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33263) Configurable StateStore compression codec

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33263: Assignee: Apache Spark (was: L. C. Hsieh) > Configurable StateStore compression codec >

[jira] [Assigned] (SPARK-33263) Configurable StateStore compression codec

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33263: Assignee: L. C. Hsieh (was: Apache Spark) > Configurable StateStore compression codec >

[jira] [Created] (SPARK-33263) Configurable StateStore compression codec

2020-10-27 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-33263: --- Summary: Configurable StateStore compression codec Key: SPARK-33263 URL: https://issues.apache.org/jira/browse/SPARK-33263 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33260) SortExec produces incorrect results if sortOrder is a Stream

2020-10-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221725#comment-17221725 ] Dongjoon Hyun commented on SPARK-33260: --- I added `correctness` label. > SortExec produces

[jira] [Updated] (SPARK-33260) SortExec produces incorrect results if sortOrder is a Stream

2020-10-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33260: -- Labels: correctness (was: ) > SortExec produces incorrect results if sortOrder is a Stream >

[jira] [Resolved] (SPARK-33260) SortExec produces incorrect results if sortOrder is a Stream

2020-10-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33260. --- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by

[jira] [Resolved] (SPARK-33262) Keep pending pods in account while scheduling new pods

2020-10-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33262. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30155

[jira] [Resolved] (SPARK-33231) Make podCreationTimeout configurable

2020-10-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33231. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30155

[jira] [Assigned] (SPARK-33231) Make podCreationTimeout configurable

2020-10-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33231: - Assignee: Holden Karau > Make podCreationTimeout configurable >

[jira] [Assigned] (SPARK-33262) Keep pending pods in account while scheduling new pods

2020-10-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33262: - Assignee: Holden Karau > Keep pending pods in account while scheduling new pods >

[jira] [Assigned] (SPARK-33262) Keep pending pods in account while scheduling new pods

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33262: Assignee: (was: Apache Spark) > Keep pending pods in account while scheduling new

[jira] [Commented] (SPARK-33262) Keep pending pods in account while scheduling new pods

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221644#comment-17221644 ] Apache Spark commented on SPARK-33262: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33262) Keep pending pods in account while scheduling new pods

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33262: Assignee: Apache Spark > Keep pending pods in account while scheduling new pods >

[jira] [Created] (SPARK-33262) Keep pending pods in account while scheduling new pods

2020-10-27 Thread Holden Karau (Jira)
Holden Karau created SPARK-33262: Summary: Keep pending pods in account while scheduling new pods Key: SPARK-33262 URL: https://issues.apache.org/jira/browse/SPARK-33262 Project: Spark Issue

[jira] [Created] (SPARK-33261) Allow people to extend the pod feature steps

2020-10-27 Thread Holden Karau (Jira)
Holden Karau created SPARK-33261: Summary: Allow people to extend the pod feature steps Key: SPARK-33261 URL: https://issues.apache.org/jira/browse/SPARK-33261 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33230) FileOutputWriter jobs have duplicate JobIDs if launched in same second

2020-10-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221581#comment-17221581 ] Steve Loughran commented on SPARK-33230: Thanks. Still got some changes to work through my side

[jira] [Assigned] (SPARK-22390) Aggregate push down

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22390: Assignee: Apache Spark > Aggregate push down > --- > >

[jira] [Assigned] (SPARK-22390) Aggregate push down

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22390: Assignee: (was: Apache Spark) > Aggregate push down > --- > >

[jira] [Commented] (SPARK-22390) Aggregate push down

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221548#comment-17221548 ] Apache Spark commented on SPARK-22390: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-22390) Aggregate push down

2020-10-27 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221546#comment-17221546 ] Huaxin Gao commented on SPARK-22390: Hi [~baibaichen], I am working on this. I put it under a

[jira] [Assigned] (SPARK-33246) Spark SQL null semantics documentation is incorrect

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33246: Assignee: Apache Spark > Spark SQL null semantics documentation is incorrect >

[jira] [Commented] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-27 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221506#comment-17221506 ] Erik Krogen commented on SPARK-33138: - I see, thanks for clarifying [~cloud_fan]. > unify temp

[jira] [Commented] (SPARK-33246) Spark SQL null semantics documentation is incorrect

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221507#comment-17221507 ] Apache Spark commented on SPARK-33246: -- User 'stwhit' has created a pull request for this issue:

[jira] [Updated] (SPARK-33246) Spark SQL null semantics documentation is incorrect

2020-10-27 Thread Stuart White (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stuart White updated SPARK-33246: - Attachment: (was: null-semantics.patch) > Spark SQL null semantics documentation is

[jira] [Commented] (SPARK-33246) Spark SQL null semantics documentation is incorrect

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221505#comment-17221505 ] Apache Spark commented on SPARK-33246: -- User 'stwhit' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33246) Spark SQL null semantics documentation is incorrect

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33246: Assignee: (was: Apache Spark) > Spark SQL null semantics documentation is incorrect

[jira] [Assigned] (SPARK-33137) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (PostgreSQL dialect)

2020-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33137: --- Assignee: Huaxin Gao > Support ALTER TABLE in JDBC v2 Table Catalog: update type and

[jira] [Resolved] (SPARK-33137) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (PostgreSQL dialect)

2020-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33137. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30089

[jira] [Assigned] (SPARK-20044) Support Spark UI behind front-end reverse proxy using a path prefix

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20044: Assignee: Gengliang Wang (was: Apache Spark) > Support Spark UI behind front-end

[jira] [Commented] (SPARK-20044) Support Spark UI behind front-end reverse proxy using a path prefix

2020-10-27 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221457#comment-17221457 ] Gengliang Wang commented on SPARK-20044: I am taking this over in

[jira] [Assigned] (SPARK-20044) Support Spark UI behind front-end reverse proxy using a path prefix

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20044: Assignee: Apache Spark (was: Gengliang Wang) > Support Spark UI behind front-end

[jira] [Updated] (SPARK-20044) Support Spark UI behind front-end reverse proxy using a path prefix

2020-10-27 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-20044: --- Affects Version/s: 2.2.0 2.3.0 2.4.0

[jira] [Reopened] (SPARK-20044) Support Spark UI behind front-end reverse proxy using a path prefix

2020-10-27 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reopened SPARK-20044: Assignee: Gengliang Wang > Support Spark UI behind front-end reverse proxy using a path

[jira] [Resolved] (SPARK-33225) Extract AliasHelper trait

2020-10-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-33225. -- Fix Version/s: 3.1.0 Assignee: Tanel Kiis Resolution: Fixed Resolved

[jira] [Commented] (SPARK-33259) Joining 3 streams results in incorrect output

2020-10-27 Thread Michael (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221425#comment-17221425 ] Michael commented on SPARK-33259: - Oh I see, thanks for pointing to the section in the documentation!

[jira] [Commented] (SPARK-33259) Joining 3 streams results in incorrect output

2020-10-27 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221409#comment-17221409 ] Jungtaek Lim commented on SPARK-33259: -- As you already figured out, this is a known limitation, and

[jira] [Commented] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221392#comment-17221392 ] Wenchen Fan commented on SPARK-33138: - > It seems to me that the current behavior of re-optimizing

[jira] [Assigned] (SPARK-33140) make Analyzer rules using SQLConf.get

2020-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33140: --- Assignee: Leanken.Lin > make Analyzer rules using SQLConf.get >

[jira] [Resolved] (SPARK-33140) make Analyzer rules using SQLConf.get

2020-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33140. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30097

[jira] [Commented] (SPARK-23539) Add support for Kafka headers in Structured Streaming

2020-10-27 Thread Dongjin Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221334#comment-17221334 ] Dongjin Lee commented on SPARK-23539: - [~ckessler] Hi Calvin, It is totally up to the committers. >

[jira] [Commented] (SPARK-33260) SortExec produces incorrect results if sortOrder is a Stream

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221333#comment-17221333 ] Apache Spark commented on SPARK-33260: -- User 'ankurdave' has created a pull request for this issue:

[jira] [Commented] (SPARK-33260) SortExec produces incorrect results if sortOrder is a Stream

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221332#comment-17221332 ] Apache Spark commented on SPARK-33260: -- User 'ankurdave' has created a pull request for this issue:

[jira] [Updated] (SPARK-33260) SortExec produces incorrect results if sortOrder is a Stream

2020-10-27 Thread Ankur Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-33260: --- Affects Version/s: (was: 2.4.7) 3.0.0 > SortExec produces incorrect

[jira] [Created] (SPARK-33260) SortExec produces incorrect results if sortOrder is a Stream

2020-10-27 Thread Ankur Dave (Jira)
Ankur Dave created SPARK-33260: -- Summary: SortExec produces incorrect results if sortOrder is a Stream Key: SPARK-33260 URL: https://issues.apache.org/jira/browse/SPARK-33260 Project: Spark

[jira] [Commented] (SPARK-22390) Aggregate push down

2020-10-27 Thread Chang chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221286#comment-17221286 ] Chang chen commented on SPARK-22390: Spark 3.0 already supported JDBC DataSource v2, so is there any

[jira] [Updated] (SPARK-33259) Joining 3 streams results in incorrect output

2020-10-27 Thread Michael (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael updated SPARK-33259: Description: I encountered an issue with Structured Streaming when doing a ((A LEFT JOIN B) INNER JOIN

[jira] [Commented] (SPARK-33258) Add asc_nulls_* and desc_nulls_* methods to SparkR

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221280#comment-17221280 ] Apache Spark commented on SPARK-33258: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33258) Add asc_nulls_* and desc_nulls_* methods to SparkR

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33258: Assignee: Apache Spark > Add asc_nulls_* and desc_nulls_* methods to SparkR >

[jira] [Assigned] (SPARK-33258) Add asc_nulls_* and desc_nulls_* methods to SparkR

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33258: Assignee: (was: Apache Spark) > Add asc_nulls_* and desc_nulls_* methods to SparkR >

[jira] [Updated] (SPARK-33259) Joining 3 streams results in incorrect output

2020-10-27 Thread Michael (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael updated SPARK-33259: Description: I encountered an issue with Structured Streaming when doing a ((A LEFT JOIN B) JOIN C)

[jira] [Commented] (SPARK-33258) Add asc_nulls_* and desc_nulls_* methods to SparkR

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221279#comment-17221279 ] Apache Spark commented on SPARK-33258: -- User 'zero323' has created a pull request for this issue:

[jira] [Updated] (SPARK-33259) Joining 3 streams results in incorrect output

2020-10-27 Thread Michael (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael updated SPARK-33259: Description: I encountered an issue with Structured Streaming when doing a ((A LEFT JOIN B) JOIN C)

[jira] [Updated] (SPARK-33259) Joining 3 streams results in incorrect output

2020-10-27 Thread Michael (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael updated SPARK-33259: Description: I encountered an issue with Structured Streaming when doing a ((A LEFT JOIN B) JOIN C)

[jira] [Created] (SPARK-33259) Joining 3 streams results in incorrect output

2020-10-27 Thread Michael (Jira)
Michael created SPARK-33259: --- Summary: Joining 3 streams results in incorrect output Key: SPARK-33259 URL: https://issues.apache.org/jira/browse/SPARK-33259 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-33258) Add asc_nulls_* and desc_nulls_* methods to SparkR

2020-10-27 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33258: -- Summary: Add asc_nulls_* and desc_nulls_* methods to SparkR Key: SPARK-33258 URL: https://issues.apache.org/jira/browse/SPARK-33258 Project: Spark

[jira] [Created] (SPARK-33257) Support Column inputs in PySpark ordering functions (asc*, desc*)

2020-10-27 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33257: -- Summary: Support Column inputs in PySpark ordering functions (asc*, desc*) Key: SPARK-33257 URL: https://issues.apache.org/jira/browse/SPARK-33257

[jira] [Commented] (SPARK-33249) Add status plugin for live application

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221244#comment-17221244 ] Apache Spark commented on SPARK-33249: -- User 'kuwii' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33249) Add status plugin for live application

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33249: Assignee: Apache Spark > Add status plugin for live application >

[jira] [Commented] (SPARK-33249) Add status plugin for live application

2020-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221243#comment-17221243 ] Apache Spark commented on SPARK-33249: -- User 'kuwii' has created a pull request for this issue:

  1   2   >