[jira] [Created] (SPARK-33279) Spark 3.0 failure due to lack of Arrow dependency

2020-10-28 Thread Liya Fan (Jira)
Liya Fan created SPARK-33279: Summary: Spark 3.0 failure due to lack of Arrow dependency Key: SPARK-33279 URL: https://issues.apache.org/jira/browse/SPARK-33279 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-33278) Improve the performance for FIRST_VALUE

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33278: Assignee: (was: Apache Spark) > Improve the performance for FIRST_VALUE >

[jira] [Commented] (SPARK-33278) Improve the performance for FIRST_VALUE

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222689#comment-17222689 ] Apache Spark commented on SPARK-33278: -- User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-33278) Improve the performance for FIRST_VALUE

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222688#comment-17222688 ] Apache Spark commented on SPARK-33278: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33278) Improve the performance for FIRST_VALUE

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33278: Assignee: Apache Spark > Improve the performance for FIRST_VALUE >

[jira] [Commented] (SPARK-33186) Respect ANSI standard and change FIRST_VALUE as pure window function.

2020-10-28 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222690#comment-17222690 ] jiaan.geng commented on SPARK-33186: Because this will lead to break change for first_value, we

[jira] [Assigned] (SPARK-33208) Update the document of SparkSession#sql

2020-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33208: --- Assignee: zhoukeyong > Update the document of SparkSession#sql >

[jira] [Updated] (SPARK-33278) Improve the performance for FIRST_VALUE

2020-10-28 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-33278: --- Description: https://issues.apache.org/jira/browse/SPARK-32934 provides a performance improvement

[jira] [Updated] (SPARK-33278) Improve the performance for FIRST_VALUE

2020-10-28 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-33278: --- Description: https://issues.apache.org/jira/browse/SPARK-32934 provides a performance improvement

[jira] [Updated] (SPARK-33278) Improve the performance for FIRST_VALUE

2020-10-28 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-33278: --- Description: https://issues.apache.org/jira/browse/SPARK-32934 provides a performance improvement

[jira] [Created] (SPARK-33278) Improve the performance for FIRST_VALUE

2020-10-28 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-33278: -- Summary: Improve the performance for FIRST_VALUE Key: SPARK-33278 URL: https://issues.apache.org/jira/browse/SPARK-33278 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-33208) Update the document of SparkSession#sql

2020-10-28 Thread zhoukeyong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222668#comment-17222668 ] zhoukeyong commented on SPARK-33208: I will work on this issue. > Update the document of

[jira] [Updated] (SPARK-33277) Python/Pandas UDF right after off-heap vectorized reader could cause executor crash.

2020-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33277: - Affects Version/s: 2.4.7 > Python/Pandas UDF right after off-heap vectorized reader could cause

[jira] [Resolved] (SPARK-33270) from_json() cannot parse schema from schema_of_json()

2020-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33270. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30172

[jira] [Assigned] (SPARK-33270) from_json() cannot parse schema from schema_of_json()

2020-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33270: Assignee: Maxim Gekk > from_json() cannot parse schema from schema_of_json() >

[jira] [Commented] (SPARK-33277) Python/Pandas UDF right after off-heap vectorized reader could cause executor crash.

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222634#comment-17222634 ] Apache Spark commented on SPARK-33277: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-33277) Python/Pandas UDF right after off-heap vectorized reader could cause executor crash.

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222635#comment-17222635 ] Apache Spark commented on SPARK-33277: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33277) Python/Pandas UDF right after off-heap vectorized reader could cause executor crash.

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33277: Assignee: Apache Spark > Python/Pandas UDF right after off-heap vectorized reader could

[jira] [Assigned] (SPARK-33277) Python/Pandas UDF right after off-heap vectorized reader could cause executor crash.

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33277: Assignee: (was: Apache Spark) > Python/Pandas UDF right after off-heap vectorized

[jira] [Created] (SPARK-33277) Python/Pandas UDF right after off-heap vectorized reader could cause executor crash.

2020-10-28 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-33277: - Summary: Python/Pandas UDF right after off-heap vectorized reader could cause executor crash. Key: SPARK-33277 URL: https://issues.apache.org/jira/browse/SPARK-33277

[jira] [Updated] (SPARK-32661) Spark executors on K8S do not request extra memory for off-heap allocations

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32661: -- Affects Version/s: 3.1.0 3.0.1 > Spark executors on K8S do not request

[jira] [Updated] (SPARK-32661) Spark executors on K8S do not request extra memory for off-heap allocations

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32661: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Bug) > Spark executors on K8S do not

[jira] [Created] (SPARK-33276) Fix K8s IT Flakiness

2020-10-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33276: - Summary: Fix K8s IT Flakiness Key: SPARK-33276 URL: https://issues.apache.org/jira/browse/SPARK-33276 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-33272) prune the attributes mapping in QueryPlan.transformUpWithNewOutput

2020-10-28 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-33272. -- Fix Version/s: 3.1.0 Resolution: Fixed Resolved by 

[jira] [Updated] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32119: -- Affects Version/s: 3.0.0 > ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes

[jira] [Updated] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32119: -- Fix Version/s: 3.0.2 > ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes

[jira] [Updated] (SPARK-33071) Join with ambiguous column succeeding but giving wrong output

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33071: -- Priority: Critical (was: Major) > Join with ambiguous column succeeding but giving wrong

[jira] [Updated] (SPARK-33071) Join with ambiguous column succeeding but giving wrong output

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33071: -- Affects Version/s: 3.1.0 > Join with ambiguous column succeeding but giving wrong output >

[jira] [Commented] (SPARK-33027) Add DisableUnnecessaryBucketedScan rule to AQE queryStagePreparationRules

2020-10-28 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222310#comment-17222310 ] Cheng Su commented on SPARK-33027: -- [~cloud_fan] - will add in this week, sorry for delaying. > Add

[jira] [Updated] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32119: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Bug) > ExecutorPlugin doesn't work

[jira] [Commented] (SPARK-33008) Division by zero on divide-like operations returns incorrect result

2020-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1797#comment-1797 ] Wenchen Fan commented on SPARK-33008: - I've changed the parent ticket, as this task is more related

[jira] [Updated] (SPARK-33008) Division by zero on divide-like operations returns incorrect result

2020-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-33008: Parent Issue: SPARK-33275 (was: SPARK-30374) > Division by zero on divide-like operations

[jira] [Updated] (SPARK-33275) ANSI mode: runtime errors instead of returning null

2020-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-33275: Description: We should respect the ANSI mode in more places. What we have done so far are mostly

[jira] [Updated] (SPARK-33275) ANSI mode: runtime errors instead of returning null

2020-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-33275: Description: We should respect the ANSI mode in more places. What we have done so far are mostly

[jira] [Created] (SPARK-33275) ANSI mode: runtime errors instead of returning null

2020-10-28 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-33275: --- Summary: ANSI mode: runtime errors instead of returning null Key: SPARK-33275 URL: https://issues.apache.org/jira/browse/SPARK-33275 Project: Spark Issue

[jira] [Resolved] (SPARK-33267) Query with having null in "in" condition against data source V2 source table supporting push down filter fails with NPE

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33267. --- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-33267) Query with having null in "in" condition against data source V2 source table supporting push down filter fails with NPE

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33267: - Assignee: Jungtaek Lim > Query with having null in "in" condition against data source

[jira] [Assigned] (SPARK-33274) [SS] Fix job hang in cp mode when total cores less than total kafka partition

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33274: Assignee: Apache Spark > [SS] Fix job hang in cp mode when total cores less than total

[jira] [Assigned] (SPARK-33274) [SS] Fix job hang in cp mode when total cores less than total kafka partition

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33274: Assignee: (was: Apache Spark) > [SS] Fix job hang in cp mode when total cores less

[jira] [Commented] (SPARK-33274) [SS] Fix job hang in cp mode when total cores less than total kafka partition

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1763#comment-1763 ] Apache Spark commented on SPARK-33274: -- User 'warrenzhu25' has created a pull request for this

[jira] [Created] (SPARK-33274) [SS] Fix job hang in cp mode when total cores less than total kafka partition

2020-10-28 Thread Zhongwei Zhu (Jira)
Zhongwei Zhu created SPARK-33274: Summary: [SS] Fix job hang in cp mode when total cores less than total kafka partition Key: SPARK-33274 URL: https://issues.apache.org/jira/browse/SPARK-33274

[jira] [Commented] (SPARK-33150) Groupby key may not be unique when using window

2020-10-28 Thread Dieter De Paepe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1748#comment-1748 ] Dieter De Paepe commented on SPARK-33150: - I think the core of the problem is that the internal

[jira] [Commented] (SPARK-33271) Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1739#comment-1739 ] Apache Spark commented on SPARK-33271: -- User 'BigaDev' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33271) Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33271: Assignee: (was: Apache Spark) > Hadoop Free Build Setup for Spark 2.4.7 on

[jira] [Assigned] (SPARK-33271) Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33271: Assignee: Apache Spark > Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes >

[jira] [Created] (SPARK-33273) Fix Flaky Test: ThriftServerQueryTestSuite. subquery_scalar_subquery_scalar_subquery_select_sql

2020-10-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33273: - Summary: Fix Flaky Test: ThriftServerQueryTestSuite. subquery_scalar_subquery_scalar_subquery_select_sql Key: SPARK-33273 URL: https://issues.apache.org/jira/browse/SPARK-33273

[jira] [Commented] (SPARK-33272) prune the attributes mapping in QueryPlan.transformUpWithNewOutput

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1726#comment-1726 ] Apache Spark commented on SPARK-33272: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33272) prune the attributes mapping in QueryPlan.transformUpWithNewOutput

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33272: Assignee: Apache Spark (was: Wenchen Fan) > prune the attributes mapping in

[jira] [Assigned] (SPARK-33272) prune the attributes mapping in QueryPlan.transformUpWithNewOutput

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33272: Assignee: Wenchen Fan (was: Apache Spark) > prune the attributes mapping in

[jira] [Created] (SPARK-33272) prune the attributes mapping in QueryPlan.transformUpWithNewOutput

2020-10-28 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-33272: --- Summary: prune the attributes mapping in QueryPlan.transformUpWithNewOutput Key: SPARK-33272 URL: https://issues.apache.org/jira/browse/SPARK-33272 Project: Spark

[jira] [Resolved] (SPARK-33268) Fix bugs for casting data from/to PythonUserDefinedType

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33268. --- Fix Version/s: 3.1.0 Assignee: Takeshi Yamamuro Resolution: Fixed This is

[jira] [Updated] (SPARK-33267) Query with having null in "in" condition against data source V2 source table supporting push down filter fails with NPE

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33267: -- Priority: Critical (was: Major) > Query with having null in "in" condition against data

[jira] [Created] (SPARK-33271) Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes

2020-10-28 Thread Ahmed Gaber (Jira)
Ahmed Gaber created SPARK-33271: --- Summary: Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes Key: SPARK-33271 URL: https://issues.apache.org/jira/browse/SPARK-33271 Project: Spark Issue

[jira] [Assigned] (SPARK-33270) from_json() cannot parse schema from schema_of_json()

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33270: Assignee: (was: Apache Spark) > from_json() cannot parse schema from

[jira] [Commented] (SPARK-33270) from_json() cannot parse schema from schema_of_json()

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222197#comment-17222197 ] Apache Spark commented on SPARK-33270: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33270) from_json() cannot parse schema from schema_of_json()

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33270: Assignee: Apache Spark > from_json() cannot parse schema from schema_of_json() >

[jira] [Created] (SPARK-33270) from_json() cannot parse schema from schema_of_json()

2020-10-28 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33270: -- Summary: from_json() cannot parse schema from schema_of_json() Key: SPARK-33270 URL: https://issues.apache.org/jira/browse/SPARK-33270 Project: Spark Issue

[jira] [Resolved] (SPARK-33208) Update the document of SparkSession#sql

2020-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33208. - Fix Version/s: 3.1.0 3.0.2 Resolution: Fixed > Update the document of

[jira] [Resolved] (SPARK-33269) Ignore ".bsp/" directory in Git

2020-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33269. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30171

[jira] [Assigned] (SPARK-33269) Ignore ".bsp/" directory in Git

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33269: Assignee: Gengliang Wang (was: Apache Spark) > Ignore ".bsp/" directory in Git >

[jira] [Assigned] (SPARK-33269) Ignore ".bsp/" directory in Git

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33269: Assignee: Apache Spark (was: Gengliang Wang) > Ignore ".bsp/" directory in Git >

[jira] [Commented] (SPARK-33269) Ignore ".bsp/" directory in Git

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222132#comment-17222132 ] Apache Spark commented on SPARK-33269: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-33267) Query with having null in "in" condition against data source V2 source table supporting push down filter fails with NPE

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222131#comment-17222131 ] Apache Spark commented on SPARK-33267: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-33267) Query with having null in "in" condition against data source V2 source table supporting push down filter fails with NPE

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33267: Assignee: Apache Spark > Query with having null in "in" condition against data source V2

[jira] [Assigned] (SPARK-33267) Query with having null in "in" condition against data source V2 source table supporting push down filter fails with NPE

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33267: Assignee: (was: Apache Spark) > Query with having null in "in" condition against

[jira] [Commented] (SPARK-33268) Fix bugs for casting data from/to PythonUserDefinedType

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222128#comment-17222128 ] Apache Spark commented on SPARK-33268: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33268) Fix bugs for casting data from/to PythonUserDefinedType

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33268: Assignee: (was: Apache Spark) > Fix bugs for casting data from/to

[jira] [Commented] (SPARK-33268) Fix bugs for casting data from/to PythonUserDefinedType

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222127#comment-17222127 ] Apache Spark commented on SPARK-33268: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33268) Fix bugs for casting data from/to PythonUserDefinedType

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33268: Assignee: Apache Spark > Fix bugs for casting data from/to PythonUserDefinedType >

[jira] [Created] (SPARK-33268) Fix bugs for casting data from/to PythonUserDefinedType

2020-10-28 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-33268: Summary: Fix bugs for casting data from/to PythonUserDefinedType Key: SPARK-33268 URL: https://issues.apache.org/jira/browse/SPARK-33268 Project: Spark

[jira] [Updated] (SPARK-33269) Ignore ".bsp/" directory in Git

2020-10-28 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-33269: --- Priority: Minor (was: Major) > Ignore ".bsp/" directory in Git >

[jira] [Created] (SPARK-33269) Ignore ".bsp/" directory in Git

2020-10-28 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-33269: -- Summary: Ignore ".bsp/" directory in Git Key: SPARK-33269 URL: https://issues.apache.org/jira/browse/SPARK-33269 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-33269) Ignore ".bsp/" directory in Git

2020-10-28 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-33269: --- Issue Type: Improvement (was: Bug) > Ignore ".bsp/" directory in Git >

[jira] [Commented] (SPARK-33267) Query with having null in "in" condition against data source V2 source table supporting push down filter fails with NPE

2020-10-28 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222122#comment-17222122 ] Jungtaek Lim commented on SPARK-33267: -- Will submit a PR soon. > Query with having null in "in"

[jira] [Updated] (SPARK-33267) Query with having null in "in" condition against data source V2 source table supporting push down filter fails with NPE

2020-10-28 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-33267: - Description: The query with having null in "in" condition against data source V2 source table

[jira] [Updated] (SPARK-33267) Query with having null in "in" condition against data source V2 source table supporting push down filter fails with NPE

2020-10-28 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-33267: - Summary: Query with having null in "in" condition against data source V2 source table

[jira] [Created] (SPARK-33267) Query with having null in "in" condition against data source V2 source table fails with NPE

2020-10-28 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-33267: Summary: Query with having null in "in" condition against data source V2 source table fails with NPE Key: SPARK-33267 URL: https://issues.apache.org/jira/browse/SPARK-33267

[jira] [Commented] (SPARK-33027) Add DisableUnnecessaryBucketedScan rule to AQE queryStagePreparationRules

2020-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222041#comment-17222041 ] Wenchen Fan commented on SPARK-33027: - [~chengsu] any progress on it? > Add

[jira] [Commented] (SPARK-33208) Update the document of SparkSession#sql

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222013#comment-17222013 ] Apache Spark commented on SPARK-33208: -- User 'waitinfuture' has created a pull request for this

[jira] [Assigned] (SPARK-33208) Update the document of SparkSession#sql

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33208: Assignee: Apache Spark > Update the document of SparkSession#sql >

[jira] [Assigned] (SPARK-33208) Update the document of SparkSession#sql

2020-10-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33208: Assignee: (was: Apache Spark) > Update the document of SparkSession#sql >

[jira] [Updated] (SPARK-33183) Bug in optimizer rule EliminateSorts

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33183: -- Labels: correctness (was: ) > Bug in optimizer rule EliminateSorts >

[jira] [Assigned] (SPARK-32934) Improve the performance for NTH_VALUE and Reactor the OffsetWindowFunction

2020-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32934: --- Assignee: jiaan.geng > Improve the performance for NTH_VALUE and Reactor the

[jira] [Resolved] (SPARK-32934) Improve the performance for NTH_VALUE and Reactor the OffsetWindowFunction

2020-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32934. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29800

[jira] [Commented] (SPARK-33183) Bug in optimizer rule EliminateSorts

2020-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221964#comment-17221964 ] Dongjoon Hyun commented on SPARK-33183: --- [~cloud_fan]. Could you label this as `correctness`? >