Liya Fan created SPARK-33279:
Summary: Spark 3.0 failure due to lack of Arrow dependency
Key: SPARK-33279
URL: https://issues.apache.org/jira/browse/SPARK-33279
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33278:
Assignee: (was: Apache Spark)
> Improve the performance for FIRST_VALUE
>
[
https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222689#comment-17222689
]
Apache Spark commented on SPARK-33278:
--
User 'beliefer' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222688#comment-17222688
]
Apache Spark commented on SPARK-33278:
--
User 'beliefer' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33278:
Assignee: Apache Spark
> Improve the performance for FIRST_VALUE
>
[
https://issues.apache.org/jira/browse/SPARK-33186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222690#comment-17222690
]
jiaan.geng commented on SPARK-33186:
Because this will lead to break change for first_value, we
[
https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-33208:
---
Assignee: zhoukeyong
> Update the document of SparkSession#sql
>
[
https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiaan.geng updated SPARK-33278:
---
Description:
https://issues.apache.org/jira/browse/SPARK-32934 provides a performance
improvement
[
https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiaan.geng updated SPARK-33278:
---
Description:
https://issues.apache.org/jira/browse/SPARK-32934 provides a performance
improvement
[
https://issues.apache.org/jira/browse/SPARK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiaan.geng updated SPARK-33278:
---
Description: https://issues.apache.org/jira/browse/SPARK-32934 provides a
performance improvement
jiaan.geng created SPARK-33278:
--
Summary: Improve the performance for FIRST_VALUE
Key: SPARK-33278
URL: https://issues.apache.org/jira/browse/SPARK-33278
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222668#comment-17222668
]
zhoukeyong commented on SPARK-33208:
I will work on this issue.
> Update the document of
[
https://issues.apache.org/jira/browse/SPARK-33277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-33277:
-
Affects Version/s: 2.4.7
> Python/Pandas UDF right after off-heap vectorized reader could cause
[
https://issues.apache.org/jira/browse/SPARK-33270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-33270.
--
Fix Version/s: 3.1.0
Resolution: Fixed
Issue resolved by pull request 30172
[
https://issues.apache.org/jira/browse/SPARK-33270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-33270:
Assignee: Maxim Gekk
> from_json() cannot parse schema from schema_of_json()
>
[
https://issues.apache.org/jira/browse/SPARK-33277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222634#comment-17222634
]
Apache Spark commented on SPARK-33277:
--
User 'ueshin' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-33277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222635#comment-17222635
]
Apache Spark commented on SPARK-33277:
--
User 'ueshin' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-33277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33277:
Assignee: Apache Spark
> Python/Pandas UDF right after off-heap vectorized reader could
[
https://issues.apache.org/jira/browse/SPARK-33277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33277:
Assignee: (was: Apache Spark)
> Python/Pandas UDF right after off-heap vectorized
Takuya Ueshin created SPARK-33277:
-
Summary: Python/Pandas UDF right after off-heap vectorized reader
could cause executor crash.
Key: SPARK-33277
URL: https://issues.apache.org/jira/browse/SPARK-33277
[
https://issues.apache.org/jira/browse/SPARK-32661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-32661:
--
Affects Version/s: 3.1.0
3.0.1
> Spark executors on K8S do not request
[
https://issues.apache.org/jira/browse/SPARK-32661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-32661:
--
Parent: SPARK-33005
Issue Type: Sub-task (was: Bug)
> Spark executors on K8S do not
Dongjoon Hyun created SPARK-33276:
-
Summary: Fix K8s IT Flakiness
Key: SPARK-33276
URL: https://issues.apache.org/jira/browse/SPARK-33276
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-33272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro resolved SPARK-33272.
--
Fix Version/s: 3.1.0
Resolution: Fixed
Resolved by
[
https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-32119:
--
Affects Version/s: 3.0.0
> ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes
[
https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-32119:
--
Fix Version/s: 3.0.2
> ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes
[
https://issues.apache.org/jira/browse/SPARK-33071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-33071:
--
Priority: Critical (was: Major)
> Join with ambiguous column succeeding but giving wrong
[
https://issues.apache.org/jira/browse/SPARK-33071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-33071:
--
Affects Version/s: 3.1.0
> Join with ambiguous column succeeding but giving wrong output
>
[
https://issues.apache.org/jira/browse/SPARK-33027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222310#comment-17222310
]
Cheng Su commented on SPARK-33027:
--
[~cloud_fan] - will add in this week, sorry for delaying.
> Add
[
https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-32119:
--
Parent: SPARK-33005
Issue Type: Sub-task (was: Bug)
> ExecutorPlugin doesn't work
[
https://issues.apache.org/jira/browse/SPARK-33008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1797#comment-1797
]
Wenchen Fan commented on SPARK-33008:
-
I've changed the parent ticket, as this task is more related
[
https://issues.apache.org/jira/browse/SPARK-33008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan updated SPARK-33008:
Parent Issue: SPARK-33275 (was: SPARK-30374)
> Division by zero on divide-like operations
[
https://issues.apache.org/jira/browse/SPARK-33275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan updated SPARK-33275:
Description: We should respect the ANSI mode in more places. What we have
done so far are mostly
[
https://issues.apache.org/jira/browse/SPARK-33275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan updated SPARK-33275:
Description: We should respect the ANSI mode in more places. What we have
done so far are mostly
Wenchen Fan created SPARK-33275:
---
Summary: ANSI mode: runtime errors instead of returning null
Key: SPARK-33275
URL: https://issues.apache.org/jira/browse/SPARK-33275
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-33267.
---
Fix Version/s: 3.0.2
3.1.0
Resolution: Fixed
Issue resolved by
[
https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-33267:
-
Assignee: Jungtaek Lim
> Query with having null in "in" condition against data source
[
https://issues.apache.org/jira/browse/SPARK-33274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33274:
Assignee: Apache Spark
> [SS] Fix job hang in cp mode when total cores less than total
[
https://issues.apache.org/jira/browse/SPARK-33274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33274:
Assignee: (was: Apache Spark)
> [SS] Fix job hang in cp mode when total cores less
[
https://issues.apache.org/jira/browse/SPARK-33274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1763#comment-1763
]
Apache Spark commented on SPARK-33274:
--
User 'warrenzhu25' has created a pull request for this
Zhongwei Zhu created SPARK-33274:
Summary: [SS] Fix job hang in cp mode when total cores less than
total kafka partition
Key: SPARK-33274
URL: https://issues.apache.org/jira/browse/SPARK-33274
[
https://issues.apache.org/jira/browse/SPARK-33150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1748#comment-1748
]
Dieter De Paepe commented on SPARK-33150:
-
I think the core of the problem is that the internal
[
https://issues.apache.org/jira/browse/SPARK-33271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1739#comment-1739
]
Apache Spark commented on SPARK-33271:
--
User 'BigaDev' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-33271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33271:
Assignee: (was: Apache Spark)
> Hadoop Free Build Setup for Spark 2.4.7 on
[
https://issues.apache.org/jira/browse/SPARK-33271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33271:
Assignee: Apache Spark
> Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes
>
Dongjoon Hyun created SPARK-33273:
-
Summary: Fix Flaky Test: ThriftServerQueryTestSuite.
subquery_scalar_subquery_scalar_subquery_select_sql
Key: SPARK-33273
URL: https://issues.apache.org/jira/browse/SPARK-33273
[
https://issues.apache.org/jira/browse/SPARK-33272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1726#comment-1726
]
Apache Spark commented on SPARK-33272:
--
User 'cloud-fan' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-33272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33272:
Assignee: Apache Spark (was: Wenchen Fan)
> prune the attributes mapping in
[
https://issues.apache.org/jira/browse/SPARK-33272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33272:
Assignee: Wenchen Fan (was: Apache Spark)
> prune the attributes mapping in
Wenchen Fan created SPARK-33272:
---
Summary: prune the attributes mapping in
QueryPlan.transformUpWithNewOutput
Key: SPARK-33272
URL: https://issues.apache.org/jira/browse/SPARK-33272
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-33268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-33268.
---
Fix Version/s: 3.1.0
Assignee: Takeshi Yamamuro
Resolution: Fixed
This is
[
https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-33267:
--
Priority: Critical (was: Major)
> Query with having null in "in" condition against data
Ahmed Gaber created SPARK-33271:
---
Summary: Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes
Key: SPARK-33271
URL: https://issues.apache.org/jira/browse/SPARK-33271
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-33270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33270:
Assignee: (was: Apache Spark)
> from_json() cannot parse schema from
[
https://issues.apache.org/jira/browse/SPARK-33270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222197#comment-17222197
]
Apache Spark commented on SPARK-33270:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-33270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33270:
Assignee: Apache Spark
> from_json() cannot parse schema from schema_of_json()
>
Maxim Gekk created SPARK-33270:
--
Summary: from_json() cannot parse schema from schema_of_json()
Key: SPARK-33270
URL: https://issues.apache.org/jira/browse/SPARK-33270
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-33208.
-
Fix Version/s: 3.1.0
3.0.2
Resolution: Fixed
> Update the document of
[
https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-33269.
--
Fix Version/s: 3.1.0
Resolution: Fixed
Issue resolved by pull request 30171
[
https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33269:
Assignee: Gengliang Wang (was: Apache Spark)
> Ignore ".bsp/" directory in Git
>
[
https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33269:
Assignee: Apache Spark (was: Gengliang Wang)
> Ignore ".bsp/" directory in Git
>
[
https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222132#comment-17222132
]
Apache Spark commented on SPARK-33269:
--
User 'gengliangwang' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222131#comment-17222131
]
Apache Spark commented on SPARK-33267:
--
User 'HeartSaVioR' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33267:
Assignee: Apache Spark
> Query with having null in "in" condition against data source V2
[
https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33267:
Assignee: (was: Apache Spark)
> Query with having null in "in" condition against
[
https://issues.apache.org/jira/browse/SPARK-33268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222128#comment-17222128
]
Apache Spark commented on SPARK-33268:
--
User 'maropu' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-33268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33268:
Assignee: (was: Apache Spark)
> Fix bugs for casting data from/to
[
https://issues.apache.org/jira/browse/SPARK-33268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222127#comment-17222127
]
Apache Spark commented on SPARK-33268:
--
User 'maropu' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-33268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33268:
Assignee: Apache Spark
> Fix bugs for casting data from/to PythonUserDefinedType
>
Takeshi Yamamuro created SPARK-33268:
Summary: Fix bugs for casting data from/to PythonUserDefinedType
Key: SPARK-33268
URL: https://issues.apache.org/jira/browse/SPARK-33268
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gengliang Wang updated SPARK-33269:
---
Priority: Minor (was: Major)
> Ignore ".bsp/" directory in Git
>
Gengliang Wang created SPARK-33269:
--
Summary: Ignore ".bsp/" directory in Git
Key: SPARK-33269
URL: https://issues.apache.org/jira/browse/SPARK-33269
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-33269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gengliang Wang updated SPARK-33269:
---
Issue Type: Improvement (was: Bug)
> Ignore ".bsp/" directory in Git
>
[
https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222122#comment-17222122
]
Jungtaek Lim commented on SPARK-33267:
--
Will submit a PR soon.
> Query with having null in "in"
[
https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim updated SPARK-33267:
-
Description:
The query with having null in "in" condition against data source V2 source
table
[
https://issues.apache.org/jira/browse/SPARK-33267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim updated SPARK-33267:
-
Summary: Query with having null in "in" condition against data source V2
source table
Jungtaek Lim created SPARK-33267:
Summary: Query with having null in "in" condition against data
source V2 source table fails with NPE
Key: SPARK-33267
URL: https://issues.apache.org/jira/browse/SPARK-33267
[
https://issues.apache.org/jira/browse/SPARK-33027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222041#comment-17222041
]
Wenchen Fan commented on SPARK-33027:
-
[~chengsu] any progress on it?
> Add
[
https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222013#comment-17222013
]
Apache Spark commented on SPARK-33208:
--
User 'waitinfuture' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33208:
Assignee: Apache Spark
> Update the document of SparkSession#sql
>
[
https://issues.apache.org/jira/browse/SPARK-33208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-33208:
Assignee: (was: Apache Spark)
> Update the document of SparkSession#sql
>
[
https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-33183:
--
Labels: correctness (was: )
> Bug in optimizer rule EliminateSorts
>
[
https://issues.apache.org/jira/browse/SPARK-32934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-32934:
---
Assignee: jiaan.geng
> Improve the performance for NTH_VALUE and Reactor the
[
https://issues.apache.org/jira/browse/SPARK-32934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-32934.
-
Fix Version/s: 3.1.0
Resolution: Fixed
Issue resolved by pull request 29800
[
https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221964#comment-17221964
]
Dongjoon Hyun commented on SPARK-33183:
---
[~cloud_fan]. Could you label this as `correctness`?
>
85 matches
Mail list logo