Emil Ejbyfeldt created SPARK-38681:
--
Summary: Support nested generic case classes
Key: SPARK-38681
URL: https://issues.apache.org/jira/browse/SPARK-38681
Project: Spark
Issue Type: Bug
JacobZheng created SPARK-38682:
--
Summary: Complex calculations with lead to driver oom
Key: SPARK-38682
URL: https://issues.apache.org/jira/browse/SPARK-38682
Project: Spark
Issue Type: Improve
[
https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JacobZheng updated SPARK-38682:
---
Attachment: screenshot-1.png
> Complex calculations with lead to driver oom
> -
[
https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JacobZheng updated SPARK-38682:
---
Attachment: 20220329164645.jpg
> Complex calculations with lead to driver oom
> ---
[
https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JacobZheng updated SPARK-38682:
---
Description:
My spark job is working fine in version 3.0.1. After I upgraded to 3.2, the
driver wou
[
https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JacobZheng updated SPARK-38682:
---
Attachment: (was: 20220329164645.jpg)
> Complex calculations with lead to driver oom
>
weixiuli created SPARK-38683:
Summary: It is unnecessary to release the
ShuffleManagedBufferIterator or ShuffleChunkManagedBufferIterator or
ManagedBufferIterator buffers when the client channel's connection is terminated
Key: SPARK-
Jungtaek Lim created SPARK-38684:
Summary: Stream-stream outer join has a possible correctness issue
due to weakly read consistent on outer iterators
Key: SPARK-38684
URL: https://issues.apache.org/jira/browse/SPA
[
https://issues.apache.org/jira/browse/SPARK-38684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513949#comment-17513949
]
Jungtaek Lim commented on SPARK-38684:
--
Will submit a PR sooner.
> Stream-stream o
[
https://issues.apache.org/jira/browse/SPARK-38561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513962#comment-17513962
]
Apache Spark commented on SPARK-38561:
--
User 'Yikun' has created a pull request for
[
https://issues.apache.org/jira/browse/SPARK-38683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38683:
Assignee: Apache Spark
> It is unnecessary to release the ShuffleManagedBufferIterator or
[
https://issues.apache.org/jira/browse/SPARK-38683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38683:
Assignee: (was: Apache Spark)
> It is unnecessary to release the ShuffleManagedBuffer
[
https://issues.apache.org/jira/browse/SPARK-38683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513963#comment-17513963
]
Apache Spark commented on SPARK-38683:
--
User 'weixiuli' has created a pull request
[
https://issues.apache.org/jira/browse/SPARK-38684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38684:
Assignee: (was: Apache Spark)
> Stream-stream outer join has a possible correctness i
[
https://issues.apache.org/jira/browse/SPARK-38684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513994#comment-17513994
]
Apache Spark commented on SPARK-38684:
--
User 'HeartSaVioR' has created a pull reque
[
https://issues.apache.org/jira/browse/SPARK-38684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38684:
Assignee: Apache Spark
> Stream-stream outer join has a possible correctness issue due to
[
https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JacobZheng updated SPARK-38682:
---
Description:
My spark job is working fine in version 3.0.1. After I upgraded to 3.2, the
driver wou
[
https://issues.apache.org/jira/browse/SPARK-38670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim resolved SPARK-38670.
--
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by pull request 35985
[https://gi
[
https://issues.apache.org/jira/browse/SPARK-38670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim reassigned SPARK-38670:
Assignee: Boyang Jerry Peng
> Add offset commit time to streaming query listener
> --
jiaan.geng created SPARK-38685:
--
Summary: Improve the implement of percentile_cont
Key: SPARK-38685
URL: https://issues.apache.org/jira/browse/SPARK-38685
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-38685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38685:
Assignee: Apache Spark
> Improve the implement of percentile_cont
> -
[
https://issues.apache.org/jira/browse/SPARK-38685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38685:
Assignee: (was: Apache Spark)
> Improve the implement of percentile_cont
> --
[
https://issues.apache.org/jira/browse/SPARK-38685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514062#comment-17514062
]
Apache Spark commented on SPARK-38685:
--
User 'beliefer' has created a pull request
[
https://issues.apache.org/jira/browse/SPARK-38685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514060#comment-17514060
]
Apache Spark commented on SPARK-38685:
--
User 'beliefer' has created a pull request
[
https://issues.apache.org/jira/browse/SPARK-38681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514064#comment-17514064
]
Apache Spark commented on SPARK-38681:
--
User 'eejbyfeldt' has created a pull reques
[
https://issues.apache.org/jira/browse/SPARK-38681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38681:
Assignee: (was: Apache Spark)
> Support nested generic case classes
> ---
[
https://issues.apache.org/jira/browse/SPARK-38681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38681:
Assignee: Apache Spark
> Support nested generic case classes
> --
[
https://issues.apache.org/jira/browse/SPARK-38674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-38674.
-
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by pull request 35989
[https://gith
[
https://issues.apache.org/jira/browse/SPARK-38674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-38674:
---
Assignee: Yuming Wang
> Remove useless deduplicate in SubqueryBroadcastExec
> -
[
https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38506:
Assignee: Apache Spark
> Push partial aggregation through join
>
[
https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38506:
Assignee: (was: Apache Spark)
> Push partial aggregation through join
> -
[
https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514133#comment-17514133
]
Apache Spark commented on SPARK-38506:
--
User 'wangyum' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-38562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-38562.
---
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by pull request 35870
[https://
[
https://issues.apache.org/jira/browse/SPARK-38562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-38562:
-
Assignee: Yikun Jiang
> Add doc for Volcano scheduler
> -
>
[
https://issues.apache.org/jira/browse/SPARK-38562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-38562:
--
Fix Version/s: 3.3.0
(was: 3.4.0)
> Add doc for Volcano scheduler
> ---
Xinrong Meng created SPARK-38686:
Summary: Implement `keep` parameter of
`(Index/MultiIndex).drop_duplicates`
Key: SPARK-38686
URL: https://issues.apache.org/jira/browse/SPARK-38686
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-37982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk resolved SPARK-37982.
--
Fix Version/s: 3.4.0
(was: 3.3.0)
Resolution: Fixed
Issue resolved by pu
[
https://issues.apache.org/jira/browse/SPARK-37982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk reassigned SPARK-37982:
Assignee: leesf
> Use error classes in the execution errors related to unsupported input type
> -
[
https://issues.apache.org/jira/browse/SPARK-38686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514275#comment-17514275
]
Apache Spark commented on SPARK-38686:
--
User 'xinrong-databricks' has created a pul
[
https://issues.apache.org/jira/browse/SPARK-38686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38686:
Assignee: (was: Apache Spark)
> Implement `keep` parameter of `(Index/MultiIndex).dro
[
https://issues.apache.org/jira/browse/SPARK-38686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38686:
Assignee: Apache Spark
> Implement `keep` parameter of `(Index/MultiIndex).drop_duplicate
[
https://issues.apache.org/jira/browse/SPARK-38687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38687:
-
Affects Version/s: 3.4.0
(was: 3.3.0)
> Use error classes in the compilation
Max Gekk created SPARK-38687:
Summary: Use error classes in the compilation errors of generators
Key: SPARK-38687
URL: https://issues.apache.org/jira/browse/SPARK-38687
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-38687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38687:
-
Description:
Migrate the following errors in QueryCompilationErrors:
* nestedGeneratorError
* moreThanOn
[
https://issues.apache.org/jira/browse/SPARK-38688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38688:
-
Description:
Migrate the following errors in QueryCompilationErrors:
* dataTypeMismatchForDeserializerEr
Max Gekk created SPARK-38688:
Summary: Use error classes in the compilation errors of
deserializer
Key: SPARK-38688
URL: https://issues.apache.org/jira/browse/SPARK-38688
Project: Spark
Issue Ty
[
https://issues.apache.org/jira/browse/SPARK-38689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38689:
-
Affects Version/s: 3.4.0
(was: 3.3.0)
> Use error classes in the compilation
Max Gekk created SPARK-38689:
Summary: Use error classes in the compilation errors of not
allowed DESC PARTITION
Key: SPARK-38689
URL: https://issues.apache.org/jira/browse/SPARK-38689
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-38689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38689:
-
Description:
Migrate the following errors in QueryCompilationErrors:
* descPartitionNotAllowedOnTempView
[
https://issues.apache.org/jira/browse/SPARK-38689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38689:
-
Description:
Migrate the following errors in QueryCompilationErrors:
* descPartitionNotAllowedOnTempView
Max Gekk created SPARK-38690:
Summary: Use error classes in the compilation errors of SHOW
CREATE TABLE
Key: SPARK-38690
URL: https://issues.apache.org/jira/browse/SPARK-38690
Project: Spark
Iss
[
https://issues.apache.org/jira/browse/SPARK-38690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38690:
-
Description:
Migrate the following errors in QueryCompilationErrors:
* showCreateTableAsSerdeNotSupporte
Max Gekk created SPARK-38691:
Summary: Use error classes in the compilation errors of
column/attr resolving
Key: SPARK-38691
URL: https://issues.apache.org/jira/browse/SPARK-38691
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-38691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38691:
-
Description:
Migrate the following errors in QueryCompilationErrors:
* cannotResolveUserSpecifiedColumns
Max Gekk created SPARK-38692:
Summary: Use error classes in the compilation errors of function
args
Key: SPARK-38692
URL: https://issues.apache.org/jira/browse/SPARK-38692
Project: Spark
Issue T
[
https://issues.apache.org/jira/browse/SPARK-38692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38692:
-
Description:
Migrate the following errors in QueryCompilationErrors:
* invalidFunctionArgumentsError
* i
Brad Solomon created SPARK-38693:
Summary: Spark does not use SessionManager
Key: SPARK-38693
URL: https://issues.apache.org/jira/browse/SPARK-38693
Project: Spark
Issue Type: Bug
C
[
https://issues.apache.org/jira/browse/SPARK-38693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad Solomon updated SPARK-38693:
-
Description:
Spark's failure to use a `SessionManager` causes
`java.lang.IllegalStateException:
[
https://issues.apache.org/jira/browse/SPARK-38693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad Solomon updated SPARK-38693:
-
Description:
Spark's failure to use a `SessionManager` causes
`java.lang.IllegalStateException:
[
https://issues.apache.org/jira/browse/SPARK-38693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad Solomon updated SPARK-38693:
-
Description:
Spark's failure to use a `SessionManager` causes
`java.lang.IllegalStateException:
[
https://issues.apache.org/jira/browse/SPARK-35803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Rabinowitz updated SPARK-35803:
-
Issue Type: Bug (was: New Feature)
> Spark SQL does not support creating views using Da
[
https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514343#comment-17514343
]
Dongjoon Hyun commented on SPARK-38652:
---
Any update, [~dcoliversun]?
> K8S IT Tes
[
https://issues.apache.org/jira/browse/SPARK-33349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514364#comment-17514364
]
Dongjoon Hyun commented on SPARK-33349:
---
Hi, All. Could you try to use the latest
[
https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514379#comment-17514379
]
Dongjoon Hyun commented on SPARK-38652:
---
BTW, [~dcoliversun]. K8s IT itself doesn'
[
https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514390#comment-17514390
]
qian commented on SPARK-38652:
--
[~dongjoon] Hi. DepsTestsSuite has tests as follow
* Launc
[
https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514398#comment-17514398
]
Dongjoon Hyun commented on SPARK-38652:
---
Got it, [~dcoliversun].
> K8S IT Test De
[
https://issues.apache.org/jira/browse/SPARK-38320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-38320:
--
Labels: correctness (was: )
> (flat)MapGroupsWithState can timeout groups which just received
Yang Jie created SPARK-38694:
Summary: Simplify Java UT code with Junit `assertThrows`
Key: SPARK-38694
URL: https://issues.apache.org/jira/browse/SPARK-38694
Project: Spark
Issue Type: Improveme
[
https://issues.apache.org/jira/browse/SPARK-38694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514412#comment-17514412
]
Apache Spark commented on SPARK-38694:
--
User 'LuciferYang' has created a pull reque
[
https://issues.apache.org/jira/browse/SPARK-38694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38694:
Assignee: Apache Spark
> Simplify Java UT code with Junit `assertThrows`
> --
[
https://issues.apache.org/jira/browse/SPARK-38694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514413#comment-17514413
]
Apache Spark commented on SPARK-38694:
--
User 'LuciferYang' has created a pull reque
[
https://issues.apache.org/jira/browse/SPARK-38694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38694:
Assignee: (was: Apache Spark)
> Simplify Java UT code with Junit `assertThrows`
> ---
[
https://issues.apache.org/jira/browse/SPARK-38349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim reassigned SPARK-38349:
Assignee: nyingping
> No need to filter events when sessionwindow gapDuration greater tha
[
https://issues.apache.org/jira/browse/SPARK-38349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim resolved SPARK-38349.
--
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by pull request 35680
[https://gi
[
https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514431#comment-17514431
]
Dongjoon Hyun commented on SPARK-38652:
---
I also confirmed this regression and rais
[
https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-38652:
--
Priority: Blocker (was: Major)
> K8S IT Test DepsTestsSuite blocks with PathIOException in ha
[
https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-38652:
--
Component/s: (was: Tests)
> K8S IT Test DepsTestsSuite blocks with PathIOException in hado
[
https://issues.apache.org/jira/browse/SPARK-38605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L. C. Hsieh resolved SPARK-38605.
-
Resolution: Won't Fix
> Retrying on file manager operation in HDFSMetadataLog
>
78 matches
Mail list logo