Zhe Dong created SPARK-41386:
Summary: Resolve small files with REBALANCE Hint
Key: SPARK-41386
URL: https://issues.apache.org/jira/browse/SPARK-41386
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Priority: Minor (was: Blocker)
> Resolve small files with REBALANCE Hint
>
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Description: todo: (was: For compatibility, we need support for
expressing window functions. Window fu
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Description: Sorry, A wrong location of Issue (was: todo: )
> Resolve small files with REBALANCE Hint
>
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Issue Type: Sub-task (was: Technical task)
> Resolve small files with REBALANCE Hint
>
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Component/s: Spark Core
(was: Connect)
Issue Type: Technical task (was: Sub-ta
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Parent: (was: SPARK-41282)
Issue Type: Improvement (was: Sub-task)
> Resolve small files wi
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Epic Link: SPARK-39375
> Resolve small files with REBALANCE Hint
> -
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Description: TODO: (was: Sorry, A wrong location of Issue)
> Resolve small files with REBALANCE Hint
>
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Summary: There are some small files when using rebalance(column) (was:
Resolve small files with REBALAN
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Component/s: SQL
(was: Spark Core)
> There are some small files when using rebalanc
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Description:
{*}Problem (/*+ REBALANCE(bot_mid) */){*}:
sparksession config:
{noformat}
config("spark.
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Description:
*Problem ( REBALANCE(column)* {*}){*}:
SparkSession config:
{noformat}
config("spark.s
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Description:
*Problem ( REBALANCE(column)* {*}){*}:
SparkSession config:
{noformat}
config("spark.sql.
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Description:
*Problem ( REBALANCE(column)* {*}){*}:
SparkSession config:
{noformat}
config("spark.sql.
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Description:
*Problem ( REBALANCE(column)* {*}){*}:
SparkSession config:
{noformat}
config("spark.sql.
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644131#comment-17644131
]
Zhe Dong commented on SPARK-41386:
--
we may change this part to avoid files that are sma
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Description:
*Problem ( REBALANCE(column)* {*}){*}:
SparkSession config:
{noformat}
config("spark.sql.
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644141#comment-17644141
]
Zhe Dong commented on SPARK-41386:
--
{noformat}
if (mapStats.isEmpty ||
mapS
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Affects Version/s: 3.3.1
(was: 3.4.0)
> There are some small files when using
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Dong updated SPARK-41386:
-
Epic Link: (was: SPARK-39375)
> There are some small files when using rebalance(column)
>
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644175#comment-17644175
]
Zhe Dong commented on SPARK-41386:
--
Hi. [~podongfeng]
That was my mistake. I removed
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644141#comment-17644141
]
Zhe Dong edited comment on SPARK-41386 at 12/7/22 8:29 AM:
---
Op
[
https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644141#comment-17644141
]
Zhe Dong edited comment on SPARK-41386 at 12/7/22 8:31 AM:
---
Op
24 matches
Mail list logo
