[
https://issues.apache.org/jira/browse/SPARK-39915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582457#comment-17582457
]
Shixiong Zhu commented on SPARK-39915:
--------------------------------------
Yeah. I would consider this is a bug since the doc of `repartition` explicitly
says
{code:java}
Returns a new Dataset that has exactly `numPartitions` partitions.
{code}
> Dataset.repartition(N) may not create N partitions
> --------------------------------------------------
>
> Key: SPARK-39915
> URL: https://issues.apache.org/jira/browse/SPARK-39915
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: Shixiong Zhu
> Priority: Major
>
> Looks like there is a behavior change in Dataset.repartition in 3.3.0. For
> example, `spark.range(10, 0).repartition(5).rdd.getNumPartitions` returns 5
> in Spark 3.2.0, but 0 in Spark 3.3.0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
