[ https://issues.apache.org/jira/browse/SPARK-3580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-3580: ----------------------------------- Assignee: (was: Apache Spark) > Add Consistent Method To Get Number of RDD Partitions Across Different > Languages > -------------------------------------------------------------------------------- > > Key: SPARK-3580 > URL: https://issues.apache.org/jira/browse/SPARK-3580 > Project: Spark > Issue Type: Improvement > Components: PySpark, Spark Core > Affects Versions: 1.1.0 > Reporter: Pat McDonough > Labels: starter > > Programmatically retrieving the number of partitions is not consistent > between python and scala. A consistent method should be defined and made > public across both languages. > RDD.partitions.size is also used quite frequently throughout the internal > code, so that might be worth refactoring as well once the new method is > available. > What we have today is below. > In Scala: > {code} > scala> someRDD.partitions.size > res0: Int = 30 > {code} > In Python: > {code} > In [2]: someRDD.getNumPartitions() > Out[2]: 30 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org