[ https://issues.apache.org/jira/browse/SPARK-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-8048. ------------------------------ Resolution: Duplicate > Explicit partitionning of an RDD with 0 partition will yield empty outer join > ----------------------------------------------------------------------------- > > Key: SPARK-8048 > URL: https://issues.apache.org/jira/browse/SPARK-8048 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 1.3.1 > Reporter: Olivier Toupin > Priority: Minor > > Check this code => > https://gist.github.com/anonymous/0f935915f2bc182841f0 > Because of this => {{.partitionBy(new HashPartitioner(0))}} > The join will return empty result. > Here a normal expected behaviour would the join to crash, cause error, or to > return unjoined results, but instead will yield an empty RDD. > This a trivial exemple, but imagine: > {{.partitionBy(new HashPartitioner(previous.partitions.length))}}. > You join on an empty "previous" rdd, the lookup table is empty, Spark will > you lose all your results, instead of returning unjoined results, and this > without warnings or errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org