Nick Young created SPARK-48656: ---------------------------------- Summary: ArrayIndexOutOfBoundsException in CartesianRDD getPartitions Key: SPARK-48656 URL: https://issues.apache.org/jira/browse/SPARK-48656 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 4.0.0 Reporter: Nick Young
```val rdd1 = spark.sparkContext.parallelize(Seq(1, 2, 3), numSlices = 65536) val rdd2 = spark.sparkContext.parallelize(Seq(1, 2, 3), numSlices = 65536)rdd2.cartesian(rdd1).partitions``` Throws `ArrayIndexOutOfBoundsException: 0` at CartesianRDD.scala:69 because `s1.index * numPartitionsInRdd2 + s2.index` overflows and wraps to 0. We should provide a better error message which indicates the number of partition overflows so it's easier for the user to debug. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org