[ https://issues.apache.org/jira/browse/SPARK-48656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kent Yao resolved SPARK-48656. ------------------------------ Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47019 [https://github.com/apache/spark/pull/47019] > ArrayIndexOutOfBoundsException in CartesianRDD getPartitions > ------------------------------------------------------------ > > Key: SPARK-48656 > URL: https://issues.apache.org/jira/browse/SPARK-48656 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 4.0.0 > Reporter: Nick Young > Assignee: Wei Guo > Priority: Major > Fix For: 4.0.0 > > > ```val rdd1 = spark.sparkContext.parallelize(Seq(1, 2, 3), numSlices = 65536) > val rdd2 = spark.sparkContext.parallelize(Seq(1, 2, 3), numSlices = > 65536)rdd2.cartesian(rdd1).partitions``` > Throws `ArrayIndexOutOfBoundsException: 0` at CartesianRDD.scala:69 because > `s1.index * numPartitionsInRdd2 + s2.index` overflows and wraps to 0. We > should provide a better error message which indicates the number of partition > overflows so it's easier for the user to debug. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org