Re: strange behavior of pyspark RDD zip

2016-04-11 Thread Davies Liu
It seems like a bug, could you file a JIRA for this? (also post a way to reproduce it) On Fri, Apr 1, 2016 at 11:08 AM, Sergey wrote: > Hi! > > I'm on Spark 1.6.1 in local mode on Windows. > > And have issue with zip of zip'pping of two RDDs of __equal__ size and > __equal__

strange behavior of pyspark RDD zip

2016-04-01 Thread Sergey
Hi! I'm on Spark 1.6.1 in local mode on Windows. And have issue with zip of zip'pping of two RDDs of __equal__ size and __equal__ partitions number (I also tried to repartition both RDDs to one partition). I get such exception when I do rdd1.zip(rdd2).count(): File