Re: Issues with JavaRDD.subtract(JavaRDD) method in local vs. cluster mode

2015-07-31 Thread Sebastian Kalix
Thanks for the quick reply. I will be unable to collect more data until Monday though, but I will update the thread accordingly. I am using Spark 1.4.0. Were there any related issues reported? I wasn't able to find any, but I may have overlooked something. I have also updated the original

Re: Issues with JavaRDD.subtract(JavaRDD) method in local vs. cluster mode

2015-07-31 Thread Ted Yu
Can you call collect() and log the output to get more clue what is left ? Which Spark release are you using ? Cheers On Fri, Jul 31, 2015 at 9:01 AM, Warfish sebastian.ka...@gmail.com wrote: Hi everyone, I work with Spark for a little while now and have encountered a strange problem that

Issues with JavaRDD.subtract(JavaRDD) method in local vs. cluster mode

2015-07-31 Thread Warfish
Hi everyone, I work with Spark for a little while now and have encountered a strange problem that gives me headaches, which has to do with the JavaRDD.subtract method. Consider the following piece of code: public static void main(String[] args) { //context is of type