Set comparison
Hi, I have a Spark method that returns RDD[String], which I am converting to a set and then comparing it to the expected output as shown in the following code. 1. val expected_res = Set(ID1, ID2, ID3) // expected output 2. val result:RDD[String] = getData(input) //method returns RDD[String] 3. val set_val = result.collect().toSet // convert to set. 4. println(set_val) // prints: Set(ID1, ID2, ID3) 5. println(expected_res)// prints: Set(ID1, ID2, ID3) // verify output 6. if( set_val == expected_res) 7.println(true) // this does not get printed The value returned by the method is almost same as expected output, but the verification is failing. I am not sure why the expected_res in Line 5 does not print the quotes even though Line 1 has them. Could that be the reason the comparison is failing? What is the right way to do the above comparison? thanks -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Set-comparison-tp7696.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: Set comparison
On Mon, Jun 16, 2014 at 10:09 PM, SK skrishna...@gmail.com wrote: The value returned by the method is almost same as expected output, but the verification is failing. I am not sure why the expected_res in Line 5 does not print the quotes even though Line 1 has them. Could that be the reason the comparison is failing? What is the right way to do the above comparison? Most likely because the strings in set_val do actually start and end with double-quotes? That is just what the output suggests. It has a string like ID1, not ID1
Re: Set comparison
In Line 1, I have expected_res as a set of strings with quotes. So I thought it would include the quotes during comparison. Anyway I modified expected_res = Set(\ID1\, \ID2\, \ID3\) and that seems to work. thanks. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Set-comparison-tp7696p7699.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: Set comparison
If you want string with quotes, you have to escape it with '\'. It's exactly what you did in the modified version. Sent from my iPhone On 2014年6月17日, at 5:43, SK skrishna...@gmail.com wrote: In Line 1, I have expected_res as a set of strings with quotes. So I thought it would include the quotes during comparison. Anyway I modified expected_res = Set(\ID1\, \ID2\, \ID3\) and that seems to work. thanks. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Set-comparison-tp7696p7699.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
