Re: unable to make a custom class as a key in a pairrdd

2014-10-24 Thread Gerard Maas
There's an issue in the way case classes are handled on the REPL and you won't be able to use a case class as a key. See: https://issues.apache.org/jira/browse/SPARK-2620 BTW, case classes already implement equals and hashCode. It's not needed to implement those again. Given that you already impl

Re: unable to make a custom class as a key in a pairrdd

2014-10-24 Thread Jaonary Rabarisoa
In the documentation it's said that we need to override the hashCode and equals methods. Without overriding it does't work too. I get this error on REPL and stand alone application On Fri, Oct 24, 2014 at 3:29 AM, Prashant Sharma wrote: > Are you doing this in REPL ? Then there is a bug filed fo

Re: unable to make a custom class as a key in a pairrdd

2014-10-23 Thread Prashant Sharma
Are you doing this in REPL ? Then there is a bug filed for this, I just can't recall the bug ID at the moment. Prashant Sharma On Fri, Oct 24, 2014 at 4:07 AM, Niklas Wilcke < 1wil...@informatik.uni-hamburg.de> wrote: > Hi Jao, > > I don't really know why this doesn't work but I have two hint

Re: unable to make a custom class as a key in a pairrdd

2014-10-23 Thread Niklas Wilcke
Hi Jao, I don't really know why this doesn't work but I have two hints. You don't need to override hashCode and equals. The modifier case is doing that for you. Writing case class PersonID(id: String) would be enough to get the class you want I think. If I change the type of the id param to Int

unable to make a custom class as a key in a pairrdd

2014-10-23 Thread Jaonary Rabarisoa
Hi all, I have the following case class that I want to use as a key in a key-value rdd. I defined the equals and hashCode methode but it's not working. What I'm doing wrong ? *case class PersonID(id: String) {* * override def hashCode = id.hashCode* * override def equals(other: Any) = o