Re: Secondar Sorting in spark using clojure/flambo

2016-07-11 Thread Punit Naik
So I finally figured it out on my own. I had to basically write my custom ordering function as a separate scala project and then call that in clojure. I had my scala file written in this manner: import org.apache.spark.Partitionerimport org.apache.spark.rdd.RDD case class RFMCKey(cId: String,

Re: Secondar Sorting in spark using clojure/flambo

2016-07-11 Thread Punit Naik
Hi Black Thanks for the reply but figured it out on my own. Posting the answer after this. On Monday, July 11, 2016 at 11:42:10 PM UTC+5:30, Blake Miller wrote: > > Hi Punit > > The behavior you are referring to is a feature of the Scala compiler, > which is why it does not happen automaticall

Re: Secondar Sorting in spark using clojure/flambo

2016-07-11 Thread Blake Miller
Hi Punit The behavior you are referring to is a feature of the Scala compiler, which is why it does not happen automatically when you try to use it from Clojure. Please see the note here: https://github.com/t6/from-scala/blob/4e1752aaa2ef835dd67a8404273bee067510a431/test/t6/from_scala/guide.clj

Re: Secondar Sorting in spark using clojure/flambo

2016-07-08 Thread Punit Naik
Hi Ashish The "package" is indeed the full package name. On 09-Jul-2016 11:02 AM, "Ashish Negi" wrote: > Should not be `package` in `:import` be the actual package name of ` > RFMCPartitioner` ? > > see examples at https://clojuredocs.org/clojure.core/import > > like : > > (ns foo.bar > (:imp

Re: Secondar Sorting in spark using clojure/flambo

2016-07-08 Thread Ashish Negi
Should not be `package` in `:import` be the actual package name of ` RFMCPartitioner` ? see examples at https://clojuredocs.org/clojure.core/import like : (ns foo.bar (:import (java.util Date Calendar) (java.util.logging Logger Le

Secondar Sorting in spark using clojure/flambo

2016-07-08 Thread Punit Naik
I have a scala program in which I have implemented a secondary sort which works perfectly. The way I have written that program is: object rfmc { // Custom Key and partitioner case class RFMCKey(cId: String, R: Double, F: Double, M: Double, C: Double) class RFMCPartitioner(part