Re: How to implement top() and filter() on object List for JavaRDD

2015-07-07 Thread Ted Yu
bq. my class has already implemented the java.io.Serializable Can you show the code for Model.User class ? Cheers On Tue, Jul 7, 2015 at 8:18 AM, Hafsa Asif hafsa.a...@matchinguu.com wrote: Thank u so much for the solution. I run the code like this, JavaRDDUser rdd =

Re: How to implement top() and filter() on object List for JavaRDD

2015-07-07 Thread Hafsa Asif
Thank u so much for the solution. I run the code like this, JavaRDDUser rdd = context.parallelize(usersList); JavaRDDUser rdd_sorted_users= rdd.sortBy(new FunctionUser,String(){ @Override public String call(User usr1) throws Exception {

Re: How to implement top() and filter() on object List for JavaRDD

2015-07-07 Thread Akhil Das
Here's a simplified example: SparkConf conf = new SparkConf().setAppName( Sigmoid).setMaster(local); JavaSparkContext sc = new JavaSparkContext(conf); ListString user = new ArrayListString(); user.add(Jack); user.add(Jill);

Re: How to implement top() and filter() on object List for JavaRDD

2015-07-07 Thread Hafsa Asif
I have also tried this stupid code snippet, only thinking that it may even compile code Function1User, Object FILTER_USER = new AbstractFunction1User, Object () { public Object apply(User user){ return user; } }; FILTER_USER is fine but cannot be applied to the

Re: How to implement top() and filter() on object List for JavaRDD

2015-07-07 Thread ayan guha
I would suggest you take alook to DataFrames. Also, I do not think you should implement comparators for user class as a whole, rather you should get the attribute to sort/compar on and delete sorting to data type of inherent attribute. Eg. sorting can be done by name and if so, it should be string

How to implement top() and filter() on object List for JavaRDD

2015-07-07 Thread Hafsa Asif
Hi, I have an object list of Users and I want to implement top() and filter() methods on the object list. Let me explain you the whole scenario: 1. I have User object list named usersList. I fill it during record set. User user = new User();

Re: How to implement top() and filter() on object List for JavaRDD

2015-07-07 Thread Hafsa Asif
Thank u for your quick response. But, I tried this and get the error as shown in pic error.jpg http://apache-spark-user-list.1001560.n3.nabble.com/file/n23676/error.jpg -- View this message in context:

Re: How to implement top() and filter() on object List for JavaRDD

2015-07-07 Thread Hafsa Asif
Rusty, I am very thankful for your help. Actually, I am facing difficulty in objects. My plan is that, I have an object list containing list of User objects. After parallelizing it through spark context, I apply comparator on user.getUserName(). As usernames are sorted, their related user object

Re: How to implement top() and filter() on object List for JavaRDD

2015-07-07 Thread Hafsa Asif
Sorry ignore my last reply. Rusty, I am very thankful for your help. Actually, I am facing difficulty in objects. My plan is that, I have an object list containing list of User objects. After parallelizing it through spark context, I apply comparator on user.getUserName(). As usernames are