Re: DataFrame Filter Inside Another Data Frame Map

2015-07-02 Thread Raghavendra Pandey
You can collect the dataframe as array n then create map out of it..., On Jul 2, 2015 9:23 AM, asoni.le...@gmail.com wrote: Any example how can i return a Hashmap from data frame ? Thanks , Ashish On Jul 1, 2015, at 11:34 PM, Holden Karau hol...@pigscanfly.ca wrote: Collecting it as a

Re: DataFrame Filter Inside Another Data Frame Map

2015-07-01 Thread Holden Karau
Collecting it as a regular (Java/scala/Python) map. You can also broadcast the map if your going to use it multiple times. On Wednesday, July 1, 2015, Ashish Soni asoni.le...@gmail.com wrote: Thanks , So if i load some static data from database and then i need to use than in my map function to

Re: DataFrame Filter Inside Another Data Frame Map

2015-07-01 Thread asoni . learn
Any example how can i return a Hashmap from data frame ? Thanks , Ashish On Jul 1, 2015, at 11:34 PM, Holden Karau hol...@pigscanfly.ca wrote: Collecting it as a regular (Java/scala/Python) map. You can also broadcast the map if your going to use it multiple times. On Wednesday, July 1,

Re: DataFrame Filter Inside Another Data Frame Map

2015-07-01 Thread Raghavendra Pandey
You cannot refer to one rdd inside another rdd.map function... Rdd object is not serialiable. Whatever objects you use inside map function should be serializable as they get transferred to executor nodes. On Jul 2, 2015 6:13 AM, Ashish Soni asoni.le...@gmail.com wrote: Hi All , I am not sure

Re: DataFrame Filter Inside Another Data Frame Map

2015-07-01 Thread Ashish Soni
Thanks , So if i load some static data from database and then i need to use than in my map function to filter records what will be the best way to do it, Ashish On Wed, Jul 1, 2015 at 10:45 PM, Raghavendra Pandey raghavendra.pan...@gmail.com wrote: You cannot refer to one rdd inside another

DataFrame Filter Inside Another Data Frame Map

2015-07-01 Thread Ashish Soni
Hi All , I am not sure what is the wrong with below code as it give below error when i access inside the map but it works outside JavaRDDCharge rdd2 = rdd.map(new FunctionCharge, Charge() { @Override public Charge call(Charge ch) throws Exception { *