How to debug Spark source using IntelliJ/ Eclipse

Hi, I am trying to understand Spark internal code and wanted to debug Spark source, to add a new feature. I have tried the steps lined out here on the Spark Wiki page IDE setup , but they do

Re: [ML] Missing documentation for the IndexToString feature transformer

I'd be more than happy to help review the docs if that would be useful :) On Sat, Dec 5, 2015 at 2:21 PM, Joseph Bradley wrote: > Thanks for reporting this! I just added a JIRA: > https://issues.apache.org/jira/browse/SPARK-12159 > That would be great if you could send a PR for it; thanks! > Jo

Re: [ML] Missing documentation for the IndexToString feature transformer

Thanks for reporting this! I just added a JIRA: https://issues.apache.org/jira/browse/SPARK-12159 That would be great if you could send a PR for it; thanks! Joseph On Sat, Dec 5, 2015 at 5:02 AM, Benjamin Fradet wrote: > Hi, > > I was wondering why the IndexToString >

Re: Returning numpy types from udfs

Filed here: https://issues.apache.org/jira/browse/SPARK-12157 On Sat, Dec 5, 2015 at 3:08 PM Reynold Xin wrote: > Not aware of any jira ticket, but it does sound like a great idea. > > > On Sat, Dec 5, 2015 at 11:03 PM, Justin Uang > wrote: > >> Hi, >> >> I have fallen into the trap of returni

Re: Returning numpy types from udfs

Not aware of any jira ticket, but it does sound like a great idea. On Sat, Dec 5, 2015 at 11:03 PM, Justin Uang wrote: > Hi, > > I have fallen into the trap of returning numpy types from udfs, such as > np.float64 and np.int. It's hard to find the issue because they behave > pretty much as regu

Returning numpy types from udfs

Hi, I have fallen into the trap of returning numpy types from udfs, such as np.float64 and np.int. It's hard to find the issue because they behave pretty much as regular pure Python floats and doubles, so can we make PYSPARK automatically translate them? If so, I'll create a Jira ticket. Justin

[ML] Missing documentation for the IndexToString feature transformer

Hi, I was wondering why the IndexToString label transformer was not documented in ml-features.md . If it's not i