Re: [Spark SQL] Java map/flatMap api broken with DataFrame in 1.3.{0,1}

2015-04-17 Thread Olivier Girardot
and the PR: https://github.com/apache/spark/pull/5564 Thank you ! Olivier. Le ven. 17 avr. 2015 à 23:00, Reynold Xin a écrit : > No there isn't a convention. Although if you want to show java 8, you > should also show java 6/7 syntax since there are still more 7 users than 8. > > > On Fri, Apr

Re: [Spark SQL] Java map/flatMap api broken with DataFrame in 1.3.{0,1}

2015-04-17 Thread Olivier Girardot
another PR I guess :) here's the associated Jira https://issues.apache.org/jira/browse/SPARK-6988 Le ven. 17 avr. 2015 à 23:00, Reynold Xin a écrit : > No there isn't a convention. Although if you want to show java 8, you > should also show java 6/7 syntax since there are still more 7 users than

Re: [Spark SQL] Java map/flatMap api broken with DataFrame in 1.3.{0,1}

2015-04-17 Thread Reynold Xin
No there isn't a convention. Although if you want to show java 8, you should also show java 6/7 syntax since there are still more 7 users than 8. On Fri, Apr 17, 2015 at 3:36 PM, Olivier Girardot < o.girar...@lateral-thoughts.com> wrote: > Is there any convention *not* to show java 8 versions in

Re: [Spark SQL] Java map/flatMap api broken with DataFrame in 1.3.{0,1}

2015-04-17 Thread Olivier Girardot
Is there any convention *not* to show java 8 versions in the documentation ? Le ven. 17 avr. 2015 à 21:39, Reynold Xin a écrit : > Please do! Thanks. > > > On Fri, Apr 17, 2015 at 2:36 PM, Olivier Girardot < > o.girar...@lateral-thoughts.com> wrote: > >> Ok, do you want me to open a pull request

Re: [Spark SQL] Java map/flatMap api broken with DataFrame in 1.3.{0,1}

2015-04-17 Thread Reynold Xin
Please do! Thanks. On Fri, Apr 17, 2015 at 2:36 PM, Olivier Girardot < o.girar...@lateral-thoughts.com> wrote: > Ok, do you want me to open a pull request to fix the dedicated > documentation ? > > Le ven. 17 avr. 2015 à 18:14, Reynold Xin a écrit : > >> I think in 1.3 and above, you'd need to

Re: [Spark SQL] Java map/flatMap api broken with DataFrame in 1.3.{0,1}

2015-04-17 Thread Olivier Girardot
Ok, do you want me to open a pull request to fix the dedicated documentation ? Le ven. 17 avr. 2015 à 18:14, Reynold Xin a écrit : > I think in 1.3 and above, you'd need to do > > .sql(...).javaRDD().map(..) > > On Fri, Apr 17, 2015 at 9:22 AM, Olivier Girardot < > o.girar...@lateral-thoughts.co

Re: [Spark SQL] Java map/flatMap api broken with DataFrame in 1.3.{0,1}

2015-04-17 Thread Reynold Xin
I think in 1.3 and above, you'd need to do .sql(...).javaRDD().map(..) On Fri, Apr 17, 2015 at 9:22 AM, Olivier Girardot < o.girar...@lateral-thoughts.com> wrote: > Yes thanks ! > > Le ven. 17 avr. 2015 à 16:20, Ted Yu a écrit : > > > The image didn't go through. > > > > I think you were referr

Re: [Spark SQL] Java map/flatMap api broken with DataFrame in 1.3.{0,1}

2015-04-17 Thread Olivier Girardot
Yes thanks ! Le ven. 17 avr. 2015 à 16:20, Ted Yu a écrit : > The image didn't go through. > > I think you were referring to: > override def map[R: ClassTag](f: Row => R): RDD[R] = rdd.map(f) > > Cheers > > On Fri, Apr 17, 2015 at 6:07 AM, Olivier Girardot < > o.girar...@lateral-thoughts.com>

Re: [Spark SQL] Java map/flatMap api broken with DataFrame in 1.3.{0,1}

2015-04-17 Thread Ted Yu
The image didn't go through. I think you were referring to: override def map[R: ClassTag](f: Row => R): RDD[R] = rdd.map(f) Cheers On Fri, Apr 17, 2015 at 6:07 AM, Olivier Girardot < o.girar...@lateral-thoughts.com> wrote: > Hi everyone, > I had an issue trying to use Spark SQL from Java (8 o