Re: Will higher order functions in spark SQL be pushed upstream?

2017-06-09 Thread Antoine HOM
Good news :) Thx Sameer. On Friday, June 9, 2017, Sameer Agarwal wrote: > * As a heavy user of complex data types I was wondering if there was >> any plan to push those changes upstream? >> > > Yes, we intend to contribute this to open source. > > >> * In addition, I was wondering if as part of

Re: [VOTE] Apache Spark 2.2.0 (RC4)

2017-06-09 Thread Sean Owen
Different errors as in https://issues.apache.org/jira/browse/SPARK-20520 but that's also reporting R test failures. I went back and tried to run the R tests and they passed, at least on Ubuntu 17 / R 3.3. On Fri, Jun 9, 2017 at 9:12 AM Nick Pentreath wrote: > All Scala, Python tests pass. ML QA

Re: Will higher order functions in spark SQL be pushed upstream?

2017-06-09 Thread Sameer Agarwal
> > * As a heavy user of complex data types I was wondering if there was > any plan to push those changes upstream? > Yes, we intend to contribute this to open source. > * In addition, I was wondering if as part of this change it also tries > to solve the column pruning / filter pushdown issues

Re: Will higher order functions in spark SQL be pushed upstream?

2017-06-09 Thread Olivier Girardot
+1 for the question 2017-06-07 19:50 GMT+02:00 Antoine HOM : > Hey guys, > > Databricks released higher order functions as part of their runtime > 3.0 beta (https://databricks.com/blog/2017/05/24/working-with- > nested-data-using-higher-order-functions-in-sql-on-databricks.html), > which helps wo

Re: [VOTE] Apache Spark 2.2.0 (RC4)

2017-06-09 Thread Dong Joon Hyun
Hi, Nick. Could you give us more information on your environment like R/JDK/OS? Bests, Dongjoon. From: Nick Pentreath Date: Friday, June 9, 2017 at 1:12 AM To: dev Subject: Re: [VOTE] Apache Spark 2.2.0 (RC4) All Scala, Python tests pass. ML QA and doc issues are resolved (as well as R it se

Re: [VOTE] Apache Spark 2.2.0 (RC4)

2017-06-09 Thread Felix Cheung
Hmm, that's odd. This test would be in Jenkins too - let me double check _ From: Nick Pentreath mailto:nick.pentre...@gmail.com>> Sent: Friday, June 9, 2017 1:12 AM Subject: Re: [VOTE] Apache Spark 2.2.0 (RC4) To: dev mailto:dev@spark.apache.org>> All Scala, Python te

Re: [VOTE] Apache Spark 2.2.0 (RC4)

2017-06-09 Thread Nick Pentreath
All Scala, Python tests pass. ML QA and doc issues are resolved (as well as R it seems). However, I'm seeing the following test failure on R consistently: https://gist.github.com/MLnick/5f26152f97ae8473f807c6895817cf72 On Thu, 8 Jun 2017 at 08:48 Denny Lee wrote: > +1 non-binding > > Tested on

Re: Are release docs part of a release?

2017-06-09 Thread Sean Owen
@Ryan yes you're speaking about the non-version-specific docs, and I agree there seems to be no particular reason not to update those continuously. (I also don't know how that relates to the doc release policy.) But it's not quite the issue here. @Matei I agree with you, except you seem to say it'