RE: Release Spark 2.3.4

2019-08-22 Thread Kazuaki Ishizaki
The following PRs regarding SPARK-28699 have been merged into branch-2.3. https://github.com/apache/spark/pull/25491 https://github.com/apache/spark/pull/25498 -> https://github.com/apache/spark/pull/25508 (backport to 2.3) I will cut `2.3.4-rc1` tag during weekend and starts 2.3.1 RC1 on next

Re: [VOTE] Release Apache Spark 2.4.4 (RC1)

2019-08-22 Thread Dongjoon Hyun
Hi, All. This 2.4.4 RC1 vote fails and the reported PRs are merged to `branch-2.4`. The following is the commit list since `v2.4.4-rc1` tag. Preparing development version 2.4.5-SNAPSHOT [SPARK-28749][TEST][BRANCH-2.4] Fix PySpark tests not to require kafka-0-8 [SPARK-28775][CORE][TESTS] Skip

Re: Unmarking most things as experimental, evolving for 3.0?

2019-08-22 Thread Sean Owen
Proposal at https://github.com/apache/spark/pull/25558 On Thu, Aug 22, 2019 at 2:33 AM Marco Gaido wrote: > > Thanks for bringing this out Sean. > +1 from me as well! > > Thanks, > Marco > > Il giorno gio 22 ago 2019 alle ore 08:21 Dongjoon Hyun > ha scritto: >> >> +1 for unmarking old ones

How to support hive built in functions in saprk

2019-08-22 Thread Shiva Kumar
HI i want to use the hive built-in functions in saprk. i tried current_username() function in spark local mode. i am getting the below error. Error: org.apache.spark.sql.AnalysisException: Undefined function: 'current_user'. This function is neither a registered temporary function nor a

Re: Unmarking most things as experimental, evolving for 3.0?

2019-08-22 Thread Marco Gaido
Thanks for bringing this out Sean. +1 from me as well! Thanks, Marco Il giorno gio 22 ago 2019 alle ore 08:21 Dongjoon Hyun < dongjoon.h...@gmail.com> ha scritto: > +1 for unmarking old ones (made in `2.3.x` and before). > Thank you, Sean. > > Bests, > Dongjoon. > > On Wed, Aug 21, 2019 at 6:46

Re: Unmarking most things as experimental, evolving for 3.0?

2019-08-22 Thread Dongjoon Hyun
+1 for unmarking old ones (made in `2.3.x` and before). Thank you, Sean. Bests, Dongjoon. On Wed, Aug 21, 2019 at 6:46 PM Sean Owen wrote: > There are currently about 130 things marked as 'experimental' in > Spark, and some have been around since Spark 1.x. A few may be > legitimately still