Re: Time to Remove Hive-on-Spark

2022-04-12 Thread Peter Vary
+1 from my side too. I have created PR against the current branch. Still needs some work, and as many reviews as possible, because it is quite big, and I might made some mistakes https://issues.apache.org/jira/browse/HIVE-26134 https://github.com/apache/hive/pull/3201 Thanks, Peter On Thu, 10

Re: Time to Remove Hive-on-Spark

2022-02-10 Thread Zoltan Haindrich
Hey, I think there is no real interest in this feature; we don't have users/contributors backing it - last development was around 2018 October; there were ~2 bugfix commits ever since that...we should stop carrying dead weight...another 2 weeks went by since Stamatis have reminded us that after

Re: Time to Remove Hive-on-Spark

2022-01-28 Thread Stamatis Zampetakis
Hi team, Almost one year has passed since the last exchange in this discussion and if I am not wrong there has been no effort to revive Hive-on-Spark. To be more precise, I don't think I have seen any Spark related JIRA for quite some time now and although I don't want to rush into conclusions,

Re: Time to Remove Hive-on-Spark

2021-02-26 Thread Edward Capriolo
I do not know how it works for most of the world. But in cloudera where the TEZ options were never popular hive-on-spark represents a solid way to get things done for small datasets lower latency. As for the spark adoption. You know a while ago I came up with some ways to make hive more spark

Re: Time to Remove Hive-on-Spark

2020-07-27 Thread David
Hello Xuefu, I am not part of the Cloudera Hive product team, though I volunteer to work on small projects from time to time. Perhaps someone from that team can chime in with some of their thoughts, but personally, I think that in the long run, there will be more of a merge between

Re: Time to Remove Hive-on-Spark

2020-07-23 Thread Xuefu Zhang
Previous reasoning seemed to suggest a lack of user adoption. Now we are concerned about ongoing maintenance effort. Both are valid considerations. However, I think we should have ways to find out the answers. Therefore, I suggest the following be carried out: 1. Send out the proposal (removing

Re: Time to Remove Hive-on-Spark

2020-07-22 Thread Alan Gates
An important point here is I don't believe David is proposing to remove Hive on Spark from the 2 or 3 lines, but only from trunk. Continuing to support it in existing 2 and 3 lines makes sense, but since no one has maintained it on trunk for some time and it does not work with many of the newer

Re: Time to Remove Hive-on-Spark

2020-07-21 Thread Chao Sun
Thanks David. FWIW Uber is still running Hive on Spark (2.3.4) on a very large scale in production right now and I don't think we have any plan to change it soon. On Tue, Jul 21, 2020 at 11:28 AM David wrote: > Hello, > > Thanks for the feedback. > > Just a quick recap: I did propose this

Re: Time to Remove Hive-on-Spark

2020-07-21 Thread David
Hello, Thanks for the feedback. Just a quick recap: I did propose this @dev and I received unanimous +1's from the community. After a couple months, I created the PR. Certainly open to discussion, but there hasn't been any discussion thus far because there have been no objections until this

Re: Time to Remove Hive-on-Spark

2020-07-21 Thread Xuefu Zhang
Hi David, While a vendor may not support a component in an open source project, removing it or not is a decision by and for the community. I certainly understand that the vendor you mentioned has contributed a great deal (including my personal effort while working there), it's not up to the

Re: Time to Remove Hive-on-Spark

2020-07-21 Thread David
Hey, Thanks for the input. FYI. Cloudera (Cloudera + Hortonworks) have removed HoS from their latest offering. "Tez is now the only supported execution engine, existing queries that change execution mode to Spark or MapReduce within a session, for example, fail."

Re: Time to Remove Hive-on-Spark

2020-07-21 Thread Xuefu Z
Sorry for chiming in late. However, I don't think we should remove Hive on Spark just because of a technical problem. This is rather a big decision that we need to be careful about. There are users that will be left high and dry by this move. If the community decides to desupport and eventually

Re: Time to Remove Hive-on-Spark

2020-07-21 Thread David
Hello Team, https://github.com/apache/hive/pull/1285 Thanks. On Wed, Jun 3, 2020 at 11:49 PM Gopal V wrote: > > +1 > > Cheers, > Gopal > > On 6/3/20 7:48 PM, Jesus Camacho Rodriguez wrote: > > +1 > > > > -Jesús > > > > On Wed, Jun 3, 2020 at 1:58 PM Alan Gates wrote: > > > >> +1. > >> > >>

Re: Time to Remove Hive-on-Spark

2020-06-03 Thread Gopal V
+1 Cheers, Gopal On 6/3/20 7:48 PM, Jesus Camacho Rodriguez wrote: +1 -Jesús On Wed, Jun 3, 2020 at 1:58 PM Alan Gates wrote: +1. Alan. On Wed, Jun 3, 2020 at 1:40 PM Prasanth Jayachandran wrote: +1 On Jun 3, 2020, at 1:38 PM, Ashutosh Chauhan wrote: +1 On Wed, Jun 3, 2020

Re: Time to Remove Hive-on-Spark

2020-06-03 Thread Jesus Camacho Rodriguez
+1 -Jesús On Wed, Jun 3, 2020 at 1:58 PM Alan Gates wrote: > +1. > > Alan. > > On Wed, Jun 3, 2020 at 1:40 PM Prasanth Jayachandran > wrote: > > > +1 > > > > > On Jun 3, 2020, at 1:38 PM, Ashutosh Chauhan > > wrote: > > > > > > +1 > > > > > > On Wed, Jun 3, 2020 at 1:23 PM David Mollitor >

Re: Time to Remove Hive-on-Spark

2020-06-03 Thread Alan Gates
+1. Alan. On Wed, Jun 3, 2020 at 1:40 PM Prasanth Jayachandran wrote: > +1 > > > On Jun 3, 2020, at 1:38 PM, Ashutosh Chauhan > wrote: > > > > +1 > > > > On Wed, Jun 3, 2020 at 1:23 PM David Mollitor wrote: > > > >> Hello Gang, > >> > >> I have spent some time working on upgrading Avro (far

Re: Time to Remove Hive-on-Spark

2020-06-03 Thread Prasanth Jayachandran
+1 > On Jun 3, 2020, at 1:38 PM, Ashutosh Chauhan wrote: > > +1 > > On Wed, Jun 3, 2020 at 1:23 PM David Mollitor wrote: > >> Hello Gang, >> >> I have spent some time working on upgrading Avro (far less than others): >> >> https://issues.apache.org/jira/browse/HIVE-21737 >> >> This should

Re: Time to Remove Hive-on-Spark

2020-06-03 Thread Ashutosh Chauhan
+1 On Wed, Jun 3, 2020 at 1:23 PM David Mollitor wrote: > Hello Gang, > > I have spent some time working on upgrading Avro (far less than others): > > https://issues.apache.org/jira/browse/HIVE-21737 > > This should be a relatively easy thing to do, but is blocked by > Hive-on-Spark. HoS has a

Time to Remove Hive-on-Spark

2020-06-03 Thread David Mollitor
Hello Gang, I have spent some time working on upgrading Avro (far less than others): https://issues.apache.org/jira/browse/HIVE-21737 This should be a relatively easy thing to do, but is blocked by Hive-on-Spark. HoS has a weird thing where it downloads some cloud-storage-hosted file of