Re: [ANNOUNCE] New PMC Member : Jesus

Congrats Jesüs! --Vaibhav From: Vineet Garg Sent: Monday, July 18, 2016 6:51 PM To: d...@hive.apache.org; user@hive.apache.org Subject: Re: [ANNOUNCE] New PMC Member : Jesus Congrats Jesus ! On 7/18/16, 10:27 AM, "Jesus Camacho

Re: Hive External Storage Handlers

So not use a self-compiled hive or Spark version, but only the ones supplied by distributions (cloudera, Hortonworks, Bigtop...) You will face performance problems, strange errors etc when building and testing your code using self-compiled versions. If you use the Hive APIs then the engine

Re: Hive on TEZ + LLAP

> These looks pretty impressive. What execution mode were you running >these? Yarn client may be? There is no other mode - everything runs on YARN. > 53 times The factor is actually bigger in actual execution. The MRv2 version takes 2.47s to prep a query, while the LLAP version takes 1.64s.

Re: Hive on TEZ + LLAP

These looks pretty impressive. What execution mode were you running these? Yarn client may be? *QueryMR/sec TEZ/sec TEZ+LLAP/sec* 203.317 13.681 3.809 *Order of Magnitude*--- 15 times

Re: ORC does not support type conversion from INT to STRING.

Hi Mathew, In layman's term if I create the source ORC table column as INT and then create a target ORC table but that column has now been defined as STRING and do an INSERT/SELECT from source table how data is internally stored? Is it implicitly converted into new format using CAST function or

Re: Hive on TEZ + LLAP

> Also has there been simple benchmarks to compare: > > 1. Hive on MR > 2. Hine on Tez > 3. Hive on Tez with LLAP I ran one today, with a small BI query in my test suite against a 1Tb data-set. TL;DR - MRv2 (203.317 seconds), Tez (13.681s), LLAP (3.809s). *Warning*: This is not a historical

Re: ORC does not support type conversion from INT to STRING.

Hi Mahender, Schema Evolution is available on the latest recent version of Hive. For example, if you set hive.metastore.disallow.incompatible.col.type.changes=false;? on master (i.e. hive2) it will support INT to STRING conversion. If you need to remain on an older version, then you are

Re: Hive External Storage Handlers

Hi, You can move up to Hive 2 that works fine and pretty stable. You can opt for Hive 1.2.1 if yoy wish. If you want to use Spark (the replacement for Shark) as the execution engine for Hive then the version that works (that I have managed to make it work with Hive is Spark 1.3.1) that you will

Hive External Storage Handlers

Hello, I am working with an external storage handler written for Hive 0.11 and run on a Shark execution engine. I’d like to move forward and upgrade to hive 1.2.1 on spark 1.6 or even 2.0. This storage has a need to run queries across tables existing in different databases in the

Re: Yarn Application ID for Hive query

> be nice to have access to a command or API call in HiveServer2 similar >to MySQL¹s ³SHOW PROCESSLIST² (and equivalent commands in most other >databases). There is one - if you have the HiveServer2 UI (in 2.0), that can be seen. It would take 10-15 line JSP script to export that as a JSON

RE: Yarn Application ID for Hive query

I am running hive on Tez. I am able to get the Yarn application ID for the hive query by submitting the query through Hive JDBC and using HiveStatement. Connection con = DriverManager.getConnection("jdbc:hive2://abc:1/default","xyz", ""); HiveStatement stmt = (HiveStatement)

RE: Yarn Application ID for Hive query

Making Hive look like a normal SQL database is the goal of libraries like this, so it make sense that that abstraction wouldn't leak a concept like application ID. Especially because not all Hive queries generate a YARN application. That said, we went through this with JDBC access to Hive a

Re: [ANNOUNCE] New PMC Member : Pengcheng

Thanks for everyone. I feel really excited. Wish to contribute more and more to the community. Thanks again. On Mon, Jul 18, 2016 at 11:17 AM, Hari Sivarama Subramaniyan < hsubramani...@hortonworks.com> wrote: > Hi Pengcheng > Congrats for the well deserved achievement! > > Regards > Hari >

Re: Want to be one contributor

I believe the answer is yes, you need cygwin to develop Hive on Windows. Many of the Hadoop family of projects run on Windows natively, but require Cygwin for development. Alan. > On Jul 16, 2016, at 18:15, Alpesh Patel wrote: > > ​I am facing below mentioned issue

Test failure

Hi all, I went through Hive developer guide, built hive code successfully and then ran "mvn test". But I got the following error: --- T E S T S ---

Re: [ANNOUNCE] New PMC Member : Pengcheng

Hi Pengcheng Congrats for the well deserved achievement! Regards Hari From: Jesus Camacho Rodriguez Sent: Monday, July 18, 2016 10:26 AM To: user@hive.apache.org Cc: d...@hive.apache.org Subject: Re: [ANNOUNCE] New PMC

Re: [ANNOUNCE] New PMC Member : Jesus

Hi Jesus Congrats for the well deserved achievement. Regards Hari From: Jesus Camacho Rodriguez Sent: Monday, July 18, 2016 10:27 AM To: user@hive.apache.org Cc: d...@hive.apache.org Subject: Re: [ANNOUNCE] New PMC

Re: [ANNOUNCE] New PMC Member : Jesus

Thanks everybody! Looking forward to continue contributing to the project! -- Jesús On 7/18/16, 6:21 PM, "Prasanth Jayachandran" wrote: >Congratulations Jesus! > >> On Jul 18, 2016, at 10:10 AM, Jimmy Xiang wrote: >> >> Congrats!! >> >>

Re: [ANNOUNCE] New PMC Member : Pengcheng

Congrats Pengcheng, well deserved! :) On 7/18/16, 6:25 PM, "Vaibhav Gumashta" wrote: >Congrats Pengcheng! > >From: Prasanth Jayachandran >Sent: Monday, July 18, 2016 10:21 AM >To:

Re: [ANNOUNCE] New PMC Member : Pengcheng

Congrats Pengcheng! From: Prasanth Jayachandran Sent: Monday, July 18, 2016 10:21 AM To: user@hive.apache.org Cc: d...@hive.apache.org Subject: Re: [ANNOUNCE] New PMC Member : Pengcheng Congratulations Pengcheng! > On Jul

Re: [ANNOUNCE] New PMC Member : Jesus

Congratulations Jesus! > On Jul 18, 2016, at 10:10 AM, Jimmy Xiang wrote: > > Congrats!! > > On Mon, Jul 18, 2016 at 9:54 AM, Vihang Karajgaonkar > wrote: >> Congratulations Jesus! >> >>> On Jul 18, 2016, at 8:30 AM, Sergio Pena

Re: [ANNOUNCE] New PMC Member : Pengcheng

Congratulations Pengcheng! > On Jul 18, 2016, at 10:10 AM, Jimmy Xiang wrote: > > Congrats!! > > On Mon, Jul 18, 2016 at 9:55 AM, Vihang Karajgaonkar > wrote: >> Congratulations! >> >>> On Jul 18, 2016, at 5:28 AM, Peter Vary wrote:

Re: [ANNOUNCE] New PMC Member : Pengcheng

Congrats!! On Mon, Jul 18, 2016 at 9:55 AM, Vihang Karajgaonkar wrote: > Congratulations! > >> On Jul 18, 2016, at 5:28 AM, Peter Vary wrote: >> >> Congratulations Pengcheng! >> >> >>> On Jul 18, 2016, at 6:55 AM, Wei Zheng

Re: [ANNOUNCE] New PMC Member : Jesus

Congrats!! On Mon, Jul 18, 2016 at 9:54 AM, Vihang Karajgaonkar wrote: > Congratulations Jesus! > >> On Jul 18, 2016, at 8:30 AM, Sergio Pena wrote: >> >> Congrats Jesus !!! >> >> On Mon, Jul 18, 2016 at 7:28 AM, Peter Vary

Re: [ANNOUNCE] New PMC Member : Pengcheng

Congratulations! > On Jul 18, 2016, at 5:28 AM, Peter Vary wrote: > > Congratulations Pengcheng! > > >> On Jul 18, 2016, at 6:55 AM, Wei Zheng wrote: >> >> Congrats Pengcheng! >> >> Thanks, >> >> Wei >> >> >> >> >> >> >> On 7/17/16, 16:01,

Re: [ANNOUNCE] New PMC Member : Jesus

Congratulations Jesus! > On Jul 18, 2016, at 8:30 AM, Sergio Pena wrote: > > Congrats Jesus !!! > > On Mon, Jul 18, 2016 at 7:28 AM, Peter Vary wrote: > >> Congratulations Jesus! >> >>> On Jul 18, 2016, at 6:55 AM, Wei Zheng

Re: [ANNOUNCE] New PMC Member : Jesus

Congratulations Jesus! On Mon, Jul 18, 2016 at 8:30 AM, Sergio Pena wrote: > Congrats Jesus !!! > > On Mon, Jul 18, 2016 at 7:28 AM, Peter Vary wrote: > > > Congratulations Jesus! > > > > > On Jul 18, 2016, at 6:55 AM, Wei Zheng

Re: Query Performance Issue : Group By and Distinct and load on reducer

Hi Dudu, Thanks for your help and proactive response on it. Today I have verified all solution you provided. it worked for me for given table with 6 billion records. Before I conclude anything, i want to check if there is any reference document/link available for these algorithm / approach It

Re: [ANNOUNCE] New PMC Member : Pengcheng

Congratulations Pengcheng! > On Jul 18, 2016, at 6:55 AM, Wei Zheng wrote: > > Congrats Pengcheng! > > Thanks, > > Wei > > > > > > > On 7/17/16, 16:01, "Xuefu Zhang" wrote: > >> Congrats, PengCheng! >> >> On Sun, Jul 17, 2016 at 2:28 PM,

Re: [ANNOUNCE] New PMC Member : Jesus

Congratulations Jesus! > On Jul 18, 2016, at 6:55 AM, Wei Zheng wrote: > > Congrats Jesus! > > Thanks, > > Wei > > > > > > > > On 7/17/16, 14:29, "Sushanth Sowmyan" wrote: > >> Good to have you onboard, Jesus! :) >> >> On Jul 17, 2016