Re: [ANNOUNCE] New PMC Member : John Pullokkaran

[Devopam Mittra]

+1
Congratulations !

regards
Dev


On Wed, Nov 25, 2015 at 4:29 AM, Ashutosh Chauhan 
wrote:

> On behalf of the Hive PMC I am delighted to announce John Pullokkaran is
> joining Hive PMC.
> John is a long time contributor in Hive and is focusing on compiler and
> optimizer areas these days.
> Please give John a warm welcome to the project!
>
> Ashutosh
>



-- 
Devopam Mittra
Life and Relations are not binary

Re: Upgrading from Hive 0.14.0 to Hive 1.2.1

[Lefty Leverenz]

Here's the documentation for Hive's schema tool:
https://cwiki.apache.org/confluence/display/Hive/Hive+Schema+Tool.

-- Lefty

On Tue, Nov 24, 2015 at 2:12 PM, Xuefu Zhang  wrote:

> This upgrade should be no different from other upgrade. You can use Hive's
> schema tool to upgrade your existing metadata.
>
> Thanks,
> Xuefu
>
> On Tue, Nov 24, 2015 at 10:05 AM, Mich Talebzadeh 
> wrote:
>
>> Hi,
>>
>>
>>
>> I would like to upgrade to Hive 1.2.1 as I understand one cannot deploy
>> Spark execution engine on 0.14
>>
>>
>>
>> *Chooses execution engine. Options are: **mr** (Map reduce, default), *
>> *tez** (Tez
>>  execution,
>> for Hadoop 2 only), or **spark** (Spark
>>  execution,
>> for Hive 1.1.0 onward).*
>>
>>
>>
>> Is there any upgrade path (I don’t want to lose my existing databases in
>> Hive) or I have to start from new including generating new metatsore etc?
>>
>>
>>
>> Thanks,
>>
>>
>>
>>
>>
>> Mich Talebzadeh
>>
>>
>>
>> *Sybase ASE 15 Gold Medal Award 2008*
>>
>> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>>
>>
>> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>>
>> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
>> 15", ISBN 978-0-9563693-0-7*.
>>
>> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
>> 978-0-9759693-0-4*
>>
>> *Publications due shortly:*
>>
>> *Complex Event Processing in Heterogeneous Environments*, ISBN:
>> 978-0-9563693-3-8
>>
>> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
>> one out shortly
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>>
>> NOTE: The information in this email is proprietary and confidential. This
>> message is for the designated recipient only, if you are not the intended
>> recipient, you should destroy it immediately. Any information in this
>> message shall not be understood as given or endorsed by Peridale Technology
>> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
>> the responsibility of the recipient to ensure that this email is virus
>> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
>> accept any responsibility.
>>
>>
>>
>
>

RE: [ANNOUNCE] New PMC Member : John Pullokkaran

[Bright Ling]

Congrats! John!

From: Pengcheng Xiong [mailto:pxi...@apache.org]
Sent: Wednesday, 25 November 2015 11:11 AM
To: user@hive.apache.org
Cc: d...@hive.apache.org
Subject: Re: [ANNOUNCE] New PMC Member : John Pullokkaran

Congrats! John!

On Wed, Nov 25, 2015 at 6:59 AM, Ashutosh Chauhan 
mailto:hashut...@apache.org>> wrote:
On behalf of the Hive PMC I am delighted to announce John Pullokkaran is 
joining Hive PMC.
John is a long time contributor in Hive and is focusing on compiler and 
optimizer areas these days.
Please give John a warm welcome to the project!

Ashutosh

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

[Pengcheng Xiong]

Congrats! John!

On Wed, Nov 25, 2015 at 6:59 AM, Ashutosh Chauhan 
wrote:

> On behalf of the Hive PMC I am delighted to announce John Pullokkaran is
> joining Hive PMC.
> John is a long time contributor in Hive and is focusing on compiler and
> optimizer areas these days.
> Please give John a warm welcome to the project!
>
> Ashutosh
>

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

[Lefty Leverenz]

Congratulations John!

-- Lefty

On Tue, Nov 24, 2015 at 6:31 PM, Navis Ryu  wrote:

> Congratulations!
>
> 2015년 11월 25일 수요일, Hari Sivarama Subramaniyan<
> hsubramani...@hortonworks.com>님이 작성한 메시지:
>
> Congrats, John!
>> --
>> *From:* Eugene Koifman 
>> *Sent:* Tuesday, November 24, 2015 3:14 PM
>> *To:* user@hive.apache.org; d...@hive.apache.org
>> *Subject:* Re: [ANNOUNCE] New PMC Member : John Pullokkaran
>>
>> Congrats!
>>
>> From: Sergey Shelukhin 
>> Reply-To: "user@hive.apache.org" 
>> Date: Tuesday, November 24, 2015 at 3:13 PM
>> To: "user@hive.apache.org" , "d...@hive.apache.org"
>> 
>> Subject: Re: [ANNOUNCE] New PMC Member : John Pullokkaran
>>
>> Congrats!
>>
>> From: Jimmy Xiang 
>> Reply-To: "user@hive.apache.org" 
>> Date: Tuesday, November 24, 2015 at 15:07
>> To: "d...@hive.apache.org" 
>> Cc: "user@hive.apache.org" 
>> Subject: Re: [ANNOUNCE] New PMC Member : John Pullokkaran
>>
>> Congrats!!
>>
>> On Tue, Nov 24, 2015 at 3:04 PM, Szehon Ho  wrote:
>>
>>> Congratulations!
>>>
>>> On Tue, Nov 24, 2015 at 3:02 PM, Xuefu Zhang 
>>> wrote:
>>>
>>> > Congratulations, John!
>>> >
>>> > --Xuefu
>>> >
>>> > On Tue, Nov 24, 2015 at 3:01 PM, Prasanth J 
>>> > wrote:
>>> >
>>> >> Congratulations and Welcome John!
>>> >>
>>> >> Thanks
>>> >> Prasanth
>>> >>
>>> >> On Nov 24, 2015, at 4:59 PM, Ashutosh Chauhan 
>>> >> wrote:
>>> >>
>>> >> On behalf of the Hive PMC I am delighted to announce John Pullokkaran
>>> is
>>> >> joining Hive PMC.
>>> >> John is a long time contributor in Hive and is focusing on compiler
>>> and
>>> >> optimizer areas these days.
>>> >> Please give John a warm welcome to the project!
>>> >>
>>> >> Ashutosh
>>> >>
>>> >>
>>> >>
>>> >
>>>
>>
>>

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

[Navis Ryu]

Congratulations!

2015년 11월 25일 수요일, Hari Sivarama Subramaniyan님이
작성한 메시지:

> Congrats, John!
> --
> *From:* Eugene Koifman  >
> *Sent:* Tuesday, November 24, 2015 3:14 PM
> *To:* user@hive.apache.org
> ;
> d...@hive.apache.org 
> *Subject:* Re: [ANNOUNCE] New PMC Member : John Pullokkaran
>
> Congrats!
>
> From: Sergey Shelukhin  >
> Reply-To: "user@hive.apache.org
> " <
> user@hive.apache.org
> >
> Date: Tuesday, November 24, 2015 at 3:13 PM
> To: "user@hive.apache.org
> " <
> user@hive.apache.org
> >, "
> d...@hive.apache.org "
> 
> >
> Subject: Re: [ANNOUNCE] New PMC Member : John Pullokkaran
>
> Congrats!
>
> From: Jimmy Xiang  >
> Reply-To: "user@hive.apache.org
> " <
> user@hive.apache.org
> >
> Date: Tuesday, November 24, 2015 at 15:07
> To: "d...@hive.apache.org
> " <
> d...@hive.apache.org >
> Cc: "user@hive.apache.org
> " <
> user@hive.apache.org
> >
> Subject: Re: [ANNOUNCE] New PMC Member : John Pullokkaran
>
> Congrats!!
>
> On Tue, Nov 24, 2015 at 3:04 PM, Szehon Ho  > wrote:
>
>> Congratulations!
>>
>> On Tue, Nov 24, 2015 at 3:02 PM, Xuefu Zhang > > wrote:
>>
>> > Congratulations, John!
>> >
>> > --Xuefu
>> >
>> > On Tue, Nov 24, 2015 at 3:01 PM, Prasanth J > >
>> > wrote:
>> >
>> >> Congratulations and Welcome John!
>> >>
>> >> Thanks
>> >> Prasanth
>> >>
>> >> On Nov 24, 2015, at 4:59 PM, Ashutosh Chauhan > >
>> >> wrote:
>> >>
>> >> On behalf of the Hive PMC I am delighted to announce John Pullokkaran
>> is
>> >> joining Hive PMC.
>> >> John is a long time contributor in Hive and is focusing on compiler and
>> >> optimizer areas these days.
>> >> Please give John a warm welcome to the project!
>> >>
>> >> Ashutosh
>> >>
>> >>
>> >>
>> >
>>
>
>

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

[Hari Sivarama Subramaniyan]

Congrats, John!


From: Eugene Koifman 
Sent: Tuesday, November 24, 2015 3:14 PM
To: user@hive.apache.org; d...@hive.apache.org
Subject: Re: [ANNOUNCE] New PMC Member : John Pullokkaran

Congrats!

From: Sergey Shelukhin mailto:ser...@hortonworks.com>>
Reply-To: "user@hive.apache.org" 
mailto:user@hive.apache.org>>
Date: Tuesday, November 24, 2015 at 3:13 PM
To: "user@hive.apache.org" 
mailto:user@hive.apache.org>>, 
"d...@hive.apache.org" 
mailto:d...@hive.apache.org>>
Subject: Re: [ANNOUNCE] New PMC Member : John Pullokkaran

Congrats!

From: Jimmy Xiang mailto:jxi...@cloudera.com>>
Reply-To: "user@hive.apache.org" 
mailto:user@hive.apache.org>>
Date: Tuesday, November 24, 2015 at 15:07
To: "d...@hive.apache.org" 
mailto:d...@hive.apache.org>>
Cc: "user@hive.apache.org" 
mailto:user@hive.apache.org>>
Subject: Re: [ANNOUNCE] New PMC Member : John Pullokkaran

Congrats!!

On Tue, Nov 24, 2015 at 3:04 PM, Szehon Ho 
mailto:sze...@cloudera.com>> wrote:
Congratulations!

On Tue, Nov 24, 2015 at 3:02 PM, Xuefu Zhang 
mailto:xzh...@cloudera.com>> wrote:

> Congratulations, John!
>
> --Xuefu
>
> On Tue, Nov 24, 2015 at 3:01 PM, Prasanth J 
> mailto:j.prasant...@gmail.com>>
> wrote:
>
>> Congratulations and Welcome John!
>>
>> Thanks
>> Prasanth
>>
>> On Nov 24, 2015, at 4:59 PM, Ashutosh Chauhan 
>> mailto:hashut...@apache.org>>
>> wrote:
>>
>> On behalf of the Hive PMC I am delighted to announce John Pullokkaran is
>> joining Hive PMC.
>> John is a long time contributor in Hive and is focusing on compiler and
>> optimizer areas these days.
>> Please give John a warm welcome to the project!
>>
>> Ashutosh
>>
>>
>>
>

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

[Eugene Koifman]

Congrats!

From: Sergey Shelukhin mailto:ser...@hortonworks.com>>
Reply-To: "user@hive.apache.org" 
mailto:user@hive.apache.org>>
Date: Tuesday, November 24, 2015 at 3:13 PM
To: "user@hive.apache.org" 
mailto:user@hive.apache.org>>, 
"d...@hive.apache.org" 
mailto:d...@hive.apache.org>>
Subject: Re: [ANNOUNCE] New PMC Member : John Pullokkaran

Congrats!

From: Jimmy Xiang mailto:jxi...@cloudera.com>>
Reply-To: "user@hive.apache.org" 
mailto:user@hive.apache.org>>
Date: Tuesday, November 24, 2015 at 15:07
To: "d...@hive.apache.org" 
mailto:d...@hive.apache.org>>
Cc: "user@hive.apache.org" 
mailto:user@hive.apache.org>>
Subject: Re: [ANNOUNCE] New PMC Member : John Pullokkaran

Congrats!!

On Tue, Nov 24, 2015 at 3:04 PM, Szehon Ho 
mailto:sze...@cloudera.com>> wrote:
Congratulations!

On Tue, Nov 24, 2015 at 3:02 PM, Xuefu Zhang 
mailto:xzh...@cloudera.com>> wrote:

> Congratulations, John!
>
> --Xuefu
>
> On Tue, Nov 24, 2015 at 3:01 PM, Prasanth J 
> mailto:j.prasant...@gmail.com>>
> wrote:
>
>> Congratulations and Welcome John!
>>
>> Thanks
>> Prasanth
>>
>> On Nov 24, 2015, at 4:59 PM, Ashutosh Chauhan 
>> mailto:hashut...@apache.org>>
>> wrote:
>>
>> On behalf of the Hive PMC I am delighted to announce John Pullokkaran is
>> joining Hive PMC.
>> John is a long time contributor in Hive and is focusing on compiler and
>> optimizer areas these days.
>> Please give John a warm welcome to the project!
>>
>> Ashutosh
>>
>>
>>
>

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

[Sergey Shelukhin]

Congrats!

From: Jimmy Xiang mailto:jxi...@cloudera.com>>
Reply-To: "user@hive.apache.org" 
mailto:user@hive.apache.org>>
Date: Tuesday, November 24, 2015 at 15:07
To: "d...@hive.apache.org" 
mailto:d...@hive.apache.org>>
Cc: "user@hive.apache.org" 
mailto:user@hive.apache.org>>
Subject: Re: [ANNOUNCE] New PMC Member : John Pullokkaran

Congrats!!

On Tue, Nov 24, 2015 at 3:04 PM, Szehon Ho 
mailto:sze...@cloudera.com>> wrote:
Congratulations!

On Tue, Nov 24, 2015 at 3:02 PM, Xuefu Zhang 
mailto:xzh...@cloudera.com>> wrote:

> Congratulations, John!
>
> --Xuefu
>
> On Tue, Nov 24, 2015 at 3:01 PM, Prasanth J 
> mailto:j.prasant...@gmail.com>>
> wrote:
>
>> Congratulations and Welcome John!
>>
>> Thanks
>> Prasanth
>>
>> On Nov 24, 2015, at 4:59 PM, Ashutosh Chauhan 
>> mailto:hashut...@apache.org>>
>> wrote:
>>
>> On behalf of the Hive PMC I am delighted to announce John Pullokkaran is
>> joining Hive PMC.
>> John is a long time contributor in Hive and is focusing on compiler and
>> optimizer areas these days.
>> Please give John a warm welcome to the project!
>>
>> Ashutosh
>>
>>
>>
>

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

[Jimmy Xiang]

Congrats!!

On Tue, Nov 24, 2015 at 3:04 PM, Szehon Ho  wrote:

> Congratulations!
>
> On Tue, Nov 24, 2015 at 3:02 PM, Xuefu Zhang  wrote:
>
> > Congratulations, John!
> >
> > --Xuefu
> >
> > On Tue, Nov 24, 2015 at 3:01 PM, Prasanth J 
> > wrote:
> >
> >> Congratulations and Welcome John!
> >>
> >> Thanks
> >> Prasanth
> >>
> >> On Nov 24, 2015, at 4:59 PM, Ashutosh Chauhan 
> >> wrote:
> >>
> >> On behalf of the Hive PMC I am delighted to announce John Pullokkaran is
> >> joining Hive PMC.
> >> John is a long time contributor in Hive and is focusing on compiler and
> >> optimizer areas these days.
> >> Please give John a warm welcome to the project!
> >>
> >> Ashutosh
> >>
> >>
> >>
> >
>

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

[Szehon Ho]

Congratulations!

On Tue, Nov 24, 2015 at 3:02 PM, Xuefu Zhang  wrote:

> Congratulations, John!
>
> --Xuefu
>
> On Tue, Nov 24, 2015 at 3:01 PM, Prasanth J 
> wrote:
>
>> Congratulations and Welcome John!
>>
>> Thanks
>> Prasanth
>>
>> On Nov 24, 2015, at 4:59 PM, Ashutosh Chauhan 
>> wrote:
>>
>> On behalf of the Hive PMC I am delighted to announce John Pullokkaran is
>> joining Hive PMC.
>> John is a long time contributor in Hive and is focusing on compiler and
>> optimizer areas these days.
>> Please give John a warm welcome to the project!
>>
>> Ashutosh
>>
>>
>>
>

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

[Xuefu Zhang]

Congratulations, John!

--Xuefu

On Tue, Nov 24, 2015 at 3:01 PM, Prasanth J  wrote:

> Congratulations and Welcome John!
>
> Thanks
> Prasanth
>
> On Nov 24, 2015, at 4:59 PM, Ashutosh Chauhan 
> wrote:
>
> On behalf of the Hive PMC I am delighted to announce John Pullokkaran is
> joining Hive PMC.
> John is a long time contributor in Hive and is focusing on compiler and
> optimizer areas these days.
> Please give John a warm welcome to the project!
>
> Ashutosh
>
>
>

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

[Prasanth J]

Congratulations and Welcome John!

Thanks
Prasanth
> On Nov 24, 2015, at 4:59 PM, Ashutosh Chauhan  wrote:
> 
> On behalf of the Hive PMC I am delighted to announce John Pullokkaran is 
> joining Hive PMC. 
> John is a long time contributor in Hive and is focusing on compiler and 
> optimizer areas these days. 
> Please give John a warm welcome to the project!
> 
> Ashutosh

[ANNOUNCE] New PMC Member : John Pullokkaran

[Ashutosh Chauhan]

On behalf of the Hive PMC I am delighted to announce John Pullokkaran is
joining Hive PMC.
John is a long time contributor in Hive and is focusing on compiler and
optimizer areas these days.
Please give John a warm welcome to the project!

Ashutosh

Re: Upgrading from Hive 0.14.0 to Hive 1.2.1

[Xuefu Zhang]

This upgrade should be no different from other upgrade. You can use Hive's
schema tool to upgrade your existing metadata.

Thanks,
Xuefu

On Tue, Nov 24, 2015 at 10:05 AM, Mich Talebzadeh 
wrote:

> Hi,
>
>
>
> I would like to upgrade to Hive 1.2.1 as I understand one cannot deploy
> Spark execution engine on 0.14
>
>
>
> *Chooses execution engine. Options are: **mr** (Map reduce, default), *
> *tez** (Tez
>  execution,
> for Hadoop 2 only), or **spark** (Spark
>  execution,
> for Hive 1.1.0 onward).*
>
>
>
> Is there any upgrade path (I don’t want to lose my existing databases in
> Hive) or I have to start from new including generating new metatsore etc?
>
>
>
> Thanks,
>
>
>
>
>
> Mich Talebzadeh
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>

Re: ERROR util.SparkUncaughtExceptionHandler: Uncaught exception in thread Thread

[Dasun Hegoda]

I'm using *apache-hive-1.2.1-bin *with *spark-1.5.1-bin-hadoop2.6 *on top
of
*hadoop-2.7.1 *. Any idea how to fix this error?

On Tue, Nov 24, 2015 at 11:43 PM, Mich Talebzadeh 
wrote:

> Which version of Hive are you using?
>
>
>
> Mich Talebzadeh
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
> *From:* Dasun Hegoda [mailto:dasunheg...@gmail.com]
> *Sent:* 24 November 2015 12:01
> *To:* user@hive.apache.org
> *Subject:* ERROR util.SparkUncaughtExceptionHandler: Uncaught exception
> in thread Thread
>
>
>
> Hi,
>
>
>
> I get below error when I try to run Hive on Spark, Any idea how to fix
> this?
>
>
>
>
>
> 15/11/24 06:33:47 ERROR util.SparkUncaughtExceptionHandler: Uncaught
> exception in thread Thread[appclient-registration-retry-thread,5,main]
>
> java.util.concurrent.RejectedExecutionException: Task
> java.util.concurrent.FutureTask@34d6330a rejected from
> java.util.concurrent.ThreadPoolExecutor@5ab0f09f[Running, pool size = 1,
> active threads = 1, queued tasks = 0, completed tasks = 0]
>
> at
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
>
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
>
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
>
> at
> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
>
> at
> org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anonfun$tryRegisterAllMasters$1.apply(AppClient.scala:96)
>
> at
> org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anonfun$tryRegisterAllMasters$1.apply(AppClient.scala:95)
>
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>
> at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
>
> at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
>
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
>
> at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:108)
>
> at
> org.apache.spark.deploy.client.AppClient$ClientEndpoint.tryRegisterAllMasters(AppClient.scala:95)
>
> at
> org.apache.spark.deploy.client.AppClient$ClientEndpoint.org$apache$spark$deploy$client$AppClient$ClientEndpoint$$registerWithMaster(AppClient.scala:121)
>
> at
> org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2$$anonfun$run$1.apply$mcV$sp(AppClient.scala:132)
>
> at org.apache.spark.util.Utils$.tryOrExit(Utils.scala:1119)
>
> at
> org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2.run(AppClient.scala:124)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> 15/11/24 06:33:47 INFO storage.DiskBlockManager: Shutdown hook called
>
> 15/11/24 06:33:47 INFO util.ShutdownHookManager: Shutdown hook called
>
> 15/11/24 06:33:47 INFO util.ShutdownHookManager: Deleting directory
> /tmp/spark-dabf9338-0423-4190-83ed-036e9e61c770
>
> 15/11/24 06:33:47 INFO util.ShutdownHookManager: Deleting directory
> /tmp/spark-c97cfee4-8bff-4b7d-8af2-59e9fd56965
>
>
>
> --
>
> Regards,
>
> Dasun Hegoda, Software Engineer
> www.dasunhegoda.com | dasunheg...@gmail.com
>



-- 
R

RE: ERROR util.SparkUncaughtExceptionHandler: Uncaught exception in thread Thread

[Mich Talebzadeh]

Which version of Hive are you using?

 

Mich Talebzadeh

 

Sybase ASE 15 Gold Medal Award 2008

A Winning Strategy: Running the most Critical Financial Data on ASE 15

http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf

Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", 
ISBN 978-0-9563693-0-7. 

co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 
978-0-9759693-0-4

Publications due shortly:

Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8

Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one 
out shortly

 

http://talebzadehmich.wordpress.com  

 

NOTE: The information in this email is proprietary and confidential. This 
message is for the designated recipient only, if you are not the intended 
recipient, you should destroy it immediately. Any information in this message 
shall not be understood as given or endorsed by Peridale Technology Ltd, its 
subsidiaries or their employees, unless expressly so stated. It is the 
responsibility of the recipient to ensure that this email is virus free, 
therefore neither Peridale Ltd, its subsidiaries nor their employees accept any 
responsibility.

 

From: Dasun Hegoda [mailto:dasunheg...@gmail.com] 
Sent: 24 November 2015 12:01
To: user@hive.apache.org
Subject: ERROR util.SparkUncaughtExceptionHandler: Uncaught exception in thread 
Thread

 

Hi,

 

I get below error when I try to run Hive on Spark, Any idea how to fix this?

 

 

15/11/24 06:33:47 ERROR util.SparkUncaughtExceptionHandler: Uncaught exception 
in thread Thread[appclient-registration-retry-thread,5,main]

java.util.concurrent.RejectedExecutionException: Task 
java.util.concurrent.FutureTask@34d6330a 
  rejected from 
java.util.concurrent.ThreadPoolExecutor@5ab0f09f[Running 
 , pool size = 
1, active threads = 1, queued tasks = 0, completed tasks = 0]

at 
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)

at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)

at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)

at 
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)

at 
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anonfun$tryRegisterAllMasters$1.apply(AppClient.scala:96)

at 
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anonfun$tryRegisterAllMasters$1.apply(AppClient.scala:95)

at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)

at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)

at 
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)

at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)

at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)

at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:108)

at 
org.apache.spark.deploy.client.AppClient$ClientEndpoint.tryRegisterAllMasters(AppClient.scala:95)

at 
org.apache.spark.deploy.client.AppClient$ClientEndpoint.org$apache$spark$deploy$client$AppClient$ClientEndpoint$$registerWithMaster(AppClient.scala:121)

at 
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2$$anonfun$run$1.apply$mcV$sp(AppClient.scala:132)

at org.apache.spark.util.Utils$.tryOrExit(Utils.scala:1119)

at 
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2.run(AppClient.scala:124)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

15/11/24 06:33:47 INFO storage.DiskBlockManager: Shutdown hook called

15/11/24 06:33:47 INFO util.ShutdownHookManager: Shutdown hook called

15/11/24 06:33:47 INFO util.ShutdownHookManager: Deleting directory 
/tmp/spark-dabf9338-0423-4190-83ed-036e9e61c770

15/11/24 06:33:47 INFO util.ShutdownHookManager: Deleting directory 
/tmp/spark-c97cfee4-8bff-4b7d-8af2-59e9fd56965

 

-- 

Regards,

Dasun Hegoda, Software Engineer  
www.dasunhegoda.com   | dasunheg...@gmail.com 
 

Upgrading from Hive 0.14.0 to Hive 1.2.1

[Mich Talebzadeh]

Hi,

 

I would like to upgrade to Hive 1.2.1 as I understand one cannot deploy
Spark execution engine on 0.14

 

Chooses execution engine. Options are: mr (Map reduce, default), tez (Tez
  execution,
for Hadoop 2 only), or spark (Spark
  execution,
for Hive 1.1.0 onward).

 

Is there any upgrade path (I don't want to lose my existing databases in
Hive) or I have to start from new including generating new metatsore etc?

 

Thanks,

 

 

Mich Talebzadeh

 

Sybase ASE 15 Gold Medal Award 2008

A Winning Strategy: Running the most Critical Financial Data on ASE 15

 

http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.
pdf

Author of the books "A Practitioner's Guide to Upgrading to Sybase ASE 15",
ISBN 978-0-9563693-0-7. 

co-author "Sybase Transact SQL Guidelines Best Practices", ISBN
978-0-9759693-0-4

Publications due shortly:

Complex Event Processing in Heterogeneous Environments, ISBN:
978-0-9563693-3-8

Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume
one out shortly

 

  http://talebzadehmich.wordpress.com

 

NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Technology
Ltd, its subsidiaries or their employees, unless expressly so stated. It is
the responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Ltd, its subsidiaries nor their employees accept
any responsibility.

 

RE: Hive CLI not starting

[Vishwakarma, Chhaya]

Hi Arpan,

I got beeline working, can you please suggest me how can I skip giving password?
I want to run “ beeline –f test.hql “ and every time it asks for password, what 
if I want to schedule this script ?

From: Vishwakarma, Chhaya [mailto:chhaya.vishwaka...@thinkbiganalytics.com]
Sent: 24 November 2015 18:07
To: user@hive.apache.org
Subject: RE: Hive CLI not starting

Yes it is Kerberized, how can I make sure ticket is not expired and TGT has not 
expired?

Hive CLI works from edge node, We are trying to have another machine with Hive 
client thats where its failing.

Any suggestion?

From: Arpan Rajani [mailto:arpan.raj...@whishworks.com]
Sent: 24 November 2015 17:29
To: user@hive.apache.org
Subject: Re: Hive CLI not starting

Chhaya,

Is your server Kerberized? For the kerberized cluster this may when your ticket 
is expired and your TGT has expired.

Try using beeline client instead.

Regards,
Arpan

On 24 November 2015 at 11:33, Vishwakarma, Chhaya 
mailto:chhaya.vishwaka...@thinkbiganalytics.com>>
 wrote:
Hi All,

I’m trying to run Hive client from Linux box(Separate from Hadoop cluster 
nodes) but when I start Hive CLI from command I get below error.
Can somebody please suggest ? I’m not getting anything relevant on internet.

hive
WARNING: Use "yarn jar" to launch YARN applications.

Logging initialized using configuration in 
file:/etc/hive/2.3.0.0-2525/0/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: 
org.apache.tez.dag.api.TezException: 
org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit 
application_1436575073312_0041 to YARN : Failed to renew token: Kind: 
TIMELINE_DELEGATION_TOKEN, Service: 
10.25.100.118:8188, Ident: (owner=hduser, 
renewer=yarn, realUser=, issueDate=1436827743392, maxDate=1437432543392, 
sequenceNumber=36, masterKeyId=16)
   at 
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:535)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.tez.dag.api.TezException: 
org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit 
application_1436575073312_0041 to YARN : Failed to renew token: Kind: 
TIMELINE_DELEGATION_TOKEN, Service: 
10.25.100.118:8188, Ident: (owner=hduser, 
renewer=yarn, realUser=, issueDate=1436827743392, maxDate=1437432543392, 
sequenceNumber=36, masterKeyId=16)
at org.apache.tez.client.TezClient.start(TezClient.java:370)
at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:184)
at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:116)
at 
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:532)
... 8 more
Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit 
application_1436575073312_0041 to YARN : Failed to renew token: Kind: 
TIMELINE_DELEGATION_TOKEN, Service: 
10.25.100.118:8188, Ident: (owner=hduser, 
renewer=yarn, realUser=, issueDate=1436827743392, maxDate=1437432543392, 
sequenceNumber=36, masterKeyId=16)
at 
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:271)
at 
org.apache.tez.client.TezYarnClient.submitApplication(TezYarnClient.java:72)
at org.apache.tez.client.TezClient.start(TezClient.java:365)


Regards,
Chhaya



--


[http://www.whishworks.com/images/whishworks/WWlogotm.png]


Arpan Rajani, Architect Big Data

1000, Great West Road, Brentford, London TW8 9DW

T:  +44 20 3475 7980
M: +44 7492 306685

W: www.whishworks.com

[http://www.whishworks.com/images/whishworks/icons/social/linkedin.png]
 [http://www.whishworks.com/images/whishworks/icons/social/blog.png] 
  
[http://www.whishworks.com/images/whishworks/icons/social/twitter.png] 
  
[http://www.whishworks.com/images/whishworks/icons/social/facebook.png] 



The contents of this e-mail are confidential and for the exclusive use of the 
intended recipient. If you receive this e-mail in error please delete it from 
your system immediately and notify us either by e-mail or telephone. You should 
not copy, forward or otherwise disclose the content of the 

HCatStorer error

[Raj hadoop]

We are facing below mentioned error on storing dataset using HCatStorer.Can
someone please help us



STORE F INTO 'default.CONTENT_SVC_USED' using
org.apache.hive.hcatalog.pig.HCatStorer();



ERROR hive.log - Got exception: java.net.URISyntaxException Malformed
escape pair at index 9: thrift://%HOSTGROUP::host_group_master1%:9933

java.net.URISyntaxException: Malformed escape pair at index 9:
thrift://%HOSTGROUP::host_group_master1%:9933


Thanks,

Raj

RE: Hive CLI not starting

[Vishwakarma, Chhaya]

Yes it is Kerberized, how can I make sure ticket is not expired and TGT has not 
expired?

Hive CLI works from edge node, We are trying to have another machine with Hive 
client thats where its failing.

Any suggestion?

From: Arpan Rajani [mailto:arpan.raj...@whishworks.com]
Sent: 24 November 2015 17:29
To: user@hive.apache.org
Subject: Re: Hive CLI not starting

Chhaya,

Is your server Kerberized? For the kerberized cluster this may when your ticket 
is expired and your TGT has expired.

Try using beeline client instead.

Regards,
Arpan

On 24 November 2015 at 11:33, Vishwakarma, Chhaya 
mailto:chhaya.vishwaka...@thinkbiganalytics.com>>
 wrote:
Hi All,

I’m trying to run Hive client from Linux box(Separate from Hadoop cluster 
nodes) but when I start Hive CLI from command I get below error.
Can somebody please suggest ? I’m not getting anything relevant on internet.

hive
WARNING: Use "yarn jar" to launch YARN applications.

Logging initialized using configuration in 
file:/etc/hive/2.3.0.0-2525/0/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: 
org.apache.tez.dag.api.TezException: 
org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit 
application_1436575073312_0041 to YARN : Failed to renew token: Kind: 
TIMELINE_DELEGATION_TOKEN, Service: 
10.25.100.118:8188, Ident: (owner=hduser, 
renewer=yarn, realUser=, issueDate=1436827743392, maxDate=1437432543392, 
sequenceNumber=36, masterKeyId=16)
   at 
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:535)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.tez.dag.api.TezException: 
org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit 
application_1436575073312_0041 to YARN : Failed to renew token: Kind: 
TIMELINE_DELEGATION_TOKEN, Service: 
10.25.100.118:8188, Ident: (owner=hduser, 
renewer=yarn, realUser=, issueDate=1436827743392, maxDate=1437432543392, 
sequenceNumber=36, masterKeyId=16)
at org.apache.tez.client.TezClient.start(TezClient.java:370)
at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:184)
at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:116)
at 
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:532)
... 8 more
Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit 
application_1436575073312_0041 to YARN : Failed to renew token: Kind: 
TIMELINE_DELEGATION_TOKEN, Service: 
10.25.100.118:8188, Ident: (owner=hduser, 
renewer=yarn, realUser=, issueDate=1436827743392, maxDate=1437432543392, 
sequenceNumber=36, masterKeyId=16)
at 
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:271)
at 
org.apache.tez.client.TezYarnClient.submitApplication(TezYarnClient.java:72)
at org.apache.tez.client.TezClient.start(TezClient.java:365)


Regards,
Chhaya



--


[http://www.whishworks.com/images/whishworks/WWlogotm.png]


Arpan Rajani, Architect Big Data

1000, Great West Road, Brentford, London TW8 9DW

T:  +44 20 3475 7980
M: +44 7492 306685

W: www.whishworks.com

[http://www.whishworks.com/images/whishworks/icons/social/linkedin.png]
 [http://www.whishworks.com/images/whishworks/icons/social/blog.png] 
  
[http://www.whishworks.com/images/whishworks/icons/social/twitter.png] 
  
[http://www.whishworks.com/images/whishworks/icons/social/facebook.png] 



The contents of this e-mail are confidential and for the exclusive use of the 
intended recipient. If you receive this e-mail in error please delete it from 
your system immediately and notify us either by e-mail or telephone. You should 
not copy, forward or otherwise disclose the content of the e-mail. The views 
expressed in this communication may not necessarily be the view held by 
WHISHWORKS.

Re: Unable to connect to beeline after kerberos is installed

[Kumar Jayapal]

Hello Arpan and Neeraj,

Thanks for the response.  I have checked the setting in hive and sentry on
my production servers they are exactly the same. It works find in Prod but
not in my test environment. Is there any thing else I am missing.

Thanks
Jay









Thanks
Jay

On Tue, Nov 24, 2015 at 3:24 AM, Arpan Rajani 
wrote:

> Hi Jay,
>
> I faced this and fixed yesterday.
> There are two things.
>
> 1)Before connecting to Hive server do you have a valid TGT?
> Do $ kinit u...@realm.com and get a valid TGT
>
> 2) in the jdbc connect string use the hive principle
> Use  "!connect
> jdbc:hive2://:/;principal=”.
>
>
> Please make sure it is the hive’s principal, not the user's.
>
> Restart the affected services. And it should work.
>
> Regards,
> Arpan
> On 24 Nov 2015 1:55 am, "Kumar Jayapal"  wrote:
>
>> Unable to connect to beeline after kerberos is installed I am getting
>> this error. Let me know if an one has resolution for it.
>>
>> *Error: Could not open client transport with JDBC Uri:
>> jdbc:hive2://hiven001.np.mmc.com:1/
>> : Peer indicated failure: Unsupported
>> mechanism type PLAIN (state=08S01,code=0)*
>>
>>
>> 2015-11-24 00:54:21,869 ERROR org.apache.thrift.server.TThreadPoolServer:
>> Error occurred during processing of message.
>> java.lang.RuntimeException:
>> org.apache.thrift.transport.TTransportException: Unsupported mechanism type
>> PLAIN
>> at
>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
>> at
>> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:739)
>> at
>> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:736)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:356)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1651)
>> at
>> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:736)
>> at
>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.thrift.transport.TTransportException: Unsupported
>> mechanism type PLAIN
>> at
>> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
>> at
>> org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:138)
>> at
>> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
>> at
>> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
>> at
>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
>> ... 10 more
>>
>>
>>
>>
>>
>>
>>
>> Thanks
>> Jay
>>
>
> The contents of this e-mail are confidential and for the exclusive use of
> the intended recipient. If you receive this e-mail in error please delete
> it from your system immediately and notify us either by e-mail or
> telephone. You should not copy, forward or otherwise disclose the content
> of the e-mail. The views expressed in this communication may not
> necessarily be the view held by WHISHWORKS.

ERROR util.SparkUncaughtExceptionHandler: Uncaught exception in thread Thread

[Dasun Hegoda]

Hi,

I get below error when I try to run Hive on Spark, Any idea how to fix this?


15/11/24 06:33:47 ERROR util.SparkUncaughtExceptionHandler: Uncaught
exception in thread Thread[appclient-registration-retry-thread,5,main]
java.util.concurrent.RejectedExecutionException: Task
java.util.concurrent.FutureTask@34d6330a rejected from
java.util.concurrent.ThreadPoolExecutor@5ab0f09f[Running, pool size = 1,
active threads = 1, queued tasks = 0, completed tasks = 0]
at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anonfun$tryRegisterAllMasters$1.apply(AppClient.scala:96)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anonfun$tryRegisterAllMasters$1.apply(AppClient.scala:95)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:108)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint.tryRegisterAllMasters(AppClient.scala:95)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint.org$apache$spark$deploy$client$AppClient$ClientEndpoint$$registerWithMaster(AppClient.scala:121)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2$$anonfun$run$1.apply$mcV$sp(AppClient.scala:132)
at org.apache.spark.util.Utils$.tryOrExit(Utils.scala:1119)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2.run(AppClient.scala:124)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
15/11/24 06:33:47 INFO storage.DiskBlockManager: Shutdown hook called
15/11/24 06:33:47 INFO util.ShutdownHookManager: Shutdown hook called
15/11/24 06:33:47 INFO util.ShutdownHookManager: Deleting directory
/tmp/spark-dabf9338-0423-4190-83ed-036e9e61c770
15/11/24 06:33:47 INFO util.ShutdownHookManager: Deleting directory
/tmp/spark-c97cfee4-8bff-4b7d-8af2-59e9fd56965

-- 
Regards,
Dasun Hegoda, Software Engineer
www.dasunhegoda.com | dasunheg...@gmail.com

Re: Hive CLI not starting

[Arpan Rajani]

Chhaya,

Is your server Kerberized? For the kerberized cluster this may when your
ticket is expired and your TGT has expired.

Try using beeline client instead.

Regards,
Arpan

On 24 November 2015 at 11:33, Vishwakarma, Chhaya <
chhaya.vishwaka...@thinkbiganalytics.com> wrote:

> Hi All,
>
>
>
> I’m trying to run Hive client from Linux box(Separate from Hadoop cluster
> nodes) but when I start Hive CLI from command I get below error.
>
> Can somebody please suggest ? I’m not getting anything relevant on
> internet.
>
>
>
> hive
>
> WARNING: Use "yarn jar" to launch YARN applications.
>
>
>
> Logging initialized using configuration in
> file:/etc/hive/2.3.0.0-2525/0/hive-log4j.properties
>
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.tez.dag.api.TezException:
> org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit
> application_1436575073312_0041 to YARN : Failed to renew token: Kind:
> TIMELINE_DELEGATION_TOKEN, Service: 10.25.100.118:8188, Ident:
> (owner=hduser, renewer=yarn, realUser=, issueDate=1436827743392,
> maxDate=1437432543392, sequenceNumber=36, masterKeyId=16)
>
>at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:535)
>
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
>
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:483)
>
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>
> Caused by: org.apache.tez.dag.api.TezException:
> org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit
> application_1436575073312_0041 to YARN : Failed to renew token: Kind:
> TIMELINE_DELEGATION_TOKEN, Service: 10.25.100.118:8188, Ident:
> (owner=hduser, renewer=yarn, realUser=, issueDate=1436827743392,
> maxDate=1437432543392, sequenceNumber=36, masterKeyId=16)
>
> at org.apache.tez.client.TezClient.start(TezClient.java:370)
>
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:184)
>
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:116)
>
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:532)
>
> ... 8 more
>
> Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to
> submit application_1436575073312_0041 to YARN : Failed to renew token:
> Kind: TIMELINE_DELEGATION_TOKEN, Service: 10.25.100.118:8188, Ident:
> (owner=hduser, renewer=yarn, realUser=, issueDate=1436827743392,
> maxDate=1437432543392, sequenceNumber=36, masterKeyId=16)
>
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:271)
>
> at
> org.apache.tez.client.TezYarnClient.submitApplication(TezYarnClient.java:72)
>
> at org.apache.tez.client.TezClient.start(TezClient.java:365)
>
>
>
>
>
> Regards,
>
> Chhaya
>



-- 

Arpan Rajani, Architect Big Data

1000, Great West Road, Brentford, London TW8 9DW

*T:  +44 20 3475 7980*
*M: **+44 *7492 306685

*W: *ww w.whishworks.com



  


-- 
The contents of this e-mail are confidential and for the exclusive use of 
the intended recipient. If you receive this e-mail in error please delete 
it from your system immediately and notify us either by e-mail or 
telephone. You should not copy, forward or otherwise disclose the content 
of the e-mail. The views expressed in this communication may not 
necessarily be the view held by WHISHWORKS.

Hive CLI not starting

[Vishwakarma, Chhaya]

Hi All,

I'm trying to run Hive client from Linux box(Separate from Hadoop cluster 
nodes) but when I start Hive CLI from command I get below error.
Can somebody please suggest ? I'm not getting anything relevant on internet.

hive
WARNING: Use "yarn jar" to launch YARN applications.

Logging initialized using configuration in 
file:/etc/hive/2.3.0.0-2525/0/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: 
org.apache.tez.dag.api.TezException: 
org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit 
application_1436575073312_0041 to YARN : Failed to renew token: Kind: 
TIMELINE_DELEGATION_TOKEN, Service: 10.25.100.118:8188, Ident: (owner=hduser, 
renewer=yarn, realUser=, issueDate=1436827743392, maxDate=1437432543392, 
sequenceNumber=36, masterKeyId=16)
   at 
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:535)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.tez.dag.api.TezException: 
org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit 
application_1436575073312_0041 to YARN : Failed to renew token: Kind: 
TIMELINE_DELEGATION_TOKEN, Service: 10.25.100.118:8188, Ident: (owner=hduser, 
renewer=yarn, realUser=, issueDate=1436827743392, maxDate=1437432543392, 
sequenceNumber=36, masterKeyId=16)
at org.apache.tez.client.TezClient.start(TezClient.java:370)
at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:184)
at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:116)
at 
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:532)
... 8 more
Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit 
application_1436575073312_0041 to YARN : Failed to renew token: Kind: 
TIMELINE_DELEGATION_TOKEN, Service: 10.25.100.118:8188, Ident: (owner=hduser, 
renewer=yarn, realUser=, issueDate=1436827743392, maxDate=1437432543392, 
sequenceNumber=36, masterKeyId=16)
at 
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:271)
at 
org.apache.tez.client.TezYarnClient.submitApplication(TezYarnClient.java:72)
at org.apache.tez.client.TezClient.start(TezClient.java:365)


Regards,
Chhaya

RE: starting spark-shell throws /tmp/hive on HDFS should be writable error

[Mich Talebzadeh]

Ok I solved this problem.

 

/tmp/hive is temporary directory. Only temporary files are kept in this
location. No problem even if we delete this directory, will be created when
required with proper permissions.

 

1)In hdfs I removed /tmp/hive directory

 

hdfs dfs -rm -r /tmp/hive

 

2)at OS level I did

 

rm -rf /tmp/hive

 

 

and then started spark-shell 

 

and it worked fine

 

 

HTH,

 

 

Mich Talebzadeh

 

Sybase ASE 15 Gold Medal Award 2008

A Winning Strategy: Running the most Critical Financial Data on ASE 15

 

http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.
pdf

Author of the books "A Practitioner's Guide to Upgrading to Sybase ASE 15",
ISBN 978-0-9563693-0-7. 

co-author "Sybase Transact SQL Guidelines Best Practices", ISBN
978-0-9759693-0-4

Publications due shortly:

Complex Event Processing in Heterogeneous Environments, ISBN:
978-0-9563693-3-8

Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume
one out shortly

 

  http://talebzadehmich.wordpress.com

 

NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Technology
Ltd, its subsidiaries or their employees, unless expressly so stated. It is
the responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Ltd, its subsidiaries nor their employees accept
any responsibility.

 

From: Mich Talebzadeh [mailto:m...@peridale.co.uk] 
Sent: 20 November 2015 21:14
To: user@hive.apache.org
Subject: starting spark-shell throws /tmp/hive on HDFS should be writable
error

 

Hi,

 

Has this been resolved. I don't think this has anything to do with /tmp/hive
directory permission

 

spark-shell

log4j:WARN No appenders could be found for logger
(org.apache.hadoop.metrics2.lib.MutableMetricsFactory).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.

Using Spark's repl log4j profile:
org/apache/spark/log4j-defaults-repl.properties

To adjust logging level use sc.setLogLevel("INFO")

Welcome to

    __

 / __/__  ___ _/ /__

_\ \/ _ \/ _ `/ __/  '_/

   /___/ .__/\_,_/_/ /_/\_\   version 1.5.2

  /_/

 

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java
1.7.0_25)

Type in expressions to have them evaluated.

Type :help for more information.

java.lang.RuntimeException: java.lang.RuntimeException: The root scratch
dir: /tmp/hive on HDFS should be writable. Current permissions are:
rwx--

 

 

:10: error: not found: value sqlContext

   import sqlContext.implicits._

  ^

:10: error: not found: value sqlContext

   import sqlContext.sql

  ^

 

scala>

 

Thanks,

 

 

Mich Talebzadeh

 

Sybase ASE 15 Gold Medal Award 2008

A Winning Strategy: Running the most Critical Financial Data on ASE 15

 

http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.
pdf

Author of the books "A Practitioner's Guide to Upgrading to Sybase ASE 15",
ISBN 978-0-9563693-0-7. 

co-author "Sybase Transact SQL Guidelines Best Practices", ISBN
978-0-9759693-0-4

Publications due shortly:

Complex Event Processing in Heterogeneous Environments, ISBN:
978-0-9563693-3-8

Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume
one out shortly

 

  http://talebzadehmich.wordpress.com

 

NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Technology
Ltd, its subsidiaries or their employees, unless expressly so stated. It is
the responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Ltd, its subsidiaries nor their employees accept
any responsibility.

 

Re: Unable to connect to beeline after kerberos is installed

[Arpan Rajani]

Hi Jay,

I faced this and fixed yesterday.
There are two things.

1)Before connecting to Hive server do you have a valid TGT?
Do $ kinit u...@realm.com and get a valid TGT

2) in the jdbc connect string use the hive principle
Use  "!connect
jdbc:hive2://:/;principal=”.


Please make sure it is the hive’s principal, not the user's.

Restart the affected services. And it should work.

Regards,
Arpan
On 24 Nov 2015 1:55 am, "Kumar Jayapal"  wrote:

> Unable to connect to beeline after kerberos is installed I am getting this
> error. Let me know if an one has resolution for it.
>
> *Error: Could not open client transport with JDBC Uri:
> jdbc:hive2://hiven001.np.mmc.com:1/
> : Peer indicated failure: Unsupported
> mechanism type PLAIN (state=08S01,code=0)*
>
>
> 2015-11-24 00:54:21,869 ERROR org.apache.thrift.server.TThreadPoolServer:
> Error occurred during processing of message.
> java.lang.RuntimeException:
> org.apache.thrift.transport.TTransportException: Unsupported mechanism type
> PLAIN
> at
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:739)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:736)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:356)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1651)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:736)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift.transport.TTransportException: Unsupported
> mechanism type PLAIN
> at
> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
> at
> org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:138)
> at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
> at
> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
> at
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
> ... 10 more
>
>
>
>
>
>
>
> Thanks
> Jay
>

-- 
The contents of this e-mail are confidential and for the exclusive use of 
the intended recipient. If you receive this e-mail in error please delete 
it from your system immediately and notify us either by e-mail or 
telephone. You should not copy, forward or otherwise disclose the content 
of the e-mail. The views expressed in this communication may not 
necessarily be the view held by WHISHWORKS.