Re: Cumulative sum with Hive

[Meri-Tuulia Rautavuori]

Hei,

Thank you very much! Cool!

I make it work like this:

SELECT r.day, r.a, SUM(r.a) OVER part_month AS cumulative_a
FROM
(
SELECT day, month, COUNT (DISTINCT (a))
FROM table
WHERE month='xx'
GROUP BY day, month
)r
WINDOW part_month as (PARTITION BY r.month order by r.day)

Greetings from Finland!
M-T

On Tue, Oct 14, 2014 at 4:48 PM, João Alves  wrote:

> Hey,
>
> You could try something like
>
> SELECT sum(a) OVER (PARTITION BY dummy ROWS UNBOUNDED PRECEDING) FROM
> (SELECT a, “dummy” AS dummy FROM table) AS aux;
>
> The smaller solution of :
>
> SELECT sum(A) OVER (PARTITION BY “dummy” ROWS UNBOUNDED PRECEDING) FROM
> table;
>
> For me gets stuck without presenting any results.
>
> You can explore more possibilities with analytical and windowing functions
> here:
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+WindowingAndAnalytics
>
>
> Best,
> João Alves
>
> On 14 Oct 2014, at 14:47, Meri-Tuulia Rautavuori <
> merituulia.rautavu...@songhi.com> wrote:
>
> Hi,
>
> Could you please help me how to calculate cumulative sum with Hive? Like
> RSUM in Oracle?
>
> Thanks.
>
> --
> Best Regards,
>
> *Meri-Tuulia Rautavuori*
> *Business Analyst*
>
> SongHi Entertainment Ltd
> *Särkiniementie 5 C 7*
> *00210 Helsinki*
> +358 45 271 2171
>
> *merituulia.rautavu...@songhi.com *
>
> http://www.songhi.com - Tune in and Play
>
> This transmission is confidential and intended solely for the person or
> organisation to whom it is addressed. It may contain privileged and
> confidential information. If you are not the intended recipient, you should
> not copy, distribute, reproduce, publish or disclose the information, take
> any action in reliance on it or use it for your own benefit. If you have
> received this transmission in error, please notify us immediately by E-mail 
> *merituulia.rautavu...@soghi.com
> *
>
> SongHi Entertainment Ltd has all publishing and copyrights 2013.
>
>
>


-- 
Best Regards,

*Meri-Tuulia Rautavuori*
*Business Analyst*

SongHi Entertainment Ltd
*Särkiniementie 5 C 7*
*00210 Helsinki*
+358 45 271 2171

*merituulia.rautavu...@songhi.com *

http://www.songhi.com - Tune in and Play

This transmission is confidential and intended solely for the person or
organisation to whom it is addressed. It may contain privileged and
confidential information. If you are not the intended recipient, you should
not copy, distribute, reproduce, publish or disclose the information, take
any action in reliance on it or use it for your own benefit. If you have
received this transmission in error, please notify us immediately by
E-mail *merituulia.rautavu...@soghi.com
*

SongHi Entertainment Ltd has all publishing and copyrights 2013.

Re: DDL wiki GRANT

[Lefty Leverenz]

Well done.  Thanks very much.

-- Lefty

On Tue, Oct 14, 2014 at 11:48 PM, Suhas Gogate  wrote:

> Done! Thx. That where Brett's question originated :)
>
> On Tue, Oct 14, 2014 at 8:45 PM, Lefty Leverenz 
> wrote:
>
>> Looks good, except that you forgot to change the GRANT syntax
>> 
>> -- it still says object_type.
>>
>> -- Lefty
>>
>> On Tue, Oct 14, 2014 at 11:41 PM, Suhas Gogate 
>> wrote:
>>
>>> Lefty/Brett,  I did change the wiki.. check if it looks okay.  --Suhas
>>>
>>> On Tue, Oct 14, 2014 at 7:34 PM, Lefty Leverenz >> > wrote:
>>>
 One question remains:  in object_specification, are the keywords TABLE
 and DATABASE optional?

 At least for TABLE I've seen queries in the test suite that omitted it,
 but that was probably for SQL standards based authorization.  So I guess we
 should assume TABLE and DATABASE are required unless someone says 
 otherwise.

 -- Lefty

 On Tue, Oct 14, 2014 at 4:48 PM, Lefty Leverenz <
 leftylever...@gmail.com> wrote:

> +1
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 4:37 PM, Brett Randall 
> wrote:
>
>> +1
>>
>> On 15 October 2014 07:23, Suhas Gogate  wrote:
>> > Agree w/ Brett.. so may be instead of "object_type", we can use
>> > "object_specification" (similar to principal specification)?
>> >
>> > GRANT
>> > priv_type [(column_list)]
>> >   [, priv_type [(column_list)]] ...
>> > [ON object_specification]
>> > TO principal_specification [, principal_specification] ...
>> > [WITH GRANT OPTION]
>> >
>> > REVOKE [GRANT OPTION FOR]
>> > priv_type [(column_list)]
>> >   [, priv_type [(column_list)]] ...
>> > [ON object_specification]
>> > FROM principal_specification [, principal_specification] ...
>> >
>> > REVOKE ALL PRIVILEGES, GRANT OPTION
>> > FROM user [, user] ...
>> >
>> > priv_type:
>> > ALL | ALTER | UPDATE | CREATE | DROP
>> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
>> >
>> > object_specification:
>> > TABLE tbl_name |
>> > DATABASE db_name
>> >
>> > principal_specification:
>> > USER user
>> >   | GROUP group
>> >   | ROLE role
>> >
>> >
>> > On Tue, Oct 14, 2014 at 11:06 AM, Lefty Leverenz <
>> leftylever...@gmail.com>
>> > wrote:
>> >>
>> >> I'll correct it as soon as we reach consensus.  (Perhaps Thejas
>> will chime
>> >> in.)
>> >>
>> >> If you want to do it yourself, you can get wiki edit privilege
>> quite
>> >> easily.
>> >>
>> >> -- Lefty
>> >>
>> >> On Tue, Oct 14, 2014 at 7:57 AM, Brett Randall <
>> javabr...@gmail.com>
>> >> wrote:
>> >>>
>> >>> I agree that the use of priv_level is confusing when it is
>> actually
>> >>> referring to object_name (of type TABLE or DATABASE).  I don't
>> mind
>> >>> the rolling-up of tbl_name or db_name into object_type, although
>> it
>> >>> then makes object_type: somewhat misleading.  "[ON object_type
>> >>> object_name]" reads well for me.
>> >>>
>> >>> Anything to correct the incorrect syntax on the wiki page (it is
>> not
>> >>> open for edits).
>> >>>
>> >>> Thanks
>> >>> Brett
>> >>>
>> >>> On 13 October 2014 18:18, Suhas Gogate 
>> wrote:
>> >>> > Hmm.. looking at the syntax priv_level does not seem to be a
>> keyword
>> >>> > but
>> >>> > rather actual name of a table or database.. so why it appears
>> like a
>> >>> > keyword
>> >>> > Also priv_level is confusing and rather clear syntax would
>> should look
>> >>> > like
>> >>> > below...
>> >>> >
>> >>> > Again answer to original question from Brett, yes GRANT syntax
>> should
>> >>> > be
>> >>> > similar to REVOKE but rather priv_level should be removed from
>> REVOKE
>> >>> > as
>> >>> > well.. :)
>> >>> >
>> >>> > GRANT
>> >>> > priv_type [(column_list)]
>> >>> >   [, priv_type [(column_list)]] ...
>> >>> > [ON object_type]
>> >>> > TO principal_specification [, principal_specification] ...
>> >>> > [WITH GRANT OPTION]
>> >>> >
>> >>> > REVOKE [GRANT OPTION FOR]
>> >>> > priv_type [(column_list)]
>> >>> >   [, priv_type [(column_list)]] ...
>> >>> > [ON object_type]
>> >>> > FROM principal_specification [, principal_specification] ...
>> >>> >
>> >>> > REVOKE ALL PRIVILEGES, GRANT OPTION
>> >>> > FROM user [, user] ...
>> >>> >
>> >>> > priv_type:
>> >>> > ALL | ALTER | UPDATE | CREATE | DROP
>> >>> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
>> >>> >
>> >>> > object_type:
>> >>> >

Re: DDL wiki GRANT

[Suhas Gogate]

Done! Thx. That where Brett's question originated :)

On Tue, Oct 14, 2014 at 8:45 PM, Lefty Leverenz 
wrote:

> Looks good, except that you forgot to change the GRANT syntax
> 
> -- it still says object_type.
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 11:41 PM, Suhas Gogate  wrote:
>
>> Lefty/Brett,  I did change the wiki.. check if it looks okay.  --Suhas
>>
>> On Tue, Oct 14, 2014 at 7:34 PM, Lefty Leverenz 
>> wrote:
>>
>>> One question remains:  in object_specification, are the keywords TABLE
>>> and DATABASE optional?
>>>
>>> At least for TABLE I've seen queries in the test suite that omitted it,
>>> but that was probably for SQL standards based authorization.  So I guess we
>>> should assume TABLE and DATABASE are required unless someone says otherwise.
>>>
>>> -- Lefty
>>>
>>> On Tue, Oct 14, 2014 at 4:48 PM, Lefty Leverenz >> > wrote:
>>>
 +1

 -- Lefty

 On Tue, Oct 14, 2014 at 4:37 PM, Brett Randall 
 wrote:

> +1
>
> On 15 October 2014 07:23, Suhas Gogate  wrote:
> > Agree w/ Brett.. so may be instead of "object_type", we can use
> > "object_specification" (similar to principal specification)?
> >
> > GRANT
> > priv_type [(column_list)]
> >   [, priv_type [(column_list)]] ...
> > [ON object_specification]
> > TO principal_specification [, principal_specification] ...
> > [WITH GRANT OPTION]
> >
> > REVOKE [GRANT OPTION FOR]
> > priv_type [(column_list)]
> >   [, priv_type [(column_list)]] ...
> > [ON object_specification]
> > FROM principal_specification [, principal_specification] ...
> >
> > REVOKE ALL PRIVILEGES, GRANT OPTION
> > FROM user [, user] ...
> >
> > priv_type:
> > ALL | ALTER | UPDATE | CREATE | DROP
> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
> >
> > object_specification:
> > TABLE tbl_name |
> > DATABASE db_name
> >
> > principal_specification:
> > USER user
> >   | GROUP group
> >   | ROLE role
> >
> >
> > On Tue, Oct 14, 2014 at 11:06 AM, Lefty Leverenz <
> leftylever...@gmail.com>
> > wrote:
> >>
> >> I'll correct it as soon as we reach consensus.  (Perhaps Thejas
> will chime
> >> in.)
> >>
> >> If you want to do it yourself, you can get wiki edit privilege quite
> >> easily.
> >>
> >> -- Lefty
> >>
> >> On Tue, Oct 14, 2014 at 7:57 AM, Brett Randall  >
> >> wrote:
> >>>
> >>> I agree that the use of priv_level is confusing when it is actually
> >>> referring to object_name (of type TABLE or DATABASE).  I don't mind
> >>> the rolling-up of tbl_name or db_name into object_type, although it
> >>> then makes object_type: somewhat misleading.  "[ON object_type
> >>> object_name]" reads well for me.
> >>>
> >>> Anything to correct the incorrect syntax on the wiki page (it is
> not
> >>> open for edits).
> >>>
> >>> Thanks
> >>> Brett
> >>>
> >>> On 13 October 2014 18:18, Suhas Gogate  wrote:
> >>> > Hmm.. looking at the syntax priv_level does not seem to be a
> keyword
> >>> > but
> >>> > rather actual name of a table or database.. so why it appears
> like a
> >>> > keyword
> >>> > Also priv_level is confusing and rather clear syntax would
> should look
> >>> > like
> >>> > below...
> >>> >
> >>> > Again answer to original question from Brett, yes GRANT syntax
> should
> >>> > be
> >>> > similar to REVOKE but rather priv_level should be removed from
> REVOKE
> >>> > as
> >>> > well.. :)
> >>> >
> >>> > GRANT
> >>> > priv_type [(column_list)]
> >>> >   [, priv_type [(column_list)]] ...
> >>> > [ON object_type]
> >>> > TO principal_specification [, principal_specification] ...
> >>> > [WITH GRANT OPTION]
> >>> >
> >>> > REVOKE [GRANT OPTION FOR]
> >>> > priv_type [(column_list)]
> >>> >   [, priv_type [(column_list)]] ...
> >>> > [ON object_type]
> >>> > FROM principal_specification [, principal_specification] ...
> >>> >
> >>> > REVOKE ALL PRIVILEGES, GRANT OPTION
> >>> > FROM user [, user] ...
> >>> >
> >>> > priv_type:
> >>> > ALL | ALTER | UPDATE | CREATE | DROP
> >>> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
> >>> >
> >>> > object_type:
> >>> > TABLE tbl_name
> >>> >   | DATABASE db_name
> >>> >
> >>> > principal_specification:
> >>> > USER user
> >>> >   | GROUP group
> >>> >   | ROLE role
> >>> >
> >>> >
> >>> > On Sat, Oct 11, 2014 at 7:55 PM, Lefty Leverenz
> >>> > 
> >>> > wrote:
> >>> >>
> >>> >> Good catch, Brett

Re: DDL wiki GRANT

[Lefty Leverenz]

Looks good, except that you forgot to change the GRANT syntax

-- it still says object_type.

-- Lefty

On Tue, Oct 14, 2014 at 11:41 PM, Suhas Gogate  wrote:

> Lefty/Brett,  I did change the wiki.. check if it looks okay.  --Suhas
>
> On Tue, Oct 14, 2014 at 7:34 PM, Lefty Leverenz 
> wrote:
>
>> One question remains:  in object_specification, are the keywords TABLE
>> and DATABASE optional?
>>
>> At least for TABLE I've seen queries in the test suite that omitted it,
>> but that was probably for SQL standards based authorization.  So I guess we
>> should assume TABLE and DATABASE are required unless someone says otherwise.
>>
>> -- Lefty
>>
>> On Tue, Oct 14, 2014 at 4:48 PM, Lefty Leverenz 
>> wrote:
>>
>>> +1
>>>
>>> -- Lefty
>>>
>>> On Tue, Oct 14, 2014 at 4:37 PM, Brett Randall 
>>> wrote:
>>>
 +1

 On 15 October 2014 07:23, Suhas Gogate  wrote:
 > Agree w/ Brett.. so may be instead of "object_type", we can use
 > "object_specification" (similar to principal specification)?
 >
 > GRANT
 > priv_type [(column_list)]
 >   [, priv_type [(column_list)]] ...
 > [ON object_specification]
 > TO principal_specification [, principal_specification] ...
 > [WITH GRANT OPTION]
 >
 > REVOKE [GRANT OPTION FOR]
 > priv_type [(column_list)]
 >   [, priv_type [(column_list)]] ...
 > [ON object_specification]
 > FROM principal_specification [, principal_specification] ...
 >
 > REVOKE ALL PRIVILEGES, GRANT OPTION
 > FROM user [, user] ...
 >
 > priv_type:
 > ALL | ALTER | UPDATE | CREATE | DROP
 >   | INDEX | LOCK | SELECT | SHOW_DATABASE
 >
 > object_specification:
 > TABLE tbl_name |
 > DATABASE db_name
 >
 > principal_specification:
 > USER user
 >   | GROUP group
 >   | ROLE role
 >
 >
 > On Tue, Oct 14, 2014 at 11:06 AM, Lefty Leverenz <
 leftylever...@gmail.com>
 > wrote:
 >>
 >> I'll correct it as soon as we reach consensus.  (Perhaps Thejas will
 chime
 >> in.)
 >>
 >> If you want to do it yourself, you can get wiki edit privilege quite
 >> easily.
 >>
 >> -- Lefty
 >>
 >> On Tue, Oct 14, 2014 at 7:57 AM, Brett Randall 
 >> wrote:
 >>>
 >>> I agree that the use of priv_level is confusing when it is actually
 >>> referring to object_name (of type TABLE or DATABASE).  I don't mind
 >>> the rolling-up of tbl_name or db_name into object_type, although it
 >>> then makes object_type: somewhat misleading.  "[ON object_type
 >>> object_name]" reads well for me.
 >>>
 >>> Anything to correct the incorrect syntax on the wiki page (it is not
 >>> open for edits).
 >>>
 >>> Thanks
 >>> Brett
 >>>
 >>> On 13 October 2014 18:18, Suhas Gogate  wrote:
 >>> > Hmm.. looking at the syntax priv_level does not seem to be a
 keyword
 >>> > but
 >>> > rather actual name of a table or database.. so why it appears
 like a
 >>> > keyword
 >>> > Also priv_level is confusing and rather clear syntax would should
 look
 >>> > like
 >>> > below...
 >>> >
 >>> > Again answer to original question from Brett, yes GRANT syntax
 should
 >>> > be
 >>> > similar to REVOKE but rather priv_level should be removed from
 REVOKE
 >>> > as
 >>> > well.. :)
 >>> >
 >>> > GRANT
 >>> > priv_type [(column_list)]
 >>> >   [, priv_type [(column_list)]] ...
 >>> > [ON object_type]
 >>> > TO principal_specification [, principal_specification] ...
 >>> > [WITH GRANT OPTION]
 >>> >
 >>> > REVOKE [GRANT OPTION FOR]
 >>> > priv_type [(column_list)]
 >>> >   [, priv_type [(column_list)]] ...
 >>> > [ON object_type]
 >>> > FROM principal_specification [, principal_specification] ...
 >>> >
 >>> > REVOKE ALL PRIVILEGES, GRANT OPTION
 >>> > FROM user [, user] ...
 >>> >
 >>> > priv_type:
 >>> > ALL | ALTER | UPDATE | CREATE | DROP
 >>> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
 >>> >
 >>> > object_type:
 >>> > TABLE tbl_name
 >>> >   | DATABASE db_name
 >>> >
 >>> > principal_specification:
 >>> > USER user
 >>> >   | GROUP group
 >>> >   | ROLE role
 >>> >
 >>> >
 >>> > On Sat, Oct 11, 2014 at 7:55 PM, Lefty Leverenz
 >>> > 
 >>> > wrote:
 >>> >>
 >>> >> Good catch, Brett.  Can we have confirmation from an expert?
 >>> >>
 >>> >> Also, is object_type optional?
 >>> >>
 >>> >> It isn't clear to me why priv_level isn't called object_name.
 >>> >>
 >>> >> -- Lefty
 >>> >>
 >>> >> On Thu, Oct 9, 2014 at 8:23 AM, Brett

Re: Need write access to Hive Wiki

[Suhas Gogate]

Lefty Thanks for the access!   --Suhas

On Tue, Oct 14, 2014 at 7:26 PM, Lefty Leverenz 
wrote:

> Oh, Brett -- you gave an email address but I only used "javabrett" --
> woohoo!  You've got edit privileges now.
>
> Whew, what a struggle.  Now who wants first dibs on fixing the GRANT
> syntax?
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 10:17 PM, Lefty Leverenz 
> wrote:
>
>> Okay, Suhas, you've got edit privileges.  Welcome to the wiki team!
>>
>> But it didn't work for Brett.
>>
>> Suspecting user error on my part, I entered javabrett despite the "No
>> matches found" message and got a bright red error message.  In exasperation
>> I entered vitthal_gogate expecting the same error, but it worked.  (Go
>> figure.)  So I'll try again with javabrett
>>
>> -- Lefty
>>
>> On Tue, Oct 14, 2014 at 9:43 PM, Brett Randall 
>> wrote:
>>
>>> Also feel free to attempt to grant wiki write access to my account,
>>> javabrett at gmail dot com .
>>>
>>> Regards
>>> Brett
>>>
>>> On 15 October 2014 12:13, Lefty Leverenz 
>>> wrote:
>>> > Still "No matches found" and one existing user has an underscore in
>>> his ID,
>>> > so that isn't the problem.
>>> >
>>> > Would someone else who can grant wiki edit privileges please try?
>>> >
>>> > -- Lefty
>>> >
>>> > On Tue, Oct 14, 2014 at 5:54 PM, Lefty Leverenz <
>>> leftylever...@gmail.com>
>>> > wrote:
>>> >>
>>> >> Still no joy.  I'll give it a few hours.  (Or someone else can try.)
>>> >>
>>> >> -- Lefty
>>> >>
>>> >> On Tue, Oct 14, 2014 at 5:12 PM, Lefty Leverenz <
>>> leftylever...@gmail.com>
>>> >> wrote:
>>> >>>
>>> >>> Hm, still "No matches found" (using cut-&-paste) -- I'll keep trying
>>> in
>>> >>> case it's a sync issue.  Thanks.
>>> >>>
>>> >>> -- Lefty
>>> >>>
>>> >>> On Tue, Oct 14, 2014 at 5:04 PM, Suhas Gogate 
>>> wrote:
>>> 
>>>  Thanks Lefty, I can actually login using this id to confluence wiki
>>>  (https://cwiki.apache.org/confluence/dashboard.action).  I created
>>> this id
>>>  just half hour back before sending over to this mailing list, so
>>> could be
>>>  sync issue?  It is underscore (_) between my first and last name.
>>> 
>>>  id:   vitthal_gogate
>>> 
>>>  Regards,
>>>  --Suhas (this is my call name, just to avoid confusion :)
>>> 
>>>  On Tue, Oct 14, 2014 at 1:51 PM, Lefty Leverenz
>>>   wrote:
>>> >
>>> > Confluence didn't recognize that user id.  Might it have a typo?
>>> >
>>> > -- Lefty
>>> >
>>> > On Tue, Oct 14, 2014 at 4:30 PM, Suhas Gogate 
>>> > wrote:
>>> >>
>>> >> My confluence user id is  vitthal_gogate
>>> >>
>>> >> Thanks
>>> >> --Suhas
>>> >
>>> >
>>> 
>>> >>>
>>> >>
>>> >
>>>
>>
>>
>

Re: DDL wiki GRANT

[Suhas Gogate]

Lefty/Brett,  I did change the wiki.. check if it looks okay.  --Suhas

On Tue, Oct 14, 2014 at 7:34 PM, Lefty Leverenz 
wrote:

> One question remains:  in object_specification, are the keywords TABLE and
> DATABASE optional?
>
> At least for TABLE I've seen queries in the test suite that omitted it,
> but that was probably for SQL standards based authorization.  So I guess we
> should assume TABLE and DATABASE are required unless someone says otherwise.
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 4:48 PM, Lefty Leverenz 
> wrote:
>
>> +1
>>
>> -- Lefty
>>
>> On Tue, Oct 14, 2014 at 4:37 PM, Brett Randall 
>> wrote:
>>
>>> +1
>>>
>>> On 15 October 2014 07:23, Suhas Gogate  wrote:
>>> > Agree w/ Brett.. so may be instead of "object_type", we can use
>>> > "object_specification" (similar to principal specification)?
>>> >
>>> > GRANT
>>> > priv_type [(column_list)]
>>> >   [, priv_type [(column_list)]] ...
>>> > [ON object_specification]
>>> > TO principal_specification [, principal_specification] ...
>>> > [WITH GRANT OPTION]
>>> >
>>> > REVOKE [GRANT OPTION FOR]
>>> > priv_type [(column_list)]
>>> >   [, priv_type [(column_list)]] ...
>>> > [ON object_specification]
>>> > FROM principal_specification [, principal_specification] ...
>>> >
>>> > REVOKE ALL PRIVILEGES, GRANT OPTION
>>> > FROM user [, user] ...
>>> >
>>> > priv_type:
>>> > ALL | ALTER | UPDATE | CREATE | DROP
>>> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
>>> >
>>> > object_specification:
>>> > TABLE tbl_name |
>>> > DATABASE db_name
>>> >
>>> > principal_specification:
>>> > USER user
>>> >   | GROUP group
>>> >   | ROLE role
>>> >
>>> >
>>> > On Tue, Oct 14, 2014 at 11:06 AM, Lefty Leverenz <
>>> leftylever...@gmail.com>
>>> > wrote:
>>> >>
>>> >> I'll correct it as soon as we reach consensus.  (Perhaps Thejas will
>>> chime
>>> >> in.)
>>> >>
>>> >> If you want to do it yourself, you can get wiki edit privilege quite
>>> >> easily.
>>> >>
>>> >> -- Lefty
>>> >>
>>> >> On Tue, Oct 14, 2014 at 7:57 AM, Brett Randall 
>>> >> wrote:
>>> >>>
>>> >>> I agree that the use of priv_level is confusing when it is actually
>>> >>> referring to object_name (of type TABLE or DATABASE).  I don't mind
>>> >>> the rolling-up of tbl_name or db_name into object_type, although it
>>> >>> then makes object_type: somewhat misleading.  "[ON object_type
>>> >>> object_name]" reads well for me.
>>> >>>
>>> >>> Anything to correct the incorrect syntax on the wiki page (it is not
>>> >>> open for edits).
>>> >>>
>>> >>> Thanks
>>> >>> Brett
>>> >>>
>>> >>> On 13 October 2014 18:18, Suhas Gogate  wrote:
>>> >>> > Hmm.. looking at the syntax priv_level does not seem to be a
>>> keyword
>>> >>> > but
>>> >>> > rather actual name of a table or database.. so why it appears like
>>> a
>>> >>> > keyword
>>> >>> > Also priv_level is confusing and rather clear syntax would should
>>> look
>>> >>> > like
>>> >>> > below...
>>> >>> >
>>> >>> > Again answer to original question from Brett, yes GRANT syntax
>>> should
>>> >>> > be
>>> >>> > similar to REVOKE but rather priv_level should be removed from
>>> REVOKE
>>> >>> > as
>>> >>> > well.. :)
>>> >>> >
>>> >>> > GRANT
>>> >>> > priv_type [(column_list)]
>>> >>> >   [, priv_type [(column_list)]] ...
>>> >>> > [ON object_type]
>>> >>> > TO principal_specification [, principal_specification] ...
>>> >>> > [WITH GRANT OPTION]
>>> >>> >
>>> >>> > REVOKE [GRANT OPTION FOR]
>>> >>> > priv_type [(column_list)]
>>> >>> >   [, priv_type [(column_list)]] ...
>>> >>> > [ON object_type]
>>> >>> > FROM principal_specification [, principal_specification] ...
>>> >>> >
>>> >>> > REVOKE ALL PRIVILEGES, GRANT OPTION
>>> >>> > FROM user [, user] ...
>>> >>> >
>>> >>> > priv_type:
>>> >>> > ALL | ALTER | UPDATE | CREATE | DROP
>>> >>> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
>>> >>> >
>>> >>> > object_type:
>>> >>> > TABLE tbl_name
>>> >>> >   | DATABASE db_name
>>> >>> >
>>> >>> > principal_specification:
>>> >>> > USER user
>>> >>> >   | GROUP group
>>> >>> >   | ROLE role
>>> >>> >
>>> >>> >
>>> >>> > On Sat, Oct 11, 2014 at 7:55 PM, Lefty Leverenz
>>> >>> > 
>>> >>> > wrote:
>>> >>> >>
>>> >>> >> Good catch, Brett.  Can we have confirmation from an expert?
>>> >>> >>
>>> >>> >> Also, is object_type optional?
>>> >>> >>
>>> >>> >> It isn't clear to me why priv_level isn't called object_name.
>>> >>> >>
>>> >>> >> -- Lefty
>>> >>> >>
>>> >>> >> On Thu, Oct 9, 2014 at 8:23 AM, Brett Randall <
>>> javabr...@gmail.com>
>>> >>> >> wrote:
>>> >>> >>>
>>> >>> >>> Hi,
>>> >>> >>>
>>> >>> >>> On
>>> >>> >>>
>>> >>> >>>
>>> https://cwiki.apache.org/confluence/display/Hive/Hive+Default+Authorization+-+Legacy+Mode#HiveDefaultAuthorization-LegacyMode-Grant/RevokePrivileges
>>> >>> >>> , GRANT shows as:
>>> >>> >>>
>>> >>> >>> GRANT
>>> >>> >>> priv_type [(column_list)]
>>> >>> >>>   [, priv_type [(column_list)]] ...
>>> >>> >>> [ON obje

Re: DDL wiki GRANT

[Lefty Leverenz]

One question remains:  in object_specification, are the keywords TABLE and
DATABASE optional?

At least for TABLE I've seen queries in the test suite that omitted it, but
that was probably for SQL standards based authorization.  So I guess we
should assume TABLE and DATABASE are required unless someone says otherwise.

-- Lefty

On Tue, Oct 14, 2014 at 4:48 PM, Lefty Leverenz 
wrote:

> +1
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 4:37 PM, Brett Randall 
> wrote:
>
>> +1
>>
>> On 15 October 2014 07:23, Suhas Gogate  wrote:
>> > Agree w/ Brett.. so may be instead of "object_type", we can use
>> > "object_specification" (similar to principal specification)?
>> >
>> > GRANT
>> > priv_type [(column_list)]
>> >   [, priv_type [(column_list)]] ...
>> > [ON object_specification]
>> > TO principal_specification [, principal_specification] ...
>> > [WITH GRANT OPTION]
>> >
>> > REVOKE [GRANT OPTION FOR]
>> > priv_type [(column_list)]
>> >   [, priv_type [(column_list)]] ...
>> > [ON object_specification]
>> > FROM principal_specification [, principal_specification] ...
>> >
>> > REVOKE ALL PRIVILEGES, GRANT OPTION
>> > FROM user [, user] ...
>> >
>> > priv_type:
>> > ALL | ALTER | UPDATE | CREATE | DROP
>> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
>> >
>> > object_specification:
>> > TABLE tbl_name |
>> > DATABASE db_name
>> >
>> > principal_specification:
>> > USER user
>> >   | GROUP group
>> >   | ROLE role
>> >
>> >
>> > On Tue, Oct 14, 2014 at 11:06 AM, Lefty Leverenz <
>> leftylever...@gmail.com>
>> > wrote:
>> >>
>> >> I'll correct it as soon as we reach consensus.  (Perhaps Thejas will
>> chime
>> >> in.)
>> >>
>> >> If you want to do it yourself, you can get wiki edit privilege quite
>> >> easily.
>> >>
>> >> -- Lefty
>> >>
>> >> On Tue, Oct 14, 2014 at 7:57 AM, Brett Randall 
>> >> wrote:
>> >>>
>> >>> I agree that the use of priv_level is confusing when it is actually
>> >>> referring to object_name (of type TABLE or DATABASE).  I don't mind
>> >>> the rolling-up of tbl_name or db_name into object_type, although it
>> >>> then makes object_type: somewhat misleading.  "[ON object_type
>> >>> object_name]" reads well for me.
>> >>>
>> >>> Anything to correct the incorrect syntax on the wiki page (it is not
>> >>> open for edits).
>> >>>
>> >>> Thanks
>> >>> Brett
>> >>>
>> >>> On 13 October 2014 18:18, Suhas Gogate  wrote:
>> >>> > Hmm.. looking at the syntax priv_level does not seem to be a keyword
>> >>> > but
>> >>> > rather actual name of a table or database.. so why it appears like a
>> >>> > keyword
>> >>> > Also priv_level is confusing and rather clear syntax would should
>> look
>> >>> > like
>> >>> > below...
>> >>> >
>> >>> > Again answer to original question from Brett, yes GRANT syntax
>> should
>> >>> > be
>> >>> > similar to REVOKE but rather priv_level should be removed from
>> REVOKE
>> >>> > as
>> >>> > well.. :)
>> >>> >
>> >>> > GRANT
>> >>> > priv_type [(column_list)]
>> >>> >   [, priv_type [(column_list)]] ...
>> >>> > [ON object_type]
>> >>> > TO principal_specification [, principal_specification] ...
>> >>> > [WITH GRANT OPTION]
>> >>> >
>> >>> > REVOKE [GRANT OPTION FOR]
>> >>> > priv_type [(column_list)]
>> >>> >   [, priv_type [(column_list)]] ...
>> >>> > [ON object_type]
>> >>> > FROM principal_specification [, principal_specification] ...
>> >>> >
>> >>> > REVOKE ALL PRIVILEGES, GRANT OPTION
>> >>> > FROM user [, user] ...
>> >>> >
>> >>> > priv_type:
>> >>> > ALL | ALTER | UPDATE | CREATE | DROP
>> >>> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
>> >>> >
>> >>> > object_type:
>> >>> > TABLE tbl_name
>> >>> >   | DATABASE db_name
>> >>> >
>> >>> > principal_specification:
>> >>> > USER user
>> >>> >   | GROUP group
>> >>> >   | ROLE role
>> >>> >
>> >>> >
>> >>> > On Sat, Oct 11, 2014 at 7:55 PM, Lefty Leverenz
>> >>> > 
>> >>> > wrote:
>> >>> >>
>> >>> >> Good catch, Brett.  Can we have confirmation from an expert?
>> >>> >>
>> >>> >> Also, is object_type optional?
>> >>> >>
>> >>> >> It isn't clear to me why priv_level isn't called object_name.
>> >>> >>
>> >>> >> -- Lefty
>> >>> >>
>> >>> >> On Thu, Oct 9, 2014 at 8:23 AM, Brett Randall > >
>> >>> >> wrote:
>> >>> >>>
>> >>> >>> Hi,
>> >>> >>>
>> >>> >>> On
>> >>> >>>
>> >>> >>>
>> https://cwiki.apache.org/confluence/display/Hive/Hive+Default+Authorization+-+Legacy+Mode#HiveDefaultAuthorization-LegacyMode-Grant/RevokePrivileges
>> >>> >>> , GRANT shows as:
>> >>> >>>
>> >>> >>> GRANT
>> >>> >>> priv_type [(column_list)]
>> >>> >>>   [, priv_type [(column_list)]] ...
>> >>> >>> [ON object_type]
>> >>> >>> TO principal_specification [, principal_specification] ...
>> >>> >>> [WITH GRANT OPTION]
>> >>> >>>
>> >>> >>> Should that not be [ON object_type priv_level], same as REVOKE,
>> >>> >>> where:
>> >>> >>>
>> >>> >>> object_type:
>> >>> >>> TABLE
>> >>> >>>   | DATABASE
>> >>> >>>
>> >>> >>> pr

Re: Need write access to Hive Wiki

[Lefty Leverenz]

Oh, Brett -- you gave an email address but I only used "javabrett" --
woohoo!  You've got edit privileges now.

Whew, what a struggle.  Now who wants first dibs on fixing the GRANT syntax?

-- Lefty

On Tue, Oct 14, 2014 at 10:17 PM, Lefty Leverenz 
wrote:

> Okay, Suhas, you've got edit privileges.  Welcome to the wiki team!
>
> But it didn't work for Brett.
>
> Suspecting user error on my part, I entered javabrett despite the "No
> matches found" message and got a bright red error message.  In exasperation
> I entered vitthal_gogate expecting the same error, but it worked.  (Go
> figure.)  So I'll try again with javabrett
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 9:43 PM, Brett Randall 
> wrote:
>
>> Also feel free to attempt to grant wiki write access to my account,
>> javabrett at gmail dot com .
>>
>> Regards
>> Brett
>>
>> On 15 October 2014 12:13, Lefty Leverenz  wrote:
>> > Still "No matches found" and one existing user has an underscore in his
>> ID,
>> > so that isn't the problem.
>> >
>> > Would someone else who can grant wiki edit privileges please try?
>> >
>> > -- Lefty
>> >
>> > On Tue, Oct 14, 2014 at 5:54 PM, Lefty Leverenz <
>> leftylever...@gmail.com>
>> > wrote:
>> >>
>> >> Still no joy.  I'll give it a few hours.  (Or someone else can try.)
>> >>
>> >> -- Lefty
>> >>
>> >> On Tue, Oct 14, 2014 at 5:12 PM, Lefty Leverenz <
>> leftylever...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Hm, still "No matches found" (using cut-&-paste) -- I'll keep trying
>> in
>> >>> case it's a sync issue.  Thanks.
>> >>>
>> >>> -- Lefty
>> >>>
>> >>> On Tue, Oct 14, 2014 at 5:04 PM, Suhas Gogate 
>> wrote:
>> 
>>  Thanks Lefty, I can actually login using this id to confluence wiki
>>  (https://cwiki.apache.org/confluence/dashboard.action).  I created
>> this id
>>  just half hour back before sending over to this mailing list, so
>> could be
>>  sync issue?  It is underscore (_) between my first and last name.
>> 
>>  id:   vitthal_gogate
>> 
>>  Regards,
>>  --Suhas (this is my call name, just to avoid confusion :)
>> 
>>  On Tue, Oct 14, 2014 at 1:51 PM, Lefty Leverenz
>>   wrote:
>> >
>> > Confluence didn't recognize that user id.  Might it have a typo?
>> >
>> > -- Lefty
>> >
>> > On Tue, Oct 14, 2014 at 4:30 PM, Suhas Gogate 
>> > wrote:
>> >>
>> >> My confluence user id is  vitthal_gogate
>> >>
>> >> Thanks
>> >> --Suhas
>> >
>> >
>> 
>> >>>
>> >>
>> >
>>
>
>

Re: Need write access to Hive Wiki

[Lefty Leverenz]

Okay, Suhas, you've got edit privileges.  Welcome to the wiki team!

But it didn't work for Brett.

Suspecting user error on my part, I entered javabrett despite the "No
matches found" message and got a bright red error message.  In exasperation
I entered vitthal_gogate expecting the same error, but it worked.  (Go
figure.)  So I'll try again with javabrett

-- Lefty

On Tue, Oct 14, 2014 at 9:43 PM, Brett Randall  wrote:

> Also feel free to attempt to grant wiki write access to my account,
> javabrett at gmail dot com .
>
> Regards
> Brett
>
> On 15 October 2014 12:13, Lefty Leverenz  wrote:
> > Still "No matches found" and one existing user has an underscore in his
> ID,
> > so that isn't the problem.
> >
> > Would someone else who can grant wiki edit privileges please try?
> >
> > -- Lefty
> >
> > On Tue, Oct 14, 2014 at 5:54 PM, Lefty Leverenz  >
> > wrote:
> >>
> >> Still no joy.  I'll give it a few hours.  (Or someone else can try.)
> >>
> >> -- Lefty
> >>
> >> On Tue, Oct 14, 2014 at 5:12 PM, Lefty Leverenz <
> leftylever...@gmail.com>
> >> wrote:
> >>>
> >>> Hm, still "No matches found" (using cut-&-paste) -- I'll keep trying in
> >>> case it's a sync issue.  Thanks.
> >>>
> >>> -- Lefty
> >>>
> >>> On Tue, Oct 14, 2014 at 5:04 PM, Suhas Gogate 
> wrote:
> 
>  Thanks Lefty, I can actually login using this id to confluence wiki
>  (https://cwiki.apache.org/confluence/dashboard.action).  I created
> this id
>  just half hour back before sending over to this mailing list, so
> could be
>  sync issue?  It is underscore (_) between my first and last name.
> 
>  id:   vitthal_gogate
> 
>  Regards,
>  --Suhas (this is my call name, just to avoid confusion :)
> 
>  On Tue, Oct 14, 2014 at 1:51 PM, Lefty Leverenz
>   wrote:
> >
> > Confluence didn't recognize that user id.  Might it have a typo?
> >
> > -- Lefty
> >
> > On Tue, Oct 14, 2014 at 4:30 PM, Suhas Gogate 
> > wrote:
> >>
> >> My confluence user id is  vitthal_gogate
> >>
> >> Thanks
> >> --Suhas
> >
> >
> 
> >>>
> >>
> >
>

Re: Need write access to Hive Wiki

[Brett Randall]

Also feel free to attempt to grant wiki write access to my account,
javabrett at gmail dot com .

Regards
Brett

On 15 October 2014 12:13, Lefty Leverenz  wrote:
> Still "No matches found" and one existing user has an underscore in his ID,
> so that isn't the problem.
>
> Would someone else who can grant wiki edit privileges please try?
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 5:54 PM, Lefty Leverenz 
> wrote:
>>
>> Still no joy.  I'll give it a few hours.  (Or someone else can try.)
>>
>> -- Lefty
>>
>> On Tue, Oct 14, 2014 at 5:12 PM, Lefty Leverenz 
>> wrote:
>>>
>>> Hm, still "No matches found" (using cut-&-paste) -- I'll keep trying in
>>> case it's a sync issue.  Thanks.
>>>
>>> -- Lefty
>>>
>>> On Tue, Oct 14, 2014 at 5:04 PM, Suhas Gogate  wrote:

 Thanks Lefty, I can actually login using this id to confluence wiki
 (https://cwiki.apache.org/confluence/dashboard.action).  I created this id
 just half hour back before sending over to this mailing list, so could be
 sync issue?  It is underscore (_) between my first and last name.

 id:   vitthal_gogate

 Regards,
 --Suhas (this is my call name, just to avoid confusion :)

 On Tue, Oct 14, 2014 at 1:51 PM, Lefty Leverenz
  wrote:
>
> Confluence didn't recognize that user id.  Might it have a typo?
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 4:30 PM, Suhas Gogate 
> wrote:
>>
>> My confluence user id is  vitthal_gogate
>>
>> Thanks
>> --Suhas
>
>

>>>
>>
>

error while running a hive view

[Vidya Sujeet]

Hello,

I am getting the following error while running a view. However, the hive
script that is used in the creation of hive view runs without error. It's
only when I select * from hive_view I am getting the error saying

*Caused by: java.lang.RuntimeException: Cannot serialize object*

Could anyone please help with this issue? The underlying base tables are
bucketed..but not partitioned.


Thanks,

Re: Need write access to Hive Wiki

[Lefty Leverenz]

Still "No matches found" and one existing user has an underscore in his ID,
so that isn't the problem.

Would someone else who can grant wiki edit privileges please try?

-- Lefty

On Tue, Oct 14, 2014 at 5:54 PM, Lefty Leverenz 
wrote:

> Still no joy.  I'll give it a few hours.  (Or someone else can try.)
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 5:12 PM, Lefty Leverenz 
> wrote:
>
>> Hm, still "No matches found" (using cut-&-paste) -- I'll keep trying in
>> case it's a sync issue.  Thanks.
>>
>> -- Lefty
>>
>> On Tue, Oct 14, 2014 at 5:04 PM, Suhas Gogate  wrote:
>>
>>> Thanks Lefty, I can actually login using this id to confluence wiki (
>>> https://cwiki.apache.org/confluence/dashboard.action).  I created this
>>> id just half hour back before sending over to this mailing list, so could
>>> be sync issue?  It is underscore (_) between my first and last name.
>>>
>>> id:   vitthal_gogate
>>>
>>> Regards,
>>> --Suhas (this is my call name, just to avoid confusion :)
>>>
>>> On Tue, Oct 14, 2014 at 1:51 PM, Lefty Leverenz >> > wrote:
>>>
 Confluence didn't recognize that user id.  Might it have a typo?

 -- Lefty

 On Tue, Oct 14, 2014 at 4:30 PM, Suhas Gogate 
 wrote:

> My confluence user id is  vitthal_gogate
>
> Thanks
> --Suhas
>


>>>
>>
>

Re: ALTER TABLE T1 PARTITION(P1) CONCATENATE bug?

[Navis류승우]

Could you tell the version number of hive?

Thanks,
Navis

2014-10-15 2:00 GMT+09:00 Time Less :

> I have found a work-around for this bug. After you issue the ALTER
> TABLE...CONCATENATE command, issue:
>
> ALTER TABLE T1 PARTITION (P1) SET LOCATION
> ".../apps/hive/warehouse/DB1/T1/P1";
>
> This will fix the metadata that CONCATENATE breaks.
>
>
> ––
> *Tim Ellis:* 510-761-6610
>
>
> On Mon, Oct 13, 2014 at 10:37 PM, Time Less 
> wrote:
>
>> Has anyone seen anything like this? Google searches turned up nothing, so
>> I thought I'd ask here, then file a JIRA if no-one thinks I'm doing it
>> wrong.
>>
>> If I ALTER a particular table with three partitions once, it works.
>> Second time it works, too, but reports it is moving a directory to the
>> Trash that doesn't exist (still, this doesn't kill it). The third time I
>> ALTER the table, it crashes, because the directory structure has been
>> modified to something invalid.
>>
>> Here's a nearly-full output of the 2nd and 3rd runs. The ALTER is exactly
>> the same both times (I just press UP ARROW):
>>
>>
>> *HQL, 2nd Run:*hive (analytics)> alter table bidtmp partition
>> (log_type='bidder',dt='2014-05-01',hour=11) concatenate ;
>>
>>
>> *Output:*Starting Job = job_1412894367814_0017, Tracking URL =
>> application_1412894367814_0017/
>> Kill Command = /usr/lib/hadoop/bin/hadoop job  -kill
>> job_1412894367814_0017
>> Hadoop job information for null: number of mappers: 97; number of
>> reducers: 0
>> 2014-10-13 20:28:23,143 null map = 0%,  reduce = 0%
>> 2014-10-13 20:28:36,042 null map = 1%,  reduce = 0%, Cumulative CPU 49.69
>> sec
>> ...
>> 2014-10-13 20:31:56,415 null map = 99%,  reduce = 0%, Cumulative CPU
>> 812.65 sec
>> 2014-10-13 20:31:57,458 null map = 100%,  reduce = 0%, Cumulative CPU
>> 813.88 sec
>> MapReduce Total cumulative CPU time: 13 minutes 33 seconds 880 msec
>> Ended Job = job_1412894367814_0017
>> Loading data to table analytics.bidtmp partition (log_type=bidder,
>> dt=2014-05-01, hour=11)
>> rmr: DEPRECATED: Please use 'rm -r' instead.
>> Moved: '.../apps/hive/warehouse/analytics.db/bidtmp/
>> *dt=2014-05-01/hour=11/log_type=bidder*' to trash at:
>> .../user/hdfs/.Trash/Current
>> *// (note the bold-faced path doesn't exist, the partition is specified
>> as log_type first, then dt, then hour)*
>> Partition analytics.bidtmp*{log_type=bidder, dt=2014-05-01, hour=11}*
>> stats: [numFiles=0, numRows=0, totalSize=0, rawDataSize=0]
>> *(here, the partition ordering is correct!)*
>> MapReduce Jobs Launched:
>> Job 0: Map: 97   Cumulative CPU: 813.88 sec   HDFS Read: 30298871932 HDFS
>> Write: 28746848923 SUCCESS
>> Total MapReduce CPU Time Spent: 13 minutes 33 seconds 880 msec
>> OK
>> Time taken: 224.128 seconds
>>
>>
>> *HQL, 3rd Run:*hive (analytics)> alter table bidtmp partition
>> (log_type='bidder',dt='2014-05-01',hour=11) concatenate ;
>>
>>
>> *Output:*java.io.FileNotFoundException: File does not exist:
>> .../apps/hive/warehouse/analytics.db/bidtmp/dt=2014-05-01/hour=11/log_type=bidder
>> *(because it should be log_type=.../dt=.../hour=... - not this order)*
>> at org.apache.hadoop.hdfs.
>> DistributedFileSystem$17.doCall(DistributedFileSystem.java:1128)
>> at
>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
>> at
>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>> at
>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1120)
>> at
>> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:419)
>> at
>> org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:520)
>> at
>> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:512)
>> at
>> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
>> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
>> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
>> at
>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
>> at
>> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
>> at
>> org.apache.hadoop.hive.ql.io.rcfile.merge.BlockMergeTask.execu

Re: Java api for connecting to hiveserver2

[Lefty Leverenz]

That's an older version of the HCatalog javadocs.  You can find current
javadocs here:  http://hive.apache.org/javadocs/r0.13.1/api/index.html.

-- Lefty

On Sun, Oct 12, 2014 at 3:05 PM, Suhas Gogate  wrote:

> Edward/Hanish, this is a reasonable use case to justify Java API. I
> initially thought need for Java API to avoid knowing SQL :) but that you
> can't for any DML operations :)  So I guess for DDL operations you should
> transition to HCatalog API.. See the design document where first para
> explicitly advises it..
>
>
> https://cwiki.apache.org/confluence/display/HCATALOG/Design+Document+-+Java+APIs+for+HCatalog+DDL+Commands
>
> I have not played as much with HCatalog APIs yet and quick look tells me
> that it is not at par in terms of all DDL/auth operations yet!
>
> https://hive.apache.org/javadocs/hcat-r0.5.0/api/index.html
> (org.apache.hcatalog.api.HCatClient).
>
> --Suhas
>
>
> On Sat, Oct 11, 2014 at 2:52 PM, Hanish Bansal <
> hanish.bansal.agar...@gmail.com> wrote:
>
>> Thanks all for response !!
>>
>> Totally agreed with Edward.
>>
>> Also the reason why we don't want to use jdbc client is: Writing sql
>> statement for everything is little bit complex rather than calling direct
>> java apis.
>> With java metastore client api it was quite easy.
>>
>> Regards,
>> Hanish
>>  On 12/10/2014 2:25 am, "Edward Capriolo"  wrote:
>>
>>> Actually I do not love the jdbc api for all things.
>>>
>>> I used to have several scripts that looped over all database, all
>>> tables, and looked at specific meta store properties or locations. These
>>> scripts were very small and to the point sometimes 5 line loops. Now I have
>>> to write things like this:
>>>
>>> ResultSet r = Statement.executeQuery("describe exteneded "+table)
>>> while (r.next){
>>> if (r.get(1).getString().equals("location")...
>>> ...
>>>
>>> It is more clunky then using the metastore api directly:
>>>
>>> hiveClient.getTable("database", "table").getSD().getLocation();
>>>
>>>
>>>
>>> On Sat, Oct 11, 2014 at 4:29 PM, Vaibhav Gumashta <
>>> vgumas...@hortonworks.com> wrote:
>>>
 Hanish,

 I agree with Suhas and would strongly encourage you to use the JDBC API
 for HiveServer2. HiveServer2 has a thrift api for the client-server RPC,
 but that is *not* intended for end user consumption and could end up
 breaking your current code in future.

 Is there any specific feature you are looking at which Hive's JDBC
 driver doesn't implement?

 Thanks,
 --Vaibhav

 On Sat, Oct 11, 2014 at 12:43 PM, Suhas Gogate 
 wrote:

> Sorry Anish, but being database programmer in the past, I always used
> embedded SQL interface... I was wondering should we really need direct 
> Java
> interface w/ HiveServer2? May be I am wrong, but would like to know your
> view on what are the limitations of using embedded SQL vs direct Java API.
>
> --Suhas
>
> On Sat, Oct 11, 2014 at 12:38 PM, Suhas Gogate 
> wrote:
>
>> Hanish, this is interesting question and I also faced similar
>> limitation lately. Although as Hive getting more closer to relational 
>> model
>> with richer SQL interface (DDL/Authorization, DML) and HiveServer2 as a 
>> way
>> to invoke embedded SQL in Java, the real question is should Hive 
>> Metastore
>> Client (java) API should at all be used by user, rather all the existing
>> Hive Client interfaces be talking to Hive Metastore internally?
>>
>> --Suhas
>>
>>
>>
>> On Fri, Oct 10, 2014 at 9:21 PM, Hanish Bansal <
>> hanish.bansal.agar...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> I am planning to use sql based authorization that is recently
>>> introduced in hive 0.13.0.
>>>
>>> I was using hive metastore client java api early which has direct
>>> apis for all operations like grant, revoke etc.
>>> But for using new authorization I ll have to use hiveserver2 and
>>> pass all requests through hiveserver2. I came up with JDBC client which 
>>> can
>>> be used to connect hiveserver2. The restriction there is we must have to
>>> write SQL statements.
>>>
>>> I want to know is there any java api to connect hiveserver2, that
>>> have direct java methods to perform operations and we don't need to 
>>> write
>>> SQL statements ??
>>>
>>> Thanks,
>>> Hanish
>>>
>>
>>
>

 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or
 entity to which it is addressed and may contain information that is
 confidential, privileged and exempt from disclosure under applicable law.
 If the reader of this message is not the intended recipient, you are hereby
 notified that any printing, copying, dissemination, distribution,
 disclosure or forwarding of this communication is strictly prohibited. If
 you have received this communication in e

Re: Need write access to Hive Wiki

[Lefty Leverenz]

Still no joy.  I'll give it a few hours.  (Or someone else can try.)

-- Lefty

On Tue, Oct 14, 2014 at 5:12 PM, Lefty Leverenz 
wrote:

> Hm, still "No matches found" (using cut-&-paste) -- I'll keep trying in
> case it's a sync issue.  Thanks.
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 5:04 PM, Suhas Gogate  wrote:
>
>> Thanks Lefty, I can actually login using this id to confluence wiki (
>> https://cwiki.apache.org/confluence/dashboard.action).  I created this
>> id just half hour back before sending over to this mailing list, so could
>> be sync issue?  It is underscore (_) between my first and last name.
>>
>> id:   vitthal_gogate
>>
>> Regards,
>> --Suhas (this is my call name, just to avoid confusion :)
>>
>> On Tue, Oct 14, 2014 at 1:51 PM, Lefty Leverenz 
>> wrote:
>>
>>> Confluence didn't recognize that user id.  Might it have a typo?
>>>
>>> -- Lefty
>>>
>>> On Tue, Oct 14, 2014 at 4:30 PM, Suhas Gogate 
>>> wrote:
>>>
 My confluence user id is  vitthal_gogate

 Thanks
 --Suhas

>>>
>>>
>>
>

Re: subscribe

[Venkata D]

How can I stop th JIRA emails? Should I send an email to
u...@apache.hive.org or develo...@apache.hive.org


On Sat, Oct 11, 2014 at 8:39 PM, Lefty Leverenz 
wrote:

> Arthur, there's a separate email address for subscribing to
> u...@apache.hive.org -- see http://hive.apache.org/mailing_lists.html.
>
> -- Lefty
>
> On Sat, Oct 11, 2014 at 6:47 PM, arthur.hk.c...@gmail.com <
> arthur.hk.c...@gmail.com> wrote:
>
>> subscribe
>>
>
>

Re: Need write access to Hive Wiki

[Lefty Leverenz]

Hm, still "No matches found" (using cut-&-paste) -- I'll keep trying in
case it's a sync issue.  Thanks.

-- Lefty

On Tue, Oct 14, 2014 at 5:04 PM, Suhas Gogate  wrote:

> Thanks Lefty, I can actually login using this id to confluence wiki (
> https://cwiki.apache.org/confluence/dashboard.action).  I created this id
> just half hour back before sending over to this mailing list, so could be
> sync issue?  It is underscore (_) between my first and last name.
>
> id:   vitthal_gogate
>
> Regards,
> --Suhas (this is my call name, just to avoid confusion :)
>
> On Tue, Oct 14, 2014 at 1:51 PM, Lefty Leverenz 
> wrote:
>
>> Confluence didn't recognize that user id.  Might it have a typo?
>>
>> -- Lefty
>>
>> On Tue, Oct 14, 2014 at 4:30 PM, Suhas Gogate  wrote:
>>
>>> My confluence user id is  vitthal_gogate
>>>
>>> Thanks
>>> --Suhas
>>>
>>
>>
>

Re: Need write access to Hive Wiki

[Suhas Gogate]

Thanks Lefty, I can actually login using this id to confluence wiki (
https://cwiki.apache.org/confluence/dashboard.action).  I created this id
just half hour back before sending over to this mailing list, so could be
sync issue?  It is underscore (_) between my first and last name.

id:   vitthal_gogate

Regards,
--Suhas (this is my call name, just to avoid confusion :)

On Tue, Oct 14, 2014 at 1:51 PM, Lefty Leverenz 
wrote:

> Confluence didn't recognize that user id.  Might it have a typo?
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 4:30 PM, Suhas Gogate  wrote:
>
>> My confluence user id is  vitthal_gogate
>>
>> Thanks
>> --Suhas
>>
>
>

Re: Need write access to Hive Wiki

[Lefty Leverenz]

Confluence didn't recognize that user id.  Might it have a typo?

-- Lefty

On Tue, Oct 14, 2014 at 4:30 PM, Suhas Gogate  wrote:

> My confluence user id is  vitthal_gogate
>
> Thanks
> --Suhas
>

Re: DDL wiki GRANT

[Lefty Leverenz]

+1

-- Lefty

On Tue, Oct 14, 2014 at 4:37 PM, Brett Randall  wrote:

> +1
>
> On 15 October 2014 07:23, Suhas Gogate  wrote:
> > Agree w/ Brett.. so may be instead of "object_type", we can use
> > "object_specification" (similar to principal specification)?
> >
> > GRANT
> > priv_type [(column_list)]
> >   [, priv_type [(column_list)]] ...
> > [ON object_specification]
> > TO principal_specification [, principal_specification] ...
> > [WITH GRANT OPTION]
> >
> > REVOKE [GRANT OPTION FOR]
> > priv_type [(column_list)]
> >   [, priv_type [(column_list)]] ...
> > [ON object_specification]
> > FROM principal_specification [, principal_specification] ...
> >
> > REVOKE ALL PRIVILEGES, GRANT OPTION
> > FROM user [, user] ...
> >
> > priv_type:
> > ALL | ALTER | UPDATE | CREATE | DROP
> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
> >
> > object_specification:
> > TABLE tbl_name |
> > DATABASE db_name
> >
> > principal_specification:
> > USER user
> >   | GROUP group
> >   | ROLE role
> >
> >
> > On Tue, Oct 14, 2014 at 11:06 AM, Lefty Leverenz <
> leftylever...@gmail.com>
> > wrote:
> >>
> >> I'll correct it as soon as we reach consensus.  (Perhaps Thejas will
> chime
> >> in.)
> >>
> >> If you want to do it yourself, you can get wiki edit privilege quite
> >> easily.
> >>
> >> -- Lefty
> >>
> >> On Tue, Oct 14, 2014 at 7:57 AM, Brett Randall 
> >> wrote:
> >>>
> >>> I agree that the use of priv_level is confusing when it is actually
> >>> referring to object_name (of type TABLE or DATABASE).  I don't mind
> >>> the rolling-up of tbl_name or db_name into object_type, although it
> >>> then makes object_type: somewhat misleading.  "[ON object_type
> >>> object_name]" reads well for me.
> >>>
> >>> Anything to correct the incorrect syntax on the wiki page (it is not
> >>> open for edits).
> >>>
> >>> Thanks
> >>> Brett
> >>>
> >>> On 13 October 2014 18:18, Suhas Gogate  wrote:
> >>> > Hmm.. looking at the syntax priv_level does not seem to be a keyword
> >>> > but
> >>> > rather actual name of a table or database.. so why it appears like a
> >>> > keyword
> >>> > Also priv_level is confusing and rather clear syntax would should
> look
> >>> > like
> >>> > below...
> >>> >
> >>> > Again answer to original question from Brett, yes GRANT syntax should
> >>> > be
> >>> > similar to REVOKE but rather priv_level should be removed from REVOKE
> >>> > as
> >>> > well.. :)
> >>> >
> >>> > GRANT
> >>> > priv_type [(column_list)]
> >>> >   [, priv_type [(column_list)]] ...
> >>> > [ON object_type]
> >>> > TO principal_specification [, principal_specification] ...
> >>> > [WITH GRANT OPTION]
> >>> >
> >>> > REVOKE [GRANT OPTION FOR]
> >>> > priv_type [(column_list)]
> >>> >   [, priv_type [(column_list)]] ...
> >>> > [ON object_type]
> >>> > FROM principal_specification [, principal_specification] ...
> >>> >
> >>> > REVOKE ALL PRIVILEGES, GRANT OPTION
> >>> > FROM user [, user] ...
> >>> >
> >>> > priv_type:
> >>> > ALL | ALTER | UPDATE | CREATE | DROP
> >>> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
> >>> >
> >>> > object_type:
> >>> > TABLE tbl_name
> >>> >   | DATABASE db_name
> >>> >
> >>> > principal_specification:
> >>> > USER user
> >>> >   | GROUP group
> >>> >   | ROLE role
> >>> >
> >>> >
> >>> > On Sat, Oct 11, 2014 at 7:55 PM, Lefty Leverenz
> >>> > 
> >>> > wrote:
> >>> >>
> >>> >> Good catch, Brett.  Can we have confirmation from an expert?
> >>> >>
> >>> >> Also, is object_type optional?
> >>> >>
> >>> >> It isn't clear to me why priv_level isn't called object_name.
> >>> >>
> >>> >> -- Lefty
> >>> >>
> >>> >> On Thu, Oct 9, 2014 at 8:23 AM, Brett Randall 
> >>> >> wrote:
> >>> >>>
> >>> >>> Hi,
> >>> >>>
> >>> >>> On
> >>> >>>
> >>> >>>
> https://cwiki.apache.org/confluence/display/Hive/Hive+Default+Authorization+-+Legacy+Mode#HiveDefaultAuthorization-LegacyMode-Grant/RevokePrivileges
> >>> >>> , GRANT shows as:
> >>> >>>
> >>> >>> GRANT
> >>> >>> priv_type [(column_list)]
> >>> >>>   [, priv_type [(column_list)]] ...
> >>> >>> [ON object_type]
> >>> >>> TO principal_specification [, principal_specification] ...
> >>> >>> [WITH GRANT OPTION]
> >>> >>>
> >>> >>> Should that not be [ON object_type priv_level], same as REVOKE,
> >>> >>> where:
> >>> >>>
> >>> >>> object_type:
> >>> >>> TABLE
> >>> >>>   | DATABASE
> >>> >>>
> >>> >>> priv_level:
> >>> >>> db_name
> >>> >>>   | tbl_name
> >>> >>>
> >>> >>> Thanks
> >>> >>> Brett
> >>> >>
> >>> >>
> >>> >
> >>
> >>
> >
>

Re: DDL wiki GRANT

[Brett Randall]

+1

On 15 October 2014 07:23, Suhas Gogate  wrote:
> Agree w/ Brett.. so may be instead of "object_type", we can use
> "object_specification" (similar to principal specification)?
>
> GRANT
> priv_type [(column_list)]
>   [, priv_type [(column_list)]] ...
> [ON object_specification]
> TO principal_specification [, principal_specification] ...
> [WITH GRANT OPTION]
>
> REVOKE [GRANT OPTION FOR]
> priv_type [(column_list)]
>   [, priv_type [(column_list)]] ...
> [ON object_specification]
> FROM principal_specification [, principal_specification] ...
>
> REVOKE ALL PRIVILEGES, GRANT OPTION
> FROM user [, user] ...
>
> priv_type:
> ALL | ALTER | UPDATE | CREATE | DROP
>   | INDEX | LOCK | SELECT | SHOW_DATABASE
>
> object_specification:
> TABLE tbl_name |
> DATABASE db_name
>
> principal_specification:
> USER user
>   | GROUP group
>   | ROLE role
>
>
> On Tue, Oct 14, 2014 at 11:06 AM, Lefty Leverenz 
> wrote:
>>
>> I'll correct it as soon as we reach consensus.  (Perhaps Thejas will chime
>> in.)
>>
>> If you want to do it yourself, you can get wiki edit privilege quite
>> easily.
>>
>> -- Lefty
>>
>> On Tue, Oct 14, 2014 at 7:57 AM, Brett Randall 
>> wrote:
>>>
>>> I agree that the use of priv_level is confusing when it is actually
>>> referring to object_name (of type TABLE or DATABASE).  I don't mind
>>> the rolling-up of tbl_name or db_name into object_type, although it
>>> then makes object_type: somewhat misleading.  "[ON object_type
>>> object_name]" reads well for me.
>>>
>>> Anything to correct the incorrect syntax on the wiki page (it is not
>>> open for edits).
>>>
>>> Thanks
>>> Brett
>>>
>>> On 13 October 2014 18:18, Suhas Gogate  wrote:
>>> > Hmm.. looking at the syntax priv_level does not seem to be a keyword
>>> > but
>>> > rather actual name of a table or database.. so why it appears like a
>>> > keyword
>>> > Also priv_level is confusing and rather clear syntax would should look
>>> > like
>>> > below...
>>> >
>>> > Again answer to original question from Brett, yes GRANT syntax should
>>> > be
>>> > similar to REVOKE but rather priv_level should be removed from REVOKE
>>> > as
>>> > well.. :)
>>> >
>>> > GRANT
>>> > priv_type [(column_list)]
>>> >   [, priv_type [(column_list)]] ...
>>> > [ON object_type]
>>> > TO principal_specification [, principal_specification] ...
>>> > [WITH GRANT OPTION]
>>> >
>>> > REVOKE [GRANT OPTION FOR]
>>> > priv_type [(column_list)]
>>> >   [, priv_type [(column_list)]] ...
>>> > [ON object_type]
>>> > FROM principal_specification [, principal_specification] ...
>>> >
>>> > REVOKE ALL PRIVILEGES, GRANT OPTION
>>> > FROM user [, user] ...
>>> >
>>> > priv_type:
>>> > ALL | ALTER | UPDATE | CREATE | DROP
>>> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
>>> >
>>> > object_type:
>>> > TABLE tbl_name
>>> >   | DATABASE db_name
>>> >
>>> > principal_specification:
>>> > USER user
>>> >   | GROUP group
>>> >   | ROLE role
>>> >
>>> >
>>> > On Sat, Oct 11, 2014 at 7:55 PM, Lefty Leverenz
>>> > 
>>> > wrote:
>>> >>
>>> >> Good catch, Brett.  Can we have confirmation from an expert?
>>> >>
>>> >> Also, is object_type optional?
>>> >>
>>> >> It isn't clear to me why priv_level isn't called object_name.
>>> >>
>>> >> -- Lefty
>>> >>
>>> >> On Thu, Oct 9, 2014 at 8:23 AM, Brett Randall 
>>> >> wrote:
>>> >>>
>>> >>> Hi,
>>> >>>
>>> >>> On
>>> >>>
>>> >>> https://cwiki.apache.org/confluence/display/Hive/Hive+Default+Authorization+-+Legacy+Mode#HiveDefaultAuthorization-LegacyMode-Grant/RevokePrivileges
>>> >>> , GRANT shows as:
>>> >>>
>>> >>> GRANT
>>> >>> priv_type [(column_list)]
>>> >>>   [, priv_type [(column_list)]] ...
>>> >>> [ON object_type]
>>> >>> TO principal_specification [, principal_specification] ...
>>> >>> [WITH GRANT OPTION]
>>> >>>
>>> >>> Should that not be [ON object_type priv_level], same as REVOKE,
>>> >>> where:
>>> >>>
>>> >>> object_type:
>>> >>> TABLE
>>> >>>   | DATABASE
>>> >>>
>>> >>> priv_level:
>>> >>> db_name
>>> >>>   | tbl_name
>>> >>>
>>> >>> Thanks
>>> >>> Brett
>>> >>
>>> >>
>>> >
>>
>>
>

Need write access to Hive Wiki

[Suhas Gogate]

My confluence user id is  vitthal_gogate

Thanks
--Suhas

Re: DDL wiki GRANT

[Suhas Gogate]

Agree w/ Brett.. so may be instead of "object_type", we can use
"object_specification" (similar to principal specification)?

GRANT
priv_type [(column_list)]
  [, priv_type [(column_list)]] ...
[ON object_specification]
TO principal_specification [, principal_specification] ...
[WITH GRANT OPTION]

REVOKE [GRANT OPTION FOR]
priv_type [(column_list)]
  [, priv_type [(column_list)]] ...
[ON object_specification]
FROM principal_specification [, principal_specification] ...

REVOKE ALL PRIVILEGES, GRANT OPTION
FROM user [, user] ...

priv_type:
ALL | ALTER | UPDATE | CREATE | DROP
  | INDEX | LOCK | SELECT | SHOW_DATABASE

object_specification:
TABLE tbl_name |
DATABASE db_name

principal_specification:
USER user
  | GROUP group
  | ROLE role


On Tue, Oct 14, 2014 at 11:06 AM, Lefty Leverenz 
wrote:

> I'll correct it as soon as we reach consensus.  (Perhaps Thejas will chime
> in.)
>
> If you want to do it yourself, you can get wiki edit privilege
> 
>  quite
> easily.
>
> -- Lefty
>
> On Tue, Oct 14, 2014 at 7:57 AM, Brett Randall 
> wrote:
>
>> I agree that the use of priv_level is confusing when it is actually
>> referring to object_name (of type TABLE or DATABASE).  I don't mind
>> the rolling-up of tbl_name or db_name into object_type, although it
>> then makes object_type: somewhat misleading.  "[ON object_type
>> object_name]" reads well for me.
>>
>> Anything to correct the incorrect syntax on the wiki page (it is not
>> open for edits).
>>
>> Thanks
>> Brett
>>
>> On 13 October 2014 18:18, Suhas Gogate  wrote:
>> > Hmm.. looking at the syntax priv_level does not seem to be a keyword but
>> > rather actual name of a table or database.. so why it appears like a
>> keyword
>> > Also priv_level is confusing and rather clear syntax would should look
>> like
>> > below...
>> >
>> > Again answer to original question from Brett, yes GRANT syntax should be
>> > similar to REVOKE but rather priv_level should be removed from REVOKE as
>> > well.. :)
>> >
>> > GRANT
>> > priv_type [(column_list)]
>> >   [, priv_type [(column_list)]] ...
>> > [ON object_type]
>> > TO principal_specification [, principal_specification] ...
>> > [WITH GRANT OPTION]
>> >
>> > REVOKE [GRANT OPTION FOR]
>> > priv_type [(column_list)]
>> >   [, priv_type [(column_list)]] ...
>> > [ON object_type]
>> > FROM principal_specification [, principal_specification] ...
>> >
>> > REVOKE ALL PRIVILEGES, GRANT OPTION
>> > FROM user [, user] ...
>> >
>> > priv_type:
>> > ALL | ALTER | UPDATE | CREATE | DROP
>> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
>> >
>> > object_type:
>> > TABLE tbl_name
>> >   | DATABASE db_name
>> >
>> > principal_specification:
>> > USER user
>> >   | GROUP group
>> >   | ROLE role
>> >
>> >
>> > On Sat, Oct 11, 2014 at 7:55 PM, Lefty Leverenz <
>> leftylever...@gmail.com>
>> > wrote:
>> >>
>> >> Good catch, Brett.  Can we have confirmation from an expert?
>> >>
>> >> Also, is object_type optional?
>> >>
>> >> It isn't clear to me why priv_level isn't called object_name.
>> >>
>> >> -- Lefty
>> >>
>> >> On Thu, Oct 9, 2014 at 8:23 AM, Brett Randall 
>> wrote:
>> >>>
>> >>> Hi,
>> >>>
>> >>> On
>> >>>
>> https://cwiki.apache.org/confluence/display/Hive/Hive+Default+Authorization+-+Legacy+Mode#HiveDefaultAuthorization-LegacyMode-Grant/RevokePrivileges
>> >>> , GRANT shows as:
>> >>>
>> >>> GRANT
>> >>> priv_type [(column_list)]
>> >>>   [, priv_type [(column_list)]] ...
>> >>> [ON object_type]
>> >>> TO principal_specification [, principal_specification] ...
>> >>> [WITH GRANT OPTION]
>> >>>
>> >>> Should that not be [ON object_type priv_level], same as REVOKE, where:
>> >>>
>> >>> object_type:
>> >>> TABLE
>> >>>   | DATABASE
>> >>>
>> >>> priv_level:
>> >>> db_name
>> >>>   | tbl_name
>> >>>
>> >>> Thanks
>> >>> Brett
>> >>
>> >>
>> >
>>
>
>

Re: DDL wiki GRANT

[Lefty Leverenz]

I'll correct it as soon as we reach consensus.  (Perhaps Thejas will chime
in.)

If you want to do it yourself, you can get wiki edit privilege

quite
easily.

-- Lefty

On Tue, Oct 14, 2014 at 7:57 AM, Brett Randall  wrote:

> I agree that the use of priv_level is confusing when it is actually
> referring to object_name (of type TABLE or DATABASE).  I don't mind
> the rolling-up of tbl_name or db_name into object_type, although it
> then makes object_type: somewhat misleading.  "[ON object_type
> object_name]" reads well for me.
>
> Anything to correct the incorrect syntax on the wiki page (it is not
> open for edits).
>
> Thanks
> Brett
>
> On 13 October 2014 18:18, Suhas Gogate  wrote:
> > Hmm.. looking at the syntax priv_level does not seem to be a keyword but
> > rather actual name of a table or database.. so why it appears like a
> keyword
> > Also priv_level is confusing and rather clear syntax would should look
> like
> > below...
> >
> > Again answer to original question from Brett, yes GRANT syntax should be
> > similar to REVOKE but rather priv_level should be removed from REVOKE as
> > well.. :)
> >
> > GRANT
> > priv_type [(column_list)]
> >   [, priv_type [(column_list)]] ...
> > [ON object_type]
> > TO principal_specification [, principal_specification] ...
> > [WITH GRANT OPTION]
> >
> > REVOKE [GRANT OPTION FOR]
> > priv_type [(column_list)]
> >   [, priv_type [(column_list)]] ...
> > [ON object_type]
> > FROM principal_specification [, principal_specification] ...
> >
> > REVOKE ALL PRIVILEGES, GRANT OPTION
> > FROM user [, user] ...
> >
> > priv_type:
> > ALL | ALTER | UPDATE | CREATE | DROP
> >   | INDEX | LOCK | SELECT | SHOW_DATABASE
> >
> > object_type:
> > TABLE tbl_name
> >   | DATABASE db_name
> >
> > principal_specification:
> > USER user
> >   | GROUP group
> >   | ROLE role
> >
> >
> > On Sat, Oct 11, 2014 at 7:55 PM, Lefty Leverenz  >
> > wrote:
> >>
> >> Good catch, Brett.  Can we have confirmation from an expert?
> >>
> >> Also, is object_type optional?
> >>
> >> It isn't clear to me why priv_level isn't called object_name.
> >>
> >> -- Lefty
> >>
> >> On Thu, Oct 9, 2014 at 8:23 AM, Brett Randall 
> wrote:
> >>>
> >>> Hi,
> >>>
> >>> On
> >>>
> https://cwiki.apache.org/confluence/display/Hive/Hive+Default+Authorization+-+Legacy+Mode#HiveDefaultAuthorization-LegacyMode-Grant/RevokePrivileges
> >>> , GRANT shows as:
> >>>
> >>> GRANT
> >>> priv_type [(column_list)]
> >>>   [, priv_type [(column_list)]] ...
> >>> [ON object_type]
> >>> TO principal_specification [, principal_specification] ...
> >>> [WITH GRANT OPTION]
> >>>
> >>> Should that not be [ON object_type priv_level], same as REVOKE, where:
> >>>
> >>> object_type:
> >>> TABLE
> >>>   | DATABASE
> >>>
> >>> priv_level:
> >>> db_name
> >>>   | tbl_name
> >>>
> >>> Thanks
> >>> Brett
> >>
> >>
> >
>

Re: ALTER TABLE T1 PARTITION(P1) CONCATENATE bug?

[Time Less]

I have found a work-around for this bug. After you issue the ALTER
TABLE...CONCATENATE command, issue:

ALTER TABLE T1 PARTITION (P1) SET LOCATION
".../apps/hive/warehouse/DB1/T1/P1";

This will fix the metadata that CONCATENATE breaks.


––
*Tim Ellis:* 510-761-6610


On Mon, Oct 13, 2014 at 10:37 PM, Time Less  wrote:

> Has anyone seen anything like this? Google searches turned up nothing, so
> I thought I'd ask here, then file a JIRA if no-one thinks I'm doing it
> wrong.
>
> If I ALTER a particular table with three partitions once, it works. Second
> time it works, too, but reports it is moving a directory to the Trash that
> doesn't exist (still, this doesn't kill it). The third time I ALTER the
> table, it crashes, because the directory structure has been modified to
> something invalid.
>
> Here's a nearly-full output of the 2nd and 3rd runs. The ALTER is exactly
> the same both times (I just press UP ARROW):
>
>
> *HQL, 2nd Run:*hive (analytics)> alter table bidtmp partition
> (log_type='bidder',dt='2014-05-01',hour=11) concatenate ;
>
>
> *Output:*Starting Job = job_1412894367814_0017, Tracking URL =
> application_1412894367814_0017/
> Kill Command = /usr/lib/hadoop/bin/hadoop job  -kill job_1412894367814_0017
> Hadoop job information for null: number of mappers: 97; number of
> reducers: 0
> 2014-10-13 20:28:23,143 null map = 0%,  reduce = 0%
> 2014-10-13 20:28:36,042 null map = 1%,  reduce = 0%, Cumulative CPU 49.69
> sec
> ...
> 2014-10-13 20:31:56,415 null map = 99%,  reduce = 0%, Cumulative CPU
> 812.65 sec
> 2014-10-13 20:31:57,458 null map = 100%,  reduce = 0%, Cumulative CPU
> 813.88 sec
> MapReduce Total cumulative CPU time: 13 minutes 33 seconds 880 msec
> Ended Job = job_1412894367814_0017
> Loading data to table analytics.bidtmp partition (log_type=bidder,
> dt=2014-05-01, hour=11)
> rmr: DEPRECATED: Please use 'rm -r' instead.
> Moved: '.../apps/hive/warehouse/analytics.db/bidtmp/
> *dt=2014-05-01/hour=11/log_type=bidder*' to trash at:
> .../user/hdfs/.Trash/Current
> *// (note the bold-faced path doesn't exist, the partition is specified as
> log_type first, then dt, then hour)*
> Partition analytics.bidtmp*{log_type=bidder, dt=2014-05-01, hour=11}*
> stats: [numFiles=0, numRows=0, totalSize=0, rawDataSize=0]
> *(here, the partition ordering is correct!)*
> MapReduce Jobs Launched:
> Job 0: Map: 97   Cumulative CPU: 813.88 sec   HDFS Read: 30298871932 HDFS
> Write: 28746848923 SUCCESS
> Total MapReduce CPU Time Spent: 13 minutes 33 seconds 880 msec
> OK
> Time taken: 224.128 seconds
>
>
> *HQL, 3rd Run:*hive (analytics)> alter table bidtmp partition
> (log_type='bidder',dt='2014-05-01',hour=11) concatenate ;
>
>
> *Output:*java.io.FileNotFoundException: File does not exist:
> .../apps/hive/warehouse/analytics.db/bidtmp/dt=2014-05-01/hour=11/log_type=bidder
> *(because it should be log_type=.../dt=.../hour=... - not this order)*
> at org.apache.hadoop.hdfs.
> DistributedFileSystem$17.doCall(DistributedFileSystem.java:1128)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1120)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:419)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:520)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:512)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
> at
> org.apache.hadoop.hive.ql.io.rcfile.merge.BlockMergeTask.execute(BlockMergeTask.java:214)
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.mergeFiles(DDLTask.java:511)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:458)
> at org.apache.hadoop.hi

Re: Cumulative sum with Hive

[João Alves]

Hey,

You could try something like

SELECT sum(a) OVER (PARTITION BY dummy ROWS UNBOUNDED PRECEDING) FROM (SELECT 
a, “dummy” AS dummy FROM table) AS aux;

The smaller solution of :

SELECT sum(A) OVER (PARTITION BY “dummy” ROWS UNBOUNDED PRECEDING) FROM table;

For me gets stuck without presenting any results.

You can explore more possibilities with analytical and windowing functions 
here: 
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+WindowingAndAnalytics


Best,
João Alves

On 14 Oct 2014, at 14:47, Meri-Tuulia Rautavuori 
 wrote:

> Hi,
> 
> Could you please help me how to calculate cumulative sum with Hive? Like RSUM 
> in Oracle?
> 
> Thanks.
> 
> -- 
> Best Regards,
> 
> Meri-Tuulia Rautavuori
> Business Analyst
> 
> SongHi Entertainment Ltd
> Särkiniementie 5 C 7
> 00210 Helsinki
> +358 45 271 2171
> 
> merituulia.rautavu...@songhi.com
> 
> http://www.songhi.com - Tune in and Play
> 
> This transmission is confidential and intended solely for the person or 
> organisation to whom it is addressed. It may contain privileged and 
> confidential information. If you are not the intended recipient, you should 
> not copy, distribute, reproduce, publish or disclose the information, take 
> any action in reliance on it or use it for your own benefit. If you have 
> received this transmission in error, please notify us immediately by E-mail 
> merituulia.rautavu...@soghi.com
> 
> SongHi Entertainment Ltd has all publishing and copyrights 2013.

Cumulative sum with Hive

[Meri-Tuulia Rautavuori]

Hi,

Could you please help me how to calculate cumulative sum with Hive? Like
RSUM in Oracle?

Thanks.

-- 
Best Regards,

*Meri-Tuulia Rautavuori*
*Business Analyst*

SongHi Entertainment Ltd
*Särkiniementie 5 C 7*
*00210 Helsinki*
+358 45 271 2171

*merituulia.rautavu...@songhi.com *

http://www.songhi.com - Tune in and Play

This transmission is confidential and intended solely for the person or
organisation to whom it is addressed. It may contain privileged and
confidential information. If you are not the intended recipient, you should
not copy, distribute, reproduce, publish or disclose the information, take
any action in reliance on it or use it for your own benefit. If you have
received this transmission in error, please notify us immediately by
E-mail *merituulia.rautavu...@soghi.com
*

SongHi Entertainment Ltd has all publishing and copyrights 2013.

Re: DDL wiki GRANT

[Brett Randall]

I agree that the use of priv_level is confusing when it is actually
referring to object_name (of type TABLE or DATABASE).  I don't mind
the rolling-up of tbl_name or db_name into object_type, although it
then makes object_type: somewhat misleading.  "[ON object_type
object_name]" reads well for me.

Anything to correct the incorrect syntax on the wiki page (it is not
open for edits).

Thanks
Brett

On 13 October 2014 18:18, Suhas Gogate  wrote:
> Hmm.. looking at the syntax priv_level does not seem to be a keyword but
> rather actual name of a table or database.. so why it appears like a keyword
> Also priv_level is confusing and rather clear syntax would should look like
> below...
>
> Again answer to original question from Brett, yes GRANT syntax should be
> similar to REVOKE but rather priv_level should be removed from REVOKE as
> well.. :)
>
> GRANT
> priv_type [(column_list)]
>   [, priv_type [(column_list)]] ...
> [ON object_type]
> TO principal_specification [, principal_specification] ...
> [WITH GRANT OPTION]
>
> REVOKE [GRANT OPTION FOR]
> priv_type [(column_list)]
>   [, priv_type [(column_list)]] ...
> [ON object_type]
> FROM principal_specification [, principal_specification] ...
>
> REVOKE ALL PRIVILEGES, GRANT OPTION
> FROM user [, user] ...
>
> priv_type:
> ALL | ALTER | UPDATE | CREATE | DROP
>   | INDEX | LOCK | SELECT | SHOW_DATABASE
>
> object_type:
> TABLE tbl_name
>   | DATABASE db_name
>
> principal_specification:
> USER user
>   | GROUP group
>   | ROLE role
>
>
> On Sat, Oct 11, 2014 at 7:55 PM, Lefty Leverenz 
> wrote:
>>
>> Good catch, Brett.  Can we have confirmation from an expert?
>>
>> Also, is object_type optional?
>>
>> It isn't clear to me why priv_level isn't called object_name.
>>
>> -- Lefty
>>
>> On Thu, Oct 9, 2014 at 8:23 AM, Brett Randall  wrote:
>>>
>>> Hi,
>>>
>>> On
>>> https://cwiki.apache.org/confluence/display/Hive/Hive+Default+Authorization+-+Legacy+Mode#HiveDefaultAuthorization-LegacyMode-Grant/RevokePrivileges
>>> , GRANT shows as:
>>>
>>> GRANT
>>> priv_type [(column_list)]
>>>   [, priv_type [(column_list)]] ...
>>> [ON object_type]
>>> TO principal_specification [, principal_specification] ...
>>> [WITH GRANT OPTION]
>>>
>>> Should that not be [ON object_type priv_level], same as REVOKE, where:
>>>
>>> object_type:
>>> TABLE
>>>   | DATABASE
>>>
>>> priv_level:
>>> db_name
>>>   | tbl_name
>>>
>>> Thanks
>>> Brett
>>
>>
>

Re: Question re: Concurrency

[Bing Jiang]

conf/hive-default.xml.template:  hive.support.concurrency

conf/hive-default.xml.template-  false


Hive does not support this feature as default. But you can enable it thru
some settings.

2014-10-14 10:38 GMT+08:00 Time Less :

> I am looking at this page:
> https://cwiki.apache.org/confluence/display/Hive/Locking
>
> I have two questions. First, what is the shared/exclusive locking
> behaviour of "alter table T1 partition(P1) concatenate"? It seems likely it
> should be the same as for touch partition.
>
> And see this quote (emphasis mine):
> """
> The default Hive behavior *will not be changed*, and concurrency *will
> not be supported*.
> """
>
> What does that mean? Is this page merely documenting a proposal that was
> rejected?
>
> ––
> *Tim Ellis:* 510-761-6610
>
>


-- 
Bing Jiang