RE: Is there any monitoring tool available for hiveserver2

[shouvanik.haldar]

That will be great! Thanks in advance.






Thanks,
Shouvanik

From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com]
Sent: Thursday, February 20, 2014 8:37 PM
To: user@hive.apache.org
Subject: RE: Is there any monitoring tool available for hiveserver2


I could share the script that does it. I will be able to do it by 12:30 . Stuck 
in a meeting till that time.

Regards
Biswa
On 21 Feb 2014 10:02, 
mailto:shouvanik.hal...@accenture.com>> wrote:
Hi Biswajit,

Could you give an idea of how to do it, please?






Thanks,
Shouvanik

From: Biswajit Nayak 
[mailto:biswajit.na...@inmobi.com]
Sent: Thursday, February 20, 2014 8:30 PM
To: user@hive.apache.org
Subject: Re: Is there any monitoring tool available for hiveserver2


I have built up a customized script for alerting and monitoring.
Could not find any default way to do it.

Thanks
Biswajit
On 21 Feb 2014 05:17, 
mailto:shouvanik.hal...@accenture.com>> wrote:
Hi,

It might happen that hiveserver2 memory gets exhausted. Similarly there would 
be many other things to  monitor for hiveserver2.

Is there any monitoring tool available in the market?

I am using EMR, for FYI.






Thanks,
Shouvanik




This message is for the designated recipient only and may contain privileged, 
proprietary, or otherwise confidential information. If you have received it in 
error, please notify the sender immediately and delete the original. Any other 
use of the e-mail by you is prohibited. Where allowed by local law, electronic 
communications with Accenture and its affiliates, including e-mail and instant 
messaging (including content), may be scanned by our systems for the purposes 
of information security and assessment of internal compliance with Accenture 
policy. .
__

www.accenture.com

_
The information contained in this communication is intended solely for the use 
of the individual or entity to whom it is addressed and others authorized to 
receive it. It may contain confidential or legally privileged information. If 
you are not the intended recipient you are hereby notified that any disclosure, 
copying, distribution or taking any action in reliance on the contents of this 
information is strictly prohibited and may be unlawful. If you have received 
this communication in error, please notify us immediately by responding to this 
email and then delete it from your system. The firm is neither liable for the 
proper and complete transmission of the information contained in this 
communication nor for any delay in its receipt.

_
The information contained in this communication is intended solely for the use 
of the individual or entity to whom it is addressed and others authorized to 
receive it. It may contain confidential or legally privileged information. If 
you are not the intended recipient you are hereby notified that any disclosure, 
copying, distribution or taking any action in reliance on the contents of this 
information is strictly prohibited and may be unlawful. If you have received 
this communication in error, please notify us immediately by responding to this 
email and then delete it from your system. The firm is neither liable for the 
proper and complete transmission of the information contained in this 
communication nor for any delay in its receipt.

RE: Is there any monitoring tool available for hiveserver2

[Biswajit Nayak]

I could share the script that does it. I will be able to do it by 12:30 .
Stuck in a meeting till that time.

Regards
Biswa
On 21 Feb 2014 10:02,  wrote:

>  Hi Biswajit,
>
>
>
> Could you give an idea of how to do it, please?
>
>
>
>
>
>
>
>
>
>
>
>
>
> Thanks,
>
> Shouvanik
>
>
>
> *From:* Biswajit Nayak [mailto:biswajit.na...@inmobi.com]
> *Sent:* Thursday, February 20, 2014 8:30 PM
> *To:* user@hive.apache.org
> *Subject:* Re: Is there any monitoring tool available for hiveserver2
>
>
>
> I have built up a customized script for alerting and monitoring.
> Could not find any default way to do it.
>
> Thanks
> Biswajit
>
> On 21 Feb 2014 05:17,  wrote:
>
> Hi,
>
>
>
> It might happen that hiveserver2 memory gets exhausted. Similarly there
> would be many other things to  monitor for hiveserver2.
>
>
>
> Is there any monitoring tool available in the market?
>
>
>
> I am using EMR, for FYI.
>
>
>
>
>
>
>
>
>
>
>
>
>
> Thanks,
>
> Shouvanik
>
>
>
>
>  --
>
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited. Where allowed
> by local law, electronic communications with Accenture and its affiliates,
> including e-mail and instant messaging (including content), may be scanned
> by our systems for the purposes of information security and assessment of
> internal compliance with Accenture policy. .
>
> __
>
> www.accenture.com
>
>
>
> _
>
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify
> us immediately by responding to this email and then delete it from your
> system. The firm is neither liable for the proper and complete transmission
> of the information contained in this communication nor for any delay in its
> receipt.
>

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Query regarding Hive Parallel Orderby

[Vaibhav Jain]

Hi,

Hive 12 has added the functionality of parallel order by. I have a few
queries regarding the working of it.
>From the source code I have figured out that to do a parallel orderby , a
partition table needs to created
which is provided as an input to TotalOrderPartitioner.  To create the
partition table, a sample of
the hive table is stored as ArrayList of byte arrays and then sorted.

So I have the following queries :

1)  Is my understanding correct?

2) Isn't it a possibility that storing the entire sample in memory would
become a bottleneck when the sample size is large?


-- 
Thanks
Vaibhav Jain

RE: Is there any monitoring tool available for hiveserver2

[shouvanik.haldar]

Hi Biswajit,

Could you give an idea of how to do it, please?






Thanks,
Shouvanik

From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com]
Sent: Thursday, February 20, 2014 8:30 PM
To: user@hive.apache.org
Subject: Re: Is there any monitoring tool available for hiveserver2


I have built up a customized script for alerting and monitoring.
Could not find any default way to do it.

Thanks
Biswajit
On 21 Feb 2014 05:17, 
mailto:shouvanik.hal...@accenture.com>> wrote:
Hi,

It might happen that hiveserver2 memory gets exhausted. Similarly there would 
be many other things to  monitor for hiveserver2.

Is there any monitoring tool available in the market?

I am using EMR, for FYI.






Thanks,
Shouvanik




This message is for the designated recipient only and may contain privileged, 
proprietary, or otherwise confidential information. If you have received it in 
error, please notify the sender immediately and delete the original. Any other 
use of the e-mail by you is prohibited. Where allowed by local law, electronic 
communications with Accenture and its affiliates, including e-mail and instant 
messaging (including content), may be scanned by our systems for the purposes 
of information security and assessment of internal compliance with Accenture 
policy. .
__

www.accenture.com

_
The information contained in this communication is intended solely for the use 
of the individual or entity to whom it is addressed and others authorized to 
receive it. It may contain confidential or legally privileged information. If 
you are not the intended recipient you are hereby notified that any disclosure, 
copying, distribution or taking any action in reliance on the contents of this 
information is strictly prohibited and may be unlawful. If you have received 
this communication in error, please notify us immediately by responding to this 
email and then delete it from your system. The firm is neither liable for the 
proper and complete transmission of the information contained in this 
communication nor for any delay in its receipt.

Re: Is there any monitoring tool available for hiveserver2

[Biswajit Nayak]

I have built up a customized script for alerting and monitoring.
Could not find any default way to do it.

Thanks
Biswajit
On 21 Feb 2014 05:17,  wrote:

>  Hi,
>
>
>
> It might happen that hiveserver2 memory gets exhausted. Similarly there
> would be many other things to  monitor for hiveserver2.
>
>
>
> Is there any monitoring tool available in the market?
>
>
>
> I am using EMR, for FYI.
>
>
>
>
>
>
>
>
>
>
>
>
>
> Thanks,
>
> Shouvanik
>
>
>
> --
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited. Where allowed
> by local law, electronic communications with Accenture and its affiliates,
> including e-mail and instant messaging (including content), may be scanned
> by our systems for the purposes of information security and assessment of
> internal compliance with Accenture policy. .
>
> __
>
> www.accenture.com
>

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Is there any monitoring tool available for hiveserver2

[shouvanik.haldar]

Hi,

It might happen that hiveserver2 memory gets exhausted. Similarly there would 
be many other things to  monitor for hiveserver2.

Is there any monitoring tool available in the market?

I am using EMR, for FYI.






Thanks,
Shouvanik




This message is for the designated recipient only and may contain privileged, 
proprietary, or otherwise confidential information. If you have received it in 
error, please notify the sender immediately and delete the original. Any other 
use of the e-mail by you is prohibited. Where allowed by local law, electronic 
communications with Accenture and its affiliates, including e-mail and instant 
messaging (including content), may be scanned by our systems for the purposes 
of information security and assessment of internal compliance with Accenture 
policy. .
__

www.accenture.com

Mailing list/Forum

[Rod Asher]

Good Afternoon,

I would like to be taken off the forum and mailing list please.

Thank you
Rod 

Rod Asher & Associates
800-722-1030 / 206-315-6678
www.asher.com

Re: Output Avro result as JSON

[Andrew Mains]

One method could be to either create a custom table off of a query with 
a JSON serde (for instance, I've used 
https://github.com/rcongiu/Hive-JSON-Serde).


Something like:

CREATE EXTERNAL TABLE my_tmp_table
ROW  FORMAT  SERDE  'org.openx.data.jsonserde.JsonSerDe'
STORED  AS  TEXTFILE
LOCATION '/path/to/file'
AS



In versions of hive later than 0.10 (which is what we're using), I 
believe that you may be able to do a similar thing with INSERT 
OVERWRITE, but I'm not
sure whether it works with custom serdes (as opposed to custom 
delimiters). See https://issues.apache.org/jira/browse/HIVE-3682.


Hope this helps!

Andrew

On 2/20/14, 11:09 AM, Software Dev wrote:
When I run a query in the hive shell for an Avro field it displays it 
as json. How can I accomplish then when saving to a file?

Output Avro result as JSON

[Software Dev]

When I run a query in the hive shell for an Avro field it displays it as
json. How can I accomplish then when saving to a file?

Sort by Avro field

[Software Dev]

Is it possible to sort by a field within an Avro struct? Am I doing
something wrong?


hive> describe logs;
OK
requestheader struct from deserializer
year int
month int
day int

hive> select * from logs where year = 2014 order by requestheader.timestamp;
FAILED: ParseException line 1:68 mismatched input 'timestamp' expecting
Identifier near '.' in expression specification

Re: Reg:Hive query with mapreduce

[Nitin Pawar]

try this

http://ysmart.cse.ohio-state.edu/online.html


On Thu, Feb 20, 2014 at 5:55 PM, Ranjini Rathinam wrote:

> Hi,
>
> How to implement the Hive query such as
>
> select * from table comp;
>
> select empId from comp where sal>12000;
>
> in mapreduce.
>
> Need to use this query in mapreduce code. How to implement the above query
> in the code using mapreduce , JAVA.
>
>
> Please provide the sample code.
>
> Thanks in advance for the support
>
> Regards
>
> Ranjini
>
>
>
>
>



-- 
Nitin Pawar

Reg:Hive query with mapreduce

[Ranjini Rathinam]

Hi,

How to implement the Hive query such as

select * from table comp;

select empId from comp where sal>12000;

in mapreduce.

Need to use this query in mapreduce code. How to implement the above query
in the code using mapreduce , JAVA.


Please provide the sample code.

Thanks in advance for the support

Regards

Ranjini

Tuning Hive queries that uses underlying HBase Table

[Manjula mohapatra]

I am querying Hive table ( mapped to HBase Table ) .

What are the techniques to tune the Hive query and to avoid HBase scans.

Query uses multiple SPLIT and SUBSTR functions and WHERE  condition
something like

select  col1, col2, ...,count(*)
from hiveTable

where split( col1)[0] > timestamp1  and split( col1)[0]