RE: Is there any monitoring tool available for hiveserver2
Sure thing. Thanks Biswajit. Thanks, Shouvanik From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com] Sent: Friday, February 21, 2014 9:23 AM To: user@hive.apache.org Subject: RE: Is there any monitoring tool available for hiveserver2 This one is for monitoring of metastore only. It has to be chaged for server and monitoring. Biswa On 21 Feb 2014 22:44, mailto:shouvanik.hal...@accenture.com>> wrote: Thanks Biswajit. I will try it and let you know. Thanks, Shouvanik From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com<mailto:biswajit.na...@inmobi.com>] Sent: Friday, February 21, 2014 2:28 AM To: user@hive.apache.org<mailto:user@hive.apache.org> Subject: Re: Is there any monitoring tool available for hiveserver2 Below is the script that does my graphing of heap (usage|allocated) for hive. #!/bin/bash HIVES_PID=`jps -mlv |grep "org.apache.hadoop.hive.metastore.HiveMetaStore" |awk '{print $1}'` if [ $HIVES_PID ]; then jmap -heap $HIVES_PID |awk ' BEGIN{ gmetric="/usr/bin/gmetric";sum=0} { split($1,arr," ") if(arr[1] == "used") { sum=sum + $3; } if(arr[1] == "MaxHeapSize") { maxheap=$3; } } END {printf "%s -n hive.metastore.current_heap_capacity -v %d -t int8 -d 60 -g HiveHeapstats \n",gmetric, maxheap;printf "%s -n hive.metastore.current_heap_usage -v %d -t int8 -d 60 -g HiveHeapstats \n",gmetric, sum;}' |sh fi please let me know if it works for you.. Apologies for the delay response.. Thanks BIswa On Fri, Feb 21, 2014 at 10:41 AM, mailto:shouvanik.hal...@accenture.com>> wrote: That will be great! Thanks in advance. Thanks, Shouvanik From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com<mailto:biswajit.na...@inmobi.com>] Sent: Thursday, February 20, 2014 8:37 PM To: user@hive.apache.org<mailto:user@hive.apache.org> Subject: RE: Is there any monitoring tool available for hiveserver2 I could share the script that does it. I will be able to do it by 12:30 . Stuck in a meeting till that time. Regards Biswa On 21 Feb 2014 10:02, mailto:shouvanik.hal...@accenture.com>> wrote: Hi Biswajit, Could you give an idea of how to do it, please? Thanks, Shouvanik From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com<mailto:biswajit.na...@inmobi.com>] Sent: Thursday, February 20, 2014 8:30 PM To: user@hive.apache.org<mailto:user@hive.apache.org> Subject: Re: Is there any monitoring tool available for hiveserver2 I have built up a customized script for alerting and monitoring. Could not find any default way to do it. Thanks Biswajit On 21 Feb 2014 05:17, mailto:shouvanik.hal...@accenture.com>> wrote: Hi, It might happen that hiveserver2 memory gets exhausted. Similarly there would be many other things to monitor for hiveserver2. Is there any monitoring tool available in the market? I am using EMR, for FYI. Thanks, Shouvanik This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. . __ www.accenture.com<http://www.accenture.com> _ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. _ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the
RE: Is there any monitoring tool available for hiveserver2
This one is for monitoring of metastore only. It has to be chaged for server and monitoring. Biswa On 21 Feb 2014 22:44, wrote: > Thanks Biswajit. > > > > I will try it and let you know. > > > > > > > > Thanks, > > Shouvanik > > > > *From:* Biswajit Nayak [mailto:biswajit.na...@inmobi.com] > *Sent:* Friday, February 21, 2014 2:28 AM > *To:* user@hive.apache.org > *Subject:* Re: Is there any monitoring tool available for hiveserver2 > > > > Below is the script that does my graphing of heap (usage|allocated) for > hive. > > > > #!/bin/bash > > HIVES_PID=`jps -mlv |grep "org.apache.hadoop.hive.metastore.HiveMetaStore" > |awk '{print $1}'` > > > if [ $HIVES_PID ]; then > > jmap -heap $HIVES_PID |awk ' > > BEGIN{ gmetric="/usr/bin/gmetric";sum=0} > > { > > split($1,arr," ") > > if(arr[1] == "used") { > > sum=sum + $3; > > } > > if(arr[1] == "MaxHeapSize") { > > maxheap=$3; > > } > > } END {printf "%s -n hive.metastore.current_heap_capacity -v %d -t int8 -d > 60 -g HiveHeapstats \n",gmetric, maxheap;printf "%s -n > hive.metastore.current_heap_usage -v %d -t int8 -d 60 -g HiveHeapstats > \n",gmetric, sum;}' |sh > > fi > > > > please let me know if it works for you.. Apologies for the delay > response.. > > > > Thanks > > BIswa > > > > > On Fri, Feb 21, 2014 at 10:41 AM, wrote: > > That will be great! Thanks in advance. > > > > > > > > > > > > > > Thanks, > > Shouvanik > > > > *From:* Biswajit Nayak [mailto:biswajit.na...@inmobi.com] > *Sent:* Thursday, February 20, 2014 8:37 PM > *To:* user@hive.apache.org > *Subject:* RE: Is there any monitoring tool available for hiveserver2 > > > > I could share the script that does it. I will be able to do it by 12:30 . > Stuck in a meeting till that time. > > Regards > Biswa > > On 21 Feb 2014 10:02, wrote: > > Hi Biswajit, > > > > Could you give an idea of how to do it, please? > > > > > > > > > > > > > > Thanks, > > Shouvanik > > > > *From:* Biswajit Nayak [mailto:biswajit.na...@inmobi.com] > *Sent:* Thursday, February 20, 2014 8:30 PM > *To:* user@hive.apache.org > *Subject:* Re: Is there any monitoring tool available for hiveserver2 > > > > I have built up a customized script for alerting and monitoring. > Could not find any default way to do it. > > Thanks > Biswajit > > On 21 Feb 2014 05:17, wrote: > > Hi, > > > > It might happen that hiveserver2 memory gets exhausted. Similarly there > would be many other things to monitor for hiveserver2. > > > > Is there any monitoring tool available in the market? > > > > I am using EMR, for FYI. > > > > > > > > > > > > > > Thanks, > > Shouvanik > > > > > -- > > > This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise confidential information. If you have > received it in error, please notify the sender immediately and delete the > original. Any other use of the e-mail by you is prohibited. Where allowed > by local law, electronic communications with Accenture and its affiliates, > including e-mail and instant messaging (including content), may be scanned > by our systems for the purposes of information security and assessment of > internal compliance with Accenture policy. . > > __ > > www.accenture.com > > > > _ > > The information contained in this communication is intended solely for the > use of the individual or entity to whom it is addressed and others > authorized to receive it. It may contain confidential or legally privileged > information. If you are not the intended recipient you are hereby notified > that any disclosure, copying, distribution or taking any action in reliance > on the contents of this information is strictly prohibited and may be > unlawful. If you have received this communication in error, please notify > us immediately by responding to this email and then delete it from your > system. The firm is neither liable for the proper and complete transmission > of the information contained in this communication nor for any delay in its > receipt. > > > > _ > &
RE: Is there any monitoring tool available for hiveserver2
Thanks Biswajit. I will try it and let you know. Thanks, Shouvanik From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com] Sent: Friday, February 21, 2014 2:28 AM To: user@hive.apache.org Subject: Re: Is there any monitoring tool available for hiveserver2 Below is the script that does my graphing of heap (usage|allocated) for hive. #!/bin/bash HIVES_PID=`jps -mlv |grep "org.apache.hadoop.hive.metastore.HiveMetaStore" |awk '{print $1}'` if [ $HIVES_PID ]; then jmap -heap $HIVES_PID |awk ' BEGIN{ gmetric="/usr/bin/gmetric";sum=0} { split($1,arr," ") if(arr[1] == "used") { sum=sum + $3; } if(arr[1] == "MaxHeapSize") { maxheap=$3; } } END {printf "%s -n hive.metastore.current_heap_capacity -v %d -t int8 -d 60 -g HiveHeapstats \n",gmetric, maxheap;printf "%s -n hive.metastore.current_heap_usage -v %d -t int8 -d 60 -g HiveHeapstats \n",gmetric, sum;}' |sh fi please let me know if it works for you.. Apologies for the delay response.. Thanks BIswa On Fri, Feb 21, 2014 at 10:41 AM, mailto:shouvanik.hal...@accenture.com>> wrote: That will be great! Thanks in advance. Thanks, Shouvanik From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com<mailto:biswajit.na...@inmobi.com>] Sent: Thursday, February 20, 2014 8:37 PM To: user@hive.apache.org<mailto:user@hive.apache.org> Subject: RE: Is there any monitoring tool available for hiveserver2 I could share the script that does it. I will be able to do it by 12:30 . Stuck in a meeting till that time. Regards Biswa On 21 Feb 2014 10:02, mailto:shouvanik.hal...@accenture.com>> wrote: Hi Biswajit, Could you give an idea of how to do it, please? Thanks, Shouvanik From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com<mailto:biswajit.na...@inmobi.com>] Sent: Thursday, February 20, 2014 8:30 PM To: user@hive.apache.org<mailto:user@hive.apache.org> Subject: Re: Is there any monitoring tool available for hiveserver2 I have built up a customized script for alerting and monitoring. Could not find any default way to do it. Thanks Biswajit On 21 Feb 2014 05:17, mailto:shouvanik.hal...@accenture.com>> wrote: Hi, It might happen that hiveserver2 memory gets exhausted. Similarly there would be many other things to monitor for hiveserver2. Is there any monitoring tool available in the market? I am using EMR, for FYI. Thanks, Shouvanik This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. . __ www.accenture.com<http://www.accenture.com> _ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. _ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. _ The information contained in this communication is intended so
Re: Is there any monitoring tool available for hiveserver2
Below is the script that does my graphing of heap (usage|allocated) for hive. #!/bin/bash HIVES_PID=`jps -mlv |grep "org.apache.hadoop.hive.metastore.HiveMetaStore" |awk '{print $1}'` if [ $HIVES_PID ]; then jmap -heap $HIVES_PID |awk ' BEGIN{ gmetric="/usr/bin/gmetric";sum=0} { split($1,arr," ") if(arr[1] == "used") { sum=sum + $3; } if(arr[1] == "MaxHeapSize") { maxheap=$3; } } END {printf "%s -n hive.metastore.current_heap_capacity -v %d -t int8 -d 60 -g HiveHeapstats \n",gmetric, maxheap;printf "%s -n hive.metastore.current_heap_usage -v %d -t int8 -d 60 -g HiveHeapstats \n",gmetric, sum;}' |sh fi please let me know if it works for you.. Apologies for the delay response.. Thanks BIswa On Fri, Feb 21, 2014 at 10:41 AM, wrote: > That will be great! Thanks in advance. > > > > > > > > > > > > > > Thanks, > > Shouvanik > > > > *From:* Biswajit Nayak [mailto:biswajit.na...@inmobi.com] > *Sent:* Thursday, February 20, 2014 8:37 PM > *To:* user@hive.apache.org > *Subject:* RE: Is there any monitoring tool available for hiveserver2 > > > > I could share the script that does it. I will be able to do it by 12:30 . > Stuck in a meeting till that time. > > Regards > Biswa > > On 21 Feb 2014 10:02, wrote: > > Hi Biswajit, > > > > Could you give an idea of how to do it, please? > > > > > > > > > > > > > > Thanks, > > Shouvanik > > > > *From:* Biswajit Nayak [mailto:biswajit.na...@inmobi.com] > *Sent:* Thursday, February 20, 2014 8:30 PM > *To:* user@hive.apache.org > *Subject:* Re: Is there any monitoring tool available for hiveserver2 > > > > I have built up a customized script for alerting and monitoring. > Could not find any default way to do it. > > Thanks > Biswajit > > On 21 Feb 2014 05:17, wrote: > > Hi, > > > > It might happen that hiveserver2 memory gets exhausted. Similarly there > would be many other things to monitor for hiveserver2. > > > > Is there any monitoring tool available in the market? > > > > I am using EMR, for FYI. > > > > > > > > > > > > > > Thanks, > > Shouvanik > > > > > -- > > > This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise confidential information. If you have > received it in error, please notify the sender immediately and delete the > original. Any other use of the e-mail by you is prohibited. Where allowed > by local law, electronic communications with Accenture and its affiliates, > including e-mail and instant messaging (including content), may be scanned > by our systems for the purposes of information security and assessment of > internal compliance with Accenture policy. . > > __ > > www.accenture.com > > > > _ > > The information contained in this communication is intended solely for the > use of the individual or entity to whom it is addressed and others > authorized to receive it. It may contain confidential or legally privileged > information. If you are not the intended recipient you are hereby notified > that any disclosure, copying, distribution or taking any action in reliance > on the contents of this information is strictly prohibited and may be > unlawful. If you have received this communication in error, please notify > us immediately by responding to this email and then delete it from your > system. The firm is neither liable for the proper and complete transmission > of the information contained in this communication nor for any delay in its > receipt. > > > > _ > > The information contained in this communication is intended solely for the > use of the individual or entity to whom it is addressed and others > authorized to receive it. It may contain confidential or legally privileged > information. If you are not the intended recipient you are hereby notified > that any disclosure, copying, distribution or taking any action in reliance > on the contents of this information is strictly prohibited and may be > unlawful. If you have received this communication in error, please notify > us immediately by responding to this email and then delete it from your > system. The firm is neither liable for the proper and complete transmission > of the information contained in this communication nor for any delay in its >
RE: Is there any monitoring tool available for hiveserver2
That will be great! Thanks in advance. Thanks, Shouvanik From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com] Sent: Thursday, February 20, 2014 8:37 PM To: user@hive.apache.org Subject: RE: Is there any monitoring tool available for hiveserver2 I could share the script that does it. I will be able to do it by 12:30 . Stuck in a meeting till that time. Regards Biswa On 21 Feb 2014 10:02, mailto:shouvanik.hal...@accenture.com>> wrote: Hi Biswajit, Could you give an idea of how to do it, please? Thanks, Shouvanik From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com<mailto:biswajit.na...@inmobi.com>] Sent: Thursday, February 20, 2014 8:30 PM To: user@hive.apache.org<mailto:user@hive.apache.org> Subject: Re: Is there any monitoring tool available for hiveserver2 I have built up a customized script for alerting and monitoring. Could not find any default way to do it. Thanks Biswajit On 21 Feb 2014 05:17, mailto:shouvanik.hal...@accenture.com>> wrote: Hi, It might happen that hiveserver2 memory gets exhausted. Similarly there would be many other things to monitor for hiveserver2. Is there any monitoring tool available in the market? I am using EMR, for FYI. Thanks, Shouvanik This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. . __ www.accenture.com<http://www.accenture.com> _ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. _ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
RE: Is there any monitoring tool available for hiveserver2
I could share the script that does it. I will be able to do it by 12:30 . Stuck in a meeting till that time. Regards Biswa On 21 Feb 2014 10:02, wrote: > Hi Biswajit, > > > > Could you give an idea of how to do it, please? > > > > > > > > > > > > > > Thanks, > > Shouvanik > > > > *From:* Biswajit Nayak [mailto:biswajit.na...@inmobi.com] > *Sent:* Thursday, February 20, 2014 8:30 PM > *To:* user@hive.apache.org > *Subject:* Re: Is there any monitoring tool available for hiveserver2 > > > > I have built up a customized script for alerting and monitoring. > Could not find any default way to do it. > > Thanks > Biswajit > > On 21 Feb 2014 05:17, wrote: > > Hi, > > > > It might happen that hiveserver2 memory gets exhausted. Similarly there > would be many other things to monitor for hiveserver2. > > > > Is there any monitoring tool available in the market? > > > > I am using EMR, for FYI. > > > > > > > > > > > > > > Thanks, > > Shouvanik > > > > > -- > > > This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise confidential information. If you have > received it in error, please notify the sender immediately and delete the > original. Any other use of the e-mail by you is prohibited. Where allowed > by local law, electronic communications with Accenture and its affiliates, > including e-mail and instant messaging (including content), may be scanned > by our systems for the purposes of information security and assessment of > internal compliance with Accenture policy. . > > __ > > www.accenture.com > > > > _ > > The information contained in this communication is intended solely for the > use of the individual or entity to whom it is addressed and others > authorized to receive it. It may contain confidential or legally privileged > information. If you are not the intended recipient you are hereby notified > that any disclosure, copying, distribution or taking any action in reliance > on the contents of this information is strictly prohibited and may be > unlawful. If you have received this communication in error, please notify > us immediately by responding to this email and then delete it from your > system. The firm is neither liable for the proper and complete transmission > of the information contained in this communication nor for any delay in its > receipt. > -- _ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
RE: Is there any monitoring tool available for hiveserver2
Hi Biswajit, Could you give an idea of how to do it, please? Thanks, Shouvanik From: Biswajit Nayak [mailto:biswajit.na...@inmobi.com] Sent: Thursday, February 20, 2014 8:30 PM To: user@hive.apache.org Subject: Re: Is there any monitoring tool available for hiveserver2 I have built up a customized script for alerting and monitoring. Could not find any default way to do it. Thanks Biswajit On 21 Feb 2014 05:17, mailto:shouvanik.hal...@accenture.com>> wrote: Hi, It might happen that hiveserver2 memory gets exhausted. Similarly there would be many other things to monitor for hiveserver2. Is there any monitoring tool available in the market? I am using EMR, for FYI. Thanks, Shouvanik This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. . __ www.accenture.com<http://www.accenture.com> _ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
Re: Is there any monitoring tool available for hiveserver2
I have built up a customized script for alerting and monitoring. Could not find any default way to do it. Thanks Biswajit On 21 Feb 2014 05:17, wrote: > Hi, > > > > It might happen that hiveserver2 memory gets exhausted. Similarly there > would be many other things to monitor for hiveserver2. > > > > Is there any monitoring tool available in the market? > > > > I am using EMR, for FYI. > > > > > > > > > > > > > > Thanks, > > Shouvanik > > > > -- > > This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise confidential information. If you have > received it in error, please notify the sender immediately and delete the > original. Any other use of the e-mail by you is prohibited. Where allowed > by local law, electronic communications with Accenture and its affiliates, > including e-mail and instant messaging (including content), may be scanned > by our systems for the purposes of information security and assessment of > internal compliance with Accenture policy. . > > __ > > www.accenture.com > -- _ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.