Re: Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-08-29 Thread agateaaa
Hi All: Put some debugging code in TUGIContainingTransport.getTransport() and I tracked it down to @Override public TUGIContainingTransport getTransport(TTransport trans) { // UGI information is not available at connection setup time, it will be set later // via set_ugi() rpc.

Re: Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-08-29 Thread agateaaa
Sorry hit send too soon ... Hi All: Put some debugging code in TUGIContainingTransport.getTransport() and I tracked it down to @Override public TUGIContainingTransport getTransport(TTransport trans) { // UGI information is not available at connection setup time, it will be set later // via

Re: Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-08-29 Thread Ashutosh Chauhan
Thanks Agatea for digging in. Seems like you have hit a bug. Would you mind opening a jira and adding your findings to it. Thanks, Ashutosh On Thu, Aug 29, 2013 at 11:22 AM, agateaaa agate...@gmail.com wrote: Sorry hit send too soon ... Hi All: Put some debugging code in

Re: Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-08-29 Thread agateaaa
Thanks Ashutosh. Filed https://issues.apache.org/jira/browse/HIVE-5172 On Thu, Aug 29, 2013 at 11:53 AM, Ashutosh Chauhan hashut...@apache.orgwrote: Thanks Agatea for digging in. Seems like you have hit a bug. Would you mind opening a jira and adding your findings to it. Thanks, Ashutosh

Re: Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-08-29 Thread agateaaa
Thanks Ashutosh. Filed https://issues.apache.org/jira/browse/HIVE-5172 On Thu, Aug 29, 2013 at 11:53 AM, Ashutosh Chauhan hashut...@apache.orgwrote: Thanks Agatea for digging in. Seems like you have hit a bug. Would you mind opening a jira and adding your findings to it. Thanks, Ashutosh

Re: Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-07-31 Thread agateaaa
Thanks Nitin There arent too many connections in close_wait state only 1 or two when we run into this. Most likely its because of dropped connection. I could not find any read or write timeouts we can set for the thrift server which will tell thrift to hold on to the client connection. See this

Re: Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-07-30 Thread Nitin Pawar
The mentioned flow is called when you have unsecure mode of thrift metastore client-server connection. So one way to avoid this is have a secure way. code public boolean process(final TProtocol in, final TProtocol out) throwsTException { setIpAddress(in); ... ... ... @Override protected void

Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-07-29 Thread agateaaa
Hi All: We are running into frequent problem using HCatalog 0.4.1 (HIve Metastore Server 0.9) where we get connection reset or connection timeout errors. The hive metastore server has been allocated enough (12G) memory. This is a critical problem for us and would appreciate if anyone has any

Re: Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-07-29 Thread Nitin Pawar
Is there any chance you can do a update on test environment with hcat-0.5 and hive-0(11 or 10) and see if you can reproduce the issue? We used to see this error when there was load on hcat server or some network issue connecting to the server(second one was rare occurrence) On Mon, Jul 29, 2013

Re: Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-07-29 Thread agateaaa
Thanks Nitin! We have simiar setup (identical hcatalog and hive server versions) on a another production environment and dont see any errors (its been running ok for a few months) Unfortunately we wont be able to move to hcat 0.5 and hive 0.11 or hive 0.10 soon. I did see that the last time we

Re: Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-07-29 Thread agateaaa
Looking at the hive metastore server logs see errors like these: 2013-07-26 06:34:52,853 ERROR server.TThreadPoolServer (TThreadPoolServer.java:run(182)) - Error occurred during processing of message. java.lang.NullPointerException at

Re: Hive Metastore Server 0.9 Connection Reset and Connection Timeout errors

2013-07-29 Thread agateaaa
Looking at the hive metastore server logs see errors like these: 2013-07-26 06:34:52,853 ERROR server.TThreadPoolServer (TThreadPoolServer.java:run(182)) - Error occurred during processing of message. java.lang.NullPointerException at