[ https://issues.apache.org/jira/browse/HIVE-9893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15000698#comment-15000698 ]
Casey Brown commented on HIVE-9893: ----------------------------------- Getting the same problem here on EMR, Hive 0.13.1 Subversion http://aws.amazon.com/elasticmapreduce -r 4fad79b21cc5cc0d2a42a9a6e94c3c0bf81cb404 Compiled by EMR Team on Mon Oct 27 16:56:19 GMT 2014 >From source with checksum 83544a51266f73ac453641ea3932c726 I'm running a ton of alter table add/remove partitions using pyhs2. These errors crop up after a few hundred of them run. I can restart hiveserver2 and everything is OK for a little while. It shows up to the client as conn = pyhs2.connect( authMechanism='PLAIN', user='hadoop', password='', database='default') File "/usr/lib/python2.7/site-packages/pyhs2/__init__.py", line 7, in connect return Connection(*args, **kwargs) File "/usr/lib/python2.7/site-packages/pyhs2/connections.py", line 47, in __init__ res = self.client.OpenSession(TOpenSessionReq(username=user, password=password, configuration=configuration)) File "/usr/lib/python2.7/site-packages/pyhs2/TCLIService/TCLIService.py", line 154, in OpenSession return self.recv_OpenSession() File "/usr/lib/python2.7/site-packages/pyhs2/TCLIService/TCLIService.py", line 170, in recv_OpenSession raise x thrift.Thrift.TApplicationException: Internal error processing OpenSession In the Hive server log, I see this: 2015-11-11 16:32:37,556 INFO [pool-5-thread-7]: thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(179)) - Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V1 2015-11-11 16:32:46,022 ERROR [pool-5-thread-7]: thrift.ProcessFunction (ProcessFunction.java:process(41)) - Internal error processing OpenSession java.lang.OutOfMemoryError: Java heap space If there's any specific logging you'd like me to look for just let me know. > HiveServer2 java.lang.OutOfMemoryError: Java heap space > ------------------------------------------------------- > > Key: HIVE-9893 > URL: https://issues.apache.org/jira/browse/HIVE-9893 > Project: Hive > Issue Type: Bug > Components: HiveServer2 > Affects Versions: 0.14.0 > Environment: Vmware Pseudo cluster > HDP 2.2 Ambari vanilla install > Centos 6.5 > Reporter: Rupert Bailey > Labels: Hadoop, Hive, Tez, Thrift > > Everything runs but dies after a few days with the Java heap space memory > error - and with no activity on cluster. It failed most resently after 5 days. > I tried to fix it after noticing the SLF4J library duplication, wondering if > conflicting libraries would be causing the error, but still fails. > Nagios reports HiveServer2 "Flapping" > HiveServer2.log output: > 22488-2015-03-07 22:19:08,359 INFO [HiveServer2-Handler-Pool: Thread-13139]: > thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(232)) - Client > protocol version: HIVE_CLI_SERVICE_PROTOCOL_V6 > 22489-2015-03-07 22:19:19,056 ERROR [HiveServer2-Handler-Pool: Thread-13139]: > thrift.ProcessFunction (ProcessFunction.java:process(41)) - Internal error > processing OpenSession > 22490:java.lang.OutOfMemoryError: Java heap space > 22491-2015-03-07 22:19:22,515 INFO [Thread-6]: server.HiveServer2 > (HiveServer2.java:stop(299)) - Shutting down HiveServer2 > 22492-2015-03-07 22:19:22,516 INFO [Thread-6]: thrift.ThriftCLIService > (ThriftCLIService.java:stop(137)) - Thrift server has stopped > 22493-2015-03-07 22:19:22,516 INFO [Thread-6]: service.AbstractService > (AbstractService.java:stop(125)) - Service:ThriftBinaryCLIService is stopped. > 22494-2015-03-07 22:19:27,078 INFO [Thread-6]: service.AbstractService > (AbstractService.java:stop(125)) - Service:OperationManager is stopped. > 22495-2015-03-07 22:19:27,078 INFO [Thread-6]: service.AbstractService > (AbstractService.java:stop(125)) - Service:SessionManager is stopped. > 22496:2015-03-07 22:19:36,096 WARN [Thread-0]: util.ShutdownHookManager > (ShutdownHookManager.java:run(56)) - ShutdownHook 'ClientFinalizer' failed, > java.lang.OutOfMemoryError: Java heap space > 22497:java.lang.OutOfMemoryError: Java heap space -- This message was sent by Atlassian JIRA (v6.3.4#6332)