We had some leaking file descriptors which ended up being a problem in hadoop.
They fixed it on 0.21 but not on older versions. There is a workaround for hive
which we successfully use. By adding this to your hive-site.xml:
<!-- workaround for connection leak problem fixed in HADOOP-5476 but only
commited to hadoop 0.21.0 -->
<property>
<name>hive.fileformat.check</name>
<value>false</value>
</property>
Bennie.
________________________________
From: Dave Brondsema [mailto:dbronds...@geek.net]
Sent: Thursday, September 02, 2010 3:13 PM
To: hive-user@hadoop.apache.org
Subject: Re: Having a Connections Leak with the Hive Server
Scott, after re-reading your original email, I'm thinking maybe we didn't have
the same problem. Hive crashed for us when it ran out of file descriptors, it
didn't hang. Nonetheless, an upgrade may help.
On Wed, Sep 1, 2010 at 10:24 AM, Scott Whitecross
<swhitecr...@gmail.com<mailto:swhitecr...@gmail.com>> wrote:
Thanks Dave. We've been investigating moving to .6, and this may help make
the decision easier.
On Tue, Aug 31, 2010 at 5:02 PM, Dave Brondsema
<dbronds...@geek.net<mailto:dbronds...@geek.net>> wrote:
We had that problem on 0.4 also. 0.6 seems to be working better now, but we
only switched a day or two ago. See
https://issues.apache.org/jira/browse/HIVE-1181 and
http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/201001.mbox/%3cd35d2f55-d770-422d-b418-6bee212a6...@forward.co.uk%3e
On Mon, Aug 30, 2010 at 3:50 PM, Scott Whitecross
<swhitecr...@gmail.com<mailto:swhitecr...@gmail.com>> wrote:
Hi all -
I'm running the Hive Server to allow multiple JDBC queries and inserts over the
course of a day. What I've noticed is that there seems to be a pretty bad leak
with connections over the course of a week, to the point of making the machine
unreachable. Looking through the code being used, it appears to be closing
Hive connections appropriately. Is there a known problem with Hive and
connections now? (I'm currently running Hive .4).
Thanks.
--
Dave Brondsema
Software Engineer
Geeknet
www.geek.net<http://www.geek.net>
--
Dave Brondsema
Software Engineer
Geeknet
www.geek.net<http://www.geek.net>
