Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-23 Thread Bryan Jeffrey
Any further word on this? We're seeing a major memory leak too. On May 21, 2014 10:03 PM, Meng QingPing mqingp...@gmail.com wrote: Vaibhav, thanks for reply. It's a remote metastore, stored in mysql. 2014-05-22 5:31 GMT+08:00 Vaibhav Gumashta vgumas...@hortonworks.com: Meng, are you

Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-21 Thread Meng QingPing
There are 18908 PM-s. I post it on http://jack-meng.qiniudn.com/hive-heap2.pnghttp://jack-meng.qiniudn.com/hive-heap.png Thanks. 2014-05-20 0:41 GMT+08:00 Sergey Shelukhin ser...@hortonworks.com: What is the count of PM-s that you are seeing when it crashes? From that screenshot I don't see

Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-21 Thread Meng QingPing
There are 18908 PM-s. I post it on http://jack-meng.qiniudn.com/hive-heap2.png Thanks. 2014-05-20 0:41 GMT+08:00 Sergey Shelukhin ser...@hortonworks.com: What is the count of PM-s that you are seeing when it crashes? From that screenshot I don't see what the problem is or I'm not reading it

Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-21 Thread Meng QingPing
I post the heap bin file on http://jack-meng.qiniudn.com/heap.bin Thanks. 2014-05-21 15:41 GMT+08:00 Meng QingPing mqingp...@gmail.com: There are 18908 PM-s. I post it on http://jack-meng.qiniudn.com/hive-heap2.png Thanks. 2014-05-20 0:41 GMT+08:00 Sergey Shelukhin

Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-21 Thread Vaibhav Gumashta
Thanks a lot for the dump Meng! On Wed, May 21, 2014 at 1:41 AM, Meng QingPing mqingp...@gmail.com wrote: I post the heap bin file on http://jack-meng.qiniudn.com/heap.bin Thanks. 2014-05-21 15:41 GMT+08:00 Meng QingPing mqingp...@gmail.com: There are 18908 PM-s. I post it on

Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-21 Thread Vaibhav Gumashta
Meng, are you running a remote metastore or an embedded one (which would be true if you added this config while starting HS2: -hiveconf hive.metastore.uris= )? Thanks, --Vaibhav On Wed, May 21, 2014 at 2:28 PM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Thanks a lot for the dump Meng!

Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-21 Thread Meng QingPing
Vaibhav, thanks for reply. It's a remote metastore, stored in mysql. 2014-05-22 5:31 GMT+08:00 Vaibhav Gumashta vgumas...@hortonworks.com: Meng, are you running a remote metastore or an embedded one (which would be true if you added this config while starting HS2: -hiveconf

Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-19 Thread Meng QingPing
Thanks. I post it on http://jack-meng.qiniudn.com/hive-heap.png 2014-05-06 23:14 GMT+08:00 Sergey Shelukhin ser...@hortonworks.com: I don't think the dev list supports attachment. Can you post to some image sharing service? On Tue, May 6, 2014 at 3:20 AM, Meng QingPing

Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-19 Thread Sergey Shelukhin
What is the count of PM-s that you are seeing when it crashes? From that screenshot I don't see what the problem is or I'm not reading it right. On Mon, May 19, 2014 at 1:17 AM, Meng QingPing mqingp...@gmail.com wrote: Thanks. I post it on http://jack-meng.qiniudn.com/hive-heap.png

Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-06 Thread Meng QingPing
Weird, the picture attached in sent mail. Anyway send again. 2014-05-05 12:17 GMT+08:00 Chandra Reddy chandu...@gmail.com: seems you have missed attachment. -Chandra On Sun, May 4, 2014 at 6:57 PM, Meng QingPing mqingp...@gmail.com wrote: I run hiveserver2 with metastore in mysql. The

Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-06 Thread Sergey Shelukhin
I don't think the dev list supports attachment. Can you post to some image sharing service? On Tue, May 6, 2014 at 3:20 AM, Meng QingPing mqingp...@gmail.com wrote: Weird, the picture attached in sent mail. Anyway send again. 2014-05-05 12:17 GMT+08:00 Chandra Reddy chandu...@gmail.com:

Re: Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-05 Thread Chandra Reddy
seems you have missed attachment. -Chandra On Sun, May 4, 2014 at 6:57 PM, Meng QingPing mqingp...@gmail.com wrote: I run hiveserver2 with metastore in mysql. The hiveserver2 OOM and the heap dump show huge objects hold by org.datanucleus.api.jdo. JDOPersistenceManagerFactory as attached. It

Is there a bug in hiveserver2 metastore caused hold huge objects

2014-05-04 Thread Meng QingPing
I run hiveserver2 with metastore in mysql. The hiveserver2 OOM and the heap dump show huge objects hold by org.datanucleus.api.jdo. JDOPersistenceManagerFactory as attached. It seems not release org.datanucleus.api.jdo.JDOPersistenceManager. Hive version is 0.13. Thanks, Jack