Hi
I try to create external table and in the location there are 8960 small
files
And I am getting every time something like that:
GC pool 'PS MarkSweep' had collection(s): count=1 time=1672ms
GC pool 'PS Scavenge' had collection(s): count=1 time=45ms
2016-02-26 15:18:29,721 INFO
[org.apache.hadoop.util.JvmPauseMonitor$Monitor@3af36922]:
util.JvmPauseMonitor (JvmPauseMonitor.java:run(195)) - Detected pause in
JVM or host machine (eg GC): pause of approximately 1995ms
GC pool 'PS MarkSweep' had collection(s): count=1 time=1936ms
And all 16 cpu cores are in 100% and all 16G memory almost gone.
Helps only if I restart hive server.
I use Hive 1.2.1 from HDP-2.3 by packaged by Hortonworks
hive.tez.java.opts=-server -Djava.net.preferIPv4Stack=true
-XX:NewRatio=8 -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps
--
Margus (margusja) Roo
http://margus.roo.ee
skype: margusja
+372 51 48 780
