I don't think Spark sets UseCompressedOops in any defaults; are you setting it? It can't be used with heaps >= 32GB. It doesn't seem to cause an error if you set it with large heaps, just a warning. What's the problem?
On Mon, Mar 23, 2020 at 6:21 AM angers.zhu <angers....@gmail.com> wrote: > Hi developers, > > These day I meet a strange problem and I can’t find why > > When I start a spark thrift server with spark.driver.memory 64g, then > use jdk8/bin/jinfo pid to see vm flags got below information, > In 64g vm, UseCompressedOops should be closed by default, why spark > thrift server is -XX: +UseCompressedOops > > Non-default VM flags: -XX:CICompilerCount=15 -XX:-CMSClassUnloadingEnabled > -XX:CMSFullGCsBeforeCompaction=0 -XX:CMSInitiatingOccupancyFraction=75 > -XX:+CMSParallelRemarkEnabled -XX:-ClassUnloading -XX:+DisableExplicitGC > -XX:ErrorFile=null -XX:-ExplicitGCInvokesConcurrentAndUnloadsClasses > -XX:InitialHeapSize=2116026368 -XX:+ManagementServer > -XX:MaxDirectMemorySize=8589934592 -XX:MaxHeapSize=6442450944 > -XX:MaxNewSize=2147483648 -XX:MaxTenuringThreshold=6 > -XX:MinHeapDeltaBytes=196608 -XX:NewSize=705298432 -XX:OldPLABSize=16 > -XX:OldSize=1410727936 -XX:+PrintGC -XX:+PrintGCDateStamps > -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution > -XX:-TraceClassUnloading -XX:+UseCMSCompactAtFullCollection > -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseCompressedClassPointers > -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC > -XX:+UseFastUnorderedTimeStamps -XX:+UseParNewGCCommand line: -Xmx6g > -Djava.library.path=/home/hadoop/hadoop/lib/native > -Djavax.security.auth.useSubjectCredsOnly=false > -Dcom.sun.management.jmxremote.port=9021 > -Dcom.sun.management.jmxremote.authenticate=false > -Dcom.sun.management.jmxremote.ssl=false -XX:MaxPermSize=1024m > -XX:PermSize=256m -XX:MaxDirectMemorySize=8192m -XX:-TraceClassUnloading > -XX:+UseCompressedOops -XX:+UseParNewGC -XX:+UseConcMarkSweepGC > -XX:+CMSClassUnloadingEnabled -XX:+UseCMSCompactAtFullCollection > -XX:CMSFullGCsBeforeCompaction=0 -XX:+CMSParallelRemarkEnabled > -XX:+DisableExplicitGC -XX:+PrintTenuringDistribution > -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=75 > -Xnoclassgc -XX:+PrintGCDetails -XX:+PrintGCDateStamps > > > Since I am not a professor in VM, hope for some help > > > angers.zhu > angers....@gmail.com > > <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=angers.zhu&uid=angers.zhu%40gmail.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22angers.zhu%40gmail.com%22%5D> > 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制 > >
