#1 I not sure if I got you point, as I known, Xmx is not turn into physical memory as soon as the process running. it first loaded into virtual memory, if you heap is need more, it will gradually increase in physical memory until to the max heap. #2 Physical memory contains not only heap, but also stack, direct memory, shared lib, and perm space, and also there have VSS, RSS, PSS, USS concept, you can google. simple says:Vss = virtual set sizeRss = resident set sizePss = proportional set size Uss = unique set size Best Regards,Andy Hu（胡 珊）
Date: Fri, 29 May 2015 07:41:41 -0700 Subject: Re: Spark Executor Memory Usage From: yuzhih...@gmail.com To: valeramoisee...@gmail.com CC: user@spark.apache.org For #2, see http://unix.stackexchange.com/questions/65835/htop-reporting-much-higher-memory-usage-than-free-or-top Cheers On Fri, May 29, 2015 at 6:56 AM, Valerii Moisieienko <valeramoisee...@gmail.com> wrote: Hello! My name is Valerii. I have noticed strange memory behaivour of Spark's executor on my cluster. Cluster works in standalone mode with 3 workers. Application runs in cluster mode. From topology configuration spark.executor.memory 1536m I checked heap usage via JVisualVM: http://joxi.ru/Q2KqBMdSvYpDrj and via htop: http://joxi.ru/Vm63RWeCvG6L2Z I have 2 questions regarding Spark's executors memory usage: 1. Why does Max Heap Size change during executor work? 2. Why does Memory usage via htop greater than executor's heap size? Thank you! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Executor-Memory-Usage-tp23083.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
