Compressed OOPs are available now in 1.6.0u14: https://jdk6.dev.java.net/6uNea.html - Aaron
On Thu, Aug 20, 2009 at 10:51 AM, Raghu Angadi <rang...@yahoo-inc.com>wrote: > > Suresh had made an spreadsheet for memory consumption.. will check. > > A large portion of NN memory is taken by references. I would expect memory > savings to be very substantial (same as going from 64bit to 32bit), could be > on the order of 40%. > > The last I heard from Sun was that compressed pointers will be in very near > future JVM (certainly JDK 1.6_x). It can use compressed pointers upto 32GB > of heap. > > I would expect runtime over head on NN would be minimal in practice. > > Raghu. > > > Steve Loughran wrote: > >> >> does anyone have any up to date data on the memory consumption per >> block/file on the NN on a 64-bit JVM with compressed pointers? >> >> The best documentation on consumption is >> http://issues.apache.org/jira/browse/HADOOP-1687 -I'm just wondering if >> anyone has looked at the memory footprint on the latest Hadoop releases, on >> those latest JVMs? -and which JVM the numbers from HADOOP-1687 came from? >> >> Those compressed pointers (which BEA JRockit had for a while) save RAM >> when the pointer references are within a couple of GB of the other refs, and >> which are discussed in some papers >> http://rappist.elis.ugent.be/~leeckhou/papers/cgo06.pdf<http://rappist.elis.ugent.be/%7Eleeckhou/papers/cgo06.pdf> >> http://www.elis.ugent.be/~kvenster/papers/VenstermansKris_ORA.pdf<http://www.elis.ugent.be/%7Ekvenster/papers/VenstermansKris_ORA.pdf> >> >> sun's commentary is up here >> http://wikis.sun.com/display/HotSpotInternals/CompressedOops >> >> I'm just not sure what it means for the NameNode, and as there is no >> sizeof() operator in Java, something that will take a bit of effort to work >> out. From what I read of the Sun wiki, when you go compressed, while your >> heap is <3-4GB, there is no decompress operation; once you go above that >> there is a shift and an add, which is probably faster than fetching another >> 32 bits from $L2 or main RAM. The result could be -could be- that your NN >> takes up much less space on 64 bit JVMs than it did before, but is no >> slower. >> >> Has anyone worked out the numbers yet? >> >> -steve >> > >
