Re: NN memory consumption on 0.20/0.21 with compressed pointers/

2009-08-25 Thread Ted Dunning
On Tue, Aug 25, 2009 at 2:57 AM, Steve Loughran ste...@apache.org wrote: On petascale-level computers, the application codes' CPU instructions are about 10% floating point (that is, in scientific applications, there are less floating point instructions than in most floating point benchmarks).

Re: NN memory consumption on 0.20/0.21 with compressed pointers/

2009-08-25 Thread Edward Capriolo
On Tue, Aug 25, 2009 at 11:59 AM, Ted Dunningted.dunn...@gmail.com wrote: On Tue, Aug 25, 2009 at 2:57 AM, Steve Loughran ste...@apache.org wrote: On petascale-level computers, the application codes' CPU instructions are about 10% floating point (that is, in scientific applications, there are

Re: NN memory consumption on 0.20/0.21 with compressed pointers/

2009-08-24 Thread Steve Loughran
Raghu Angadi wrote: Suresh had made an spreadsheet for memory consumption.. will check. A large portion of NN memory is taken by references. I would expect memory savings to be very substantial (same as going from 64bit to 32bit), could be on the order of 40%. The last I heard from Sun was

Re: NN memory consumption on 0.20/0.21 with compressed pointers/

2009-08-24 Thread Brian Bockelman
On Aug 24, 2009, at 5:42 AM, Steve Loughran wrote: Raghu Angadi wrote: Suresh had made an spreadsheet for memory consumption.. will check. A large portion of NN memory is taken by references. I would expect memory savings to be very substantial (same as going from 64bit to 32bit), could

Re: NN memory consumption on 0.20/0.21 with compressed pointers/

2009-08-20 Thread Raghu Angadi
Suresh had made an spreadsheet for memory consumption.. will check. A large portion of NN memory is taken by references. I would expect memory savings to be very substantial (same as going from 64bit to 32bit), could be on the order of 40%. The last I heard from Sun was that compressed

Re: NN memory consumption on 0.20/0.21 with compressed pointers/

2009-08-20 Thread Aaron Kimball
Compressed OOPs are available now in 1.6.0u14: https://jdk6.dev.java.net/6uNea.html - Aaron On Thu, Aug 20, 2009 at 10:51 AM, Raghu Angadi rang...@yahoo-inc.comwrote: Suresh had made an spreadsheet for memory consumption.. will check. A large portion of NN memory is taken by references. I

Re: NN memory consumption on 0.20/0.21 with compressed pointers/

2009-08-20 Thread Scott Carey
On 8/20/09 3:40 AM, Steve Loughran ste...@apache.org wrote: does anyone have any up to date data on the memory consumption per block/file on the NN on a 64-bit JVM with compressed pointers? The best documentation on consumption is http://issues.apache.org/jira/browse/HADOOP-1687 -I'm