Meng, It would help if you could provide a heap dump (so we can feed it to MAT). I think that the weights of the objects is one thing, but the representation of the dependencies is another thing that takes memory.
As for the second optimization that Daniel mentions, I believe that you should be able to implement it in the Projector. Pascal On 2013-02-17, at 9:23 AM, zhu kane <[email protected]> wrote: > Hi Daniel, > > Thanks for your explanation! It's very good to know latest Sat4j provides an > optional way to reduce memory footprint. > > I indeed created a test case for investigating it. The test data makes the > patch is large, you can download it[1], then apply it to p2's source code. > > A new test case 'MemoryConsuming' is introduced, free to use any profile tool > to observe the memory footprint. > > You also mentioned second way to reduce memory footprint, is it doable way by > client code, such as p2? > > [1] > http://dl.dropbox.com/u/4064358/0001-A-test-case-for-memory-footprint-of-SAT4J.patch.gz > > Mengxin > > > On Sun, Feb 17, 2013 at 9:49 PM, Daniel Le Berre <[email protected]> wrote: > Le 17/02/2013 13:13, Mengxin Zhu a écrit : > > Hi Daniel and Pascal, > > > > I observed SAT4J library need lots of memory for resolving dependencies. > > > > In our case we want to install 677 root IUs, and there are totally > > 17000 IUs in multiple repositories. More than 240M memory was used in > > resolving process. Especially SAT4J library totally created more than > > 210M int[] instances, and held them until the calculating completed. > > Above memory usage data is obtained in VisualVM. > > > > Is it expected behavior to create and hold such large int array for > > calculating the dependencies? If it's unavoidable trade-off for the > > calculating, do we have a formula to estimate how much memory will be > > required when resolving dependencies in certain number of IUs? > > > > Thank you in advance, > > Cheers, > > Mengxin Zhu > > _______________________________________________ > > p2-dev mailing list > > [email protected] > > https://dev.eclipse.org/mailman/listinfo/p2-dev > > Hi Mengxin, > > Sat4j uses arbitrary precision arithmetic (BigInteger) to manage the > weights used to order the UIs. > We use really big weights (e.g. with 30 digits!) to make sure that those > IUs are applied whatever the consequence are (e.g. for patches). > > In some cases, the information can be stored on long, to reduce memory > usage. This is available in Sat4j 2.3.3 for instance. > > sat4j 2.3.3 is currently being considered for inclusion with Kepler. If > you can provide us with a test case, we could check if we could reduce > the memory footprint > using long instead of BigInteger. > > Another way to reduce the memory footprint would be to cache the > BigInteger object used: the objective function reuses many time the same > weight. > Reusing the same BigInteger object each time should significantly > decrease the memory footprint too. > > Cheers, > > Daniel > > _______________________________________________ > p2-dev mailing list > [email protected] > https://dev.eclipse.org/mailman/listinfo/p2-dev > > _______________________________________________ > p2-dev mailing list > [email protected] > https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________ p2-dev mailing list [email protected] https://dev.eclipse.org/mailman/listinfo/p2-dev
