Re: [dev] size: cppuhelper containers ... ~700k

[Stephan Bergmann]

Michael Meeks wrote:

Hi there,

So - in my ongoing campaign to shrink memory usage, it seems that the
cppuhelper classes use the (uber-stupid) stl::hash_map code that insists
on allocating a staggering amount of memory for an empty hash (as
previously discussed).

Dumping counts of OMultiTypeInterfaceContainerHelper and it's
Int32 variant extant post writer startup I see:

IfContainers 507
IfContainersInt32 388

I also notice that wrt. the contents of the m_pMap we normally
have very few elements:

count   num elements
282 1
   2841 2
   1569 3
 38 4

That is for IfContainers, and -incredibly- none of the
IfContainersInt32 actually have any contents at all ;-) [ I have to load
an impress document to get even a few of them with 1-2 items in them ].

A hash_map's default size is 193 elements, so we waste:

(507+388) * (193 - 4) * 4-bytes-per-ptr = 676Kb

The patch:
http://go-oo.org/ooo-build/patches/src680/size-cppuhelper.diff

switches from a hash to a vector, which given the size saving, and
small data sets, will almost certainly accelerate things however you
look at it, and saves a chunk of RAM.

The saving is around 700k by not allocating the (~all empty) Int32
container contents too, numbers from pmap (DIRTY):

Before: 19264 19256 + 19268 + 3 / p : 19262
After:  18552 18552 + 18544 + 3 / p : 18549
Saving: - p :   713(k)

"using STL hash may unexpectedly bloat your world" :-]


Be specific in your finger pointing.  ;)  There is no STL, and there is 
no STL hash.  There is C++ and there is STLport hash_map.  I vaguely 
remember you once tried to attack this space-waste problem of STLport 
hash_map more generally by modifying the STLport used by OOo; what has 
become of that?



This does of course change some cross-module API - wrt. the inlines in
interfacecontainer.hxx - but I believe this is no longer a big concern
(?).


As far as I see, the patch attached to 
 only changes 
the all-inline class template 
cppu::OMultiTypeInterfaceContainerHelperVar, so that should be OK.


-Stephan


Thanks,

Michael.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [dev] size: cppuhelper containers ... ~700k

[Thorsten Behrens]

Hi Stephan,

you asked:
> What is o3l?
>
o3tl, I guess - a module in OOo.

Cheers,

-- Thorsten

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [dev] size: cppuhelper containers ... ~700k

[Caolan McNamara]

On Wed, 2006-12-20 at 10:13 +0100, Stephan Bergmann wrote:
> Caolan McNamara wrote:
> > On Tue, 2006-12-19 at 16:36 +, Michael Meeks wrote:
> >> Hi there,
> >>
> >>So - in my ongoing campaign to shrink memory usage, it seems that the
> >> cppuhelper classes use the (uber-stupid) stl::hash_map code that insists
> >> on allocating a staggering amount of memory for an empty hash (as
> >> previously discussed).
> > 
> > Maybe it is worth jamming a TR1 alike  implementation
> > hash_map replacement with a more sensible initial allocation system into
> > o3l and convert to using that in one fell swoop ? 
> 
> What is o3l?  (Anyway, yes, replacing STLport's hash_map with something 
> more standard and future oriented looks like a good move to me---modulo 
> any potential compatibility issues, of course.)

Sorry, typo. I meant into the o3tl module which I assume expands to
"OpenOffice.org Template Library", which looked like a natural place for
something like this. 

But digging around a bit further it won't make much of a difference
really unless we implement our own better one because I see that
STLport5.1.0 has an an unordered_map implementation now, upgrading to
use that would make unordered_map available. But unordered_map still
backs onto the same hashtable that the STLport uses for hash_map and
both hash_map and unordered_map share the same default initial sizes of
100, and any explicit size setting for the hash_map will be downwardly
limited to the smallest prime allowed in hashtable.c

The good news though is that hdu's patch for reducing the smallest
allowed explicit initial size is now in stlport5

i.e. 

#  define __PRIME_LIST_BODY { \
  7ul,  23ul, \

instead of 

# define __PRIME_LIST_BODY { \
  53ul, 97ul, 193ul,   389ul,   769ul,  \

so maybe the least effort route after al is to go with hdu's two patches
to our stlport for the above upstreamed change and for the empty
hashtable optimization.

C.

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [dev] size: cppuhelper containers ... ~700k

[Stephan Bergmann]

Caolan McNamara wrote:

On Tue, 2006-12-19 at 16:36 +, Michael Meeks wrote:

Hi there,

So - in my ongoing campaign to shrink memory usage, it seems that the
cppuhelper classes use the (uber-stupid) stl::hash_map code that insists
on allocating a staggering amount of memory for an empty hash (as
previously discussed).


Maybe it is worth jamming a TR1 alike  implementation
hash_map replacement with a more sensible initial allocation system into
o3l and convert to using that in one fell swoop ? 


What is o3l?  (Anyway, yes, replacing STLport's hash_map with something 
more standard and future oriented looks like a good move to me---modulo 
any potential compatibility issues, of course.)


-Stephan


C.

(http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2003/n1456.html)


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [dev] size: cppuhelper containers ... ~700k

[Caolan McNamara]

On Tue, 2006-12-19 at 16:36 +, Michael Meeks wrote:
> Hi there,
> 
>   So - in my ongoing campaign to shrink memory usage, it seems that the
> cppuhelper classes use the (uber-stupid) stl::hash_map code that insists
> on allocating a staggering amount of memory for an empty hash (as
> previously discussed).

Maybe it is worth jamming a TR1 alike  implementation
hash_map replacement with a more sensible initial allocation system into
o3l and convert to using that in one fell swoop ? 

C.

(http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2003/n1456.html)

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]