>On Sat, Feb 27, 2010 at 12:48 PM, <casper....@sun.com> wrote:
>>
>>>well the whole install took 2 hours
>>>for 1 pkg !!
>>>it seems that disk read/write here is the bottleneck (?).
>>>iostat -D 2 shows my hd busy at around 100%
>>>iosnoop reports a lot of access by pkg :
>>
>> How much memory? =A0It is possible that you're paging to death.
>
>
>
>Hi Casper,
>
>I am subscribed to pkg and caiman.
>
>Maybe his system is swapping to death.
>But why??? If you compare memory- and CPU- usage of IPS to those
>values of *every* other system on the market, it is unbelievable at
>the first look. ^Horrific^ - to find the right term.
Yes, that is true.
I'm not sure why it needs so much memory; it is possible that using
python contributes to the memory use. Both perl and python of wonderful
primitives but you pay for how the primitives are implemented: they don't
give you a choice.
A long time ago, we re-wrote the "contents" file database in a (SQL)
database; that was a bad idea for several reasons: you have again no choice
on how the data is stored and this again added to huge memory pressure
and apart from other issues such as not properly sorting the contents file,
it made installing much slower and not faster. The contents file was then
around 15-20MB, the database used 512MB or more.
It took me around 10 days to re-write the svr4 package tools using some
form of home-grown database: but this database only needed indexing on one key,
the pathname, to me it was important to keep the memory use down as I
wanted to run it on small systems (upgrade with a shrunk miniroot on a
256MB system)
I used avl trees and stored the following:
typedef struct pkgentry {
char *line; /* The contents line for the file */
avl_node_t avl; /* The avl header */
int pkgoff; /* Where the packages live; start with SP */
int pathlen; /* The length of the pathname */
int len; /* Length of the line (incl NUL) */
} pkgentry_t;
I keep the whole line and I don't actually parse it and most of the work is
still done by the client programs.
The contents file was at the end of SXDE around 40MB(!); the daemon needs
only 60MB.
I don't know how much data the IPS needs to keep in memory, but I'm sure
that the penalty is much larger than the 50%.
It gets worse when you want to install/upgrade zones: (memory + cpu)* #zones.
You can scale if your operation is O(1) or the constant is small.
Casper
_______________________________________________
opensolaris-discuss mailing list
opensolaris-discuss@opensolaris.org
