Bug#329743: uses too much memory

2005-09-23 Thread Joey Hess
Package: cdebconf
Severity: normal

In d-i in a lowmem mode install (no translations), cdebconf uses 5 mb of
ram. With partman running the next largest memory usage, by
parted_server is only 1 mb (numbers for mipsel arch). This is largely
because it keeps all the question and template info in memory, which is
very wasteful.

We need a db backend that either stores template/question data in a form
that can be memapped, or in a way that makes it easy to load in stuff
on a per-question (or per-package) basis.

-- 
see shy jo


signature.asc
Description: Digital signature


Bug#329743: uses too much memory

2005-09-23 Thread sferriol

Joey Hess a écrit :

Package: cdebconf
Severity: normal

In d-i in a lowmem mode install (no translations), cdebconf uses 5 mb of
ram. With partman running the next largest memory usage, by
parted_server is only 1 mb (numbers for mipsel arch). This is largely
because it keeps all the question and template info in memory, which is
very wasteful.


there are some problems in lowmem:
 - there is no trimtemplates on /var/lib/cdebconf/templates.dat
 - trimtemplates do not remove extended-descriptions and indices (i'm 
working on it)

 - unable to boot netboot with 24Mb because initrd is too big

i'm testing new features:
 - add udpkg --remove option to remove old configured packages during 
installation steps. So it's a sort of installation 'in one shot', you 
can not go back to change the language for example.


 - i create two directories in main-menu: main-menu-pre.d and 
main-menu-post.d. And you can put scripts executed before and/or after a 
menu is configured. i think it will be useful for lowmem. this avoid to 
create lots of lowmem micro udebs between two menus.


sylvain


smime.p7s
Description: S/MIME Cryptographic Signature


Bug#329743: uses too much memory

2005-09-23 Thread Colin Watson
On Fri, Sep 23, 2005 at 08:31:43AM +0200, Joey Hess wrote:
 In d-i in a lowmem mode install (no translations), cdebconf uses 5 mb of
 ram. With partman running the next largest memory usage, by
 parted_server is only 1 mb (numbers for mipsel arch). This is largely
 because it keeps all the question and template info in memory, which is
 very wasteful.
 
 We need a db backend that either stores template/question data in a form
 that can be memapped, or in a way that makes it easy to load in stuff
 on a per-question (or per-package) basis.

Making it mmap()able isn't just a matter of a new db backend (there's no
real reason that rfc822db couldn't be made to do that). The problem is
more fundamental, because this structure in the core isn't amenable to
mmap()ing:

struct template_l10n_fields
{
char *language;
char *defaultval;
char *choices;
char *indices;
char *description;
char *extended_description;
struct template_l10n_fields *next;
};

My thought is that template_l10n_fields instances probably make up the
bulk of cdebconf's memory usage, and there are only a few functions that
access them directly. The set family are only called from the DATA
command implementation and from debconf-loadtemplate (the former we need
to save in cdebconf memory, but the latter can be loaded on-demand), and
the get family can just load the l10n fields on demand each time.

Making this reasonably efficient, then, would probably just require
writing a variant of rfc822_parse_stanza() which read from memory rather
than a file, and storing pointers in rfc822db to the start of the stanza
for each template in the mmap()ed templatedb, invalidating the pointers
where necessary.

I'll keep looking at all of this.

Cheers,

-- 
Colin Watson   [EMAIL PROTECTED]


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]