On 3/27/23 01:45, Jonathan Matthew wrote:
> On systems where we pull in around 100k users from ldap, ypldap uses a
> fair bit of memory (over 300MB peak) moving data from the ldapclient process
> to the main process.
>
> The ldapclient process sends each user and group record to the parent process
> in instances of struct idm_req, which includes a 1kB buffer for the user/group
> details. It currently sends the full struct through imsg, but only sending
> the used portion of the 1kB buffer reduces peak memory usage to around 100MB,
> and it turns out it's pretty easy, as in the diff below.
>
> ok?
tried it on my system and it didn't burn
ok aisha
>
> Index: ldapclient.c
> ===================================================================
> RCS file: /cvs/src/usr.sbin/ypldap/ldapclient.c,v
> retrieving revision 1.46
> diff -u -p -r1.46 ldapclient.c
> --- ldapclient.c 13 Oct 2022 04:55:33 -0000 1.46
> +++ ldapclient.c 27 Mar 2023 04:19:53 -0000
> @@ -567,7 +567,8 @@ client_search_idm(struct env *env, struc
>
> if (client_build_req(idm, &ir, m, min_attr, max_attr)
> == 0)
> imsg_compose_event(env->sc_iev, type, 0, 0, -1,
> - &ir, sizeof(ir));
> + &ir, sizeof(ir.ir_key) +
> + strlen(ir.ir_line) + 1);
>
> aldap_freemsg(m);
> }
> Index: ypldap.c
> ===================================================================
> RCS file: /cvs/src/usr.sbin/ypldap/ypldap.c,v
> retrieving revision 1.23
> diff -u -p -r1.23 ypldap.c
> --- ypldap.c 22 Aug 2022 08:02:02 -0000 1.23
> +++ ypldap.c 27 Mar 2023 04:19:53 -0000
> @@ -392,7 +392,7 @@ main_dispatch_client(int fd, short event
> if (env->update_trashed)
> break;
>
> - (void)memcpy(&ir, imsg.data, sizeof(ir));
> + (void)memcpy(&ir, imsg.data, n - IMSG_HEADER_SIZE);
> if ((ue = calloc(1, sizeof(*ue))) == NULL ||
> (ue->ue_line = strdup(ir.ir_line)) == NULL) {
> /*
> @@ -418,7 +418,7 @@ main_dispatch_client(int fd, short event
> if (env->update_trashed)
> break;
>
> - (void)memcpy(&ir, imsg.data, sizeof(ir));
> + (void)memcpy(&ir, imsg.data, n - IMSG_HEADER_SIZE);
> if ((ge = calloc(1, sizeof(*ge))) == NULL ||
> (ge->ge_line = strdup(ir.ir_line)) == NULL) {
> /*
>
