[DRBD-user] linstor-proxmox v7.0.1
Dear DRBD on PVE users, this is version 7.0.1 of the linstor-proxmox plugin. The RC phase was exceptionally long, which was a result of public holidays and me being on vacation. There have not been any reported bugs, the original announcement for your convenience: - it fixes a performance regression introduced with 7.0.0 - it requires LINSTOR 1.23.0 - all users of 7.0.0 should then upgrade to 7.0.1 The detailed version: Long story short, PVE calls status() freaking often. Depending on if you select the storage in the GUI, this can happen every second, otherwise like every 10s. To accommodate for that Proxmox's status() API has a "cache" object that is passed in for every storage type. Let's say you have 4 'drbd' storages (as in storage.cfg), then for the first call we get an empty cache which we can fill, and for the next 3 we can use that. That helps, but - we can not make use of that for more than one iteration (remember, PVE calls status an awful lot), that is why we introduced a file based persistent cache to keep some information longer than 10 seconds. - things get even worse if you have a distributed system where then every PVE node hammers the LINSTOR controller with (the same) questions all the time. Do the math, the LINSTOR controller gets too many "what is the status of my storage" calls very very quickly even in setups with a handful of nodes. Recently we improved how we display storage information, but that also rendered the built in PVE cache useless as we needed to query every RG individually. That was known, I thought we could get away with it. We did not. How was it fixed: - we can now query the storage information for all RGs once. That helps to again use the builtin cache more effectively as we again need 1 call for every iteration. - The new LINSTOR API for querying all RGs at once also got a cache parameter which also helps a lot because it avoids the pressure on the controller that resulted from the distributed nature of LINSTOR/DRBD. The http queries were/are not the problem, the problem is that the controller then needs to get the state of all satellites. Regards, rck GIT: https://github.com/LINBIT/linstor-proxmox/commit/e72f262ca0db1f875f996c22da322c28a39d7831 TGZ: https://pkg.linbit.com//downloads/connectors/linstor-proxmox-7.0.1.tar.gz signature.asc Description: PGP signature ___ Star us on GITHUB: https://github.com/LINBIT drbd-user mailing list drbd-user@lists.linbit.com https://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] linstor-proxmox v7.0.1-rc.1
On Thu, May 25, 2023 at 01:15:55PM +0200, Roland Kammerer wrote: > DEB: > https://packages.linbit.com/staging/public/dists/proxmox-7/drbd-9/pool/linstor-proxmox_7.0.1~rc.1-1_all.deb please use this link (note the switched "staging" and "public"): https://packages.linbit.com/public/staging/dists/proxmox-7/drbd-9/pool/linstor-proxmox_7.0.1~rc.1-1_all.deb Regards, rck ___ Star us on GITHUB: https://github.com/LINBIT drbd-user mailing list drbd-user@lists.linbit.com https://lists.linbit.com/mailman/listinfo/drbd-user
[DRBD-user] linstor-proxmox v7.0.1-rc.1
Dear DRBD on PVE users, this is the first RC for the upcoming 7.0.1 version. - it fixes a performance regression introduced with 7.0.0 - it requires LINSTOR 1.23.0 - all users of 7.0.0 should then upgrade to 7.0.1 The detailed version: Long story short, PVE calls status() freaking often. Depending on if you select the storage in the GUI, this can happen every second, otherwise like every 10s. To accommodate for that Proxmox's status() API has a "cache" object that is passed in for every storage type. Let's say you have 4 'drbd' storages (as in storage.cfg), then for the first call we get an empty cache which we can fill, and for the next 3 we can use that. That helps, but - we can not make use of that for more than one iteration (remember, PVE calls status an awful lot), that is why we introduced a file based persistent cache to keep some information longer than 10 seconds. - things get even worse if you have a distributed system where then every PVE node hammers the LINSTOR controller with (the same) questions all the time. Do the math, the LINSTOR controller gets too many "what is the status of my storage" calls very very quickly even in setups with a handful of nodes. Recently we improved how we display storage information, but that also rendered the built in PVE cache useless as we needed to query every RG individually. That was known, I thought we could get away with it. We did not. How was it fixed: - we can now query the storage information for all RGs once. That helps to again use the builtin cache more effectively as we again need 1 call for every iteration. - The new LINSTOR API for querying all RGs at once also got a cache parameter which also helps a lot because it avoids the pressure on the controller that resulted from the distributed nature of LINSTOR/DRBD. The http queries were/are not the problem, the problem is that the controller then needs to get the state of all satellites. Please test! If I don't get any negative feedback (positive one is welcome as well), I will release the final version in about a week from now. Regards, rck GIT: https://github.com/LINBIT/linstor-proxmox/commit/6541b7d624b543c92107d4e562ac7f9e555a3fad TGZ: https://pkg.linbit.com//downloads/connectors/linstor-proxmox-7.0.1-rc.1.tar.gz DEB: https://packages.linbit.com/staging/public/dists/proxmox-7/drbd-9/pool/linstor-proxmox_7.0.1~rc.1-1_all.deb signature.asc Description: PGP signature ___ Star us on GITHUB: https://github.com/LINBIT drbd-user mailing list drbd-user@lists.linbit.com https://lists.linbit.com/mailman/listinfo/drbd-user