I wrote a set of python bindings for libcgroup v1 about 5 years ago, and abandoned it because I couldn't find anybody interested in using cgroups at the time.
On Mon, Aug 31, 2020 at 11:40 AM Brock Palen <bro...@umich.edu> wrote: > > Thanks, > > yeah I was looking for an API that would take into consideration most cases, > like I find with hwloc-bind --get where I can find the number the process > has access to. Wether is cgroups, other sorts of affinity setting etc. > > Brock Palen > IG: brockpalen1984 > www.umich.edu/~brockp > Director Advanced Research Computing - TS > bro...@umich.edu > (734)936-1985 > > > On Mon, Aug 31, 2020 at 12:37 PM Guy Streeter <guy.stree...@gmail.com> wrote: >> >> I forgot that the cpuset value is still available in cgroups v2. You >> would want the cpuset.cpus.effective value. >> More information is available here: >> https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html >> >> On Mon, Aug 31, 2020 at 11:19 AM Guy Streeter <guy.stree...@gmail.com> wrote: >> > >> > As I said, cgroups doesn't limit the group to a number of cores, it >> > limits processing time, either as an absolute amount or as a share of >> > what is available. >> > A docker process can be restricted to a set of cores, but that is done >> > with cpu affinity, not cgroups. >> > >> > You could try to figure out an equivalency. For instance if you are >> > using cpu.shares to limit the cgroups, then figure the ratio of a >> > cgroup's share to the shares of all the cgroups at that level, and >> > apply that ratio to the number of available cores to get an estimated >> > number of threads you should start. >> > >> > On Mon, Aug 31, 2020 at 10:40 AM Brock Palen <bro...@umich.edu> wrote: >> > > >> > > Sorry if wasn't clear, I'm trying to find out what is available to my >> > > process before it starts up threads. If the user is jailed in a cgroup >> > > (docker, slurm, other) and the program tries to start 36 threads, when >> > > it only has access to 4 cores, it's probably not a huge deal, but not >> > > desirable. >> > > >> > > I do allow the user to specify number of threads, but would like to >> > > automate it for least astonishment. >> > > >> > > Brock Palen >> > > IG: brockpalen1984 >> > > www.umich.edu/~brockp >> > > Director Advanced Research Computing - TS >> > > bro...@umich.edu >> > > (734)936-1985 >> > > >> > > >> > > On Mon, Aug 31, 2020 at 11:34 AM Guy Streeter <guy.stree...@gmail.com> >> > > wrote: >> > >> >> > >> My very basic understanding of cgroups is that it can be used to limit >> > >> cpu processing time for a group, and to ensure fair distribution of >> > >> processing time within the group, but I don't know of a way to use >> > >> cgroups to limit the number of CPUs available to a cgroup. >> > >> >> > >> On Mon, Aug 31, 2020 at 8:56 AM Brock Palen <bro...@umich.edu> wrote: >> > >> > >> > >> > Hello, >> > >> > >> > >> > I have a small utility, it is currently using >> > >> > multiprocess.cpu_count() >> > >> > Which currently ignores cgroups etc. >> > >> > >> > >> > I see https://gitlab.com/guystreeter/python-hwloc >> > >> > But appears stale, >> > >> > >> > >> > How would you detect number of threads that are safe to start in a >> > >> > cgroup from Python3 ? >> > >> > >> > >> > Thanks! >> > >> > >> > >> > Brock Palen >> > >> > IG: brockpalen1984 >> > >> > www.umich.edu/~brockp >> > >> > Director Advanced Research Computing - TS >> > >> > bro...@umich.edu >> > >> > (734)936-1985 >> > >> > _______________________________________________ >> > >> > hwloc-users mailing list >> > >> > hwloc-users@lists.open-mpi.org >> > >> > https://lists.open-mpi.org/mailman/listinfo/hwloc-users >> > >> _______________________________________________ >> > >> hwloc-users mailing list >> > >> hwloc-users@lists.open-mpi.org >> > >> https://lists.open-mpi.org/mailman/listinfo/hwloc-users >> > > >> > > _______________________________________________ >> > > hwloc-users mailing list >> > > hwloc-users@lists.open-mpi.org >> > > https://lists.open-mpi.org/mailman/listinfo/hwloc-users >> _______________________________________________ >> hwloc-users mailing list >> hwloc-users@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/hwloc-users > > _______________________________________________ > hwloc-users mailing list > hwloc-users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/hwloc-users _______________________________________________ hwloc-users mailing list hwloc-users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-users
