Re: [Libhugetlbfs-devel] [PATCH 0/4] [hugetlb] Dynamic huge page pool resizing
Adam Litke wrote: > How it works > > > Upon depletion of the hugetlb pool, rather than reporting an error > immediately, > first try and allocate the needed huge pages directly from the buddy > allocator. > Care must be taken to avoid unbounded growth of the hugetlb pool, so the > hugetlb filesystem quota is used to limit overall pool size. > If I understand hugetlb correctly, there is no accounting of hugepages to the RSS of any process. Since the pool will no longer be static, should we also consider changes to the accounting of hugepages? -- Warm Regards, Balbir Singh Linux Technology Center IBM, ISTL - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 0/4] [hugetlb] Dynamic huge page pool resizing
On Tue, 2007-09-25 at 16:52 +0530, Balbir Singh wrote: > Adam Litke wrote: > > How it works > > > > > > Upon depletion of the hugetlb pool, rather than reporting an error > > immediately, > > first try and allocate the needed huge pages directly from the buddy > > allocator. > > Care must be taken to avoid unbounded growth of the hugetlb pool, so the > > hugetlb filesystem quota is used to limit overall pool size. > > > > If I understand hugetlb correctly, there is no accounting of hugepages > to the RSS of any process. Since the pool will no longer be static, > should we also consider changes to the accounting of hugepages? You're right: there is no accounting of huge pages against a process. This is also the case for the statically allocated pool so this particular issue exists unconditionally. There are several things missing: RSS accounting, counting huge pages towards locked_vm limits, etc... The plan is to address these separately and to fix them all at once. In the absence of traditional per-process huge page accounting, the kernel has provided an alternate means for restricting a process' access to the global hugetlb pool: filesystem permissions and quotas. It's not ideal, but with this patch series, the filesystem permissions and quotas remain the effective mechanism for restricting pool growth and consumption by processes. -- Adam Litke - (agl at us.ibm.com) IBM Linux Technology Center - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 0/4] [hugetlb] Dynamic huge page pool resizing
Adam Litke wrote: > On Tue, 2007-09-25 at 16:52 +0530, Balbir Singh wrote: >> Adam Litke wrote: >>> How it works >>> >>> >>> Upon depletion of the hugetlb pool, rather than reporting an error >>> immediately, >>> first try and allocate the needed huge pages directly from the buddy >>> allocator. >>> Care must be taken to avoid unbounded growth of the hugetlb pool, so the >>> hugetlb filesystem quota is used to limit overall pool size. >>> >> If I understand hugetlb correctly, there is no accounting of hugepages >> to the RSS of any process. Since the pool will no longer be static, >> should we also consider changes to the accounting of hugepages? > > You're right: there is no accounting of huge pages against a process. > This is also the case for the statically allocated pool so this > particular issue exists unconditionally. There are several things > missing: RSS accounting, counting huge pages towards locked_vm limits, > etc... The plan is to address these separately and to fix them all at > once. > I am interested in the accounting and control of hugepages as an extension to the current memory controller, we can of-course do this incrementally. > In the absence of traditional per-process huge page accounting, the > kernel has provided an alternate means for restricting a process' access > to the global hugetlb pool: filesystem permissions and quotas. It's not > ideal, but with this patch series, the filesystem permissions and quotas > remain the effective mechanism for restricting pool growth and > consumption by processes. > OK, thats what I thought. Thanks for sharing your plans -- Warm Regards, Balbir Singh Linux Technology Center IBM, ISTL - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
