On Wed, Sep 16, 2026 at 1:41 AM Christian König <[email protected]> wrote: > > On 9/15/26 19:26, T.J. Mercier wrote: > > On Tue, Sep 15, 2026 at 9:26 AM T.J. Mercier <[email protected]> wrote: > >> > >> On Tue, Sep 15, 2026 at 8:44 AM Christian König > >> <[email protected]> wrote: > ...>> Even on arm64 phones free 2M pages aren't likely to be available for > >> very long after boot, but lately we have been doing more proactive > >> reclaim triggered by userspace before launching workflows that desire > >> large dma-buf allocations (and also at other times during application > >> lifecycle in general). That somewhat increases the likelihood that > >> high order pages will be available. I agree there's no guarantee we'll > >> get any, but without __GFP_RECLAIM the attempt is pretty cheap and the > >> payoff can be pretty benficial which Davidlohr's allocation and > >> mapping measurements demonstrate. > > > > Oh I forgot to add that this behavior is similar to how slab > > allocations are done for slabs requiring more than an order 0 page per > > slab. A large optimal order is attempted first with ~__GFP_RECLAIM, > > but if that fails a fallback to a smaller min order is done. When the > > optimal order allocation attempt fails, the cost is just taking the > > zone lock for a quick peek at the free lists. > > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/slub.c?h=v7.2#n3373 > > Well from a lot of experience with AMDs ROCm stack and Intels XE driver I > think that this is a really bad idea. > > The background is that there are a lot of operation which can fragment memory > to a point where only 4k pages are available, e.g. for example storage I/O > can do that trivially (the kernel compile is a really good test case for > that). > > Now acceleration HW usually has a very specific sweet spot where it works > best. AMDs GPUs for example can work with 1G, 2M and 4k in the L2 and > everything between 2M..4k in the L1. On Intel it is only 2M and Nvidia has 1G > and 2M (but really don't nail me on that this is just as far as I know). > > So if reclaim of smaller pages into larger ones isn't requested what you end > up with is unstable performance, e.g. if you only run application A > everything is fine, if you switch to application B and then run application A > you see a performance penalty of up to 30% which eventually doesn't go away > over time. > > On the other hand when reclaim is always enabled for multiple different > orders you start to see stuttering when allocations are made because the core > memory management tries to fulfill all those different orders.
Yup I understand the tradeoff you're describing here between allocation latency and performance predictability for GPU/HPC. But for a generic allocator like the system heap with very loose constraints on what memory can be used, this lack of optimism permanently incurs a penalty that might not have been necessary. Users of the system heap don't demand any particular page order they just deal with what they're given. There is no single driver or use case for system_heap users. Opportunistic allocation without reclaim gives the upside when pages are free without stalling when they aren't. > We literally had tons of bug reports regarding those problems and it is one > of the major reasons why we doesn't use DMA-buf heaps for HPC use cases. > Instead drivers even communicate the sweet spot to for example TTMs > allocation backend. This behavior isn't an inherent property of all dma-buf heaps though, it's just how the system heap does it. You could create a different heap that always tries to allocate the sweet spot for a particular use case. > I don't have much of a problem when we say that this is just for Android, we > never do reclaim here, instead reclaim is triggered solely by userspace on > application switch etc... but the general idea of DMA-buf heaps is that it > should work for everybody. I'm not sure these patches were motivated by Android (based on x86 focus in the commit message), but Davidlohr would have to comment on that. That's just where most of my experience comes from and what I can share details about.
