Hi Gerd, thanks for the feedback.
Am 23.04.20 um 13:18 schrieb Gerd Hoffmann: > On Wed, Apr 22, 2020 at 04:40:55PM +0200, Thomas Zimmermann wrote: >> With limited VRAM available, fragmentation can lead to OOM errors. >> Alternating between bottom-up and top-down placement keeps BOs near the >> ends of the VRAM and the available pages consecutively near the middle. >> >> A real-world example with 16 MiB of VRAM is shown below. >> >> > cat /sys/kernel/debug/dri/0/vram-mm >> 0x0000000000000000-0x000000000000057f: 1407: free >> 0x000000000000057f-0x0000000000000b5b: 1500: used >> 0x0000000000000b5b-0x0000000000000ff0: 1173: free >> >> The first free area was the location of the fbdev framebuffer. The used >> area is Weston's current framebuffer of 1500 pages. Weston now cannot >> do a pageflip to another 1500 page-wide framebuffer, even though enough >> pages are available. The patch resolves this problem to >> >> > cat /sys/kernel/debug/dri/0/vram-mm >> 0x0000000000000000-0x00000000000005dc: 1500: used >> 0x00000000000005dc-0x0000000000000a14: 1080: free >> 0x0000000000000a14-0x0000000000000ff0: 1500: used >> >> with both of Weston's framebuffers located near the ends of the VRAM >> memory. > > I don't think it is that simple. > > First: How will that interact with cursor bo allocations? IIRC the > strategy for them is to allocate top-down, for similar reasons (avoid > small cursor bo allocs fragment vram memory). In ast, 2 cursor BOs are allocated during driver initialization and kept permanently at the vram's top end. I don't know about other drivers. But cursor BOs are small, so they don't make much of a difference. What is needed is space for 2 primary framebuffers during pageflips, with one of them pinned. The other framebuffer can be located anywhere. > > Second: I think ttm will move bo's from vram to system only on memory > pressure. So you can still end up with fragmented memory. To make the > scheme with one fb @ top and one @ bottom work reliable you have to be > more aggressive on pushing out framebuffers. I'm the process of converting mgag200 to atomic modesetting. The given example is what I observed. I'm not claiming that the placement scheme is perfect, but it is required to get mgag200 working with atomic modesetting's pageflip logic. So we're solving a real problem here. The bug comes from Weston's allocation strategy. Looking at the debug output: >> 0x0000000000000000-0x000000000000057f: 1407: free This was fbdev's framebuffer with 1600x900@32bpp >> 0x000000000000057f-0x0000000000000b5b: 1500: used This is Weston's framebuffer also with 1600x900@32bpp. But Weston allocates an additional, unused 60 scanlines. That is to render with tiles of 64x64px, I suppose. fbdev doesn't do that, hence Weston's second framebuffer doesn't fit into the free location of the fbdev framebuffer. The other drivers with a small amount of vram are also prone to this problem. They simply have not yet encountered such a setup. > > Third: I'd suggest make topdown allocations depending on current state > instead of simply alternating, i.e. if there is a pinned framebuffer @ > offset 0, then go for top-down. That's what the current patch does. If the last pin was at the bottom, the next goes to the top. And then the other way around. Without alternating between both end of vram, the problem would occur again when fragmentation happens near the top end. > > I also think using this scheme should be optional. In the simplest case > we can allow drivers opt-in. Or we try do to something clever > automatically: using the strategy only for framebuffers larger than 1/4 > or 1/3 of total vram memory (which makes alloc failures due to > fragmentation much more likely). I'd like to not change behavior automatically, but we can surely make this optional. Looking again at the vram helpers, this functionality could be implemented in drm_gem_vram_plane_helper_prepare_fb(). Drivers with other placement strategies could implement their own helper for prepare_fb. Best regards Thomas > > cheers, > Gerd > -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
signature.asc
Description: OpenPGP digital signature
_______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
