On Tue, Mar 15, 2011 at 9:07 AM, Robert Fekete <robert.fek...@linaro.org>wrote:
> On 8 March 2011 20:23, Laurent Pinchart > <laurent.pinch...@ideasonboard.com> wrote: > > Hi Andy, > > > > On Tuesday 08 March 2011 20:12:45 Andy Walls wrote: > >> On Tue, 2011-03-08 at 16:52 +0100, Laurent Pinchart wrote: > >> > >> [snip] > >> > >> > > > It really shouldn't be that hard to get everyone involved together > >> > > > and settle on a single solution (either based on an existing > >> > > > proposal or create a 'the best of' vendor-neutral solution). > >> > > > >> > > "Single" might be making the problem impossibly hard to solve well. > >> > > One-size-fits-all solutions have a tendency to fall short on meeting > >> > > someone's critical requirement. I will agree that "less than n", > for > >> > > some small n, is certainly desirable. > >> > > > >> > > The memory allocators and managers are ideally satisfying the > >> > > requirements imposed by device hardware, what userspace applications > >> > > are expected to do with the buffers, and system performance. (And > >> > > maybe the platform architecture, I/O bus, and dedicated video > memory?) > >> > > >> > In the embedded world, a very common use case is to capture video data > >> > from an ISP (V4L2+MC), process it in a DSP (V4L2+M2M, tidspbridge, > ...) > >> > and display it on the GPU (OpenGL/ES). We need to be able to share a > >> > data buffer between the ISP and the DSP, and another buffer between > the > >> > DSP and the GPU. If processing is not required, sharing a data buffer > >> > between the ISP and the GPU is required. Achieving zero-copy requires > a > >> > single memory management solution used by the ISP, the DSP and the > GPU. > >> > >> Ah. I guess I misunderstood what was meant by "memory provider" to some > >> extent. > >> > >> So what I read is a common way of providing in kernel persistent buffers > >> (buffer objects? buffer entities?) for drivers and userspace > >> applications to pass around by reference (no copies). Userspace may or > >> may not want to see the contents of the buffer objects. > > > > Exactly. How that memory is allocated in irrelevant here, and we can have > > several different allocators as long as the buffer objects can be managed > > through a single API. That API will probably have to expose buffer > properties > > related to allocation, in order for all components in the system to > verify > > that the buffers are suitable for their needs, but the allocation process > > itself is irrelevant. > > > >> So I understand now why a single solution is desirable. > > > > Exactly, > > It is important to know that there are 3 topics of discussion which > all are a separate topic of its own: > > 1. The actual memory allocator > 2. In-kernel API > 3. Userland API > > Explained: > 1. This is how you acquire the actual physical or virtual memory, > defrag, swap, etc. This can be enhanced by CMA, hotswap, memory > regions or whatever and the main topic for a system wide memory > allocator does not deal much with how this is done. > 2. In-kernel API is important from a device driver point of view in > order to resolve buffers, pin memory when used(enable defrag when > unpinned) > 3. Userland API deals with alloc/free, import/export(IPC), security, > and set-domain capabilities among others and is meant to pass buffers > between processes in userland and enable no-copy data paths. > > We need to resolve 2. and 3. > > GEM/TTM is mentioned in this thread and there is an overlap of what is > happening within DRM/DRI/GEM/TTM/KMS and V4L2. The whole idea behind > DRM is to have one device driver for everything (well at least 2D/3D, > video codecs, display output/composition), while on a SoC all this is > on several drivers/IP's. A V4L2 device cannot resolve a GEM handle. > GEM only lives inside one DRM device (AFAIK). GEM is also mainly for > "dedicated memory-less" graphics cards while TTM mainly targets > advanced Graphics Card with dedicated memory. From a SoC point of view > DRM looks very "fluffy" and not quite slimmed for an embedded device, > and you cannot get GEM/TTM without bringing in all of DRM/DRI. KMS on > the other hand is very attractive as a framebuffer device replacer. It > is not an easy task to decide on a multimedia user interface for a SoC > vendor. > > Uniting the frameworks within the kernel will likely fail(too big of a > task) but a common system wide memory manager would for sure make life > easier enabling the possibility to pass buffers between drivers(and > user-land as well). In order for No-copy to work on a system level the > general multimedia infrastructure in User-land (i.e. > Gstreamer/X11/wayland/stagefright/flingers/etc) must also be aware of > this memory manager and manage handles accordingly. This > infrastructure in user-land puts the requirements on the User land API > (1.). > > I know that STE and ARM has a vision to have a hwmem/ump alike API and > that Linaro is one place to resolve this. As Jesse Barker mentioned > earlier Linaro has work ongoing on this topic > ( > https://wiki.linaro.org/WorkingGroups/Middleware/Graphics/Projects/UnifiedMemoryManagement > ) > and a V4L2 brainstorming meeting in Warsaw will likely bring this up > as well. And Gstreamer is also looking at this from a user-land point > of view. > > ARM, STE seems to agree on this, V4L2 maestros seems to agree, > GStreamer as well(I believe), > How about Samsung(vcm)? TI(cmem)? Freescale? DRI community? Linus? > > Jesse! any progress? > > Robert (et al.), Based upon the requirements in the link Robert (and I) posted, we are looking into what TTM changes would be needed to support those using the current UMP API as a sort of template, as this is something we have currently to hand (linux-linaro-2.6.38 based tree containing UMP kernel and Mali 400 device driver on git.linaro.org). We believe it would be similar to a mapping onto the HWMEM API, but if there are doubts there, I will happily add some additional work items to validate that. I will also be adding the V4L2 requirement to resolve a MM handle from a virtual address (we heard that one from Samsung in a separate thread). On the KMS front, I have suggested to Scott (Linaro landing teams lead) that we push in that direction already and have proposed that we allocate resources in the graphics working group to support that (of course, contributions have to come from the SoC vendors to enable that work fully; thus the landing team involvement). I would like to add an additional request for comments on the contents of our unified memory management position to make sure that it addresses everyone's concerns. Anyone in Linaro can edit it directly, but I will happily make "proxy" edits based upon email and IRC requests. cheers, Jesse
_______________________________________________ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
