Good thread - thanks everyone for contributing. My 2 cents... CMEM (or something like it) is needed to do virt2phys (and vice-versa) address translation on arbitrary physically contiguous buffers, not just those allocated by CMEM. This is because often the buffers destined for the DSP (and/or other HW like DMA or accelerators) are allocated by device drivers. We don't want to force a memcpy from a video capture buffer into a CMEM buffer - especially considering the size of video buffers (and HD makes things worse). We just want to pass it directly to the DSP/accelerator to start encoding; but they, of course, only work on physical addresses, so we need a reliable virt2phys/phys2virt.
Also, regarding Link doing the lookup - apps do pass buffers across processors that Link doesn't know about. Ptrs to these buffers are often embedded in Link messages, but Link doesn't know the contents of those messages, so it doesn't know anything needs translated. And finally, when considering the single processor ARM architectures like DM355, there is no Link(!), but there is a need to pass these buffers into HW accelerators. In these cases as well, we'll need virt2phys and vice-versa - so we don't want a dependency on Link for these virt2phys/phys2virt services. Chris > -----Original Message----- > From: > [EMAIL PROTECTED] > p.com > [mailto:[EMAIL PROTECTED] > avincidsp.com] On Behalf Of Kevin Hilman > Sent: Friday, October 26, 2007 2:50 PM > To: Tivy, Robert > Cc: davinci-linux-open-source@linux.davincidsp.com > Subject: Re: Translating kernel virtual address to physical address > > Tivy, Robert wrote: > > Diego, > > > > Thankyou very much for your detailed analysis and > suggestions, we very > > much appreciate it. > > > > I haven't been involved in the CMEM world for very long and > therefore > > don't have much insight into its original spec or > development, and I'm > > still learning the internals of Linux, but what you suggest > sounds quite > > reasonable. I understand that the general feeling in the Linux > > community within and outside TI is that CMEM is "not ready for > > primetime", i.e., incorporation into the Linux mainline. > > > > Your analysis of the features supported by CMEM is > accurate. One thing > > you didn't mention is the requirement that the user be able > to obtain > > the physical address of CMEM buffers, for the purpose of > granting them > > to DSP codecs, where the DSP doesn't have an MMU, and for > granting to HW > > (DSP or otherwise) that doesn't sit behind an MMU (such as DMA). > > > > Another point worth mentioning is that CMEM has a history > of being used > > for buffers that were not granted by CMEM. I'm speaking of > the get_phys > > functionality, where users of CMEM will also call CMEM's get_phys > > functionality for buffers that were not obtained by CMEM. > This is not a > > supported use case, but we are almost forced to support this due to > > established legacy usage. > > These are both good examples of what I meant when I said I > thought CMEM > usage has evolved beyond it's original intent/design. > > For DSP communication, I don't believe applications pass buffers > directly to the DSP. They pass through DSPLink. IMO, It should be > DSPLink's job to do any translations necessary using get_user_pages(). > The application should not have to care about whether the DSP > can handle > contiguous buffers or not. > > As far as getting physical addresses of non-CMEM allocated memory. Do > you know much about the legacy use-cases of this? If it is not > supported, a CMEM redesign would be the perfect time to break it. :) > > Hope that helps. And thanks Robert for bringing this up in an open > forum. I'm glad to see TI seeking feedback from the community on this > stuff. > > Best regards, > > Kevin > > > > -------------------------------------------------------------- > ---------- > > *From:* Diego Dompe [mailto:[EMAIL PROTECTED] > > *Sent:* Friday, October 26, 2007 1:19 PM > > *To:* Tivy, Robert > > *Cc:* davinci-linux-open-source@linux.davincidsp.com > > *Subject:* Re: Translating kernel virtual address to > physical address > > > > Robert, > > > > I have tried to use CMEM in recent kernels and found > the problems > > you described, and I was wondering if TI plans to fix > it to make it > > work, or would be interested in re-design the driver to make it > > portable and simpler. I see unlikely that the main tree > accepts the > > driver in the current state. > > > > As far I understand from the CMEM driver available, his > functional > > objective is to create object pools to be used by Codec > Engine. To > > do so the current implementation does as follows: > > - Requires the kernel to be run with a reserved memory > area (leaving > > memory out of kernel control with the mem=XX param). > The location of > > the reserved memory is notified to the driver as parameter. > > - Creates several object pools of different sizes in > the memory area > > (parsed as a module parameters). Those objects should > > be contiguous physical memory areas so they can be > accessed by the > > EDMA engine, or the DSP. > > - The objects are requested using ioctls, and the return value > > (object handler) is the virtual address of the > allocated object into > > the memory area. > > - The mmap function is used along with the object > handler to map the > > address into the user space address space (requiring > the get_phys > > function to re-obtain the physical address that needs to be > > re-mapped). The mapping mode (cached or uncached) is controlled > > trough ioctls as well the special functions to > implement the cache > > coherency. > > > > I think there are several things that could be done in > a different > > way to make the driver simpler. > > > > - Unless I'm missing something the requirement for contiguous > > physical memory pages could be satisfied by the > __get_free_page() > > function (just as the FB driver does, as the VPBE > hardware requires > > also contiguous memory). I'm not sure why is required to take a > > complete block of memory out of kernel control. > > > > - The current driver is implementing his own object > pool handlers: > > the linux kernel provides this functionality already > with the slab > > allocator interface (kmem_cache_create). Using the flags > > SLAB_NO_REAP | SLAB_PANIC and allocating the objects > early on the > > boot process (or forcing to install the driver early > during board > > boot-up) will be enough warranty to create a driver that could > > allocate contiguous memory objects without requiring taking the > > memory out of kernel control. > > > > - The question will be then how to provide this memory > objects to > > the user space without requiring a hack like the current mmap + > > reverse translation of the virtual pointers as object handlers? > > A simple and clean way could be to create a pseudo file system > > (mount it under /dev/ticmem) where a file descriptor is > created for > > every object (/dev/ticmem/buffer4MBX, buffer2MBX, etc). > Providing > > exclusive open operations could be used for allocation > (if you can > > open it, it's yours), getting rid of the ioctls for reservation. > > A way to get rid of the ioctls for selecting cached or > non-cached > > mapping is to use the standard of /dev/mem that if you > mmap a device > > with the O_SYNC flag, then the mapping is not cached. > > > > This was just my 2 cents on how I would like to see the driver > > implemented, and I think that will make it a bit more > appealing to > > be accepted in main tree. Please feel free to comment > or question it. > > > > Diego > > RidgeRun Engineering > > > > On Oct 25, 2007, at 3:30 PM, Tivy, Robert wrote: > > > >> Engineers at TI are looking to get the TI CMEM module into the > >> Linux kernel mainline in the near future. In order to > progress to > >> this goal we need to address some items. One of the items that > >> needs addressing is the CMEM driver get_phys() function. > >> Currently it just decodes page mappings at a low level, using > >> pgd/pmd/pte low-level functions/macros. Kevin Hilman > provided an > >> improved version, one that is much more portable and correct. > >> However, there's a problem with this new version in that it > >> doesn't translate some kernel virtual addresses > correctly, so I'm > >> asking the list for help. > >> > >> CMEM is responsible for allocating contiguous memory > blocks. It > >> is granted a block of physical memory during module insertion > >> time. Typically this block is one not managed by the kernel, > >> i.e., the kernel has been "told" to not touch this memory, by > >> virtue of the "mem=" boot command line arg. In order to manage > >> this memory block, CMEM performs an ioremap() (either > >> ioremap_nocache() or ioremap_cached()), getting a > kernel virtual > >> pointer as a result. At some later time (as a result of a user > >> request) CMEM calls get_phys() on a kernel virtual > address within > >> this block (get_phys() also is called for user mappings to the > >> same block). The "new" get_phys() doesn't handle this kernel > >> translation correctly - the returned phys addr is always some > >> constant offset from the correct one. > >> > >> The ioremap functions obtain a virtual address block > large enough > >> for the physical block through the kernel > get_vm_area() function. > >> This function traverses the list 'vmlist' for an area that is > >> large enough to satisfy the request. The chosen > virtual address > >> is then mapped to the requested physical address. The problem > >> with get_phys() is that it uses the 'virt_to_phys()' API which > >> assumes a fixed, constant relationship between a > kernel virt addr > >> and the corresponding phys addr, but in this case the > virt addr is > >> some arbitrary address found to be free somewhere between > >> VMALLOC_START->VMALLOC_END. >From what I can see, the > only way to > >> do this translation is to traverse the 'vmlist' and find the > >> vm_struct for the virt addr, and return the phys_addr > element of > >> the vm_struct. However, vmlist is not exported, and I would > >> expect that there is a helper function somewhere for traversing > >> vmlist, but I can't find anything. > >> > >> The following code will return the wrong physp: > >> > >> block_virtp = (unsigned long) > ioremap_nocache(start, length); > >> ... > >> physp = get_phys(block_virtp); > >> > >> And here is the code in get_phys() that is returning > the wrong value: > >> > >> /* For kernel direct-mapped memory, take the easy way */ > >> if (virtp >= PAGE_OFFSET) { > >> physp = virt_to_phys((void *)virtp); > >> } > >> So, for example, if the physical block starts at > 0x87800000 and we do > >> block_virtp = (unsigned long) ioremap_nocache(0x87800000, > >> 0x800000); > >> we get block_virtp = 0xc80800000, and the call > >> physp = get_phys(0xc8080000); > >> returns a physp of 0x88080000, but it should be 0x87800000 (and > >> all phys addrs it returns are the same amount too high > - 0x00880000). > >> > >> Moving to a different chip w/ a different Linux, we get > >> block_virtp = 0xce000000 > >> and > >> get_phys(0xce000000) > >> returns 0x8e000000. Clearly both these systems have a > >> virt_to_phys() macro/function that simply subtracts 0x40000000, > >> but for virt addrs that were allocated by ioremap() there is no > >> such fixed relationship and the vmlist must be > consulted, but the > >> CMEM module can't get to the unexported symbol 'vmlist'. > >> > >> Please advise on how to successfully translate these types of > >> virtual addresses. > >> > >> Thanks in advance for any "pointers" you can provide :) > >> > >> Regards, > >> > >> - Rob Tivy > >> Texas Instruments, Santa Barbara > >> > >> _______________________________________________ > >> Davinci-linux-open-source mailing list > >> Davinci-linux-open-source@linux.davincidsp.com > >> <mailto:Davinci-linux-open-source@linux.davincidsp.com> > >> > http://linux.davincidsp.com/mailman/listinfo/davinci-linux-open-source > > > > > > > -------------------------------------------------------------- > ---------- > > > > _______________________________________________ > > Davinci-linux-open-source mailing list > > Davinci-linux-open-source@linux.davincidsp.com > > > http://linux.davincidsp.com/mailman/listinfo/davinci-linux-open-source > > _______________________________________________ > Davinci-linux-open-source mailing list > Davinci-linux-open-source@linux.davincidsp.com > http://linux.davincidsp.com/mailman/listinfo/davinci-linux-open-source > _______________________________________________ Davinci-linux-open-source mailing list Davinci-linux-open-source@linux.davincidsp.com http://linux.davincidsp.com/mailman/listinfo/davinci-linux-open-source
