On Fri, Nov 07, 2014 at 04:01:24PM +0800, lxu wrote:
> A multiple process DPDK application must mmap hugepages and pci resource into
> same virtual addresses. By default the virtual addresses chosen by the
> primary process automatically when calling the mmap. But sometime the virtual
> addresses chosen by the primary process isn't usable at secondary process.
> Such as the secondary process linked with more libraries than primary
> process. The library has been mapped into this virtual address. The command
> line parameter 'base-virtaddr' has been added for this situation. If it's
> configured, the hugepages will be mapped into this base address. But the
> virtual address of uio resource mapped still does not refer to the parameter.
> In that case it would still fail.
>
> This patch try to map uio resources after hugepages when the base_virtaddr is
> configured. So the error of "EAL: pci_map_resource(): cannot mmap" can be
> resolved by set base-virtaddr into free virtual address space.
>
> Signed-off-by: lxu <liang.xu at cinfotech.cn>
> ---
> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 29 ++++++++++++++++++++++++++++-
> 1 file changed, 28 insertions(+), 1 deletion(-)
>
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> index 7e62266..a2c9ab6 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> @@ -273,6 +273,24 @@ pci_get_uio_dev(struct rte_pci_device *dev, char *dstbuf,
> return uio_num;
> }
>
> +static inline const struct rte_memseg *
> +get_physmem_last(void)
> +{
> + const struct rte_memseg * seg = rte_eal_get_physmem_layout();
> + const struct rte_memseg * last = seg;
> + unsigned i = 0;
> +
> + for (i=0; i<RTE_MAX_MEMSEG; i++, seg++) {
> + if (seg->addr == NULL)
> + break;
> +
> + if(seg->addr > last->addr)
> + last = seg;
> +
> + }
> + return last;
> +}
> +
> /* map the PCI resource of a PCI device in virtual memory */
> int
> pci_uio_map_resource(struct rte_pci_device *dev)
> @@ -290,6 +308,13 @@ pci_uio_map_resource(struct rte_pci_device *dev)
> struct mapped_pci_resource *uio_res;
> struct pci_map *maps;
>
> + /* map uio resource into user required virtual address */
> + static void * requested_addr;
> + if (internal_config.base_virtaddr && NULL == requested_addr) {
> + const struct rte_memseg * last = get_physmem_last();
> + requested_addr = RTE_PTR_ADD(last->addr, last->len);
> + }
> +
Could you perhaps take anatoly's suggestion and modify this patch so the
checking
for a space right after the memsegs does not just depend on base-virtaddr being
set. Unless it causes other problems, there is no reason this code should not
always be used.
/Bruce
